Skip to content

Commit 87c0d60

Browse files
authored
Merge pull request #125 from CoreTrace/feat/read-entry-points
feat: reach advisory entry points marked read through attribute reads
2 parents a10277f + 6895ea2 commit 87c0d60

12 files changed

Lines changed: 291 additions & 41 deletions

File tree

‎CHANGELOG.md‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,14 @@
66

77
- `yaml.load_all`, `yaml.full_load_all` and `yaml.unsafe_load_all` are deserialization sinks like their single-document versions; `yaml.load_all` with a safe loader is not, and `yaml.safe_load_all` stays safe (#123).
88

9+
### Dependencies
10+
11+
- An advisory entry point marked `read` is an attribute whose getter runs the affected code, such as `request.form`: reading it is reachable, called or not — a subscript, an iteration or a method called on it reads it — with one finding per function, where it is first read (#111).
12+
13+
### Plugins
14+
15+
- The call graph records the symbols each function reads, called or not, where it first reads each: `CallGraph.reads(function)` gives them as `SymbolRead` records, modules served from the cache keep them, and the cache format is bumped (#111).
16+
917
### Precision
1018

1119
- A sink call can be made safe by one of its arguments: a `SafeArgument` model names the argument and the values that take kinds off the sink, only when the call gives one explicitly. `yaml.load` with `SafeLoader`, `BaseLoader` or their C versions, under every spelling, is no longer an insecure deserialization; an absent loader, another loader, a variable or unpacked arguments still are (#121).

‎docs/plugins.md‎

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -227,8 +227,11 @@ contributed, the policy, the project root, and each module's imports, call graph
227227
functions. `ctx.functions(module)` gives every function as the call graph names it
228228
(`Admin.get`, `outer.inner`, `<module>`), with its span, so a finding can be placed in
229229
one, and the label of the entry point it is (`http` for a route, `argv` for a command)
230-
when the models make it one. The shipped `vulnerable-dependency`,
231-
`reachable-vulnerability` and `dependency-policy` plugins are of this kind.
230+
when the models make it one. `ctx.call_graph(module)` gives each function's call sites
231+
(`sites(function)`) and the symbols it reads, called or not (`reads(function)`: reading
232+
`request.form['name']` reads `python.flask.request.form`), each where it first reads it.
233+
The shipped `vulnerable-dependency`, `reachable-vulnerability` and `dependency-policy`
234+
plugins are of this kind.
232235

233236
```python
234237
from collections.abc import Sequence

‎docs/usage.md‎

Lines changed: 10 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -151,7 +151,7 @@ sources and in configuration files.
151151
| Rule | Trigger |
152152
|---|---|
153153
| `vulnerable-dependency` | A requirement allows a version affected by an advisory, reported at its line. |
154-
| `reachable-vulnerability` | A call in the project to an API the advisory affects. |
154+
| `reachable-vulnerability` | A call in the project to an API the advisory affects, or a read of an entry point marked `read`. |
155155
| `exploitable-vulnerability` | Attacker-controlled data reaches such a call. Critical. |
156156
| `denied-dependency` | A package the policy denies. |
157157
| `unpinned-dependency` | A requirement without an exact pin when the policy requires pins. |
@@ -291,6 +291,13 @@ call to either kind of symbol is reachable. `modules` names the top-level module
291291
package installs, so the analyzer can tell an imported package from a merely required
292292
one; it defaults to the package name.
293293

294+
An entry point marked `"read": true` is an attribute whose getter runs the affected code,
295+
such as a request body parsed on first access
296+
(`{"symbol": "python.flask.request.form", "justification": "…", "read": true}`): reading
297+
it is reachable, called or not — `request.form['name']`, `for key in request.form`,
298+
`request.form.get('name')` all read it. A function reading it several times is reported
299+
once, where it first reads it.
300+
294301
An `argument` condition is decided at each call. It names the argument by keyword and,
295302
when it may be passed positionally, by `position` (counted from 0, the receiver of a
296303
method excluded); `values` are what makes the call affected, written as the analyzer
@@ -305,7 +312,8 @@ template's origin — cannot be decided and is always pending review, never drop
305312
Every dependency finding records the highest level of evidence established in its
306313
`level` metadata: `declared` (the requirement allows a vulnerable version), `imported`
307314
(a module of the package is imported somewhere), `reachable` (an entry point or affected
308-
symbol is called) or `exploitable` (attacker input reaches it). The last two carry
315+
symbol is called, or an entry point marked `read` is read) or `exploitable` (attacker
316+
input reaches it). The last two carry
309317
`entry_point`, `justification`, `conditions`, and which of them the call meets
310318
(`conditions_met`) or leaves to review (`conditions_pending_review`). A requirement whose
311319
entry points are only called with arguments that contradict a condition stays
Lines changed: 52 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -1,16 +1,24 @@
1-
"""Calls, anywhere in the project, to an API affected by a vulnerable requirement."""
1+
"""Calls, anywhere in the project, to an API affected by a vulnerable requirement, and
2+
reads of an affected attribute whose getter runs the vulnerable code."""
23

34
from __future__ import annotations
45

5-
from collections.abc import Sequence
6+
from collections.abc import Iterator, Sequence
67
from typing import ClassVar
78

89
from coretrace_python.analysis import AnyAnalysis
9-
from coretrace_python.dependency import DependencyAnalysis
10-
from coretrace_python.dependency.correlation import affected_symbols, check_conditions, evidence
10+
from coretrace_python.dependency import Advisory, DependencyAnalysis
11+
from coretrace_python.dependency.correlation import (
12+
ConditionCheck,
13+
affected_symbols,
14+
check_conditions,
15+
evidence,
16+
)
1117
from coretrace_python.findings import Confidence, Finding
1218
from coretrace_python.interprocedural import CallGraphAnalysis, ExternalSymbol
1319
from coretrace_python.plugins import ProjectContext, ProjectPlugin
20+
from coretrace_python.semantic.symbols import SymbolId
21+
from coretrace_python.source import SourceSpan
1422

1523

1624
class ReachableVulnerabilityPlugin(ProjectPlugin):
@@ -29,21 +37,46 @@ def analyze_project(self, ctx: ProjectContext) -> Sequence[Finding]:
2937
if not isinstance(site.target, ExternalSymbol):
3038
continue
3139
for advisory in affected.get(site.target.symbol, ()):
32-
check = check_conditions(advisory.entry_point(site.target.symbol), site.arguments)
40+
entry = advisory.entry_point(site.target.symbol)
41+
if entry is not None and entry.read:
42+
continue # a call reads its callee first: reported with the reads
43+
check = check_conditions(entry, site.arguments)
3344
if check.contradicted is not None:
3445
continue
35-
findings.append(
36-
Finding(
37-
rule_id="reachable-vulnerability",
38-
message=(
39-
f"{advisory.id}: {site.target.symbol} is affected in the required "
40-
f"{advisory.package} {advisory.vulnerable}: {advisory.summary}"
41-
),
42-
severity=advisory.severity,
43-
confidence=Confidence.HIGH,
44-
span=site.location,
45-
function=function,
46-
metadata=evidence(advisory, site.target.symbol, "reachable", check),
47-
)
48-
)
46+
findings.append(_reached(advisory, site.target.symbol, site.location, function, check))
47+
# Reading ``request.form.get`` reads ``request.form``: one finding per entry
48+
# point and function, where the function first reads it.
49+
reported: set[tuple[Advisory, SymbolId]] = set()
50+
for read in graph.reads(function):
51+
for symbol in _read_through(read.symbol):
52+
for advisory in affected.get(symbol, ()):
53+
entry = advisory.entry_point(symbol)
54+
if entry is None or not entry.read or (advisory, symbol) in reported:
55+
continue
56+
reported.add((advisory, symbol))
57+
check = check_conditions(entry, None)
58+
findings.append(_reached(advisory, symbol, read.location, function, check))
4959
return findings
60+
61+
62+
def _read_through(symbol: SymbolId) -> Iterator[SymbolId]:
63+
"""``symbol`` and every symbol it is an attribute of, as reading it reads them all."""
64+
65+
components = symbol.canonical_name.split(".")
66+
for end in range(len(components), 1, -1):
67+
yield SymbolId(".".join(components[:end]))
68+
69+
70+
def _reached(advisory: Advisory, symbol: SymbolId, span: SourceSpan, function: str, check: ConditionCheck) -> Finding:
71+
return Finding(
72+
rule_id="reachable-vulnerability",
73+
message=(
74+
f"{advisory.id}: {symbol} is affected in the required "
75+
f"{advisory.package} {advisory.vulnerable}: {advisory.summary}"
76+
),
77+
severity=advisory.severity,
78+
confidence=Confidence.HIGH,
79+
span=span,
80+
function=function,
81+
metadata=evidence(advisory, symbol, "reachable", check),
82+
)

‎src/coretrace_python/cache.py‎

Lines changed: 14 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -4,9 +4,10 @@
44
source text and identity, the engine, schema and plugin API versions, the plugins and
55
their code, the security models, the advisories, the dependency graph, and the keys of
66
the project modules it imports transitively. A module whose key is unchanged on a later
7-
run is served from the cache: its summaries seed the project index, its call sites serve
8-
the project plugins and its findings are reported as they were. Entries are JSON, so a
9-
tampered or foreign file can never execute anything; an unreadable entry is a miss.
7+
run is served from the cache: its summaries seed the project index, its call sites and
8+
symbol reads serve the project plugins and its findings are reported as they were.
9+
Entries are JSON, so a tampered or foreign file can never execute anything; an
10+
unreadable entry is a miss.
1011
"""
1112

1213
from __future__ import annotations
@@ -33,13 +34,14 @@
3334
Mutation,
3435
NonlocalWrite,
3536
SummaryIndex,
37+
SymbolRead,
3638
Target,
3739
UnknownTarget,
3840
)
3941
from coretrace_python.semantic.symbols import SymbolId
4042
from coretrace_python.source import SourceId, SourceSpan
4143

42-
CACHE_FORMAT = 7
44+
CACHE_FORMAT = 8
4345

4446

4547
@dataclass(frozen=True)
@@ -50,6 +52,7 @@ class CachedModule:
5052
summaries: Mapping[str, FunctionSummary]
5153
sites: tuple[CallSite, ...]
5254
findings: tuple[Finding, ...]
55+
reads: tuple[SymbolRead, ...] = ()
5356

5457
def __post_init__(self) -> None:
5558
object.__setattr__(self, "summaries", MappingProxyType(dict(self.summaries)))
@@ -134,6 +137,7 @@ def encode(module: CachedModule) -> dict[str, Any]:
134137
"summaries": {name: _encode_summary(s) for name, s in module.summaries.items()},
135138
"sites": [_encode_site(site) for site in module.sites],
136139
"findings": [_encode_finding(finding) for finding in module.findings],
140+
"reads": [[r.function, _encode_span(r.location), str(r.symbol)] for r in module.reads],
137141
}
138142

139143

@@ -145,6 +149,7 @@ def decode(data: Mapping[str, Any]) -> CachedModule:
145149
{_string(name): _decode_summary(s) for name, s in data["summaries"].items()},
146150
tuple(_decode_site(site) for site in data["sites"]),
147151
tuple(_decode_finding(finding) for finding in data["findings"]),
152+
tuple(_decode_read(read) for read in data["reads"]),
148153
)
149154

150155

@@ -194,6 +199,11 @@ def _decode_function(data: Any) -> ModuleFunction:
194199
return ModuleFunction(_string(name), _decode_span(span), None if entry_point is None else _string(entry_point))
195200

196201

202+
def _decode_read(data: Any) -> SymbolRead:
203+
function, span, symbol = data
204+
return SymbolRead(_string(function), _decode_span(span), SymbolId(_string(symbol)))
205+
206+
197207
def _encode_finding(finding: Finding) -> dict[str, Any]:
198208
return {
199209
"rule": finding.rule_id,

‎src/coretrace_python/dependency/advisories.py‎

Lines changed: 16 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -138,14 +138,7 @@ def dump_advisories(advisories: Iterable[Advisory]) -> str:
138138
"severity": a.severity.value,
139139
"affected_symbols": [str(s) for s in a.affected_symbols],
140140
"aliases": list(a.aliases),
141-
"entry_points": [
142-
{
143-
"symbol": str(e.symbol),
144-
"justification": e.justification,
145-
"conditions": [_condition_entry(c) for c in e.conditions],
146-
}
147-
for e in a.entry_points
148-
],
141+
"entry_points": [_entry_point_entry(e) for e in a.entry_points],
149142
"modules": list(a.modules),
150143
}
151144
for a in advisories
@@ -154,6 +147,17 @@ def dump_advisories(advisories: Iterable[Advisory]) -> str:
154147
return json.dumps(document, indent=2) + "\n"
155148

156149

150+
def _entry_point_entry(entry_point: AdvisoryEntryPoint) -> dict[str, Any]:
151+
entry: dict[str, Any] = {
152+
"symbol": str(entry_point.symbol),
153+
"justification": entry_point.justification,
154+
"conditions": [_condition_entry(c) for c in entry_point.conditions],
155+
}
156+
if entry_point.read:
157+
entry["read"] = True
158+
return entry
159+
160+
157161
def _condition_entry(condition: Condition) -> dict[str, Any]:
158162
entry: dict[str, Any] = {"kind": condition.kind, "text": condition.text}
159163
if condition.argument is not None:
@@ -194,10 +198,14 @@ def _advisory(entry: Mapping[str, Any]) -> Advisory:
194198

195199

196200
def _entry_point(entry: Mapping[str, Any]) -> AdvisoryEntryPoint:
201+
read = entry.get("read", False)
202+
if not isinstance(read, bool):
203+
raise TypeError(f"entry point read must be true or false, got {read!r}")
197204
return AdvisoryEntryPoint(
198205
SymbolId(str(entry["symbol"])),
199206
str(entry["justification"]),
200207
tuple(_condition(c) for c in entry.get("conditions") or []),
208+
read,
201209
)
202210

203211

‎src/coretrace_python/dependency/graph.py‎

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -168,11 +168,16 @@ def checkable(self) -> bool:
168168
@dataclass(frozen=True)
169169
class AdvisoryEntryPoint:
170170
"""A public API through which a project reaches an affected symbol, justified by the
171-
fixing commit or by a call path, with the conditions under which it is affected."""
171+
fixing commit or by a call path, with the conditions under which it is affected.
172+
173+
An entry point is reached by a call, or — when ``read`` is set, for an attribute whose
174+
getter runs the affected code, such as a lazily parsed request body — by any read of
175+
it or of an attribute of it."""
172176

173177
symbol: SymbolId
174178
justification: str
175179
conditions: tuple[Condition, ...] = ()
180+
read: bool = False
176181

177182

178183
DIRECT = "affected symbol, changed by the fix"

‎src/coretrace_python/engine.py‎

Lines changed: 13 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -76,6 +76,7 @@
7676
ProjectSummaries,
7777
SummaryAnalysis,
7878
SummaryIndex,
79+
SymbolRead,
7980
build_module_graph,
8081
discover_sources,
8182
project_symbol,
@@ -466,7 +467,15 @@ def analyze_project(
466467
sites: dict[str, list[CallSite]] = {function.name: [] for function in entry.functions}
467468
for site in entry.sites:
468469
sites.setdefault(site.caller, []).append(site)
469-
call_graphs[name] = CallGraph({}, {f: tuple(s) for f, s in sites.items()}, frozenset())
470+
reads: dict[str, list[SymbolRead]] = {}
471+
for read in entry.reads:
472+
reads.setdefault(read.function, []).append(read)
473+
call_graphs[name] = CallGraph(
474+
{},
475+
{f: tuple(s) for f, s in sites.items()},
476+
frozenset(),
477+
reads={f: tuple(r) for f, r in reads.items()},
478+
)
470479
context = ProjectContext(graph, dependencies, advisories, analysable, call_graphs, policy, root, functions)
471480
for plugin in all_plugins:
472481
if isinstance(plugin, ProjectPlugin):
@@ -651,7 +660,8 @@ def _configuration_key(
651660
def _analyse_module(
652661
manager: AnalysisManager, plugins: tuple[Plugin, ...], affected: Affected
653662
) -> CachedModule:
654-
"""One module's findings, summaries and call sites: what the cache keeps (§11)."""
663+
"""One module's findings, summaries, call sites and symbol reads: what the cache
664+
keeps (§11)."""
655665

656666
findings, supported = _check_module(manager, plugins)
657667
graph = manager.get(CallGraphAnalysis)
@@ -671,6 +681,7 @@ def _analyse_module(
671681
_summaries_of(manager),
672682
tuple(site for function in graph.functions for site in graph.sites(function)),
673683
(*findings, *correlated),
684+
tuple(read for function in graph.functions for read in graph.reads(function)),
674685
)
675686

676687

‎src/coretrace_python/interprocedural/__init__.py‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@
88
ExternalSymbol,
99
KnownFunction,
1010
ModuleFunction,
11+
SymbolRead,
1112
Target,
1213
UnknownTarget,
1314
)
@@ -46,6 +47,7 @@
4647
"SummaryAnalysis",
4748
"SummaryIndex",
4849
"SummaryTable",
50+
"SymbolRead",
4951
"Target",
5052
"UnknownTarget",
5153
"build_module_graph",

0 commit comments

Comments
 (0)