Skip to content

fix(analyzers): share Python AST parsing for environment-read detection - #332

Merged
keshprad merged 3 commits into
mainfrom
codex/fix-issue-329-env-ast
Aug 5, 2026
Merged

fix(analyzers): share Python AST parsing for environment-read detection#332
keshprad merged 3 commits into
mainfrom
codex/fix-issue-329-env-ast

Conversation

@mohgupta-ship-it

@mohgupta-ship-it mohgupta-ship-it commented Aug 2, 2026

Copy link
Copy Markdown
Member

Summary

Replaces spelling-sensitive Python E2 checks with AST-based environment-read detection and introduces shared per-scan Python AST parsing for all AST-consuming analyzers.

  • Detects complete environment materialization or enumeration through copy(), dict(os.environ), dict unpacking, .items(), direct iteration, and copy.copy() variants.
  • Resolves os and environ import aliases and is insensitive to formatting such as os . environ . copy ().
  • Shares one parsed AST per eligible Python file across E2, output-handling, behavioral-AST, and taint-tracking analyzers.
  • Keeps AST objects outside checkpointed graph state, with bounded per-scan caching and cleanup after reporting.
  • Preserves non-Python checks and regex fallback for syntactically invalid Python.
  • Does not classify single-key reads or passing os.environ to a child process as full-environment harvesting.

Motivation

E2 previously matched literal text such as os.environ.copy(), so aliases and equivalent Python syntax bypassed the rule. The shared AST layer fixes that detection gap while removing duplicate parsing across analyzer branches.

Validation

  • uv run --no-sync python -m pytest tests/nodes/analyzers/test_static_patterns.py tests/unit/test_patterns.py -q
  • uv run --with hatchling --no-sync python -m pytest -qq --disable-warnings -m "not integration and not provider" tests/
  • uv run --no-sync make lint
  • uv run --no-sync make format-check

This supersedes the regex-only coverage in #331.

Fixes #329

@mohgupta-ship-it
mohgupta-ship-it force-pushed the codex/fix-issue-329-env-ast branch from 73baca8 to 641614e Compare August 2, 2026 09:27
@keshprad

keshprad commented Aug 3, 2026

Copy link
Copy Markdown
Member

Could this PR also introduce a shared Python AST parsing utility for the existing AST analyzer, output_handling, data_exfiltration, and future AST-backed analyzers?

Each currently parses Python independently. Centralizing parsing would avoid repeated full-file parses during a scan and give us one place for syntax-error handling, alias resolution, and eventually scope-aware name resolution. E2 can still own its detection logic here; it would simply consume the shared parsed tree.

Signed-off-by: Mohit Gupta <mohgupta@nvidia.com>
Signed-off-by: Mohit Gupta <mohgupta@nvidia.com>
@mohgupta-ship-it
mohgupta-ship-it force-pushed the codex/fix-issue-329-env-ast branch from 641614e to 1bf8b72 Compare August 4, 2026 10:07
@keshprad
keshprad marked this pull request as ready for review August 4, 2026 23:07
@keshprad

keshprad commented Aug 4, 2026

Copy link
Copy Markdown
Member

Valid calls such as os.environ.get(key="API_KEY") have no positional args, so the condition at static_patterns_data_exfiltration.py:245 is false and the continue skips E2. The old regex matched this form, so this loses coverage and provides a simple bypass for sensitive environment reads. Please inspect the key keyword too and add regression coverage.

@keshprad keshprad changed the title fix(analyzer): detect full environment reads via AST fix(analyzers): share Python AST parsing for environment-read detection Aug 4, 2026
Signed-off-by: keshavp <32313895+keshprad@users.noreply.github.com>

@keshprad keshprad left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved: the keyword-form E2 regression is fixed, focused validation passes, and all PR checks are green.

@keshprad
keshprad merged commit e6ebe70 into main Aug 5, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

E2 "Env Variable Harvesting" matches one spelling, so whitespace alone flips DO_NOT_INSTALL to SAFE

2 participants