Title: feat(cli): quiet-by-default extract with --verbose - #3290
Title: feat(cli): quiet-by-default extract with --verbose#3290albertbu wants to merge 4 commits into
Conversation
Use git ls-files, Google repo manifests, and nested worktrees so ignored trees are pruned at directory granularity instead of a Python walk. Co-authored-by: Cursor <cursoragent@cursor.com>
Gating the out-of-root check on islink() admitted regular files reached by following a symlink directory, including graphify-out/memory/ which skips ignore pruning. Restore the v8 resolve()-under-root admit guard and prune those dirs during the memory walk too. Co-authored-by: Cursor <cursoragent@cursor.com>
Route CPU-heavy extract and JS/Python fact collection through graphify.parallel so worker counts follow GRAPHIFY_MAX_WORKERS. Co-authored-by: Cursor <cursoragent@cursor.com>
Keep detect/AST/resolution chatter behind --verbose (or GRAPHIFY_VERBOSE). Also escape the merge-driver for git-config on POSIX and restore pytest norecursedirs defaults so Hypothesis does not warn at collection. Co-authored-by: Cursor <cursoragent@cursor.com>
910536b to
7a16d32
Compare
![graphify-labs[bot]](https://avatars.githubusercontent.com/in/4356270?s=60&v=4){kind=small}
There was a problem hiding this comment.
Graphify reviewed this change.
Worth a look — the grounded gate found no coupling regressions or blocking issues, but 5 advisory finding(s) below merit a look before merge.
Formal verification. 4 change(s) tested, no difference found (not proven).
Graphify review — findings
Adds a --verbose flag (and GRAPHIFY_VERBOSE=1) to graphify extract, making detect/AST/resolution progress quiet by default while symbol resolution now reports JS/TS facts, Python facts, and edge-application counts; --timing is unchanged. Routes AST extraction, JS/Python symbol resolution, and cache probes through shared graphify.parallel pools (GRAPHIFY_MAX_WORKERS), and guards the _stat_index cache with an RLock so concurrent probes load, mutate, and flush the index without racing — publishing _stat_index_root last so a second thread never sees an empty index as ready. Teaches detect() to enumerate large trees via git ls-files, repo project lists, and nested worktrees/submodules to skip gitignored directories, and fixes the merge-driver config to round-trip a quoted Windows interpreter path on POSIX git (#2166).
Worth a look
- Misindented cache-hit lock block leaves an
ifwithout an indented body —graphify/cache.py· Escalate · high- agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
- Shared ignored_dirs set mutated concurrently by parallel subtree walks —
graphify/detect.py· Escalate · high- agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
- file_hash writes _stat_index_dirty without global declaration under lock —
graphify/cache.py:548· Escalate · medium- agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
- Extract progress output is now suppressed unless verbose is enabled —
graphify/cli.py:658· Escalate · medium- agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
- Merge mode no longer prunes excluded or stale graph sources —
graphify/cli.py:4415· Escalate · medium- agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
Analysis details — impact, health, verification
Impact & health
Graphify review
Impact — 3794 functions depend on the 2026 functions this change touches.
Health — this change adds coupling hotspots:
- new:
extract()— 527 callers, 43 callees - new:
_rebuild_code()— 113 callers, 50 callees - new:
detect()— 119 callers, 17 callees - new:
save_semantic_cache()— 58 callers, 9 callees - new:
_extract_generic()— 18 callers, 25 callees - new:
save_manifest()— 40 callers, 11 callees - new:
extract_files_direct()— 17 callers, 20 callees - new:
extract_js()— 85 callers, 4 callees - …and 80 more — each is listed as a finding
Verification — 3794 functions in the blast radius were not formally verified this run (proofs are advisory here).
Gate & verification
graphify gate
PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.
Advisory (not blocking):
- verification_scope: 3766 function(s) in the blast radius were not formally verified this run
Formal verification
Could not verify: Could not verify cached\_word\_count.
The verifier did not have enough to check cached\_word\_count, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: parameter `path` is annotated `Path` — outside the synthesizable primitive/collection set
Could not verify: Could not verify \_ensure\_stat\_index.
The verifier did not have enough to check \_ensure\_stat\_index, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: parameter `root` is annotated `Path` — outside the synthesizable primitive/collection set
Could not verify: Could not verify file\_hash.
The verifier did not have enough to check file\_hash, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: parameter `path` is annotated `Path` — outside the synthesizable primitive/collection set
No difference found (not proven): No behavior difference found in \_flush\_stat\_index (not a proof).
The verifier ran both versions of \_flush\_stat\_index on many inputs and saw identical behavior every time. Strong evidence the change is safe, but evidence, not a proof.
Guarantee: Empirical: differential testing (both versions run on many generated inputs). A divergence on an untested input remains possible, so this is 'no counterexample found', not 'proven equivalent'.
Note: An input the sampler did not try could still differ.
Could not verify: Could not verify dispatch\_command.
The verifier did not have enough to check dispatch\_command, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: not verifiable: all 23 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous (mostly SystemExit — names the real obstacle, not a sampling gap)
No difference found (not proven): No behavior difference found in mark (not a proof).
The verifier ran both versions of mark on many inputs and saw identical behavior every time. Strong evidence the change is safe, but evidence, not a proof.
Guarantee: Empirical: differential testing (both versions run on many generated inputs). A divergence on an untested input remains possible, so this is 'no counterexample found', not 'proven equivalent'.
Note: An input the sampler did not try could still differ.
No difference found (not proven): No behavior difference found in total (not a proof).
The verifier ran both versions of total on many inputs and saw identical behavior every time. Strong evidence the change is safe, but evidence, not a proof.
Guarantee: Empirical: differential testing (both versions run on many generated inputs). A divergence on an untested input remains possible, so this is 'no counterexample found', not 'proven equivalent'.
Note: An input the sampler did not try could still differ.
Could not verify: Could not verify detect.
The verifier did not have enough to check detect, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: parameter `root` is annotated `Path` — outside the synthesizable primitive/collection set
Could not verify: Could not verify detect\_incremental.
The verifier did not have enough to check detect\_incremental, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: parameter `root` is annotated `Path` — outside the synthesizable primitive/collection set
Could not verify: Could not verify \_git\_tracked\_path\_keys.
The verifier did not have enough to check \_git\_tracked\_path\_keys, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: parameter `root` is annotated `Path` — outside the synthesizable primitive/collection set
Could not verify: Could not verify \_is\_noise\_dir.
The verifier did not have enough to check \_is\_noise\_dir, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: parameter `parent` is annotated `'Path | None'` — outside the synthesizable primitive/collection set
Could not verify: Could not verify extract.
The verifier did not have enough to check extract, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: parameter `cache_root` is annotated `Path | None` — outside the synthesizable primitive/collection set
Could not verify: Could not verify \_extract\_parallel.
The verifier did not have enough to check \_extract\_parallel, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: parameter `root` is annotated `Path` — outside the synthesizable primitive/collection set
Could not verify: Could not verify \_extract\_sequential.
The verifier did not have enough to check \_extract\_sequential, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: parameter `root` is annotated `Path` — outside the synthesizable primitive/collection set
Could not verify: Could not verify \_augment\_symbol\_resolution\_edges.
The verifier did not have enough to check \_augment\_symbol\_resolution\_edges, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: parameter `root` is annotated `Path` — outside the synthesizable primitive/collection set
Could not verify: Could not verify \_collect\_js\_symbol\_resolution\_facts.
The verifier did not have enough to check \_collect\_js\_symbol\_resolution\_facts, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: parameter `facts` is annotated `_SymbolResolutionFacts` — outside the synthesizable primitive/collection set
Could not verify: Could not verify \_collect\_python\_symbol\_resolution\_facts.
The verifier did not have enough to check \_collect\_python\_symbol\_resolution\_facts, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: parameter `root` is annotated `Path` — outside the synthesizable primitive/collection set
Could not verify: Could not verify \_register\_merge\_driver.
The verifier did not have enough to check \_register\_merge\_driver, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: parameter `root` is annotated `Path` — outside the synthesizable primitive/collection set
No difference found (not proven): No behavior difference found in \_run\_cli (not a proof).
The verifier ran both versions of \_run\_cli on many inputs and saw identical behavior every time. Strong evidence the change is safe, but evidence, not a proof.
Guarantee: Empirical: differential testing (both versions run on many generated inputs). A divergence on an untested input remains possible, so this is 'no counterexample found', not 'proven equivalent'.
Note: An input the sampler did not try could still differ.
· 11 grounded finding(s) anchored inline below; 77 more finding(s) on lines outside this diff (see the check run).
| return name[dot:].lower() in _CORPUS_EXT_LOWER | ||
|
|
||
|
|
||
| def _is_noise_dir(part: str, parent: "Path | None" = None) -> bool: |
There was a problem hiding this comment.
_is_noise_dir()
10 callers depend on it (afferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
| return _path_is_under(worktree, root) or _path_is_under(root, worktree) | ||
|
|
||
|
|
||
| def _git_enumerate_named_worktrees( |
There was a problem hiding this comment.
_git_enumerate_named_worktrees()
fans out to 7 callees (efferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
| ) | ||
|
|
||
|
|
||
| def _git_enumerate_files( |
There was a problem hiding this comment.
_git_enumerate_files()
fans out to 8 callees (efferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
|
|
||
|
|
||
| def detect(root: Path, *, follow_symlinks: bool | None = None, google_workspace: bool | None = None, extra_excludes: list[str] | None = None, cache_root: Path | None = None, gitignore: bool = True) -> dict: | ||
| def detect(root: Path, *, follow_symlinks: bool | None = None, google_workspace: bool | None = None, extra_excludes: list[str] | None = None, cache_root: Path | None = None, gitignore: bool = True, code_only: bool = False) -> dict: |
There was a problem hiding this comment.
detect()
fans out to 17 callees (efferent coupling); 119 callers depend on it (afferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
| vprint(f"scanning {root} ...") | ||
| heartbeat = _Heartbeat("scanning") | ||
|
|
||
| def _walk_from( |
There was a problem hiding this comment.
_walk_from()
fans out to 8 callees (efferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
| return per_file, uncached_work | ||
|
|
||
|
|
||
| def _extract_parallel( |
There was a problem hiding this comment.
_extract_parallel()
high coupling complexity (Ca·Ce = 20).
Grounded coupling-delta finding (deterministic), not an LLM guess.
| _PARALLEL_THRESHOLD = PARALLEL_THRESHOLD | ||
|
|
||
|
|
||
| def extract( |
There was a problem hiding this comment.
extract()
fans out to 43 callees (efferent coupling); 527 callers depend on it (afferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
| ] | ||
| if not js_paths: | ||
| return | ||
| def _collect_js_facts_for_path(path: Path) -> _SymbolResolutionFacts: |
There was a problem hiding this comment.
_collect_js_facts_for_path()
fans out to 26 callees (efferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
| return _read_text(function_node, source) | ||
| return None | ||
|
|
||
| def _collect_python_facts_for_path(path: Path, root: Path) -> _SymbolResolutionFacts: |
There was a problem hiding this comment.
_collect_python_facts_for_path()
fans out to 11 callees (efferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
| return [fn(item) for item in batch] | ||
|
|
||
|
|
||
| def map_in_process_pool( |
There was a problem hiding this comment.
map_in_process_pool()
8 callers depend on it (afferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
1 participant
Review this slice: albertbu/graphify@albertb/extract-pools...albertb/extract-verbose
Stacked on
albertb/extract-pools(PR2). Review the compare link above, or the single commit910536b— the GitHub diff vsv8includes PR1+PR2.Summary
graphify extractis quiet by default.--verbose/GRAPHIFY_VERBOSE=1turns on detect, AST, and symbol-resolution progress.--timingis unchanged.--verbose(tests and operators need to see that mode).git configvalue is escaped so a quoted Windows interpreter path round-trips on POSIX git (hook install emits empty _PINNED on Windows uv installs, so every post-commit rebuild silently no-ops #2166).norecursedirsrestates the defaults (including.hypothesis); chunking tests assert the out-of-scope cacheRuntimeWarning.Test plan
graphify extract <dir> --code-onlyis quiet;--verboseshows detect/AST/resolving linesincremental scan of …without--verbosepytest tests/test_extract_cli.py tests/test_extract_code_only_cli.py tests/test_hooks.py tests/test_chunking.py -q