feat: project language overrides and modern native Leiden - #3309
feat: project language overrides and modern native Leiden#3309andrebrait wants to merge 2 commits into
Conversation
…Graphify-Labs#2961) `.inc` is hard-mapped to the Pascal extractor, but it is "include file" in whatever language a project happens to use: PHP on pfSense, Pascal in a Delphi tree, SQL or assembly elsewhere. A PHP `.inc` parsed as Pascal does not fail — it yields a handful of incidental nodes, so the graph looks populated while the shipped runtime is missing from it (7 nodes instead of 471 on the reporter's file). Hardcoding one more extension (Graphify-Labs#1042's fix) cannot generalise, because `.inc` has no single correct global meaning. `.graphifyrc` gains `language.<ext>=<language | .ext>`: language.inc=php language.tpl=.ts The declaration reaches every place graphify keys a decision on the suffix: * detect.classify_file — a declared extension counts as code * extract._get_extractor — dispatch, ahead of the .h/.m sniffs * _lang_is_case_insensitive / — cross-file resolution rules _lang_family / the PHP pass * resolver_registry — the declared language's resolvers wake * cache.load_cached/save_cached — a `salt` folds the target language into the key, so the same bytes cached under the old extractor are never served for the new one * the extraction pool — an initializer forwards the mapping to workers, which start with empty module state under `spawn` The parser moves from hooks.py into a dependency-free graphify.rcfile so detect/extract (and the workers) can read it; hooks._load_graphifyrc keeps its name and behaviour. detect() and extract() activate `<root>/.graphifyrc` themselves, so the CLI, `update`/`watch`, the hooks, the MCP server, and the skill runbook all pick it up with no further wiring. A malformed file is reported once on stderr and the scan continues with the defaults.
The Graphify clustering path already imports graspologic_native directly, but the leiden extra still installed the Python graspologic package, whose metadata excludes Python 3.13+. Use graspologic-native's cp39-abi3 wheels instead so the extra works on current Python versions without a compiler or a rustworkx fork.
![graphify-labs[bot]](https://avatars.githubusercontent.com/in/4356270?s=60&v=4){kind=small}
There was a problem hiding this comment.
Graphify reviewed this change.
Worth a look — the grounded gate found no coupling regressions or blocking issues, but 5 advisory finding(s) below merit a look before merge.
Formal verification. No changes could be formally verified in this run.
Graphify review — findings
Adds project-level .graphifyrc language overrides so a repo can declare what an ambiguous extension means (language.inc=php), and threads that declaration through classification, extractor dispatch, cross-file resolution keys, and the AST cache. Salts cache keys via cache_salt so a file re-parsed under a remapped language never reuses an entry produced by the old extractor, and seeds worker processes with the parent's overrides through _worker_init so dispatch and cache keys agree under spawn. A remap wins over C/C++/ObjC sniffing, a typo in the rc file is reported once on stderr and the scan falls back to graphify's defaults, and the README documents the new config file alongside the leiden abi3-wheel note.
Worth a look
- Removed closing bracket leaves uv.lock invalid TOML —
uv.lock· Escalate · high- agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
- classify_file now depends on stale process-global language overrides —
graphify/detect.py:516· Escalate · medium- agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
- Process-global language overrides can race between concurrent scans —
graphify/detect.py:1669· Escalate · medium- agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
- Global language-override state mutated per-scan without synchronization —
graphify/detect.py:1670· Escalate · medium- agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
- _get_extractor may return None for remapped suffix with no registered extractor —
graphify/extract.py:5624· Escalate · medium- agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
Analysis details — impact, health, verification
Impact & health
Graphify review
Impact — 2652 functions depend on the 570 functions this change touches.
Health — this change adds coupling hotspots:
- new:
extract()— 523 callers, 46 callees - new:
_rebuild_code()— 113 callers, 50 callees - new:
detect()— 110 callers, 16 callees - new:
save_semantic_cache()— 58 callers, 9 callees - new:
save_manifest()— 40 callers, 11 callees - new:
load_cached()— 48 callers, 8 callees - new:
extract_files_direct()— 17 callers, 20 callees - new:
extract_js()— 85 callers, 4 callees - …and 57 more — each is listed as a finding
Verification — 2652 functions in the blast radius were not formally verified this run (proofs are advisory here).
Gate & verification
graphify gate
PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.
Advisory (not blocking):
- verification_scope: 2447 function(s) in the blast radius were not formally verified this run
Formal verification
Could not verify: Could not verify load\_cached.
The verifier did not have enough to check load\_cached, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: parameter `path` is annotated `Path` — outside the synthesizable primitive/collection set
Could not verify: Could not verify save\_cached.
The verifier did not have enough to check save\_cached, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: parameter `path` is annotated `Path` — outside the synthesizable primitive/collection set
Could not verify: Could not verify classify\_file.
The verifier did not have enough to check classify\_file, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: parameter `path` is annotated `Path` — outside the synthesizable primitive/collection set
Could not verify: Could not verify detect.
The verifier did not have enough to check detect, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: parameter `root` is annotated `Path` — outside the synthesizable primitive/collection set
Could not verify: Could not verify extract.
The verifier did not have enough to check extract, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: parameter `cache_root` is annotated `Path | None` — outside the synthesizable primitive/collection set
Could not verify: Could not verify \_extract\_parallel.
The verifier did not have enough to check \_extract\_parallel, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: parameter `root` is annotated `Path` — outside the synthesizable primitive/collection set
Could not verify: Could not verify \_extract\_sequential.
The verifier did not have enough to check \_extract\_sequential, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: parameter `root` is annotated `Path` — outside the synthesizable primitive/collection set
Could not verify: Could not verify \_extract\_single\_file.
The verifier did not have enough to check \_extract\_single\_file, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: not verifiable: all 8 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous (mostly ValueError — names the real obstacle, not a sampling gap)
Could not verify: Could not verify \_get\_extractor.
The verifier did not have enough to check \_get\_extractor, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: parameter `path` is annotated `Path` — outside the synthesizable primitive/collection set
Could not verify: Could not verify \_lang\_family.
The verifier did not have enough to check \_lang\_family, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: parameter `source_file` is annotated `object` — outside the synthesizable primitive/collection set
Could not verify: Could not verify \_lang\_is\_case\_insensitive.
The verifier did not have enough to check \_lang\_is\_case\_insensitive, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: parameter `source_file` is annotated `object` — outside the synthesizable primitive/collection set
Could not verify: Could not verify \_load\_graphifyrc.
The verifier did not have enough to check \_load\_graphifyrc, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: parameter `root` is annotated `Path` — outside the synthesizable primitive/collection set
Could not verify: Could not verify run\_language\_resolvers.
The verifier did not have enough to check run\_language\_resolvers, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: parameter `paths` is annotated `Sequence` — outside the synthesizable primitive/collection set
· 6 grounded finding(s) anchored inline below; 59 more finding(s) on lines outside this diff (see the check run).
| return hashlib.sha256(f"{h}:{salt}".encode("utf-8")).hexdigest() | ||
|
|
||
|
|
||
| def load_cached(path: Path, root: Path = Path("."), kind: str = "ast", |
There was a problem hiding this comment.
load_cached()
fans out to 8 callees (efferent coupling); 48 callers depend on it (afferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
| @@ -1063,9 +1076,13 @@ def load_cached(path: Path, root: Path = Path("."), kind: str = "ast", | |||
|
|
|||
| def save_cached(path: Path, result: dict, root: Path = Path("."), kind: str = "ast", | |||
There was a problem hiding this comment.
save_cached()
fans out to 7 callees (efferent coupling); 24 callers depend on it (afferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
| @@ -1664,6 +1667,9 @@ def _resolves_under_root(path: Path, root: Path) -> bool: | |||
|
|
|||
| def detect(root: Path, *, follow_symlinks: bool | None = None, google_workspace: bool | None = None, extra_excludes: list[str] | None = None, cache_root: Path | None = None, gitignore: bool = True) -> dict: | |||
There was a problem hiding this comment.
detect()
fans out to 16 callees (efferent coupling); 110 callers depend on it (afferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
| set_language_overrides(language_overrides) | ||
|
|
||
|
|
||
| def _extract_single_file(args: tuple) -> tuple[int, dict]: |
There was a problem hiding this comment.
_extract_single_file()
fans out to 7 callees (efferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
| ) from None | ||
|
|
||
|
|
||
| def load_graphifyrc(root: Path) -> dict: |
There was a problem hiding this comment.
load_graphifyrc()
10 callers depend on it (afferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
| return dict(_ACTIVE) | ||
|
|
||
|
|
||
| def activate_language_overrides(root: Path) -> dict[str, str]: |
There was a problem hiding this comment.
activate_language_overrides()
7 callers depend on it (afferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
|
Closing this in favour of #3310, which carries only the packaging change. This branch was cut wrong on my side: its first commit is @abhay-codes07's own commit from the still-open #3075, so this PR re-proposed someone else's work under my branch rather than adding anything to it. Apologies for the noise — #3075 stands on its own and I have no claim on it. The part that is mine is the |
2 participants
Summary
This branch rebases the implementation from #3075 onto current
v8and updates the Leiden extra to install the native binding on Python 3.13+.Changes
language.inc=phpand related cache, extraction, resolver, and hook plumbing).leidenextra fromgraspologic; python_version < '3.13'tograspologic-native>=1.2.1.allextra include the same native binding.cp39-abi3wheels.graspologic-native.The current
cluster.pyalready importsgraspologic_nativedirectly before falling back to the Python wrapper and NetworkX Louvain.graspologic-native1.3.1 publishes abi3 wheels for Linux, macOS, and Windows through current Python versions, so this does not require a rustworkx fork or a Rust toolchain on users' machines.Verification
The new test fails against the old extra and passes after the metadata change. The branch is based on current
v8(33362d9) and contains the complete language-override implementation from #3075 (4686466).