Optimize graph reports and low-inference graph execution

[forhappy]

Summary

This PR covers the graph-report, inference, performance, and query-quality session scope:

• keep GRAPH_REPORT.md community labels concise, deterministic, and unique without changing opaque community identities;
• expose low|medium|high|max inference profiles while retaining max as the compatibility default;
• admit safely filterable low-inference relationships before node-link materialization and prune only unreferenced inferred placeholders;
• bound incremental reclustering to the changed region, preserving unaffected assignments and falling back to full clustering when safety limits are exceeded;
• rank source-backed, operation-specific query roots ahead of generic types and method/subword distractors;
• reduce local SQLite query startup work by reusing initial immutable snapshot metadata; and
• add an independently source-reviewed delta-rs oracle suite with symmetric Compass/Graphify scoring.

Implemented mitigations

Earlier inference admission

Inference policy is applied to coalesced raw records before node-link materialization. Calls are suppressed only when their inferred provenance is known and safe to exclude; unresolved endpoints, constructible targets, metadata relationships, hyperedge references, unknown provenance, and duplicate-evidence collisions remain conservative. Orphan cleanup is reference-aware.

Native testing found and fixed an overly broad collision key: occurrence protection now includes source and target, so true duplicate evidence remains protected without treating unrelated sparse call records as one collision group. This still does not eliminate every extraction and cross-file-resolution cost.

Bounded incremental clustering

Incremental builds reuse prior clustered artifacts for fact-neutral changes. Topology-changing updates recluster changed sources, touched communities, and their boundary closure under explicit node and ratio limits. Node deletion, missing prior assignments, invalid bounds, or oversized regions deterministically fall back to full Louvain clustering.

Query specificity and startup

Discovery separates entity-subject tokens from action predicates, restricts production roots to source-backed representation types, adds bounded composite-initialism matching, and rejects isolated generic subword noise. Immutable-store queries reuse graph identity, generation, and partial-build metadata from the first reader instead of reopening the snapshot for the same information.

Native compilation also caught and fixed operation-predicate set intersection before publication.

Independent accuracy oracles

The focused suite pins delta-rs at c27874c10043b5ccf0207d27eee148be0a033c6e and contains 20 positive symbol/file/line labels plus five negative controls, each with an independent judgment source and rationale. Compass validation checks typed identity, anchors, direction, ambiguity/no-match behavior, and bounded work. Graphify is scored against the same declared seed identity and anchor rather than against tool agreement.

Fresh post-mitigation replay

The complete focused comparison ran at Compass 54b69f39, Graphify 09a34ad8, and the pinned delta-rs commit. It used three full build repetitions, ten measured batches for every query, one unmeasured warmup per query mode, release binaries, low inference, and clustering for both tools. The harness completed and intentionally exited nonzero because declared qualification gates were missed; no threshold or oracle label was weakened.

Build workload	Compass low p50	Graphify p50	Speedup	Compass RSS	Graphify RSS
Cold	4.312 s	7.337 s	1.70x	1,101.17 MiB	149.47 MiB
Unchanged warm	0.318 s	3.833 s	12.04x	24.47 MiB	230.44 MiB
One-file incremental	1.218 s	3.810 s	3.13x	141.11 MiB	232.42 MiB

The incremental p50 improved from the earlier 4.111 s to 1.218 s, and peak RSS fell from 1,312.2 MiB to 141.11 MiB. Cold cost and memory remained essentially unchanged, showing that admission must move farther upstream of resolver allocations.

Independently labeled query metric	Compass low	Graphify
Positive Top-1	8/20 (40%)	1/20 (5%)
Positive MRR@10	0.4917	0.1125
Positive recall@10	60%	20%
Complete positive answer	6/20 (30%)	1/20 (5%)
Negative-control accuracy	5/5 (100%)	5/5 (100%)

All ten batches produced identical ranking evidence. No positive row passed both tools' strict eligibility checks, so this replay does not support a positive-query cross-tool speed claim. Both tools passed all five negative rows: Compass fresh p50 was 0.084–0.606 s versus Graphify 0.539–0.686 s. Compass exceeded 5x on one negative row and achieved 1.12x–1.33x on the other four.

The low graph contained 9,982 nodes and 25,206 relationships versus Graphify's 9,670 and 27,173. All Compass relationships carried exact evidence; only 100 (0.40%) were convention-origin, versus Graphify's 1,031 INFERRED relationships (3.79%). The source-aware compatibility comparator still reported 82 missing/two ambiguous Graphify node hypotheses and 4,957 missing/36 ambiguous Graphify edge hypotheses. These are compatibility diagnostics, not ground-truth precision or recall.

The replay therefore validates the incremental-clustering and ranking mitigations, but does not meet the 5x cold/query or cold-memory goals. The remaining priorities are earlier resolver admission, cold peak-memory reduction, the 12 failed positive rankings, and non-startup query search cost. Full evidence and claim boundaries are recorded in docs/implementation/low-inference-graphify-evaluation.md.

Compatibility

• Existing public entry points retain max inference behavior by default.
• No graph schema version or stable identity contract changes.
• Incremental clustering falls back instead of publishing a partial or ambiguous result.
• Unknown provenance and attributes remain conservative.

Validation

Native code validation at 54b69f39 passed from the clean detached PR worktree:

• cargo fmt --all -- --check
• targeted cargo clippy for graph/query/core/store with all targets, all features, and -D warnings
• cargo clippy --workspace --lib --bins --locked -- -D warnings
• cargo test --workspace --lib --bins --locked
• full compass-graph, compass-query, compass-store, and compass-core suites
• cargo test -p compass-cli --test compass_product --locked
• sh scripts/check_product_boundary.sh
• ./scripts/qualify_code_graph_v1.sh --fixtures-only
• 35 Python benchmark/oracle unit tests

Fixture qualification produced deterministic clean/warm/rebuild/restored/alternate-checkout byte equality with graph digest sha256:ede1befdb08d19a487a7d006cff15b0a9047cf1a84367eb7a65d2e5dc7d5e7f7.

All 14 GitHub checks passed at the validated code head. Commit 410ec0a0 only adds the dated replay evidence to the evaluation document.

Review focus

The highest-risk areas are conservative inference admission, deletion/boundary behavior in incremental clustering, and ranking eligibility versus ranking-only synonyms. Regression tests cover each boundary, and the repository baseline plus code-graph qualification pass at the pushed code head.