Skip to content

fix(query): rank same-hop nodes by query relevance before degree under the budget - #3284

Open
andytsai821201-spec wants to merge 1 commit into
Graphify-Labs:v8from
andytsai821201-spec:fix/query-relevance-aware-truncation
Open

fix(query): rank same-hop nodes by query relevance before degree under the budget#3284
andytsai821201-spec wants to merge 1 commit into
Graphify-Labs:v8from
andytsai821201-spec:fix/query-relevance-aware-truncation

Conversation

@andytsai821201-spec

Copy link
Copy Markdown

Summary

query already scores every node against the question to pick seeds, then throws that ranking away before rendering. Under a tight --budget, the renderer's hop-then-degree order lets an unrelated hub outrank a node that actually matched a query term in the same hop layer — so the hub survives and the answer is cut. This threads the existing ranking into _subgraph_to_text as a second sort key. ~30 lines, no new scoring pass, no new flag.

Problem

_subgraph_to_text orders non-seed nodes by (hop distance, -degree, id) (#BUG2). Given seed S with two depth-1 neighbors — RetryTimeout (degree 1, matches the term "timeout") and Logger (a hub wired to eight leaves, matches nothing) — the query "CompanySpacingGate timeout" at --budget 60 renders:

NODE CompanySpacingGate
NODE TimeoutPolicy        <- per-term seed for "timeout"
NODE Logger               <- hub, zero relevance
... (truncated — RetryTimeout cut)

RetryTimeout is a genuine match that did not win the single per-term seed seat, so it is neither protected as a seed nor ranked above the hub — the exact node the question is about is the one dropped.

Fix

  • _subgraph_to_text(..., scores: dict[str, float] | None = None): sort key becomes (hop, -score, -degree, id). Hop distance stays primary (the #BUG2 intent is preserved); relevance decides within a layer; degree and id keep the tail deterministic.
  • _query_graph_text passes {nid: score for score, nid in qs.ranked} — the ranking _score_query already computed for seed selection.

Byte-identical when no non-seed node scored: a missing/empty map or all-zero scores fall through to the previous degree order. Seeds still render first and still survive truncation. path / explain untouched.

Tests

  • test_subgraph_to_text_query_match_outranks_hub_in_same_hop_layer — unit: scored node renders before a higher-degree unscored node in the same layer.
  • test_subgraph_to_text_without_scores_keeps_degree_order — unit: scores=None, {}, and all-zero produce identical output to today.
  • test_query_graph_text_threads_relevance_scores_into_rendering — end to end: a second term-matching node that is not the per-term seed survives a tight budget ahead of the hub.

Watched all three fail before the change (the e2e failure reproduces the ['CompanySpacingGate', 'TimeoutPolicy', 'Logger'] output above). tests/test_serve.py: 149 passed. Full suite: 5,167 passed; the 14 failures present (test_ollama_retry_cap.py missing openai, test_skillgen.py audit baselines) reproduce identically on pristine v8 in this environment and are unrelated.

Relation to existing work

Deliberately narrower than #347 / #1856 (which rework retrieval end to end): this is an ordering-only change inside the existing renderer. Complementary to #1303 (degree-blind seed ties) and the resolved #897 (seed scoring) — those concern which seeds are chosen; this concerns what survives the budget after seeds are fixed.

🤖 Generated with Claude Code

…r the budget

`_query_graph_text` already scores every node against the question
(`_score_query` → `qs.ranked`) to pick seeds, then discards that ranking
before rendering. `_subgraph_to_text` orders non-seed nodes by hop
distance, then degree (#BUG2), so within one hop layer an unrelated hub
always outranks a node that matched a query term — and a tight `--budget`
cuts the node that answers the question while keeping the hub.

Thread the existing ranking into the renderer as `scores` and insert it as
the second sort key: hop distance stays primary (#BUG2 intent preserved),
query relevance decides within a layer, degree and node id keep the tail
deterministic. No new scoring pass, no new flag.

Byte-identical when no non-seed node scored (missing/empty map or all-zero
scores fall through to the previous degree order). Seeds still render
first and still survive truncation.

Tests: unit (match outranks hub in its layer; no-scores output unchanged)
and end-to-end (`query` on a graph where a second term-matching node is
not the per-term seed survives a tight budget ahead of the hub).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

@graphify-labs graphify-labs Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Graphify reviewed this change.

Looks safe to merge — no coupling regressions and no blocking issues, checked against the code graph (not a self-assessment).

Formal verification. 2 change(s) tested, no difference found (not proven).


Graphify review — findings

Makes budget-driven subgraph rendering relevance-aware: _subgraph_to_text takes an optional scores map and, within a hop layer, orders query-matching nodes ahead of higher-degree hubs so a tight token_budget drops the incidental hub rather than the node that answers the question. _query_graph_text threads the ranking it already computed (qs.ranked) into the renderer. Hop distance remains the primary sort key and an empty or missing scores map preserves the previous hop/degree ordering byte-for-byte.

No blocking issues surfaced.

Analysis details — impact, health, verification

Impact & health

Graphify review

Impact — 579 functions depend on the 337 functions this change touches.

Health — this change adds coupling hotspots:

  • new: dispatch_command() — 2 callers, 123 callees
  • new: _query_graph_text() — 21 callers, 9 callees
  • new: _score_query() — 15 callers, 5 callees
  • new: _query_terms() — 20 callers, 3 callees
  • new: run_benchmark() — 16 callers, 3 callees
  • new: _build_server() — 2 callers, 16 callees
  • new: _load_graph() — 9 callers, 3 callees
  • new: _query_subgraph_tokens() — 7 callers, 3 callees
  • …and 8 more — each is listed as a finding

Verification — 579 functions in the blast radius were not formally verified this run (proofs are advisory here).

Gate & verification

graphify gate

PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.

Advisory (not blocking):

  • verification_scope: 404 function(s) in the blast radius were not formally verified this run

Formal verification

No difference found (not proven): No behavior difference found in \_query\_graph\_text (not a proof).

The verifier ran both versions of \_query\_graph\_text on many inputs and saw identical behavior every time. Strong evidence the change is safe, but evidence, not a proof.

Guarantee: Empirical: differential testing (both versions run on many generated inputs). A divergence on an untested input remains possible, so this is 'no counterexample found', not 'proven equivalent'.

Note: An input the sampler did not try could still differ.

No difference found (not proven): No behavior difference found in \_subgraph\_to\_text (not a proof).

The verifier ran both versions of \_subgraph\_to\_text on many inputs and saw identical behavior every time. Strong evidence the change is safe, but evidence, not a proof.

Guarantee: Empirical: differential testing (both versions run on many generated inputs). A divergence on an untested input remains possible, so this is 'no counterexample found', not 'proven equivalent'.

Note: An input the sampler did not try could still differ.

· 16 more finding(s) on lines outside this diff (see the check run).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

graphify query: seed scoring returns low-relevance nodes, causing multi-call token cascade

1 participant