Skip to content

feat(hooks): strict search gate with per-agent query evidence - #3291

Open
logan683 wants to merge 12 commits into
Graphify-Labs:v8from
logan683:pr/strict-search-gate
Open

feat(hooks): strict search gate with per-agent query evidence#3291
logan683 wants to merge 12 commits into
Graphify-Labs:v8from
logan683:pr/strict-search-gate

Conversation

@logan683

@logan683 logan683 commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Stacked on #3281 (pr/hook-fail-open); follow-up to #3280. Please review #3281 first — this PR's diff against v8 includes it until it merges, after which this branch will be rebased.

Problem

After #3281 removed the graph-wide query stamp, strict Claude hooks recorded no query evidence at all. The Read deny fires once per session unconditionally and the retry is unconditional; Bash|Grep search is nudge-only by design (cli.py docstring). An agent whose harness instructs it to read and search through Bash (cat, sed, grep -r) never meets a block, and a nudge that costs nothing is learned in one turn to cost nothing. Measured on this machine: seven documented recurrences of "grep the corpus before querying" over twelve days, with a headless Claude run scoring RED for it four days after the previous fix.

Fix

  • Query evidence. New PostToolUse hook hook-guard mark-queried writes graphify-out/cache/hook_sessions/<session_id>[--agent-<agent_id>].queried after the MCP graph tools (query_graph, get_node, get_neighbors, get_community, god_nodes, shortest_path) or a Bash command that runs -m graphify query|explain|path (the project sidecar form "$(cat graphify-out/.graphify_python)" -m graphify query … included). Same identity sanitising/hashing as the existing .denied marker.
  • Search deny. hook-guard search --strict denies a recursive in-project corpus search — grep -r/-R/--recursive, rg, find, fd, ack, ag, or the Grep tool on a directory — until that marker exists. Not once-per-session: the way out is a query, not a retry. Exact-file grep, stdin grep, git grep, out-of-project targets, Glob, soft mode, missing session_id, malformed stdin and GRAPHIFY_HOOK_STRICT=0 never block. The deny text is constant; parsing never executes the command.
  • Parser. Quoted spans are protected before the operator split, so a regex containing (/| or a "$(cat …)" interpreter cannot fragment its segment. _strip_heredoc_bodies is shared with _bash_invokes_search.
  • Install. --strict now applies to Bash|Grep; the PostToolUse hook is registered and stripped with the others. README strict paragraph and CHANGELOG updated.

Evidence

  • tests/test_search_strict.py — RED on 2a83725: 12 failed / 5 passed (the five are the never-deny safety cases). GREEN: 301 passed across test_search_strict, test_hook_strict, test_search_hook, test_read_hook, test_hook_guard, test_install*, test_settings_merge, test_codebuddy.
  • Full suite on Windows: 5139 passed; the 28 failures are pre-existing Windows path-form tests in files this PR does not touch (identical set on a clean 2a83725 export).
  • python -m tools.skillgen --check 134 OK, --audit-coverage OK, --schema-singleton OK; ruff check . clean.
  • Live: a fresh claude -p --model claude-opus-5 --dangerously-skip-permissions session in a scratch project with these hooks installed project-scoped, given a neutral relationship question naming no tool, made grep -rn … . its second call → denied with the strict reason → mcp__graphify__query_graph → subsequent searches allowed → correct answer with file:line citations.

Not covered

  • PowerShell-tool searches (Select-String -Recurse) — different matcher, not designed here.
  • Claude Code only; other hosts unchanged.

🤖 Generated with Claude Code

https://claude.ai/code/session_01HNTa1HY67hwopSPQtffAPy

logan683 and others added 12 commits September 2, 2026 03:23
…se hooks

The generated hook resolved its command via shutil.which("graphify"), which
returns the console-script shim. For uv-tool and pipx installs that shim is
unsigned; on Windows with Smart App Control enabled, code-integrity policy
refuses to load it, so the hook can never run. Both ~/.local/bin/graphify.exe
and the venv Scripts/graphify.exe are NotSigned, so resolving to a different
shim directory does not help.

The hook also had no failure guard. A PreToolUse hook that exits non-zero
blocks the tool call it was meant to advise, so an advisory graph check could
break the host on any launcher failure.

Resolve to '<sys.executable> -m graphify' via the existing hooks._pinned_python(),
the same reasoning already applied to git hooks, and make both the POSIX and
Windows commands fail open with a timeout.

The Graphify-Labs#2165 regression parses the subcommand out of the hook command; it now
handles both supported grammars rather than assuming the subcommand is the
second token, so it keeps proving the same contract.

Refs Graphify-Labs#3280
The generated Windows command may be run by cmd.exe or by PowerShell, and no
bare fail-open suffix is correct in both. `& exit /b 0` is cmd syntax; run by
PowerShell the bare `&` is the background operator, so the hook spawns a job
and exits 1 - the fail-open suffix caused the very failure it was added to
prevent (Graphify-Labs#3280). `; exit 0` has the mirror problem: valid in PowerShell,
rejected by cmd.exe.

Naming the interpreter removes the ambiguity. Measured on Windows, the
generated string now exits 0 under both pwsh and cmd.exe; the previous form
exited 1 under pwsh.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PPaGbwQeyWLwbVm9uRgHtU
Strict Claude hooks recorded no query evidence after the graph-wide stamp
was removed: the Read deny fired once per session unconditionally and the
retry was unconditional, and Bash/Grep search was nudge-only by design. An
agent instructed to read and search through Bash never met a block, and a
nudge that costs nothing is learned in one turn to cost nothing.

- PostToolUse `hook-guard mark-queried` writes cache/hook_sessions/<sid>
  [--agent-<id>].queried after mcp__graphify__{query_graph,get_node,
  get_neighbors,get_community,god_nodes,shortest_path} or a Bash command
  that runs `-m graphify query|explain|path` (project sidecar form included).
- `hook-guard search --strict` denies a recursive in-project corpus search
  (grep -r/-R, rg, find, fd, ack, ag, or the Grep tool on a directory) until
  that marker exists. Not once-per-session: the way out is a query, not a
  retry. Exact-file grep, stdin grep, git grep, out-of-project targets, Glob,
  soft mode, no session_id, malformed stdin and GRAPHIFY_HOOK_STRICT=0 never
  block. Deny text is constant; parsing never executes the command.
- Quoted spans are protected before the operator split so a regex with
  parens or a "$(cat graphify-out/.graphify_python)" interpreter cannot
  fragment its segment.
- install: --strict now applies to Bash|Grep; the PostToolUse marker hook is
  registered and removed with the others.

Tests: tests/test_search_strict.py (RED on 2a83725: 12 failed / 5 passed;
GREEN: 301 passed across the hook/install suites), skillgen --check 134 OK,
ruff clean. Live proof: a fresh `claude -p --model claude-opus-5` session
in a scratch project with the candidate hooks had its second tool call
(grep -rn ... .) denied, queried the graph next, and was then allowed.

Stacked on Graphify-Labs#3281 (pr/hook-fail-open); follow-up to Graphify-Labs#3280.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HNTa1HY67hwopSPQtffAPy
Round 1 review findings on the strict search gate:

- F1: a Bash-only session never reached the 24-hour marker sweep that lived
  only inside _mark_session_denied, so .queried markers accumulated and a
  reused session id stayed pre-authorized. Both writers now call one shared
  _gc_session_markers.
- F2: Claude Code on Windows exposes a PowerShell tool (tool_name
  "PowerShell", tool_input.command; PreToolUse:PowerShell hook events were
  measured in a live transcript). Its recursive in-project searches now reach
  the same guard: matcher Bash|Grep|PowerShell, and two measured shapes are
  denied until query evidence exists — `Get-ChildItem|gci|ls|dir ... -Recurse
  ... | Select-String|sls` and `Select-String ... -Path <dir or dir\*>`.
  Exact-file -Path, Get-Content | Select-String and out-of-project targets
  only nudge; non-search PowerShell stays silent.

Tests: 5 RED on a0a4f1d, 306 passed after; skillgen --check 134 OK; ruff clean.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HNTa1HY67hwopSPQtffAPy
…ll listing

Round 3 review findings:

- F1: _session_has_queried now enforces the 24-hour TTL itself and best-effort
  unlinks an expired marker, so a stale .queried never pre-authorizes a reused
  session id while no writer happens to sweep. One _SESSION_MARKER_TTL shared
  with the GC.
- F2: a recursive PowerShell listing without a Select-String pipe
  (Get-ChildItem|gci|ls|dir ... -Recurse) is the same corpus-enumeration class
  as `find .` and now denies in-project, nudges out-of-project, and stays
  silent when not recursive.

Tests: 2 RED on 8d04a72, 326 passed after; skillgen --check 134 OK; ruff clean.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HNTa1HY67hwopSPQtffAPy

@graphify-labs graphify-labs Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Graphify reviewed this change.

Worth a look — the grounded gate found no coupling regressions or blocking issues, but 5 advisory finding(s) below merit a look before merge.

Formal verification. 2 change(s) alter behavior, breaking input(s) attached.

Behavior changes: \_claude\_pretooluse\_hooks changes behavior, here is the input that shows it.

The verifier found a concrete input on which \_claude\_pretooluse\_hooks behaves differently before and after the change. If that change is intended, ship it; if not, this is your bug.

Guarantee: This difference was REPRODUCED, the verifier actually ran both versions on that input and saw them disagree. It is real, not an artifact.

Evidence: On input \{"strict":"True","project":"True"\}, the old code produced \[\{'matcher': 'Bash\|Grep', 'hooks': \[\{'type': 'command', 'command': 'graphify hook\-guard search'\}\]\}, \{'matcher': 'Read\|Glob', 'hooks': \[\{'type': 'command', 'command': 'graphify hook\-guar… but the new code produces \[\{'matcher': 'Bash\|Grep\|PowerShell', 'hooks': \[\{'type': 'command', 'command': '"$\(cat graphify\-out/\.graphify\_python\)" \-m graphify hook\-guard search \-\-strict \|\| true', 'commandWindows…. Paste that input straight into a regression test.

Behavior changes: \_resolve\_graphify\_exe changes behavior, here is the input that shows it.

The verifier found a concrete input on which \_resolve\_graphify\_exe behaves differently before and after the change. If that change is intended, ship it; if not, this is your bug.

Guarantee: This difference was REPRODUCED, the verifier actually ran both versions on that input and saw them disagree. It is real, not an artifact.

Evidence: On input \{"project":"True"\}, the old code produced 'graphify' but the new code produces '"$\(cat graphify\-out/\.graphify\_python\)" \-m graphify'. Paste that input straight into a regression test.


Graphify review — findings

Reworks generated agent guidance to prefer the MCP query_graph tool and fall back to the graph's recorded interpreter via "$(cat graphify-out/.graphify_python)" -m graphify … (with a PowerShell variant), so hardened Windows hosts stop routing through an unsigned graphify.exe shim. Rewires project-scoped hooks to resolve Graphify through each clone's graphify-out/.graphify_python sidecar and fail open (emit no graph decision) until it exists, keeping committed configs portable on Application Control hosts. Fixes strict Claude hooks to key deny markers by session_id plus agent_id instead of one graph-wide 30-minute stamp, records per-session/agent query evidence via a PostToolUse hook-guard mark-queried on the MCP graph tools and graphify query|explain|path, GC's markers older than 24 hours, and now blocks a recursive in-project search (Bash grep -r/rg/find, the Grep tool, or PowerShell Get-ChildItem -Recurse | Select-String) until one traversal runs — while leaving exact-file grep, stdin grep, git grep, out-of-project targets, Glob, and soft mode unblocked.

Worth a look

  • hook command interpolates unquoted exe now that caller quoting removedgraphify/install.py:320 · Escalate · high
    • agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
  • Strict Claude PreToolUse hooks mask blocking failuresgraphify/install.py:324 · Escalate · high
    • agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
  • Project-scoped Windows hook executes interpreter path read from project-controlled filegraphify/install.py:1496 · Escalate · high
    • agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
  • Wrapper skip loop can drop the wrapped command's positional argsgraphify/cli.py · Escalate · medium
    • agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
  • IndexError on empty graphify command tokensgraphify/cli.py · Escalate · medium
    • agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
Analysis details — impact, health, verification

Impact & health

Graphify review

Impact — 2314 functions depend on the 2136 functions this change touches.

Health — this change adds coupling hotspots:

  • new: dispatch_command() — 2 callers, 122 callees
  • new: codebuddy_install() — 20 callers, 5 callees
  • new: claude_install() — 19 callers, 4 callees
  • new: _run_hook_guard() — 5 callers, 14 callees
  • new: gemini_install() — 10 callers, 7 callees
  • new: claude_uninstall() — 17 callers, 4 callees
  • new: _project_uninstall() — 5 callers, 13 callees
  • new: render() — 13 callers, 5 callees
  • …and 21 more — each is listed as a finding

Verification — 2314 functions in the blast radius were not formally verified this run (proofs are advisory here).

Gate & verification

graphify gate

PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.

Advisory (not blocking):

  • verification_scope: 2260 function(s) in the blast radius were not formally verified this run

Formal verification

Behavior changes: \_claude\_pretooluse\_hooks changes behavior, here is the input that shows it.

The verifier found a concrete input on which \_claude\_pretooluse\_hooks behaves differently before and after the change. If that change is intended, ship it; if not, this is your bug.

Guarantee: This difference was REPRODUCED, the verifier actually ran both versions on that input and saw them disagree. It is real, not an artifact.

Evidence: On input \{"strict":"True","project":"True"\}, the old code produced \[\{'matcher': 'Bash\|Grep', 'hooks': \[\{'type': 'command', 'command': 'graphify hook\-guard search'\}\]\}, \{'matcher': 'Read\|Glob', 'hooks': \[\{'type': 'command', 'command': 'graphify hook\-guar… but the new code produces \[\{'matcher': 'Bash\|Grep\|PowerShell', 'hooks': \[\{'type': 'command', 'command': '"$\(cat graphify\-out/\.graphify\_python\)" \-m graphify hook\-guard search \-\-strict \|\| true', 'commandWindows…. Paste that input straight into a regression test.

Behavior changes: \_resolve\_graphify\_exe changes behavior, here is the input that shows it.

The verifier found a concrete input on which \_resolve\_graphify\_exe behaves differently before and after the change. If that change is intended, ship it; if not, this is your bug.

Guarantee: This difference was REPRODUCED, the verifier actually ran both versions on that input and saw them disagree. It is real, not an artifact.

Evidence: On input \{"project":"True"\}, the old code produced 'graphify' but the new code produces '"$\(cat graphify\-out/\.graphify\_python\)" \-m graphify'. Paste that input straight into a regression test.

No difference found (not proven): No behavior difference found in \_bash\_invokes\_search (not a proof).

The verifier ran both versions of \_bash\_invokes\_search on many inputs and saw identical behavior every time. Strong evidence the change is safe, but evidence, not a proof.

Guarantee: Empirical: differential testing (both versions run on many generated inputs). A divergence on an untested input remains possible, so this is 'no counterexample found', not 'proven equivalent'.

Note: An input the sampler did not try could still differ.

Could not verify: Could not verify dispatch\_command.

The verifier did not have enough to check dispatch\_command, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: not verifiable: all 23 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous (mostly SystemExit — names the real obstacle, not a sampling gap)

No difference found (not proven): No behavior difference found in \_mark\_session\_denied (not a proof).

The verifier ran both versions of \_mark\_session\_denied on many inputs and saw identical behavior every time. Strong evidence the change is safe, but evidence, not a proof.

Guarantee: Empirical: differential testing (both versions run on many generated inputs). A divergence on an untested input remains possible, so this is 'no counterexample found', not 'proven equivalent'.

Note: An input the sampler did not try could still differ.

No difference found (not proven): No behavior difference found in \_run\_hook\_guard (not a proof).

The verifier ran both versions of \_run\_hook\_guard on many inputs and saw identical behavior every time. Strong evidence the change is safe, but evidence, not a proof.

Guarantee: Empirical: differential testing (both versions run on many generated inputs). A divergence on an untested input remains possible, so this is 'no counterexample found', not 'proven equivalent'.

Note: An input the sampler did not try could still differ.

Could not verify: Could not verify claude\_install.

The verifier did not have enough to check claude\_install, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: not verifiable: all 30 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous (mostly TypeError — names the real obstacle, not a sampling gap)

Could not verify: Could not verify \_install\_claude\_hook.

The verifier did not have enough to check \_install\_claude\_hook, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: parameter `project_dir` is annotated `Path` — outside the synthesizable primitive/collection set

Could not verify: Could not verify \_install\_codebuddy\_hook.

The verifier did not have enough to check \_install\_codebuddy\_hook, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: parameter `project_dir` is annotated `Path` — outside the synthesizable primitive/collection set

Could not verify: Could not verify \_install\_codex\_hook.

The verifier did not have enough to check \_install\_codex\_hook, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: parameter `project_dir` is annotated `Path` — outside the synthesizable primitive/collection set

No difference found (not proven): No behavior difference found in \_print\_install\_usage (not a proof).

The verifier ran both versions of \_print\_install\_usage on many inputs and saw identical behavior every time. Strong evidence the change is safe, but evidence, not a proof.

Guarantee: Empirical: differential testing (both versions run on many generated inputs). A divergence on an untested input remains possible, so this is 'no counterexample found', not 'proven equivalent'.

Note: An input the sampler did not try could still differ.

Could not verify: Could not verify \_strip\_graphify\_hook.

The verifier did not have enough to check \_strip\_graphify\_hook, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: parameter `settings_path` is annotated `Path` — outside the synthesizable primitive/collection set

Could not verify: Could not verify \_uninstall\_codebuddy\_hook.

The verifier did not have enough to check \_uninstall\_codebuddy\_hook, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: parameter `project_dir` is annotated `Path` — outside the synthesizable primitive/collection set

· 5 grounded finding(s) anchored inline below; 24 more finding(s) on lines outside this diff (see the check run).

Comment thread graphify/cli.py
return False


def _run_hook_guard(kind: str, strict: bool = False) -> None:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Health regression_run_hook_guard()

fans out to 14 callees (efferent coupling).

Grounded coupling-delta finding (deterministic), not an LLM guess.

Comment thread graphify/install.py
'powershell.exe -NoProfile -NonInteractive -Command '
f'"{inner}; exit 0"'
)
def _install_codex_hook(project_dir: Path, project: bool = False) -> None:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Health regression_install_codex_hook()

7 callers depend on it (afferent coupling).

Grounded coupling-delta finding (deterministic), not an LLM guess.

Comment thread graphify/install.py
print("Strict mode: the first raw file read per session is blocked until one")
print("graph traversal runs, and recursive in-project searches stay blocked until")
print("this session has run one (toggle with GRAPHIFY_HOOK_STRICT=0).")
def _install_claude_hook(project_dir: Path, strict: bool = False, project: bool = False) -> None:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Health regression_install_claude_hook()

fans out to 6 callees (efferent coupling); 7 callers depend on it (afferent coupling).

Grounded coupling-delta finding (deterministic), not an LLM guess.

Comment thread graphify/install.py
settings_path.write_text(json.dumps(settings, indent=2), encoding="utf-8")
print(f" .claude/{settings_path.name} -> PreToolUse hook removed")
print(f" .claude/{settings_path.name} -> graphify hooks removed")
def uninstall_all(project_dir: Path | None = None, purge: bool = False) -> None:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Health regressionuninstall_all()

fans out to 13 callees (efferent coupling).

Grounded coupling-delta finding (deterministic), not an LLM guess.

assert _is_deny(_invoke("search", _search("grep -rn foo ."), tmp_path, monkeypatch))


def test_allows_after_mcp_query(tmp_path, monkeypatch):

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Health regressiontest_allows_after_mcp_query()

fans out to 6 callees (efferent coupling).

Grounded coupling-delta finding (deterministic), not an LLM guess.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant