Skip to content

stop letting one tool claim to be another - #39

Merged
ayushcodes10 merged 1 commit into
mainfrom
fix/recalled-by-cannot-be-claimed
Sep 3, 2026
Merged

stop letting one tool claim to be another#39
ayushcodes10 merged 1 commit into
mainfrom
fix/recalled-by-cannot-be-claimed

Conversation

@ayushcodes10

Copy link
Copy Markdown
Owner

Found by testing end to end through Codex's own ~/.codex/config.toml, over the real MCP stdio protocol.

The hole

written_by has been derived from the fact's own edge since 2026-08-29 — a caller cannot assert who wrote something. I had been describing the number as trustworthy on that basis.

It was half-trustworthy. recalled_by was still a parameter:

# from a server whose agent_id is "claude-code"
record_recall_save(scope, fact_id, note, recalled_by="codex")
→ {"counts_toward_gate": true, "cross_tool_saves": 2}

One tool, both ends, counted.

Why it matters

Criterion 6 measures whether two tools were involved, and the agent supplying the evidence is the agent being graded. Neither side may come from the caller. Deriving the writer while accepting a claim about the reader closed one end of a two-ended hole.

The reader is now the server's own configured agent_id — from the config the client launched it with. recalled_by is gone from the tool contract.

Also

Both derived values are now returned on every response, not only when a save is rejected. The caller supplies neither, so the response is the only way it can see what the server concluded and confirm the save says what it meant.

Testing

415 pass, 4 added:

  • the exact call that used to work now raises TypeError
  • recalled_by is absent from the tool signature
  • one tool alone cannot reach counts_toward_gate: true
  • a genuine two-server save still counts, with written_by: codex, recalled_by: claude-code

Verified by removing the fix and watching the guards go red, not merely by watching them pass.

Worth noting: the 2048-char description guard from #38 caught this change bloating record_recall_save to 2096 chars within an hour of being added. Docstring is 1376 now.

End-to-end testing through Codex's own config found the half of criterion 6
that was still forgeable. `written_by` has been derived from the fact's edge
since 2026-08-29, so a caller cannot assert who wrote something. `recalled_by`
was still a parameter, and a claude-code server passing recalled_by="codex"
produced a counted cross-tool save:

  {"counts_toward_gate": true, "cross_tool_saves": 2}

The criterion measures whether two tools were involved, and the agent
supplying the evidence is the agent being graded, so neither side may come
from the caller. The reader is now the server's own configured agent id, which
comes from the config the client launched it with. `recalled_by` is gone from
the tool contract.

Both derived values are now returned on every response, not only when the save
is rejected: the caller supplies neither, so the response is the only way it
can see what the server concluded and check the save says what it meant.

Verified by removing the fix and watching the guards go red, not merely by
watching them pass. The 2048-char description guard added an hour earlier also
caught this change bloating record_recall_save to 2096; the docstring is 1376
now.

415 tests pass, 4 added.
@ayushcodes10
ayushcodes10 merged commit d9183ef into main Sep 3, 2026
2 checks passed
@ayushcodes10
ayushcodes10 deleted the fix/recalled-by-cannot-be-claimed branch September 3, 2026 11:27
@github-actions github-actions Bot locked and limited conversation to collaborators Sep 3, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant