stop letting one tool claim to be another - #39
Merged
Conversation
End-to-end testing through Codex's own config found the half of criterion 6
that was still forgeable. `written_by` has been derived from the fact's edge
since 2026-08-29, so a caller cannot assert who wrote something. `recalled_by`
was still a parameter, and a claude-code server passing recalled_by="codex"
produced a counted cross-tool save:
{"counts_toward_gate": true, "cross_tool_saves": 2}
The criterion measures whether two tools were involved, and the agent
supplying the evidence is the agent being graded, so neither side may come
from the caller. The reader is now the server's own configured agent id, which
comes from the config the client launched it with. `recalled_by` is gone from
the tool contract.
Both derived values are now returned on every response, not only when the save
is rejected: the caller supplies neither, so the response is the only way it
can see what the server concluded and check the save says what it meant.
Verified by removing the fix and watching the guards go red, not merely by
watching them pass. The 2048-char description guard added an hour earlier also
caught this change bloating record_recall_save to 2096; the docstring is 1376
now.
415 tests pass, 4 added.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Found by testing end to end through Codex's own
~/.codex/config.toml, over the real MCP stdio protocol.The hole
written_byhas been derived from the fact's own edge since 2026-08-29 — a caller cannot assert who wrote something. I had been describing the number as trustworthy on that basis.It was half-trustworthy.
recalled_bywas still a parameter:One tool, both ends, counted.
Why it matters
Criterion 6 measures whether two tools were involved, and the agent supplying the evidence is the agent being graded. Neither side may come from the caller. Deriving the writer while accepting a claim about the reader closed one end of a two-ended hole.
The reader is now the server's own configured
agent_id— from the config the client launched it with.recalled_byis gone from the tool contract.Also
Both derived values are now returned on every response, not only when a save is rejected. The caller supplies neither, so the response is the only way it can see what the server concluded and confirm the save says what it meant.
Testing
415 pass, 4 added:
TypeErrorrecalled_byis absent from the tool signaturecounts_toward_gate: truewritten_by: codex,recalled_by: claude-codeVerified by removing the fix and watching the guards go red, not merely by watching them pass.
Worth noting: the 2048-char description guard from #38 caught this change bloating
record_recall_saveto 2096 chars within an hour of being added. Docstring is 1376 now.