Repository navigation
Give each confidence word one meaning on the commander's board - #5
Merged
Merged
Conversation
The prompt defined confirmed and speculative and left plausible, likely and supported to the model, so the step between them was whatever it judged; one live board put two inferences at supported while the deciding evidence was outstanding. Each word now has a definition, and a hypothesis whose blocking evidence is outstanding is at most likely. The enum, its order and every other spec are unchanged. CLAUDE.md said only the Precedent Analyst's system prompt is cached and the commander's was 1,018 tokens. Counted today, the commander's root prompt was already about 1,630 and is now about 1,770, so two prompts clear both thresholds; the sentence says so. Co-Authored-By: Claude <noreply@anthropic.com>
Review found the definitions contradicting themselves: a hypothesis with a bearing observation and outstanding deciding evidence met the definition of supported and broke the cap at likely. Supported now excludes that case. The prompt also states what blocking_evidence's own description already says, that no hypothesis is confirmed while any of it is outstanding, which one measured board had broken. The caching patch's notes still said the commander's prompt is under the floor and sets no breakpoint; they now say both prompts clear it. Notes only, above the first hunk. Co-Authored-By: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The commander's prompt defined
confirmed(needs a direct observation) andspeculative(no falsifying test), and leftplausible,likelyandsupportedto the model. Each word now has a definition:While blocking evidence that would decide a hypothesis is outstanding, that hypothesis is at most
likely, and while any blocking evidence is outstanding no hypothesis isconfirmed, which is whatblocking_evidence's own description already says.supportedexcludes a hypothesis that outstanding blocking evidence would decide, so the definition and the cap cannot both apply. The enum, its order and every other spec are unchanged.The measurements below ran on the first wording, before those two clarifications, which came from review. The clarifications are measured in the next change's acceptance runs.
Measured. Three
assessboards on the bridge view with these definitions, against two counts fixed before the runs: hypotheses abovelikelywhose falsifying test is the outstanding audit log, and hypotheses abovelikelyciting nothing in the evidence array. Both were zero on all three boards; the outstanding request carried minute 161 on all three; all kept three standing conditions. The governance contract and attribution demos still pass.What it does not fix, recorded rather than claimed. One of the three boards marked a causal hypothesis
confirmed("starving the JVM and causing batch transfer delays") on evidence that establishes the CPU figure and not the cause. A further sentence telling the model that an observation does not establish its own cause was tried and dropped: three more boards each rated a causal hypothesissupported, the case the sentence forbade. The model rates a hypothesis by its observed half when the statement bundles an observation with a cause. That is a structural problem, and the next change makes it one the contract can check: a requiredclaim_typeon each hypothesis, withsupportedandconfirmedrejected on a causal claim.Also here. CLAUDE.md said only the Precedent Analyst's system prompt clears the caching thresholds and put the commander's at 1,018 tokens. Counted, the commander's root prompt was already about 1,630 tokens and is now about 1,770; the sentence now says two prompts clear both, and the caching patch's explanatory notes, which said the commander sets no breakpoint, are corrected to match. The hunks are unchanged.
Checks:
oa validate,checks/prompt_inputs.py,checks/contract_invariants.py.🤖 Generated with Claude Code