Repository navigation
Make a causal claim one the contract can hold below supported - #6
Merged
Merged
Conversation
Live boards rated a hypothesis that bundled an observation with its cause by the observed half, and a prompt sentence forbidding it was broken on every board it was tried on. Each hypothesis now declares claim_type, observation or causal, and the output schema rejects supported and confirmed on a causal one. The prompt and both governance overrides name the two values; the contract override labels its claims observations, so its rejection stays on the missing falsifying_test. Six new contract invariants cover both rejected words, the highest accepted one, an observation at confirmed, and the label omitted or misspelt; each guard was mutated out and each removal fails its own case. What the rule cannot catch, a cause labelled as an observation, is stated in the schema and the README. Three live boards: every hypothesis labelled causal, the leading one held at likely, all schema valid; both governance demos fail for their stated reasons. Co-Authored-By: Claude <noreply@anthropic.com>
…wn rule Review found the proofs weaker than the schema. The base board was causal, so a case raising confidence to confirmed was refused by the new rule too and the precedent rule and the closed hypothesis object could be deleted with the suite green; six named cases did not pin "these two words, only on a causal claim"; and a board stripped of both falsifying_test and claim_type was refused for the label, not the caveat the contract demo asserts. The base board is an observation again. The rule's cases are the whole grid, both labels against all five words read from the schema, with the verdict stated in the test and not read from the rule; the label is tried omitted, misspelt and as four other types; the closed hypothesis object has a case of its own. A case can now name the rule that refuses it, and a refusal for any other reason fails. `required` lists falsifying_test before claim_type, and a case holds that order. Every guard was removed in turn: each removal fails its own case and no other, and removing the reason check lets a schema mutation pass silently, so the check is load-bearing. Co-Authored-By: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follows #5. With the confidence definitions in place, live boards still rated a hypothesis that bundled an observation with its cause ("setroubleshootd at 99.7% CPU is starving the JVM and causing the delays") by its observed half, at
supportedorconfirmed. A prompt sentence forbidding that was broken on every board it was tried on, so the rule now lives in the contract.The change
claim_type:observation(states what was seen) orcausal(says one thing made another happen, including a statement that bundles an observation with a cause). Required, so it cannot be dodged by omission.supportedandconfirmedwhenclaim_typeiscausal.falsifying_test.What it cannot catch. A cause the board labels an observation passes. That is one explicit word a reader can challenge rather than a judgement inside a sentence, which is as far as a schema reaches. The schema comment and the README both say so.
Checks.
checks/contract_invariants.pynow has 39 cases. The causal rule is tested across the whole grid, bothclaim_typevalues against all five confidence words, with the verdict stated in the test rather than read from the rule; the label is tried omitted, misspelt, and as a boolean, number, null and list. A case can name the rule that refuses it, and a refusal for any other reason fails, so a case cannot pass because a different rule fired. The base board is an observation, so the older precedent and closed-object cases still isolate their own rules.requiredlistsfalsifying_testbeforeclaim_type, and a case holds that order, because the governance contract demo asserts the board is refused for the stripped caveat. Every guard was removed in turn and each removal fails its own case and no other; removing either value from a governance override failschecks/prompt_inputs.py.oa validateon all six specs, the confidence-scale self-test,checks/briefing_grounded.py,checks/walkthrough_claims.pyand the test suite pass.One residual risk, stated. The attribution demo's override pins the leading hypothesis at
causalandlikely. If a model ignores that and writessupported, the board is refused by the causal rule before the guard rule, and the demo fails on its asserted reason. It fails red, never green.Live. Three
assessboards on the bridge view: all schema valid; every hypothesis labelledcausaland none with causal wording labelledobservation; the leading CPU hypothesis held atlikelyon all three. None confirmed while blocking evidence was outstanding, none abovelikelyciting nothing in the evidence array, which also measures #5's two clarifications. The governance contract and attribution demos fail for their stated reasons.🤖 Generated with Claude Code