Skip to content

Make a causal claim one the contract can hold below supported - #6

Merged
sgriffiths merged 2 commits into
mainfrom
feat/claim-type
Oct 22, 2026
Merged

sgriffiths merged 2 commits into
mainfrom
feat/claim-type

Conversation

@sgriffiths

@sgriffiths sgriffiths commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

Follows #5. With the confidence definitions in place, live boards still rated a hypothesis that bundled an observation with its cause ("setroubleshootd at 99.7% CPU is starving the JVM and causing the delays") by its observed half, at supported or confirmed. A prompt sentence forbidding that was broken on every board it was tried on, so the rule now lives in the contract.

The change

  • Each hypothesis declares claim_type: observation (states what was seen) or causal (says one thing made another happen, including a statement that bundles an observation with a cause). Required, so it cannot be dodged by omission.
  • The assess output schema rejects supported and confirmed when claim_type is causal.
  • The commander's prompt defines both values and names them where the other closed sets are named. Both governance override prompts name them too; the contract override labels its claims observations, so its rejection stays on the missing falsifying_test.
  • README's contract section states the rule and what it does not catch.

What it cannot catch. A cause the board labels an observation passes. That is one explicit word a reader can challenge rather than a judgement inside a sentence, which is as far as a schema reaches. The schema comment and the README both say so.

Checks. checks/contract_invariants.py now has 39 cases. The causal rule is tested across the whole grid, both claim_type values against all five confidence words, with the verdict stated in the test rather than read from the rule; the label is tried omitted, misspelt, and as a boolean, number, null and list. A case can name the rule that refuses it, and a refusal for any other reason fails, so a case cannot pass because a different rule fired. The base board is an observation, so the older precedent and closed-object cases still isolate their own rules. required lists falsifying_test before claim_type, and a case holds that order, because the governance contract demo asserts the board is refused for the stripped caveat. Every guard was removed in turn and each removal fails its own case and no other; removing either value from a governance override fails checks/prompt_inputs.py. oa validate on all six specs, the confidence-scale self-test, checks/briefing_grounded.py, checks/walkthrough_claims.py and the test suite pass.

One residual risk, stated. The attribution demo's override pins the leading hypothesis at causal and likely. If a model ignores that and writes supported, the board is refused by the causal rule before the guard rule, and the demo fails on its asserted reason. It fails red, never green.

Live. Three assess boards on the bridge view: all schema valid; every hypothesis labelled causal and none with causal wording labelled observation; the leading CPU hypothesis held at likely on all three. None confirmed while blocking evidence was outstanding, none above likely citing nothing in the evidence array, which also measures #5's two clarifications. The governance contract and attribution demos fail for their stated reasons.

🤖 Generated with Claude Code

sgriffiths and others added 2 commits October 22, 2026 16:57
Live boards rated a hypothesis that bundled an observation with its cause
by the observed half, and a prompt sentence forbidding it was broken on every
board it was tried on. Each hypothesis now declares claim_type, observation
or causal, and the output schema rejects supported and confirmed on a causal
one. The prompt and both governance overrides name the two values; the
contract override labels its claims observations, so its rejection stays on
the missing falsifying_test.

Six new contract invariants cover both rejected words, the highest accepted
one, an observation at confirmed, and the label omitted or misspelt; each
guard was mutated out and each removal fails its own case. What the rule
cannot catch, a cause labelled as an observation, is stated in the schema and
the README.

Three live boards: every hypothesis labelled causal, the leading one held at
likely, all schema valid; both governance demos fail for their stated reasons.

Co-Authored-By: Claude <noreply@anthropic.com>
…wn rule

Review found the proofs weaker than the schema. The base board was causal,
so a case raising confidence to confirmed was refused by the new rule too and
the precedent rule and the closed hypothesis object could be deleted with the
suite green; six named cases did not pin "these two words, only on a causal
claim"; and a board stripped of both falsifying_test and claim_type was
refused for the label, not the caveat the contract demo asserts.

The base board is an observation again. The rule's cases are the whole grid,
both labels against all five words read from the schema, with the verdict
stated in the test and not read from the rule; the label is tried omitted,
misspelt and as four other types; the closed hypothesis object has a case of
its own. A case can now name the rule that refuses it, and a refusal for any
other reason fails. `required` lists falsifying_test before claim_type, and a
case holds that order. Every guard was removed in turn: each removal fails
its own case and no other, and removing the reason check lets a schema
mutation pass silently, so the check is load-bearing.

Co-Authored-By: Claude <noreply@anthropic.com>
@sgriffiths
sgriffiths merged commit 4c2e2f5 into main Oct 22, 2026
2 checks passed
@sgriffiths
sgriffiths deleted the feat/claim-type branch October 22, 2026 06:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant