From be0328f487b966f7d80590d84636505a573b568d Mon Sep 17 00:00:00 2001 From: wshallwshall Date: Sun, 2 Aug 2026 10:13:16 -0500 Subject: [PATCH] =?UTF-8?q?docs(standards):=20a=20fourth=20review=20rule?= =?UTF-8?q?=20=E2=80=94=20the=20instrument=20that=20answers=20an=20adjacen?= =?UTF-8?q?t=20question?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The three rules in Secure_Development_Standards §3 catch prose that is TRUE and misleading. They do not catch the failure that produced eleven retractions across four parallel sessions on 2026-08-02: a claim that is FALSE when written while feeling measured, because the instrument answered a question adjacent to the one asked. The eleven, each verified by the session that made it: git diff on a STAGED file "unstaged delta?" vs "is the tree dirty?" merge-base --is-ancestor "is this an ancestor?" vs "did this land?" <-- squash-merge: always no a hash INEQUALITY "are these different?" vs "is the copy WORSE?" session-start banner "who was live then?" vs "who is live now?" grep -c $'\r$' on git diff "does the diff render CR?" vs "does the FILE have CRLF?" $? after `cmd | tail` "did tail succeed?" vs "did the gate pass?" Actions ?filter=latest "latest attempt?" vs "what did the suite ever do?" JOB conclusion "did the job pass?" vs "did the STEP pass?" Two findings that make it actionable rather than a scolding: - Re-reading caught NONE of the eleven. A check that could fail caught one immediately. Re-reading confirms what you meant; it cannot test what you wrote. - None was a stale fact. Every one was wrong at birth. "#119 never merged (it died on a CI timeout)" was never true at any instant -- that PR's timeline carries exactly one `closed` event, simultaneous with `merged`. So dating a claim does not protect against this class; only re-deriving it does. Hence the rule is a PROPHYLACTIC, checkable before the sentence exists and without a peer: name the question, name what the instrument returns, confirm they are the same sentence. Also adds the one-liner to CLAUDE.md §11 alongside the other three, per the provenance note's own reasoning -- an instruction that short cannot drift, and a pointer nobody follows mid-task changes no behaviour. No version-history row: the "Reviewing security prose" subsection carries none (added in 39990f8c without one), so additions there set no bump precedent. No change to the SSDF / ASVS / HIPAA mappings. Named by the repo-security-review session, which applied it to its own four retractions and found four for four; instances contributed by the ci-margin-correction, announce-hook, sandbox-codec and ADR 0154 sessions. --- CLAUDE.md | 3 +++ docs/Secure_Development_Standards.md | 29 +++++++++++++++++++++++++++- 2 files changed, 31 insertions(+), 1 deletion(-) diff --git a/CLAUDE.md b/CLAUDE.md index 4657dbb5..cf1c76a4 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -429,6 +429,9 @@ harness process only.) - **State a load-bearing fact ONCE and link to it; never restate it.** - **A completeness claim is a liability — prefer "at least" to an enumeration.** - **A compensating control must not rest on a false premise.** +- **Confirm your instrument answers the question you asked, not one adjacent to it** — `git diff` on a + staged file, `--is-ancestor` under squash-merge, `$?` after a pipe, a *job* conclusion for a *step* + question. Name the question and what the tool returns; check they are the same sentence. --- diff --git a/docs/Secure_Development_Standards.md b/docs/Secure_Development_Standards.md index c1f916e0..5658f067 100644 --- a/docs/Secure_Development_Standards.md +++ b/docs/Secure_Development_Standards.md @@ -100,10 +100,37 @@ asking what happens to someone who acts on the sentence; **none** was caught by console's own search route. The control itself was sound; the stated reason was not, and the next person to touch it reasons from the comment. A wrong justification is worse than none. +- **Confirm your instrument answers the question you asked, not one adjacent to it.** The rules above + catch prose that is true and misleading; this one catches a claim that is **false when written, while + feeling measured**. The rule is the question — ***does my instrument answer the question I asked, or + one adjacent to it?*** — and it outlives every example below. **The pairs are dated illustrations, not + the rule:** `--is-ancestor` misleads here only because this repository squash-merges, and any of them + may stop being true without the question changing at all. On 2026-08-02 four parallel sessions + retracted eleven claims in a single night and every one traced to an instrument answering in adjacent + terms: `git diff` on a **staged** file (returns + *"is there an unstaged delta"*; the question was *"is the tree dirty"*); `merge-base --is-ancestor` + (*"is this an ancestor"* vs *"did this work land"* — **squash-merge makes the answer always no**); a + hash **inequality** (*"are these different"* vs *"is the installed copy **worse**"*); a session-start + **banner** (*"who was live when it printed"* vs *"who is live now"*); `grep -c $'\r$'` over `git diff` + output (*"does the diff **render** a CR"* vs *"does the file contain CRLF"* — it reported a + byte-perfect file as mangled); `$?` after `cmd | tail` (*"did `tail` succeed"*); the Actions + `?filter=latest` view (*"what did the **latest attempt** do"* vs *"what did the suite ever do"*); and a + **job** conclusion answering a **step** question, which drops the tightest samples by construction. + **Re-reading caught none of the eleven; a check that could fail caught one immediately.** Nor were + these facts that expired — "#119 never merged (it died on a CI timeout)" was never true at any instant: + that pull request's timeline carries exactly one `closed` event, simultaneous with `merged`. Dating a + claim does not protect against this class; only re-deriving it does. So before publishing a measured + claim, write down the question and write down what the instrument returns, and check that they are the + same sentence. + *Provenance (the evidence is the point):* the completeness-claim and false-premise rules, and the governing instruction above, came out of the 2026-07-30 public-documentation audit; the state-it-once rule was named by the parallel ASVS review session, which also supplied the -`harden_kex_groups` and `PHI.md` §5/§7 instances. `CLAUDE.md` §11 carries these as bare one-line +`harden_kex_groups` and `PHI.md` §5/§7 instances. The instrument rule came out of the 2026-08-02 +parallel-session cluster — four sessions, eleven retractions, none caught by its own author — and was +named by the repo-security-review session after applying it to its own four and finding four for four; +the remaining instances were contributed by the ci-margin-correction, announce-hook, sandbox-codec and +ADR 0154 sessions, each of which had made one. `CLAUDE.md` §11 carries these as bare one-line imperatives — deliberately duplicated, because an instruction that short cannot meaningfully drift and a pointer nobody follows mid-task changes no behaviour. **This section is the source of record for the reasoning, the evidence and the dates.**