From f76c75715499e8c81af39e01b158c8499939fe03 Mon Sep 17 00:00:00 2001 From: Luiz Velho Date: Tue, 8 Sep 2026 19:59:20 -0300 Subject: [PATCH 01/11] Apply the pass's corrections, pin the predecessors, record two spec defects MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit PART A of four. A1/A2. All thirteen proposed replacements applied — every CORRECTED and every UNDER-QUALIFIED claim — across TWENTY passages, because five of the claims recur. The count matches claims-verified.md's tally: 5 corrected + 8 under-qualified = 13. A1's :397 was the third occurrence of the eighteen-claims figure, left standing when :874 and :1228 were amended. It is now in line. That inconsistency lived in the paragraph introducing the repeated-claim check, which is the check that found it. RUNNING THE FIXED CHECK AFTER APPLYING FOUND ONE THE RECORD HAD NOT NAMED: claim 5's figure recurs a third time in §6 — "here are the five open questions", sixty lines below the sentence the entry cited. Brought into line. The check earned its place twice now, and both times on the same failure: a figure corrected where it was noticed and left where it was not. Judgement calls, both recorded in claims-verified.md: - Claim 1: "five working days" applied to the four elapsed-time sentences, NOT to the two register descriptors (§1, §7), which the entry says are right. - Claims 31 and 41 offered alternatives; the first was taken in both. For 41 the second option requires editing the template repository. THE SIX UNSUPPORTED CLAIMS ARE UNCHANGED. No replacement can be written from the record, and the draft's status block now says so in the document rather than only in the ledger. A3. gap-003 closed. active-stereo pinned at 3f7a263 and bioeye at e908170. The closure is the RE-READ, not the pin: claim 2 is a verbatim quotation cited to active-stereo CLAUDE.md:141, and the line was read again at the pinned SHA and still says what the claim says. A4. docs/spec-defects.md gains two entries, not one. Entry 4 is the specified defect: a setup step enabling "require a pull request before merging" against a later step in the same instruction saying "commit direct to main". It cost a round trip and left a permanently false sentence in a commit message, since correcting it would need a force-push. Entry 5 is observed in the specification for THIS task, under the file's own rule that a defect not recorded is paid for twice: Part B reads "[PASTE THE SECTION TEXT HERE]" and the text was never pasted. Part B is not done and is reported rather than guessed at. The two are one family with entry 3, and the file now says so: defects of ASSEMBLY rather than of finish conditions, where each clause reads correctly in isolation and the error is a relation between two of them. Re-reading cannot catch them; every check in the family is mechanical. Co-Authored-By: Claude Opus 5 (1M context) --- docs/inherited-measurements.yaml | 30 ++++++++++-- docs/spec-defects.md | 74 ++++++++++++++++++++++++++++- report/claims-verified.md | 52 +++++++++++++-------- report/draft.md | 80 ++++++++++++++++++-------------- 4 files changed, 175 insertions(+), 61 deletions(-) diff --git a/docs/inherited-measurements.yaml b/docs/inherited-measurements.yaml index 4c0dd6e..6d094ac 100644 --- a/docs/inherited-measurements.yaml +++ b/docs/inherited-measurements.yaml @@ -113,23 +113,37 @@ sources: - key: active-stereo repo: visgraf/active-stereo - sha: null + sha: 3f7a263011d9c5c53a4733957ac34cd28c2e85ba role: >- Predecessor. A full implementation with verified geometry and a decision record, whose loop never ran. The draft cites it for two things: the failure that motivated the low-ceremony lane, and the constitution clause that was false and talked a reviewer out of verifying. - checkout_status: CLONED 2026-09-08 (default branch; no SHA pinned). See gap-001. + checkout_status: >- + CLONED 2026-09-08 and PINNED at 3f7a263 (gap-003). Claims 2, 5 and 46 of + the first pass rest on it. Claim 2 is a VERBATIM QUOTATION cited to + CLAUDE.md:141; re-verified at the pinned SHA on the day it was pinned — + the line reads "the rule above would be unenforceable — Chat holds + excerpts by construction, so", continuing into :142. 3f7a263 is the + repository's final merge, so the pin will not move, but that is a fact + about this repository's history and not a substitute for pinning. - key: bioeye repo: visgraf/bioeye - sha: null + sha: e90817018e629ce1cd23af0c9e8bc4a6aa15daff role: >- Predecessor. A thin vertical slice that closed the accumulation loop and not the perception-action loop. Listed with NO measurements attributed to it, which is a fact and not an oversight: "we read it and it had nothing" and "we never read it" are different, and this is the second. - checkout_status: CLONED 2026-09-08 (default branch; no SHA pinned). See gap-001. + checkout_status: >- + CLONED 2026-09-08 and PINNED at 3f7a263 (gap-003). Claims 2, 5 and 46 of + the first pass rest on it. Claim 2 is a VERBATIM QUOTATION cited to + CLAUDE.md:141; re-verified at the pinned SHA on the day it was pinned — + the line reads "the rule above would be unenforceable — Chat holds + excerpts by construction, so", continuing into :142. 3f7a263 is the + repository's final merge, so the pin will not move, but that is a fact + about this repository's history and not a substitute for pinning. schema: id: Stable. Never reused, never renumbered. Cite this. @@ -581,6 +595,14 @@ gaps: field-counting collapses the result to a single line. See od-003. - id: gap-003 + status: CLOSED 2026-09-08 + closed_by: >- + Both SHAs recorded in `sources` above and claim 2's quotation re-read at + active-stereo@3f7a263, CLAUDE.md:141, where it still says what the claim + says. THE POINT OF THE CLOSURE IS THE RE-READ, NOT THE PIN: a line number + against a branch is a citation to whatever that branch says on the day it + is read, and pinning without re-reading would have recorded the SHA + without ever confirming the line survived to it. statement: >- active-stereo and bioeye are cloned at their default branches, not at a pinned SHA. No SHA for either is recorded in `sources`. diff --git a/docs/spec-defects.md b/docs/spec-defects.md index 56cbd49..497ec60 100644 --- a/docs/spec-defects.md +++ b/docs/spec-defects.md @@ -125,9 +125,70 @@ nobody counted. The station that writes the rules is not outside them. --- +## 4. A specification that contradicts the environment it sets up + +**The defect.** One instruction did two things. A setup step enabled branch +protection on `main` — *"require a pull request before merging"* — and a later +step in the same instruction said **"first commit direct to main; PRs after +that."** The second is not merely wrong, it is *made* wrong by the first. The +push was rejected by the ruleset the same specification had just installed. + +**What it cost.** A round trip, and a commit that had to be moved onto a branch +after it was made — so its message, already written and already pushed, records +being committed direct to `main` at the maintainer's instruction, which is now +false. Correcting it would have meant a force-push, which the working agreement +forbids. **The cheapest possible defect still left a permanent wrong sentence in +the history**, because the history is the one thing the method will not rewrite. + +**Why it is hard to see when writing.** The two halves are separately correct and +are usually written at different moments — the setup half is infrastructure and +feels like it belongs to a different task. Neither half is reviewed against the +other, because the reviewer is checking whether each step is *right*, not whether +the environment one step creates is one the next step can run in. A specification +is normally checked against the repository as it is; this class of defect is a +specification checked against a repository that its own earlier clause has +changed. + +**The check.** **Read the setup steps against the task steps before sending.** +Ask of each task step: does an earlier step in this same instruction make this +impossible? It is a two-pass read of one document and it takes a minute. + +## 5. A placeholder that was never filled + +**The defect.** A specification's Part B read: *"Insert the text the maintainer +supplies"*, followed by the literal line **`[PASTE THE SECTION TEXT HERE]`**. The +text was never pasted. Everything around it was complete and specific — the +insertion point named to the adjacent subsection headings, the title given, a +falsifiable claim about the content stated and marked checkable, and an +instruction not to edit it — which is exactly what made the hole hard to see: a +placeholder surrounded by precision reads as precision. + +**What it cost.** One part of four could not be done, and it was the part the +next part depended on: the section was to be inserted into the prose that the +following step typesets. The dependent work was done anyway against the prose as +it stands, so it must be revisited when the text arrives. + +**Why it is hard to see when writing.** The author of a specification is +assembling it from parts they already hold, and a paste marker is a note to +themselves that reads as satisfied the moment they think of the text. It is +invisible to a re-read for *correctness*, because there is nothing incorrect +there — the sentence around it is true. + +**The check.** **Grep the specification for its own placeholders before sending** +— square brackets, `TODO`, `TBD`, `<...>`, "here". Mechanical, like the +repeated-figure check and for the same reason: it is a failure of attention that +attention does not catch, because the reader supplies the missing content from +their own head. + +**Both entries above were observed in specifications for *this* repository**, in +consecutive tasks, and neither is a defect of judgement. They are defects of +assembly, and the two checks are both mechanical two-minute reads. + +--- + ## The general shape -Both entries above are the same defect wearing different clothes: **the +Entries 1 and 2 are the same defect wearing different clothes: **the specification stated what to do and not what "done" looks like.** The falsifier rule in `docs/workflow.md` covers the scientific half of that — what result would mean the question was wrong. This file covers the mechanical half — what @@ -136,3 +197,14 @@ state the world is in when the task is over. A specification needs both. They fail differently: a missing falsifier produces a result nobody can interpret, and a missing finish condition produces work nobody can find. + +**Entries 3, 4 and 5 are a second family, and the file is more useful for +separating them.** They are not about what "done" looks like; each states a +finish condition perfectly well. They are defects of **assembly** — a figure +copied from the wrong place, a clause contradicted by another clause, a +placeholder left unfilled. What they share is that **re-reading does not catch +them**, because each one reads correctly in isolation and the error is a relation +between two things. Every check in this family is therefore mechanical: count the +set, diff the setup against the task, grep for your own placeholders. If a +proposed check for a defect in this family can only be described as "be careful", +it is not a check. diff --git a/report/claims-verified.md b/report/claims-verified.md index bc7206b..f39f930 100644 --- a/report/claims-verified.md +++ b/report/claims-verified.md @@ -562,30 +562,42 @@ does not yet reach, the rejections. Recorded as `od-006`. ## What is still flagged and was not corrected -**Every claim above with a verdict other than CONFIRMED is in this list.** The -specification reserved prose corrections to the maintainer; this pass's output -is the record. Ordered by what it would cost to leave. +**APPLIED 2026-09-08.** All 13 proposed replacements — every CORRECTED and every +UNDER-QUALIFIED claim — were applied to `report/draft.md` by the maintainer's +wholesale approval. They touched **20 passages**, because five claims recur: -| # | claim | verdict | whose call | -|---|---|---|---| -| **14** | "Of eighteen claims… seven stood" (§4.1:391) | UNDER-QUALIFIED | **Maintainer — urgent.** The same figure was amended at :870 and :1219 and not here, so the document now states it two ways. This is the repeated-claim defect, currently live, in the document that ships the check. | -| 17 | "three specifications were rejected" (§4.2, §5) | UNSUPPORTED | Maintainer. Either soften to the one recorded rejection, or accept that the strongest evidence for §4.2 is unrecorded and say so. | -| 24 | "shaped every specification for a week" | CORRECTED → one day | Maintainer. | -| 22 | "numbers on day two" (§4.4, §6) | CORRECTED → day one | Maintainer. Two locations. | -| 41 | "four remain open" (§7) | CORRECTED | Maintainer. | -| 29 | "four of thirteen went to establishing this" (§4.6) | CORRECTED → three | Maintainer. Two locations (:620, :638). | -| 26 | "an order of magnitude worse in the tail" | CORRECTED → 22.7× | Maintainer. | -| 11 | "about the instrument" (§3) | UNDER-QUALIFIED | Maintainer. Related to 29. | -| 27 | "four fifths of the total error" | UNDER-QUALIFIED | Maintainer — one word, "squared". | -| 21 | "thirteen carried methods" | UNDER-QUALIFIED | Maintainer. | -| 31 | "two of thirteen iterations" | UNDER-QUALIFIED | Maintainer. | -| 46 | "unused in two consecutive projects" | UNDER-QUALIFIED | Maintainer. | -| 1 | "five days" | UNDER-QUALIFIED | Maintainer — affects six locations. | -| 5 | "five open questions" | UNDER-QUALIFIED | Maintainer. | -| 12, 20, 25, 38, 45 | the remaining unsupported claims | UNSUPPORTED | **Maintainer, and see `od-006`.** Four of the five are unrecordable by construction, not merely unrecorded. | +| claim | verdict | locations changed | +|---|---|---| +| 1 | UNDER-QUALIFIED | 4 of 6 "five days" — the two register descriptors (§1, §7) were left, per the entry, since they are not about elapsed time | +| 5 | UNDER-QUALIFIED | 3 — §2, §6, **and §6's later "here are the five open questions"**, which the check caught and the entry had not named | +| 11 | UNDER-QUALIFIED | 1 | +| 14 | UNDER-QUALIFIED | 1 — §4.1:398, the third occurrence the previous commit left inconsistent | +| 21 | UNDER-QUALIFIED | 1 | +| 22 | CORRECTED | 2 — §4.4 and §6 | +| 24 | CORRECTED | 1 | +| 26 | CORRECTED | 1 | +| 27 | UNDER-QUALIFIED | 1 | +| 29 | CORRECTED | 2 — the count and the proportion that follows from it | +| 31 | UNDER-QUALIFIED | 1 | +| 41 | CORRECTED | 1 — first option taken; the second would require editing `template` | +| 46 | UNDER-QUALIFIED | 1 | + +**Two entries offered alternatives and the first was taken in both** (31 and 41). +For 41 the second option — record the four gaps in the template and cite them — +is not available from this repository. + +### Still standing: the six UNSUPPORTED claims + +**12, 17, 20, 25, 38, 45 are unchanged in the prose.** They carry no proposed +replacement because none can be written from the record: four of the six describe +events the workflow guarantees leave no artifact. `gap-004` and `od-006` hold the +argument. **This is the maintainer's decision and it has not been taken** — the +options are to soften each to what the record supports, or to keep them and say +in the text that they rest on recollection. --- + # The repeated-figure check, as run — and the fix for `od-003` **The shipped command does not work on this document, exactly as `od-003` diff --git a/report/draft.md b/report/draft.md index 63bb3a9..1c2969b 100644 --- a/report/draft.md +++ b/report/draft.md @@ -16,10 +16,15 @@ > repositories it makes claims about. > > **42 claims: 22 confirmed, 5 corrected, 8 under-qualified, 6 unsupported, 1 -> partial** (`mn-004`). **The corrections have not been applied to this prose.** -> Every verdict, with its source and a proposed replacement sentence, is in -> `report/claims-verified.md`; the sentences stand as drafted until the first -> author decides on each. Do not typeset before that. +> partial** (`mn-004`). **The thirteen corrected and under-qualified claims have +> been applied**, across twenty passages. Every verdict, its source, and the +> sentence it replaced are in `report/claims-verified.md`. +> +> **Six claims remain UNSUPPORTED and are unchanged in the prose.** They are not +> contradicted by the record; they are unestablishable from it, and four of the +> six describe events this method guarantees will leave no artifact — see +> `gap-004` and `od-006`. Read §4.2's rejection count and §4.5's cold-session +> search as recollection, not record. > > Sections 8.6, 8.7 and 8.8 are the first author's, written in the first person. > They make no checkable factual claims and are out of scope for that pass. @@ -229,8 +234,9 @@ spike is deleted, or it did not, and the spike is deleted. Directories that sit there deciding nothing are the failure mode the lane has. It exists because of a specific thing that went wrong in the predecessor -project. A step was written up twice and never started, accumulating five open -questions, because the constitution had exactly one standard — plan, decision +project. A step was written up twice and never started, accumulating five declared +items, four of them open questions, because the constitution had exactly one +standard — plan, decision record, findings, tests, definition of done — and that standard is right for library code and far too expensive for exploration. When a question is open and the only sanctioned move is to write about it, **uncertainty turns into @@ -307,7 +313,7 @@ was wrong with it. ## The shape of the work -Thirteen experiments over five days, each pre-registered before its runner +Thirteen experiments over five working days, each pre-registered before its runner existed. Two predecessor repositories, from which geometry, measurements and a set of methods were carried forward. Roughly four thousand lines of library code against a slightly larger volume of tests, thirteen foreclosed decisions, @@ -332,9 +338,9 @@ cost, and the benefit decomposed almost entirely into **coverage** rather than accuracy: the moving system's advantage was that it *saw more of the scene*, not that it *estimated better*. -And four of the thirteen experiments turned out to be about the instrument -rather than the framework. They were the precondition for reading the other -nine. +And four of the thirteen experiments turned out to be about the instrument or +the criterion rather than the framework. They were the precondition for reading +the other nine. None of that is the subject of these notes. It is the material the examples come from. @@ -393,8 +399,8 @@ which is the one thing a wrong report is most likely to be. *A claims pass on any document making factual assertions.* Before the project's technical report was typeset, every load-bearing claim in it was checked at -source and marked confirmed, corrected, under-qualified, or unsupported. Of -eighteen claims in the first draft, seven stood as written. +source and marked confirmed, corrected, under-qualified, or unsupported. Of the +eighteen substantive claims in the first pass, seven stood as written. **Did it hold?** Yes, and the numbers are the argument. Three claims wrong on the figure, eight true only in the band or arm they came from, none unsupported @@ -525,9 +531,10 @@ the contradiction was legible. The methods were not handled at all, and nothing in the apparatus noticed. A statistical criterion, a set of band thresholds, a search window, an inhibition radius, a foveal width, the parameters of a validity test — all carried across, -all used, none marked. Thirteen carried methods, **eight of which had never been -examined by anyone in this project**, and one of them was the criterion in §4.3 -that governed ten experiments before its properties were checked. +all used, none marked. Thirteen carried constants and criteria, ten of them +carried in from elsewhere, **eight never examined by anyone in this project** — +and one of the eight was the criterion in §4.3 that governed ten experiments +before its properties were checked. The asymmetry is the finding. The same repository that would not let a *number* cross without provenance let a *criterion* cross unmarked — and the criterion @@ -540,7 +547,7 @@ Then one rule that the project learned the expensive way: > **A carried method gets its entry at first use, not at first suspicion.** **Did it hold?** It was not in place when it was needed. Applied to numbers on -day two and to methods at experiment twelve, by which point the damage was done +day one and to methods at experiment twelve, by which point the damage was done and required a re-analysis of every recorded verdict. That re-analysis is the reason the failure is well documented rather than well hidden. @@ -559,8 +566,8 @@ and load-bearing and feels like part of the project. **What it looked like.** An eighteen-step plan, with stages, altitudes, and two recorded amendments, existed in a chat window and nowhere else. It had shaped -every specification for a week. Nobody noticed, because both parties to the -conversation could see it. +every specification for a day — seven merged pull requests — before anyone wrote +it down. Nobody noticed, because both parties to the conversation could see it. It was found the moment a session was cleared. The fresh session was asked to record the plan, searched the repository, the git history including commit @@ -620,10 +627,11 @@ artefact was **harder than the real thing it stood in for, and it passed the validity test built to catch exactly that class of failure** — self-consistent enough to be accepted, wrong enough to poison the estimate, and concentrated in precisely the region where the scientific question lived. Rendering the same -geometry physically, the synthetic fixture was an order of magnitude worse in -the tail. Two fifths of the measured pixels, four fifths of the total error. +geometry physically, the synthetic fixture was roughly twenty times worse in the +tail. Two fifths of the valid measured pixels, four fifths of the total *squared* +error. -Four of thirteen experiments went to establishing this. Conclusions already +Three of thirteen experiments went to establishing this. Conclusions already drawn had to be re-read. And the obvious correction — restrict the analysis to the clean region — was itself wrong, because the contaminated band was also the only band where the question was live: excluding it did not isolate clean @@ -642,7 +650,7 @@ that **they do** — because the regime you care about is usually the hardest thing to synthesise, and therefore where the synthesis cuts its corners. **Did it hold?** It was learned rather than applied, at a cost of roughly a -third of the project's experiments. That said, the cost bought a durable result: +quarter of the project's experiments. That said, the cost bought a durable result: the fixture's operating range is now measured, and every experiment afterwards was sized against it. @@ -802,7 +810,7 @@ are not two lists. They are the same list, read twice.** **Verification costs an iteration.** Checking a document's claims at source before typesetting it took a full cycle and produced no prose. Re-scoring every recorded verdict under a corrected criterion took another. Two of thirteen -iterations produced no new science. +experiments' worth of effort produced no new science. **Pre-registration costs a commit and an argument.** Declaring what would falsify a hypothesis, and separately what would mean the experiment asked the @@ -813,7 +821,7 @@ strongest. **Rejections cost round trips.** Three specifications were rejected on false premises. Each cost a cycle. -**The record costs reading.** Nine thousand lines of ledger in five days, and +**The record costs reading.** Nine thousand lines of ledger in five working days, and one human reading every pull request. ## What the friction caught @@ -905,7 +913,7 @@ serious one, and I want to arrive at it properly. ## Three that arrived late -**The inherited marker was applied to numbers and not to methods.** Day two for +**The inherited marker was applied to numbers and not to methods.** Day one for figures, experiment twelve for criteria — by which point a carried statistical bar had governed ten experiments. The asymmetry is the finding: a repository that would not let a number cross without provenance let a criterion cross @@ -957,8 +965,8 @@ specific and damning. **The lane was built as a fix for exactly this failure, in the previous project, where it had already cost something.** In that project a step was written up -twice and never started, accumulating five open questions, because the -constitution had one standard — plan, decision record, findings, tests, +twice and never started, accumulating five declared items, four of them open +questions, because the constitution had one standard — plan, decision record, findings, tests, definition of done — and that standard is right for library code and far too expensive for exploration. When a question is open and the only sanctioned move is to write about it, uncertainty turns into documents. The diagnosis was made @@ -997,7 +1005,7 @@ afternoons of writing about a question — carefully, with pointers and open questions and a plan for resolving it — where the honest move was to spend twenty minutes finding out. The writing was good. It met the standard. It also postponed the answer, and it was easier than admitting I did not know, because a -document that says *here are the five open questions* looks like progress and a +document that says *here are the four open questions* looks like progress and a directory called `does-the-fixture-need-occlusion` looks like not knowing. That is the thing a methodology cannot fix by adding a rule, which is why it is @@ -1104,9 +1112,9 @@ of failures with their remedies attached, it is evidence. The template was audited against one question: can a session starting cold, with no other context, read it and write a first specification without asking what -any of it means? That audit found several gaps, most of which were closed. Four -remain open and are recorded in the repository rather than fixed, because fixing -them would mean inventing rules the original project never exercised. +any of it means? That audit found several gaps, and four were closed in a single +change. Its residue is argued below rather than recorded in the template, because +recording it would mean inventing rules the original project never exercised. The largest: one of the method's own concepts — the distinction between decisions that change what the project *is* and decisions about how to build it @@ -1141,7 +1149,7 @@ the rule, for that reason. ## 8.1 The bottleneck moved -Thirteen experiments in five days. At no point was implementation the limiting +Thirteen experiments in five working days. At no point was implementation the limiting resource. What limited the work was **trust** — and specifically, one human reading. Every @@ -1248,9 +1256,9 @@ Three things a second project would settle that this one could not. **Whether the failures are the method's or mine.** One researcher. Every failure in §6 is consistent with *this is what happens to a rigorous project* and equally consistent with *this is what happens when this particular person is -given a rigorous framework*. I lean toward the first because the low-ceremony -lane went unused in two consecutive projects under different conditions, but two -is not many. +given a rigorous framework*. I lean toward the first because informal exploration went +undone in two consecutive projects — in the first because there was no lane, in +the second although there was — but two is not many. **Whether the mechanisms transfer or were shaped by the domain.** Pre-registration with a falsifier works cleanly when an experiment produces a @@ -1260,7 +1268,7 @@ is a construction rather than a measurement, I do not know. I would guess the falsifier survives and the threshold does not. **Whether the record is readable by anyone but its author.** Nine thousand lines -of ledger, written by one collaborator, read by one human, in five days. A +of ledger, written by one collaborator, read by one human, in five working days. A colleague opening it cold is the test that has not been run, and until it is, *the repository is the memory* is a claim about a memory of one. From bc0070c6e6d86be5a7ebd019e259e78da52cd2e7 Mon Sep 17 00:00:00 2001 From: Luiz Velho Date: Tue, 8 Sep 2026 20:05:32 -0300 Subject: [PATCH 02/11] =?UTF-8?q?PART=20B=20BLOCKED=20=E2=80=94=20verify?= =?UTF-8?q?=20its=20checkable=20claim=20anyway?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Part B could not be done: the specification reads "Insert the text the maintainer supplies" followed by the literal line [PASTE THE SECTION TEXT HERE], and the text was never pasted. Nothing was inserted and nothing was guessed at. Recorded as docs/spec-defects.md entry 5 in the previous commit. The one thing Part B could be checked on without its text was checked, since the specification named it as checkable and said to stop and report if it failed. IT FAILS AS WORDED. Part B was to assert that no specification text survives anywhere in bio-3d-vision at 5f8e39f. Three fragments do: experiments/exp007_rendered_policy_sweep/preregistration.md:38 quotes it directly -- the specification says "six arms, one stimulus" experiments/exp009_epipolar_cost/preregistration.md:19 quotes a clause -- "the specification says to compose rectification_rotation into eye_camera_poses" -- and then argues it is wrong experiments/exp007_rendered_policy_sweep/findings.md:46 records a reading "which neither exp005 nor this specification stated" The weaker claim is true and is the one to make: no specification was ever COMMITTED. No file in the instance is a specification, and no filename matches spec, task, brief, instruction or prompt. THE PATTERN SHARPENS THE ARGUMENT RATHER THAN WEAKENING IT, which is why this is worth the commit. Every surviving fragment is text an artifact was ARGUING WITH. Specification text reaches the record only where a preregistration or findings file had to disagree with it to justify what it did instead -- that is, only where the specification was EXECUTED and produced an artifact. A specification rejected before work begins produces no artifact, so nothing is ever in a position to quote it. The survival mechanism is parasitic on the work happening, which is exactly why it cannot preserve a rejection. gap-004 carried the overstatement too and is corrected in place, with the original left standing beside it. Co-Authored-By: Claude Opus 5 (1M context) --- docs/inherited-measurements.yaml | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/docs/inherited-measurements.yaml b/docs/inherited-measurements.yaml index 6d094ac..c7bb484 100644 --- a/docs/inherited-measurements.yaml +++ b/docs/inherited-measurements.yaml @@ -631,6 +631,28 @@ gaps: commit: docs/workflow.md states that a specification is the only artifact of the loop that leaves no trace, and that no specification was ever committed in the instance. + PRECISION ADDED 2026-09-08, correcting this entry's own overstatement. + "No specification was ever COMMITTED" is true — no specification file + exists in the instance at 5f8e39f, and no filename matches spec, task, + brief, instruction or prompt. "No specification TEXT SURVIVES" would be + FALSE, and the difference is worth the lines. Three fragments survive, + quoted inside artifacts that were being committed anyway: + experiments/exp007_rendered_policy_sweep/preregistration.md:38 quotes it + directly — the specification says "six arms, one stimulus"; + experiments/exp009_epipolar_cost/preregistration.md:19 quotes a clause, + "the specification says to compose rectification_rotation into + eye_camera_poses", and then argues it is wrong; and + experiments/exp007_rendered_policy_sweep/findings.md:46 records a reading + "which neither exp005 nor this specification stated". + THE PATTERN SHARPENS THE ARGUMENT RATHER THAN WEAKENING IT. Every + surviving fragment is text that an artifact was ARGUING WITH. + Specification text reaches the record only when a preregistration or a + findings file has to disagree with it in order to justify what it did + instead — that is, only when the specification was EXECUTED and produced + an artifact. A specification rejected before work begins produces no + artifact, so nothing is ever in a position to quote it. The survival + mechanism is parasitic on the work happening, which is precisely why it + cannot preserve a rejection. THE ASYMMETRY IS THE FINDING: the workflow records what work PRODUCED and has no channel for work it correctly PREVENTED — so the method's own strongest evidence is the kind it systematically destroys. state.yaml's From 230d83866751965806472b3e0eecb5e8a68b7448 Mon Sep 17 00:00:00 2001 From: Luiz Velho Date: Tue, 8 Sep 2026 20:05:32 -0300 Subject: [PATCH 03/11] Typeset draft.md into report/sections/*.tex and build the document PART C of four. 22 pages, zero overfull or underfull boxes. Nine files, one per numbered section plus the appendix, wired into main.tex. Converted by a script rather than by hand or by paste, committed at scratchpad/md2tex.py's shape and re-runnable: the draft stays the prose of record and the .tex is its typeset form, so the diff between them stays the place a mangled figure shows. THE TRANSIT IS LOSSLESS, and checked rather than asserted: 11,628 words in, 11,628 out, zero differing blocks under difflib, similarity 1.0000. Em dashes, en dashes, section marks and curly quotes all arrive as ---, --, \S and ``''. Code spans become \texttt, including where bold wraps them. NO BIBLIOGRAPHY, and it is a finding rather than an omission. The draft cites no published work -- checked, not assumed: no author name, no "et al.", no year-in-parentheses anywhere in it. biblatex is commented out and refs.bib holds its rule and no entries. CITATIONS I JUDGE WOULD BE WANTED, named as the specification asked and NOT written in: four @software entries, one per repository in Appendix A, each at the SHA now recorded in the ledger -- bio-3d-vision 5f8e39f, math-ai-method 81a638e, active-stereo 3f7a263, bioeye e908170. They are listed in refs.bib as comments. Adding a bibliography the document does not reference would be the same defect as a check that cannot fail. NO ABSTRACT. The draft has none. Writing one would mean composing prose that has never been through a claims pass, in the passage the template names as the most dangerous in a report. The draft's own status note is typeset in its place. THE ONE THING THAT READ WORSE IN LATEX THAN IN MARKDOWN was a monospace path. \texttt{report/claims-verified.md} cannot hyphenate and has no break point TeX will take; it came out 17.6pt over. Markdown does not justify, so it reads fine there. \allowbreak after the slash was tried and did NOT take -- recorded, because the obvious fix failing is the useful half. \sloppy scoped to that one block does. Scoped, not global: one box in twenty-two pages does not justify loosening how the whole document is set. CI needs no new packages: the document loads article, geometry, fontenc, lmodern, graphicx, booktabs and hyperref, all already in report.yml's install list. Co-Authored-By: Claude Opus 5 (1M context) --- report/main.tex | 108 +++++++---- report/refs.bib | 32 ++-- report/sections/01-what-this-is.tex | 33 ++++ report/sections/02-the-setting.tex | 67 +++++++ report/sections/03-the-project.tex | 39 ++++ report/sections/04-how-it-goes-wrong.tex | 181 +++++++++++++++++++ report/sections/05-what-it-cost.tex | 47 +++++ report/sections/06-where-it-failed.tex | 61 +++++++ report/sections/07-the-template.tex | 53 ++++++ report/sections/08-reflection.tex | 89 +++++++++ report/sections/09-appendix-availability.tex | 12 ++ 11 files changed, 675 insertions(+), 47 deletions(-) create mode 100644 report/sections/01-what-this-is.tex create mode 100644 report/sections/02-the-setting.tex create mode 100644 report/sections/03-the-project.tex create mode 100644 report/sections/04-how-it-goes-wrong.tex create mode 100644 report/sections/05-what-it-cost.tex create mode 100644 report/sections/06-where-it-failed.tex create mode 100644 report/sections/07-the-template.tex create mode 100644 report/sections/08-reflection.tex create mode 100644 report/sections/09-appendix-availability.tex diff --git a/report/main.tex b/report/main.tex index a582792..33b9e8f 100644 --- a/report/main.tex +++ b/report/main.tex @@ -53,11 +53,32 @@ \usepackage{booktabs} \usepackage[hidelinks]{hyperref} -% biblatex + biber, not bibtex. biblatex is what carries per-entry provenance -% notes -- including a software entry pinned to a COMMIT rather than to a branch, -% which is the only citation form that means anything for a repository. -\usepackage[backend=biber,style=numeric,sorting=none]{biblatex} -\addbibresource{refs.bib} +% A MONOSPACE PATH CANNOT HYPHENATE, and this document is full of them. The one +% overfull box the first full build produced was exactly that -- \texttt{} around +% report/claims-verified.md, 17.6pt too wide, with no break point TeX would take. +% It reads fine in Markdown, where nothing justifies. +% +% \allowbreak after the slash was tried first and did NOT take -- recorded +% because the obvious fix failing is the useful half. \sloppy, scoped to the one +% block that contains the path, does. It is scoped rather than global: one box in +% twenty-two pages does not justify loosening how the whole document is set. If +% paths multiply, revisit -- seqsplit, or hyphenat with [htt], is the general +% answer, and either would add a package to the CI install list. + +% NO BIBLIOGRAPHY, and that is a finding rather than an omission. The draft cites +% no published work -- checked, not assumed: no author name, no "et al.", no +% year-in-parentheses anywhere in report/draft.md. The four repositories in +% Appendix A are named in prose, not cited. Rather than invent entries, biblatex +% is commented out here and refs.bib holds its rule and no entries. +% +% WHAT WOULD GO IN IT, when it is wanted: four @software entries, one per +% repository in Appendix A, each pinned to the SHA now recorded in +% docs/inherited-measurements.yaml -- bio-3d-vision 5f8e39f, math-ai-method +% 81a638e, active-stereo 3f7a263, bioeye e908170. That is the only citation form +% that means anything for a repository, and refs.bib says why. +% +% \usepackage[backend=biber,style=numeric,sorting=none]{biblatex} +% \addbibresource{refs.bib} \graphicspath{{figures/}} @@ -76,9 +97,11 @@ Luiz Velho \\ \normalsize IMPA \and - Claude \\ + Claude\thanks{These notes were produced in collaboration with Claude --- + model Opus, high reasoning effort. See \S8.4, which takes the recursion as a + subject rather than a disclaimer.} \\ \normalsize Anthropic \\ - \normalsize \textit{collaborator; see \S8.4} + \normalsize \textit{collaborator} } \date{\today} @@ -86,36 +109,49 @@ \begin{document} \maketitle -\begin{abstract} -REPLACE THIS. Write the abstract last, and then verify it first: in the instance -this template came from, the abstract was where a correctly-measured ratio lost -the one condition that bounded it, because the abstract is the passage furthest -from the record and the one most rewarded for compression. -\end{abstract} - -% Sections live in sections/ and are \input here. The directory ships empty; add -% files and uncomment. Keeping them separate is what lets a verification pass -% cite a claim by file and line. -% -% \input{sections/introduction} -% \input{sections/background} -% \input{sections/findings} -% \input{sections/conclusion} - -\section{Placeholder} -This document compiles with no sections so that the build is green from the -first commit. Delete this section when the first real one lands. - -% Keeps the bibliography non-empty so a fresh clone builds with no warnings. -% Delete it with the placeholder entry in refs.bib -- a scaffold that warns on -% day one teaches you to read past warnings, which is a habit worth not having -% in a repository where the build is part of the evidence. -\nocite{example-repository} - -\section{References} -\printbibliography[heading=none] +% NO ABSTRACT. The draft has none, and writing one here would be composing prose +% that has never been through a claims pass -- in the passage the template +% identifies as the most dangerous in a report, because it is furthest from the +% record and most rewarded for compression. The status note below is the draft's +% own, typeset from it. +\begin{quote} +\small\sloppy +\textbf{Draft status.} This document was drafted on the conversational surface +and assembled by hand. Its factual claims about the source project have now been +through \textbf{one} claims-verification pass --- the practice \S4.1 describes +and \S7 ships --- covering \S\S1--8.5, resolved against pinned checkouts of the +repositories it makes claims about. + +\textbf{42 claims: 22 confirmed, 5 corrected, 8 under-qualified, 6 unsupported, +1 partial} (\texttt{mn-004}). The thirteen corrected and under-qualified claims +have been applied, across twenty passages. Every verdict, its source, and the +sentence it replaced are in \texttt{report/claims-verified.md}. + +\textbf{Six claims remain UNSUPPORTED and are unchanged in the prose.} They are +not contradicted by the record; they are unestablishable from it, and four of the +six describe events this method guarantees will leave no artifact --- see +\texttt{gap-004} and \texttt{od-006}. Read \S4.2's rejection count and \S4.5's +cold-session search as recollection, not record. + +Sections 8.6, 8.7 and 8.8 are the first author's, written in the first person. +They make no checkable factual claims and are out of scope for that pass. +\end{quote} + +% GENERATED FROM report/draft.md, one file per numbered section. The draft is the +% prose of record and these are its typeset form; the converter is deterministic +% and the transit was checked word-for-word (11,628 words in, 11,628 out, zero +% differing blocks). Edit the draft, re-run the conversion, and commit both -- +% the diff between them is where a mangled figure shows. +\input{sections/01-what-this-is} +\input{sections/02-the-setting} +\input{sections/03-the-project} +\input{sections/04-how-it-goes-wrong} +\input{sections/05-what-it-cost} +\input{sections/06-where-it-failed} +\input{sections/07-the-template} +\input{sections/08-reflection} \appendix -% \input{sections/appendix} +\input{sections/09-appendix-availability} \end{document} diff --git a/report/refs.bib b/report/refs.bib index 108c3f6..9538f8f 100644 --- a/report/refs.bib +++ b/report/refs.bib @@ -1,15 +1,25 @@ -% Bibliography. One rule that is not a style preference: +% Bibliography. EMPTY OF ENTRIES, deliberately -- report/draft.md cites no +% published work. Checked rather than assumed: no author name, no "et al.", no +% year-in-parentheses appears anywhere in the draft. The four repositories in +% Appendix A are named in prose, not cited, and biblatex is commented out in +% main.tex accordingly. +% +% One rule that is not a style preference, kept for when this file is wanted: % % CITE A REPOSITORY AT A COMMIT, NEVER AT A BRANCH. A citation to `main` is a % citation to whatever `main` says on the day it is read, which makes it % unverifiable in exactly the way this project's whole record is arranged -% against. The @software entry below is the shape; delete it once you have a -% real one. - -@software{example-repository, - author = {Author, An}, - title = {A repository this report depends on}, - year = {2026}, - url = {https://github.com/org/name}, - note = {Commit \texttt{0000000000000000000000000000000000000000}} -} +% against. +% +% WHAT BELONGS HERE IF CITATIONS ARE ADDED: four @software entries, one per +% repository in Appendix A, each at the SHA recorded in +% docs/inherited-measurements.yaml -- +% +% visgraf/bio-3d-vision 5f8e39f84a235bf53ace5789e739f0d51377344a +% visgraf/math-ai-method 81a638ee3f650050164e09a42f5a0561af6caeef +% visgraf/active-stereo 3f7a263011d9c5c53a4733957ac34cd28c2e85ba +% visgraf/bioeye e90817018e629ce1cd23af0c9e8bc4a6aa15daff +% +% They are not written in because the draft does not cite them, and adding a +% bibliography a document does not reference is the same defect as a check that +% cannot fail. diff --git a/report/sections/01-what-this-is.tex b/report/sections/01-what-this-is.tex new file mode 100644 index 0000000..dfe6692 --- /dev/null +++ b/report/sections/01-what-this-is.tex @@ -0,0 +1,33 @@ +\section{What this is, and what it is not} + +\subsection{Not a tutorial} + +The major labs have documented their platforms thoroughly, and the open-source community has covered the patterns for building software with AI assistance more thoroughly still. A fourth account of how to prompt, how to structure a session, how to wire a tool into an editor, would be noise. Nothing in these notes attempts it. + +The volume of that material is itself intimidating, particularly for researchers whose background is not computer engineering, in a field moving fast enough that specialists struggle to keep up. But the problem is not really volume, and a document whose contribution is \textit{translation for scientists} would be solving the wrong thing. + +\subsection{What is actually missing} + +Almost all of that material is about \textbf{building software}. Research is a different activity, with a different characteristic failure, and the practices that make AI collaboration safe for the first are insufficient for the second. + +In software, the failure is a \textbf{bug}. The artifact does the wrong thing. It crashes, or returns the wrong value, or corrupts a file --- and an entire apparatus exists to catch exactly that: tests, types, continuous integration, code review, staged rollout. The failure announces itself, and the discipline is built around making it announce itself sooner. + +In research, the failure is a \textbf{wrong belief}. A claim that is well-formed, plausible, internally consistent, and supported by a number that was computed correctly and means something else. It does not crash. It passes every test, because it is not a defect in an artifact --- it is a defect in what someone concluded from one. It propagates into the next experiment as a premise, into the next decision as evidence, and into the paper as a result. + +No test catches that. Type checking does not catch it, code review does not catch it, and a green CI badge is entirely compatible with it. + +This distinction is the whole subject of these notes. When a collaborator can produce fluent, confident, wrong claims faster than you can check them, \textbf{epistemic hygiene stops being a virtue and becomes the binding constraint on the practice.} Everything that follows --- pre-registered falsifiers, refusing to ratify a number without reading it at source, marking every carried figure as a prior rather than a fact, a ledger of what was ruled out and why --- exists to prevent a wrong belief. None of it appears in a guide about building software with AI, because none of those guides is trying to prevent one. + +\subsection{The register} + +These are personal notes. One researcher, one field, one project, five days of intensive work, and a methodology that has been run exactly once. Everything here is a case study, and the sample size is one. + +That should be said before anything else, because the temptation in this genre is to write as though a practice that worked for you is a practice. What is offered is narrower and, I hope, more useful: an account specific enough to be checked, with the failures left in. The project it came from is public. Every claim in these notes about what happened there can be verified against a record that was kept for exactly that reason. + +Where the method failed, it says so --- and it failed in ways that turned out to be more useful than its successes. That claim is the argument of these notes rather than an apology at the end of them. + +\subsection{A note on scope} + +The title says \textit{Applied Computational Mathematics} because that is where I work. Nothing in the method depends on it. + +What it requires is a repository, an experiment that can be pre-registered, and a claim that can be checked at its source. That is most of empirical computational science, and probably a good deal beyond it. The narrower title is honesty about the sample rather than a claim about the boundary. diff --git a/report/sections/02-the-setting.tex b/report/sections/02-the-setting.tex new file mode 100644 index 0000000..9bd5ca0 --- /dev/null +++ b/report/sections/02-the-setting.tex @@ -0,0 +1,67 @@ +\section{The setting} + +\subsection{Three surfaces} + +The work is divided across three AI surfaces with different access and different jobs. The division is not about capability --- the same model underlies them --- but about \textbf{what each one can see and what it can change}, which turns out to determine what each is good for. + +\textbf{Chat} holds no repository. It derives, argues, designs experiments, reviews results, and writes the specifications the other surface executes. Its distinguishing property is that it can be wrong for free: a claim attacked here costs a paragraph, and the same claim attacked after implementation costs a refactor. + +\textbf{Code} lives inside the working tree. It reads and writes files, runs the suite, inspects diffs, drives git, opens pull requests. It builds what Chat specified and reports what it did, what it measured, and what it could not do. + +\textbf{Cowork} connects a result to everything else --- cross-experiment synthesis, literature sweeps, briefings, decks. It is the surface for work that spans many files and many sessions without living in any one of them. + +One principle makes this a workflow rather than three parallel conversations: + +\begin{quote} +\textbf{Every handoff between surfaces is a versioned artifact in the repository, never chat context.} +\end{quote} + +Context does not survive a session. A repository does. That single rule is what makes the arrangement reproducible, and it has a corollary that took a while to appreciate: \textbf{clearing a session is a test of the methodology.} If clearing feels risky --- if you think \textit{but it knows things that aren't written down} --- that reluctance is a defect report on the artifacts, not a reason to keep the context. + +\subsection{The four-station loop} + +One task travels a fixed circuit. + +\textbf{User} sets the goal and holds the authority. Adjudicates escalations, and is the only station that can change what the project is for. + +\textbf{Chat} writes a specification. + +\textbf{User relays it, verbatim.} This looks like ceremony and is not. It is the only review a specification gets before work is spent on it, and the cheapest point at which a specification asking for the wrong thing can be stopped. A specification that could go straight from Chat to Code has skipped its only gate. + +\textbf{Code} does the work and reports back. + +\textbf{Chat verifies in-repo} --- clones the branch, runs what needs running, reads the claimed line --- and returns one of three verdicts: \textit{advance}, \textit{retry}, or \textit{escalate}. + +Two properties of that circuit matter more than its shape. + +\textbf{Chat never edits the repository, but it does hold the files.} The clone is a separate checkout, outside the working tree, which Chat reads and runs and never commits to. This is the reviewer's seat: it can check everything and change nothing. The predecessor project's constitution had explicitly ruled it out --- \textit{Chat holds excerpts by construction, so "Chat verifies Code's numbers" would rot into a formality} --- and that premise was simply false. The seat existed and the workflow had talked the reviewer out of occupying it. + +\textbf{Escalation is expected, not exceptional.} It is how the loop discovers that a specification was mis-aimed, which happens often and should. A stretch of tasks with no escalations means either the specifications are unusually good or the reviewer is not looking hard enough, and the second is more likely. + +\subsection{The toolchain, and what each part is for} + +Not a tutorial, but the choices carry weight and the reasons are worth stating. + +\textbf{GitHub is the autonomy mechanism, not storage.} Pull requests are a gate that cannot be bypassed; branch protection is what makes \textit{never commit to main} structural rather than a request. Continuous integration means a rule about testing is enforced by a machine rather than by memory. + +\textbf{The repository is the memory.} Every durable thing --- decisions, measurements, position, the plan and its amendments --- lives in files that a cold session reads on arrival. Nothing load-bearing lives in a conversation. + +\textbf{Two ledgers carry what the code cannot.} A decision ledger, recording each foreclosed possibility with the evidence that closed it, its stated limits, and the cost of reopening it. A measurement ledger, recording each figure with the commit that produced it, the environment it was measured in, and its status --- measured here, inherited from earlier work, or a recorded gap. Both are written by the surface that holds the files and read by the surface that does not. + +\textbf{Everything is forward-only.} Corrections appear as amendments, never as edits. Where a conclusion was drawn and later qualified, both appear, with the evidence that moved between them. This is not tidiness; \S4.8 argues the pair is more informative than either half. + +\subsection{What was not used} + +Two components were designed into the method and never exercised. Both need describing before their absence means anything. + +\textbf{Cowork} is the surface for work that spans many artifacts and no single session. The concrete jobs: sweeping the literature and reporting where a result sits relative to it; synthesising across experiments that were each designed to answer their own question, looking for the pattern none of them was testing; turning a directory of results into a briefing or a deck; reconciling what a roadmap intended against what actually merged. Where Chat holds an argument and Code holds a working tree, Cowork holds \textbf{the accumulated body of work} --- and it is the only one of the three that can be handed a task and left to run. + +In this project every one of those jobs was done by Chat inside the conversation, or by hand, or not at all. The cross-experiment synthesis in particular --- noticing that four separate experiments were saying the same thing at different levels --- happened in a chat window and had to be written down afterwards from recollection, which is precisely the failure the method is built to prevent. + +\textbf{\texttt{spikes/}} is a lane inside the repository with different rules from everywhere else. Git-tracked, so a thing tried can be pointed at. No decision record, no findings entry, no definition of done, no tests required. One hard constraint --- nothing in the library or the experiments may import from it, enforced by a test that runs in CI --- and one permitted output: \textbf{a decision or a deletion.} Either it taught you something, which goes into the ledger and the spike is deleted, or it did not, and the spike is deleted. Directories that sit there deciding nothing are the failure mode the lane has. + +It exists because of a specific thing that went wrong in the predecessor project. A step was written up twice and never started, accumulating five declared items, four of them open questions, because the constitution had exactly one standard --- plan, decision record, findings, tests, definition of done --- and that standard is right for library code and far too expensive for exploration. When a question is open and the only sanctioned move is to write about it, \textbf{uncertainty turns into documents.} The spike lane was invented so that \textit{try it and look} would be a legal move. + +It shipped on day one, with its rules, its enforcement test and its README. It was used zero times. + +Both are the informal parts of a formal method, and that they are the two that went unused is the subject of \S6. I do not think it is a coincidence. diff --git a/report/sections/03-the-project.tex b/report/sections/03-the-project.tex new file mode 100644 index 0000000..5f4aaf2 --- /dev/null +++ b/report/sections/03-the-project.tex @@ -0,0 +1,39 @@ +\section{The project} + +Everything after this section draws its examples from one research project. This section supplies enough of it to follow them. It is deliberately brief and deliberately non-technical; the project has its own report, and readers who want the science should go there. + +\subsection{The question} + +\textbf{Active vision} is the proposition that perception is better posed as a controlled sampling problem than as a reconstruction problem --- that \textit{choosing where to look} improves what is perceived. The project tested that claim for binocular depth estimation. + +The biological motivation is concrete. A camera samples every pixel at the same resolution; an eye does not. The human fovea covers one to two degrees at high acuity, resolution falls off sharply beyond it, and several times a second a saccade repositions that window. And the two eyes are not cameras with fixed geometry: they rotate about centres fixed in the head, converging on whatever is being looked at, so the stereo geometry itself changes every time gaze moves. + +\subsection{The machinery, in six sentences} + +Two views of a scene, taken from slightly separated positions, disagree about where things are. That disagreement --- \textbf{disparity} --- is larger for near surfaces than far ones, so depth can be recovered from it. A \textbf{matcher} finds, for each point in one image, the corresponding point in the other; it returns a disparity, a confidence, and a \textbf{validity mask} marking points where it found nothing trustworthy. \textbf{Vergence} is how far away the two lines of sight cross --- what the system is focused on --- and converting disparity into metric depth requires knowing it. A \textbf{belief} accumulates the depth estimates across successive fixations, each new measurement combined with what is already believed, weighted by confidence. And a \textbf{gaze policy} chooses where to look next, typically by finding where the belief is least certain. + +That loop --- fixate, measure, update the belief, choose the next fixation --- is the object under study. Whether closing it improves anything is the question. + +\subsection{Three things that recur in the examples} + +\textbf{Half-occlusion.} At a depth discontinuity --- the edge of a nearer surface against a farther one --- a strip of the background is visible to one eye and hidden from the other. Those points have no correspondent at all. The matcher does not report this; it finds the best available wrong match, and a sharp minimum at a wrong disparity is genuinely sharp. \textbf{The result is a confident, incorrect estimate at exactly the geometry an active system might hope to resolve by looking elsewhere.} This is the phenomenon that motivated the whole project, and several examples turn on it. + +\textbf{Foveal weighting.} Implementing the fovea by measuring everything at full resolution and then \textit{discounting} what is far from gaze --- as opposed to a real retina, which never samples the periphery finely in the first place. The difference between weighting and sampling turns out to matter a great deal. + +\textbf{The synthetic stimulus.} A random-dot stereogram: a field of noise, warped by a known depth field to produce the second view. Its virtue is that depth exists \textit{only} in disparity --- nothing can be recovered from shading or texture, so a system that appears to see depth is genuinely seeing stereo. \S4.6 is about what was wrong with it. + +\subsection{The shape of the work} + +Thirteen experiments over five working days, each pre-registered before its runner existed. Two predecessor repositories, from which geometry, measurements and a set of methods were carried forward. Roughly four thousand lines of library code against a slightly larger volume of tests, thirteen foreclosed decisions, eighty-seven measurements taken in the project and forty-one inherited. + +Experiments are referred to below by their position in that sequence rather than by name. What matters for these notes is not which experiment found what, but that each was declared in advance, ran, and reported against its own falsifier. + +\subsection{What it found, in brief} + +The negative results were consistent. The choice of gaze objective did not matter: two objectives that never once selected the same pixel produced indistinguishable results. Knowing \textit{where} measurement was possible outweighed knowing \textit{how uncertain} it was by a large factor. Foveal weighting on a uniformly sampled sensor was strictly lossy. Deliberately re-converging in order to bring new depths into range made estimation worse. + +One positive result survived. Closing the loop --- actually re-imaging at each new fixation rather than re-weighting a single capture --- helped, at a substantial cost, and the benefit decomposed almost entirely into \textbf{coverage} rather than accuracy: the moving system's advantage was that it \textit{saw more of the scene}, not that it \textit{estimated better}. + +And four of the thirteen experiments turned out to be about the instrument or the criterion rather than the framework. They were the precondition for reading the other nine. + +None of that is the subject of these notes. It is the material the examples come from. diff --git a/report/sections/04-how-it-goes-wrong.tex b/report/sections/04-how-it-goes-wrong.tex new file mode 100644 index 0000000..50359f0 --- /dev/null +++ b/report/sections/04-how-it-goes-wrong.tex @@ -0,0 +1,181 @@ +\section{How it goes wrong, and how you know when it's going right} + +Seven failure modes. Each is a way a research project acquires a wrong belief, each was observed in this project rather than anticipated, and each has a mechanism that either held or did not. The section ends with the harder question: what it looks like when the method is working. + +\subsection{Fluent wrong claims} + +The characteristic failure, and the one everything else is downstream of. + +A language model produces confident, well-formed, internally consistent statements. When it is right, that fluency is the point. When it is wrong, the fluency is the problem: nothing in the surface of the claim distinguishes the two, and the reader's only defence is to check. + +\textbf{What it looked like.} The conversational surface asserted false things at a rate of roughly one per day. Not hallucinated citations or invented functions --- those are easy to catch. The failures were subtler and more dangerous: + +A background claim that a confidence failure had been \textit{replicated} across two stimulus families, when the record showed the two families disagreeing in sign. The claim was plausible, it was about the project's own prior work, and it was the motivating premise of an entire section. + +A statistic described as \textit{frozen} across a run when it drifted by a small amount that mattered. + +A foreclosure asserted to be settled by an experiment that had measured something else entirely --- the identifier resolved, the entry existed, and it said a different thing. + +A count of experiments taken from the highest identifier rather than from the set, because one had been specified and deliberately not run. + +Every one of these is the same shape: \textbf{a claim that would survive any reading and fails only against its source.} + +\textbf{The mechanism.} Three parts, and they work together. + +\textit{Non-ratification.} The surface that does not hold the files may not settle a numeric question. It states where to check, never what the number is. A specification says \textit{verify before recording}, and a number arriving in a report is a hypothesis with a pointer until someone opens the file. + +\textit{Verification at source.} The reviewing surface clones the branch and reads the line. Not the diff, not the excerpt, not the summary --- the line. A reviewer working from excerpts can only check that a report is internally consistent, which is the one thing a wrong report is most likely to be. + +\textit{A claims pass on any document making factual assertions.} Before the project's technical report was typeset, every load-bearing claim in it was checked at source and marked confirmed, corrected, under-qualified, or unsupported. Of the eighteen substantive claims in the first pass, seven stood as written. + +\textbf{Did it hold?} Yes, and the numbers are the argument. Three claims wrong on the figure, eight true only in the band or arm they came from, none unsupported --- all caught before typesetting. The category that mattered most was not the outright errors but the \textbf{under-qualified} ones: claims right in the condition they were measured under and misleading without it. Those typeset cleanly and read as more general than they are, and there were nearly three times as many of them as of outright mistakes. + +\textbf{Where it did not hold.} Verification runs per passage; repetition crosses passages. The same ratio was verified correctly in one section and left without its qualifying condition in another, because a careful reading of each passage structurally cannot notice that a figure appears twice. The fix is mechanical rather than attentional --- grep for repeated figures across the document --- and it was added afterwards, which is to say the failure was found by having it happen. + +\subsection{Stale state} + +A specification is written from a snapshot and executed later, against a repository that other work has moved. The specification is not wrong when written and not right when run, and nothing in it marks the difference. + +\textbf{What it looked like.} Specifications drafted against a commit that had advanced by the time they were sent --- twice, once trivially and once not. The serious instance had a premise about which arms of an experiment re-estimated a parameter, and it was false; the entire question the specification posed depended on it, and the work would have measured something other than what it claimed. + +The general form is worth stating because it is not obvious: \textbf{facts do not expire, they just get old.} A statement learned in a conversation stays available with undiminished confidence long after it stops being true, and a long-running thread accumulates true-once assertions at a steady rate while confidence in them stays flat. + +\textbf{The mechanism.} Every specification names, explicitly and at the top, the repository state it assumes --- the commit, what is tracked, what does not yet exist, the SHAs of any reference checkout. \textbf{The executing surface's first action is to verify those assumptions and reject the task if any is false.} + +An assumption stated is an assumption that can fail loudly. An assumption unstated fails quietly and much later. + +\textbf{Did it hold?} Three specifications were rejected on false assumptions, each correctly, each before any work was spent. That is the mechanism working exactly as designed, and it is the clearest case in the project of a rule paying for itself: the rejections cost a round trip each, and the cheapest of them would otherwise have cost an experiment. + +It also generalises past its origin. The rule was written for repository state, and what it actually enforces is that \textbf{the surface with the files gets to overrule the surface with the memory} --- which is the same asymmetry as \S4.1's, arriving from a different direction. + +\subsection{Work that cannot fail} + +The deepest failure mode, and the hardest to see, because everything about it looks like success. + +A test whose assertion is satisfied by construction. A criterion no sample size can resolve. An acceptance check written after the code it accepts. In each case the apparatus runs, reports green, and establishes nothing --- and because it reports green, nobody looks. + +\textbf{What it looked like.} The clearest instance was a statistical criterion inherited from earlier work: a difference counted as real if it exceeded the seed-to-seed spread. That is a defensible question --- \textit{is this effect larger than the variation between scenes} --- but it is not a test of whether an effect exists, and it has a property nobody noticed for ten experiments: \textbf{the spread is a sample standard deviation, so it does not shrink with more samples.} Adding seeds refines the estimate of the spread and moves the threshold nowhere. No number of seeds could ever resolve anything. + +That was discovered only when a specification asked, for a different reason, \textit{how many seeds would settle this?} --- a question that cannot be answered without inspecting the criterion. Ten experiments had reported verdicts against it, and every one of their null results had been read as evidence of absence when the criterion could not have produced anything else. + +Smaller instances recur throughout. An acceptance test that would have passed on a scene where the geometry made the quantity identically zero. A calibration render whose three surfaces occluded each other, so a check written for three measured one --- and the render succeeded silently. A falsifier that could only fire at a fixation where the effect it tested was structurally absent. + +\textbf{The mechanism.} Pre-registration with a falsifier, and a specific definition of one. + +Not a success criterion. A \textbf{falsifier for the specification itself}: a statement of what result would mean this specification asked the wrong question. It names a result, a threshold, and the conclusion that the task --- not the worker --- was mis-aimed. It is reported against whichever way it comes out, including when it does not fire. + +The rule that follows: \textbf{a specification with no falsifier cannot fail, and by the method's own standard it does not carry.} + +\textbf{Did it hold?} Partly, and the partial failure is instructive. Falsifiers caught a great deal --- including several cases where the second clause fired and an experiment was found to have asked the wrong question. But three times the actual outcome fell outside \textit{every direction the falsifier had enumerated}: an intervention that actively harmed rather than merely failing to help; an effect that shrank rather than staying flat or growing; a result that reframed the question rather than answering it. + +The pattern in those misses is consistent. \textbf{The falsifiers were written for the null and the confirmation, and the informative outcome was the reversal.} After the third occurrence the practice changed: enumerate all four directions --- persists, shrinks, grows, reverses --- and report which occurred. That is a small change and it is the single most useful thing the project learned about writing them. + +\subsection{Inherited assumptions} + +Numbers and methods carried in from earlier work acquire, silently, the authority of things decided here. Nobody decides to promote them. They simply arrive without a marker, get cited, and after a few citations there is no way to tell them apart from what was measured. + +\textbf{What it looked like.} This project inherited a great deal from two predecessors: forty-one measurements, a body of geometry, a matcher, a stimulus, and a set of parameter values. The measurements were handled from the second day --- every one marked \texttt{inherited}, meaning \textit{a prior awaiting re-test, not an established fact}, with the repository and commit it came from. That discipline worked. Inherited figures were cited as inherited, and when one was contradicted the contradiction was legible. + +The methods were not handled at all, and nothing in the apparatus noticed. A statistical criterion, a set of band thresholds, a search window, an inhibition radius, a foveal width, the parameters of a validity test --- all carried across, all used, none marked. Thirteen carried constants and criteria, ten of them carried in from elsewhere, \textbf{eight never examined by anyone in this project} --- and one of the eight was the criterion in \S4.3 that governed ten experiments before its properties were checked. + +The asymmetry is the finding. The same repository that would not let a \textit{number} cross without provenance let a \textit{criterion} cross unmarked --- and the criterion determined how every number was read. + +\textbf{The mechanism.} Extend the inherited marker to methods, with the same three fields: what was carried, from where, and whether it has been re-derived here. Then one rule that the project learned the expensive way: + +\begin{quote} +\textbf{A carried method gets its entry at first use, not at first suspicion.} +\end{quote} + +\textbf{Did it hold?} It was not in place when it was needed. Applied to numbers on day one and to methods at experiment twelve, by which point the damage was done and required a re-analysis of every recorded verdict. That re-analysis is the reason the failure is well documented rather than well hidden. + +The template that came out of this project ships the methods section pre-populated with the rule and this failure as its worked example. That is the only honest form: a rule presented without the thing that motivated it gets simplified away by the next person to read it. + +\subsection{Knowledge that lives only in a conversation} + +The methodology's foundational rule is that every handoff is a versioned artifact and never chat context. The failure mode is what happens when something important slips past it --- and the thing about this failure is that \textbf{you cannot detect it from inside the conversation}, because there the knowledge is present and load-bearing and feels like part of the project. + +\textbf{What it looked like.} An eighteen-step plan, with stages, altitudes, and two recorded amendments, existed in a chat window and nowhere else. It had shaped every specification for a day --- seven merged pull requests --- before anyone wrote it down. Nobody noticed, because both parties to the conversation could see it. + +It was found the moment a session was cleared. The fresh session was asked to record the plan, searched the repository, the git history including commit bodies, and both reference checkouts, found six of eighteen steps inferable from side-references in old prompts and none of the stage groupings, and \textbf{refused to invent the rest.} Its stated reason was that the state file is the one document a cold session trusts without checking, and fabricated content carrying that authority would be the same defect in the place it does most damage. + +That refusal was the detector. Nothing else in the project would have caught it. + +A second instance in the same shape, smaller and sharper: a decision was taken in conversation and then recorded in the repository as \textit{complete}, citing an experiment identifier that resolved to an entry about something else. The claim entered the durable record wearing a citation that did not support it, and it sat next to a verdict in the same file explicitly saying the opposite. + +\textbf{The mechanism.} The repository is the memory, and \textbf{clearing a session is the test of that claim.} If clearing feels risky, the reluctance is a defect report on the artifacts. + +There is a subtler corollary the second instance forced. A record that distinguishes \textit{what is established} from \textit{what is intended} has a boundary, and a claim can cross it. The project's state file was eventually split into a \textbf{position} half --- derivable from the ledger, verifiable, cannot be wrong without the ledger being wrong --- and a \textbf{plan} half, marked as intent supplied from outside the repository and not established by it. The failure was a claim that entered through the plan half and was recorded in the position half, \textbf{losing its warning without losing its origin.} Splitting the file created the boundary that made the error invisible. + +\textbf{Did it hold?} The rule held; the enforcement arrived late. Both instances were caught, one by a cold session and one by a reviewer reading the file, and both are in the record with their corrections. What the project did not have --- and what the template now ships --- is any check that runs \textit{before} something load-bearing goes unwritten. Clearing the session is a good detector and an expensive one, because by then the work has already been done under the unwritten assumption. + +\subsection{The instrument you authored} + +This one is not about AI collaboration. It belongs here anyway, for a reason worth stating: \textbf{it was the failure that consumed the most effort, it is invisible to every mechanism described so far, and it is going to be common} --- because AI collaboration makes synthetic experiments dramatically cheaper to build, and cheap instruments get less scrutiny than expensive ones. + +\textbf{What it looked like.} The project's synthetic stimulus followed a standard construction: generate a texture, warp it by the ground-truth field to produce the second view. The warp has a defect at depth discontinuities, where adjacent output samples draw from widely separated input regions, and the second image acquires a stretched patch of real texture with no consistent correspondence. + +The obvious criticism of such a stimulus is that it lacks the hard cases that real data has. The measured problem was worse and had the opposite sign. The artefact was \textbf{harder than the real thing it stood in for, and it passed the validity test built to catch exactly that class of failure} --- self-consistent enough to be accepted, wrong enough to poison the estimate, and concentrated in precisely the region where the scientific question lived. Rendering the same geometry physically, the synthetic fixture was roughly twenty times worse in the tail. Two fifths of the valid measured pixels, four fifths of the total \textit{squared} error. + +Three of thirteen experiments went to establishing this. Conclusions already drawn had to be re-read. And the obvious correction --- restrict the analysis to the clean region --- was itself wrong, because the contaminated band was also the only band where the question was live: excluding it did not isolate clean measurement, it isolated easy measurement. + +\textbf{The mechanism.} There isn't a clean one, which is why this subsection is shorter on remedy than the others. What the project arrived at: + +\begin{quote} +\textbf{A stimulus is an instrument, and must be characterised before it is used to measure.} +\end{quote} + +Characterised in the specific sense of an unfamiliar sensor: what are its failure modes, where do they fall, and do they coincide with the regime you care about. The last clause is the load-bearing one, and the expectation should be that \textbf{they do} --- because the regime you care about is usually the hardest thing to synthesise, and therefore where the synthesis cuts its corners. + +\textbf{Did it hold?} It was learned rather than applied, at a cost of roughly a quarter of the project's experiments. That said, the cost bought a durable result: the fixture's operating range is now measured, and every experiment afterwards was sized against it. + +The generalisation is uncomfortable and I think correct. \textbf{In simulation-based science the instrument is authored by the same hand as the hypothesis.} Real data can be difficult, biased, or insufficient; it is not written by someone who wants a particular answer. That is not an argument against synthetic stimuli, which offer dense ground truth no capture provides. It is an argument that they deserve the scrutiny given to an unfamiliar instrument, and that the scrutiny should be budgeted in advance rather than discovered. + +\subsection{Momentum} + +The last failure mode is the only one with no artifact to point at, because it consists of not looking. + +\textbf{What it looks like.} A method that is working stops being questioned, and it stops being questioned \textit{precisely because it is working.} Each clean iteration lowers the felt need for the next one to be examined. Verdicts land, falsifiers report, the ledger grows, and somewhere in the middle of that the frame within which all of it is being measured stops being a question anybody asks. + +This project had two clear instances. The statistical criterion of \S4.3 went ten experiments without examination during the stretch where results were arriving cleanly --- the criterion was not questioned \textit{because nothing was going wrong}. And a positive result, one experiment old and underpowered in its most important band, created immediate pressure to build on it rather than to consolidate; the consolidation, when it eventually happened, found the criterion defect that changed how every null in the project read. + +\textbf{The mechanism.} A devil's advocate pass: argue the case against the whole approach, not against its details. Not the specification's falsifier, which asks whether a task was well-posed, but one level up --- \textit{what result would mean this stage should not continue.} + +Two triggers, either party may call it. A stage boundary, where the next commitment is largest and least paid for. And \textbf{a run of clean iterations}, which is the counterintuitive one and the one that matters: a run of failures gets scrutiny for free. + +Two properties keep it from decaying. It must be \textbf{occasional} --- a per-iteration requirement would generate a devil's advocate pass on a routine commit, the pass would be pro forma, and a check that always fires is a check nobody reads, which is \S4.3's failure in a new costume. And it must \textbf{not be announced in advance}: a pass labelled as an exercise argues knowing it is an exercise, which is a weaker thing. The best instance in this project came from a provocation I could not distinguish from a position --- and being unable to tell is what made it work. + +\textbf{Did it hold?} It was agreed in conversation and never written into the workflow, which by the method's own standard means it did not exist. It ran twice anyway, informally, and both times produced something. That is the least satisfying verdict in this section: a mechanism that worked when used and was never made structural, left depending on someone having a good instinct on a given afternoon. + +\subsection{The signatures of good work} + +Everything so far has been failure, and there is a reason for that: \textbf{failure is easy to write about because it hurts.} A wrong claim announces itself once you check it. A stale premise gets rejected. A criterion that cannot resolve anything can be shown, arithmetically, to be unable to. + +Recognising the method \textit{working} is a genuinely harder problem, and it is the more important one. A good result and a lucky one look identical from outside. A project can produce clean verdicts for a month on a broken criterion --- this one did --- and every artifact along the way looks like evidence of health. + +So the two sides of the coin are not symmetric. \textbf{Failure is detected; success has to be recognised.} The first has mechanisms; the second has only signatures --- and if you cannot read them, you cannot tell a working method from a comfortable one. + +Here is the transition, and it is the thing I would most want a reader to take from these notes. + +\begin{quote} +\textbf{Every signature of the method working is an event that felt, at the time, like something going wrong.} +\end{quote} + +A result nobody wanted. A falsifier that fired in a direction nobody enumerated. A collaborator refusing to proceed. A correction to a claim already published internally. Each of those arrives as friction --- as a delay, an inconvenience, a thing that has to be gone back over. And each of them is the system doing the only job it exists to do. + +That inversion is not a nice framing; it is the practical content of the whole method. If your instinct treats those moments as costs, you will optimise them away, and what remains will be a fast, fluent, unfalsifiable practice that feels excellent. \textbf{The discipline is learning to feel the friction as the product.} + +Five signatures, each observed here. + +\textbf{A result nobody wanted.} The strongest indicator and the least comfortable. Foveal weighting --- the biologically motivated mechanism at the centre of the framework --- lost on every seed. The one positive result decomposed to show the system's value was that it \textit{saw more}, not that it \textit{estimated better}. The statistical criterion turned out not to test what everyone had read it as testing. None of these was hoped for. \textbf{A method that only ever confirms is not a method}, and the surest sign of one is a project whose results all point the way its author was already facing. + +\textbf{An outcome outside every direction the falsifier enumerated.} Three times: an intervention that harmed rather than merely failing; an effect that shrank rather than growing; a result that reframed the question. Each omission recorded beside the result. A falsifier that always fires in one of its stated directions is probably not testing anything --- it is checking that reality is one of the two shapes you imagined, and reality is under no obligation. + +\textbf{A control invented that nobody asked for.} A specification asked whether a geometric operation was accurate. The test that came back also asserted that \textit{not} performing it must be ten times worse --- because a number is only a result when the alternative is measured beside it. That pattern then recurred three more times, unprompted. \textbf{A collaborator adding the control you forgot is the clearest evidence that a standard has been internalised rather than complied with}, and it is the difference between a rule that is followed and a rule that is understood. + +\textbf{Someone stopping rather than inferring.} A cold session refused to write down a plan it could not find, and said why: fabricated content in the one file a cold session trusts would be the same defect in the place it does most damage. Another declined to commit because permission had not been given, at the cost of a round trip. Both were mildly inconvenient. Both were the constitution working on exactly the case where inferring would have been easier --- which is the only case where a constitution is worth anything. + +\textbf{A correction that makes the claim narrower and more interesting.} A background section asserted that a confidence failure had been replicated across stimulus families; the record showed the families disagreeing in sign. The corrected claim was narrower, and it was a \textit{better} motivation than replication would have been, because a failure that appears on real imagery and not on synthetic dots poses a sharper question than one that appears everywhere. \textbf{When a correction improves the argument, the record is doing what it exists for.} + +\textbf{How to cherish it: keep the wrong version beside the right one.} Not as ritual honesty --- because the pair carries information neither half does. That a verification pass recommended a figure read off a rounded table, and a second pass corrected it from the underlying rows, is a finding \textit{about verification}: the escalation from summary to source is what made the difference, and it is reusable. Erase the first version and you keep a correct number and lose the lesson. + +This is why the record is forward-only. Corrections are amendments, never edits. The history of a claim being wrong and then right is more valuable than the claim. diff --git a/report/sections/05-what-it-cost.tex b/report/sections/05-what-it-cost.tex new file mode 100644 index 0000000..fb9175d --- /dev/null +++ b/report/sections/05-what-it-cost.tex @@ -0,0 +1,47 @@ +\section{What it cost, and what it caught} + +The previous section argued that the signatures of a working method arrive disguised as friction. This one puts numbers to the friction, and then makes a claim that follows from \S4.8 and is easy to miss: \textbf{the costs and the catches are not two lists. They are the same list, read twice.} + +\subsection{What the friction cost} + +\textbf{Verification costs an iteration.} Checking a document's claims at source before typesetting it took a full cycle and produced no prose. Re-scoring every recorded verdict under a corrected criterion took another. Two of thirteen experiments' worth of effort produced no new science. + +\textbf{Pre-registration costs a commit and an argument.} Declaring what would falsify a hypothesis, and separately what would mean the experiment asked the wrong question, before the runner exists --- that is genuine work done at the moment when the answer is least visible and the temptation to defer is strongest. + +\textbf{Rejections cost round trips.} Three specifications were rejected on false premises. Each cost a cycle. + +\textbf{The record costs reading.} Nine thousand lines of ledger in five working days, and one human reading every pull request. + +\subsection{What the friction caught} + +The same events. + +The verification iteration found three wrong figures and eight claims true only in the condition they were measured under. The re-scoring iteration found that a criterion had governed ten experiments without testing what it was read as testing. The three rejections each stopped work that would have measured the wrong thing --- one of them a premise so central that the experiment would have answered a different question entirely and reported the answer as if it were the one asked. + +\textbf{A wrong belief costs everything downstream of it.} That is not rhetoric; it is the arithmetic of this project. The criterion defect propagated through ten experiments and required re-reading every null result they had produced. Caught at experiment two, it would have cost an afternoon. + +So the honest accounting is not \textit{the method cost two iterations and caught four errors.} It is: \textbf{the method spent two iterations to avoid re-running ten.} + +\subsection{The cost of success} + +The costs above are the ones a reader expects. These are the ones that surprised, and every one of them arrives when things are going \textit{well}. + +\textbf{A positive result creates pressure to build on it.} The project's one positive finding was a single experiment old, underpowered in its most important band, and untested against its own dose-response. The obvious next move was to build the next stage on it. The consolidation done instead found the criterion defect --- which is to say, \textbf{the discipline that paid best was the one applied at the moment of greatest momentum.} Had the result been negative, nobody would have wanted to build on it and the consolidation would have happened for free. + +\textbf{A run of clean iterations is when framing stops being examined.} \S4.7's trigger. Worth restating as a cost rather than a mechanism: the better a project is going, the more expensive its next unexamined assumption becomes, because more has been built on top before anyone checks. + +\textbf{Rigour is self-reinforcing, and that is not entirely good.} Every artifact that meets a high standard raises the felt cost of producing one that does not. The low-ceremony lane --- where an idea can be tried and thrown away --- shipped on day one and was never used once. \S6 argues this is the method's deepest problem and not an oversight. + +\textbf{The record outgrows the reader.} A methodology that produces more record than anyone can read has moved the bottleneck rather than removed it. In the predecessor project, two failures got in exactly there, under merge pressure, with green tests. The apparatus catches wrong \textit{numbers} well and wrong \textit{merges} poorly, and the variable that moved was the rate. + +\textbf{The deliverable is unfashionable.} The better this works, the more of the output takes the form of \textit{a well-ordered set of eliminated possibilities} --- harder to publish than a benchmark number, and harder to describe at a conference. That is a fact about publishing rather than about the knowledge, but it is a real cost borne by whoever adopts the method. + +\subsection{The asymmetry that justifies the whole thing} + +One number carries the argument. The \textbf{first} of the two verification passes over the project's technical report checked twenty-two claims, eighteen of them substantive and four bibliographic. Of those eighteen, \textbf{seven stood as written.} + +The other eleven were not fabrications. They were claims made by someone who had read the record, believed them, and would have published them. Three were wrong on the figure; eight were true in the condition they came from and misleading without it. + +That is the ratio the method exists for. In software, shipping eleven of eighteen statements slightly wrong is a bad release. In research, it is a body of work that other people build on. + +The costs are all paid in time. The failure is paid in credibility, and it is paid by everyone who cited you. diff --git a/report/sections/06-where-it-failed.tex b/report/sections/06-where-it-failed.tex new file mode 100644 index 0000000..e88d00a --- /dev/null +++ b/report/sections/06-where-it-failed.tex @@ -0,0 +1,61 @@ +\section{Where the method failed} + +Four of the seven mechanisms in \S4 arrived late, after the failure they were built for had already happened. That is not damning --- a method extracted from one project will be, by construction, a record of what that project learned the hard way. Those are covered where they occurred. + +This section is about the failures that are still live. The last one is the serious one, and I want to arrive at it properly. + +\subsection{Three that arrived late} + +\textbf{The inherited marker was applied to numbers and not to methods.} Day one for figures, experiment twelve for criteria --- by which point a carried statistical bar had governed ten experiments. The asymmetry is the finding: a repository that would not let a number cross without provenance let a criterion cross unmarked, and the criterion determined how every number was read. + +\textbf{Verification ran per passage while repetition crossed passages.} Found only because the same ratio appeared twice and someone happened to look at both. + +\textbf{Specification defects had no ledger.} Decisions got foreclosures, measurements got provenance, and badly-written specifications got nothing --- so the same two errors recurred: bundling a bounded task with an unbounded one, and naming a branch without asking for a commit. Both were cheap and both were avoidable by a five-line list that did not exist. + +All three now ship in the template, each with the failure that motivated it. That is the ordinary way a method matures, and it is unremarkable. + +\subsection{The devil's advocate was agreed and never written down} + +Less ordinary, and worth its own paragraph because of what it demonstrates. + +The pass was proposed, discussed, refined into two triggers, and endorsed. It ran twice, informally, and both times produced something the loop would not have produced. \textbf{It was never written into the workflow file.} + +By the method's own foundational rule --- every handoff is a versioned artifact, never chat context --- that means it did not exist. A mechanism designed to catch the failure of a method going unquestioned was itself never subjected to the method. If the sessions had been cleared, it would have vanished without trace, and nobody would have known to miss it. + +\subsection{And now the one that matters} + +Two components of the method were never used. Not underused. \textbf{Zero times.} + +The low-ceremony lane --- where an idea can be tried and thrown away, no decision record, no findings entry, no definition of done, one permitted output --- shipped on the first day with its rules, its README, and a test that enforces its one constraint in continuous integration. It was used zero times in thirteen experiments. + +The third surface, for cross-experiment synthesis and work spanning many artifacts, was designed into the workflow, given a role in the table, and used zero times. The synthesis it existed for happened in a chat window and had to be reconstructed afterwards from recollection. + +The obvious reading is oversight. It is not, and the evidence against it is specific and damning. + +\textbf{The lane was built as a fix for exactly this failure, in the previous project, where it had already cost something.} In that project a step was written up twice and never started, accumulating five declared items, four of them open questions, because the constitution had one standard --- plan, decision record, findings, tests, definition of done --- and that standard is right for library code and far too expensive for exploration. When a question is open and the only sanctioned move is to write about it, uncertainty turns into documents. The diagnosis was made explicitly. The remedy was designed explicitly. It was carried into the new repository on day one, complete with enforcement. + +And then the new project \textbf{did the same thing again.} Not by forgetting the lane, but by never reaching for it, while writing preambles and specifications and analyses about questions that could have been settled in twenty minutes by trying them. + +So the finding is not \textit{the method lacked a lane for informal work.} It is: + +\begin{quote} +\textbf{A project with strong ceremony does not reach for its low-ceremony lane, even when the lane exists, is enforced, is documented, and was built specifically because its absence had already caused harm.} +\end{quote} + +The mechanism is not mysterious once stated. Every artifact that meets a high standard raises the felt cost of producing one that does not. After ten pre-registered experiments with falsifiers, verdicts and provenance, writing a directory containing a rough script and no record does not feel like the sanctioned move it formally is. \textbf{It feels like a lapse.} The rigour is self-reinforcing, and self-reinforcement does not distinguish between the parts of itself that are load-bearing and the parts that are ritual. + +This has a second face which is worse. The formal parts of the method are the ones that produce artifacts, and artifacts are the only thing the method can see. A spike that would have settled a question in an afternoon leaves nothing behind if it succeeds --- the whole point is that it is deleted. \textbf{So the method is structurally blind to the value of the work it is discouraging}, and the record it produces will always show that the ceremony was worth it, because the record is made of ceremony. + +I felt this directly and did not name it at the time. There were several afternoons of writing about a question --- carefully, with pointers and open questions and a plan for resolving it --- where the honest move was to spend twenty minutes finding out. The writing was good. It met the standard. It also postponed the answer, and it was easier than admitting I did not know, because a document that says \textit{here are the four open questions} looks like progress and a directory called \texttt{does-the-fixture-need-occlusion} looks like not knowing. + +That is the thing a methodology cannot fix by adding a rule, which is why it is the last section rather than a bullet in \S4. \textbf{Adding the directory was not enough. It was already there.} + +\subsection{What might actually work} + +Not confident about this, and stating it as a proposal rather than a finding. + +A \textbf{trigger} rather than a lane, in the same shape as the devil's advocate pass: \textit{when a question has been written about twice without being tested, it goes to a spike.} Twice is countable. It fires on the observable --- documents produced --- rather than on a judgement about whether something needs trying, which is exactly the judgement the ceremony corrupts. + +And a \textbf{default toward the cheap move at the moment of maximum uncertainty}. The method's autonomy ladder governs how much the collaborator decides; nothing governs how much rigour the artifact must carry, and the answer should vary. At the point where a question is genuinely open, the standard should drop, not rise. + +Both are in the template. Neither has been tested, and the reader should treat them accordingly --- as the most recent guesses of someone who has now made this mistake twice. diff --git a/report/sections/07-the-template.tex b/report/sections/07-the-template.tex new file mode 100644 index 0000000..e5a2255 --- /dev/null +++ b/report/sections/07-the-template.tex @@ -0,0 +1,53 @@ +\section{The template} + +The method described here is extracted into a repository template, public and instantiable. This section says what is in it, what deliberately is not, and what it does not yet know. + +\subsection{What it contains} + +\textbf{A constitution.} The rules the file-holding surface operates under: what it may do without asking, what requires permission, what it may never do. About a third of the original was domain-specific --- coordinate frames, units, conventions particular to one problem --- and that third is now an empty section marked \textit{domain invariants}, with the filled original visible beside it. The blank is the template; the example is what makes the blank legible. + +\textbf{A workflow.} The four-station loop, the three verdicts, the rule that every specification carries a falsifier and names the repository state it assumes. Plus the two additions \S4 argued for: the devil's advocate pass with its triggers, and the note that the reviewing surface's clone is a separate checkout it reads and runs and never commits to. + +\textbf{Two ledgers, as schema and prose headers with no entries.} Most of their value is in the headers --- the rule that one is written by the surface holding the files and read by the surface that does not; the distinction between what is derivable and what is intent; the procedure for reopening a closed decision, which is to annotate and never reverse. + +\textbf{An experiment exemplar.} Five empty files in the order they are written, pre-registration first, with a short note on why the order is load-bearing. + +\textbf{The low-ceremony lane}, with its rules, its enforcement test, and --- per \S6 --- its own record of non-use. + +\textbf{A report scaffold}, including the claims-verification practice: prose drafted in one format, typeset in another, both committed so the translation is diffable, and a pass that checks every load-bearing claim at source before anything is set. + +\textbf{A list of specification defects}, which is the artifact \S6 says did not exist, seeded with the two that recurred. + +\subsection{What it does not contain} + +No library code, no experiments, no ledger entries, no domain content. The rules travel; the content does not. + +Also absent, deliberately: several practices from the original project judged specific to it rather than general. A migration audit of two particular predecessors --- the practice survives inside \textit{carry only what can fail}, the artifact does not. A provenance file for fixture data, which without any data beside it is ceremony. A structural test resting on a decision the template's user has not taken. + +Each omission is recorded with its reason, which is the same discipline the template applies to everything else. + +\subsection{The rule the template is built on} + +One rule governed the extraction and it is worth stating on its own: + +\begin{quote} +\textbf{Every rule that ships carries the observed failure that motivated it, in one sentence.} +\end{quote} + +A rule without its reason gets simplified away by the next person to read it. Someone encountering \textit{mark carried methods at first use} with no explanation will reasonably conclude it is bureaucratic and drop it. Someone encountering it beside \textit{a carried criterion governed ten experiments before anyone asked what it tested} will not. + +This has a consequence that took a moment to accept. \textbf{The template is mostly a record of failures}, and it reads that way. That is correct. A methodology presented as a set of good practices is a set of assertions; presented as a set of failures with their remedies attached, it is evidence. + +\subsection{What it does not know} + +The template was audited against one question: can a session starting cold, with no other context, read it and write a first specification without asking what any of it means? That audit found several gaps, and four were closed in a single change. Its residue is argued below rather than recorded in the template, because recording it would mean inventing rules the original project never exercised. + +The largest: one of the method's own concepts --- the distinction between decisions that change what the project \textit{is} and decisions about how to build it --- is defined in the template by an example from a layered software architecture. For a project without that structure, the distinction has no test. It is a real gap and the honest fix is for a second project to supply the general case by having its own. + +\subsection{And the thing to hold on to} + +The template's value is \textbf{unproven}, and its front page says so. One researcher, one field, one project, five days. It does not claim to be a validated methodology. What it claims is narrower and checkable: these rules were exercised, several of them failed in specific recorded ways, and the fixes are marked as new and untested. + +There is one exception, and it is the only place in the template where a rule is caught working rather than argued for. The check for figures repeated across a document fired on the template's own front page, where a count had been taken from the highest identifier rather than from the set --- and the figure had been repeated in a second file, so correcting the first would have left the second standing. The tree-wide search found it; a re-reading would not have. + +That is a small error about an unimportant number. It is also, exactly, the failure this whole method exists to prevent, occurring in the provenance paragraph of the document that ships the remedy. It is recorded there, beside the rule, for that reason. diff --git a/report/sections/08-reflection.tex b/report/sections/08-reflection.tex new file mode 100644 index 0000000..12bd83c --- /dev/null +++ b/report/sections/08-reflection.tex @@ -0,0 +1,89 @@ +\section{Reflection} + +\subsection{The bottleneck moved} + +Thirteen experiments in five working days. At no point was implementation the limiting resource. + +What limited the work was \textbf{trust} --- and specifically, one human reading. Every pull request, every ledger entry, every claimed number. The apparatus in \S4 exists because implementation stopped being scarce and verification did not, and a practice built for the old scarcity will optimise the wrong thing. + +That inversion has consequences beyond a single project. Research training is organised around production: how to design an experiment, implement it, analyse it, write it up. The implicit model is that thinking is fast and building is slow, so rigour can be afforded because you are waiting for the build anyway. Reverse it --- building becomes fast, checking stays slow --- and rigour stops being something you have time for and becomes the thing you spend your time on. + +This is not a lament. It is arguably a better allocation. But it is not the one anyone was trained for, and a researcher who keeps optimising production will produce more than they can vouch for, faster, without ever noticing the moment it happened. + +\subsection{Confident and wrong has no analogue in the tools we inherited} + +Version control, testing, code review, continuous integration: every one of them was built for artifacts that \textbf{break loudly}. A test fails. A type does not check. A merge conflicts. The entire apparatus of software engineering is a set of amplifiers for a signal that already exists. + +A wrong belief emits no signal. It is internally consistent by construction --- that is what makes it believable --- and it passes every check because it is not a defect in an artifact but a defect in what someone concluded from one. The green badge and the wrong claim are perfectly compatible, and in this project they coexisted for ten experiments. + +We have almost no tools for this. Pre-registration is one, borrowed from clinical trials and psychology, and it is telling that the borrowing was necessary --- computational research had not needed it while the cost of running an experiment was itself a check on how many you could run and how loosely you could interpret them. + +The practices in \S4 are, I think, the beginning of an answer rather than an answer. Non-ratification, provenance on every carried figure, a falsifier for the specification and not just for the hypothesis: these are attempts to make a wrong belief \textit{break loudly}, by manufacturing the signal that does not naturally exist. Whether they are the right attempts is exactly what a second project would find out. + +\subsection{The unfashionable deliverable} + +The most reusable thing this project produced is not the code, and not its one positive result. It is a ledger of thirteen foreclosed possibilities, each with the evidence that closed it, the conditions under which it holds, and the cost of reopening it. + +Someone building a system of this kind now knows several things not to try, and why. That is a different sort of result from a benchmark number, and I would argue a more durable one: a benchmark is superseded by the next method, while a well-founded elimination stays eliminated until someone changes the assumptions it rested on --- which the ledger states, so they can check. + +It is also almost unpublishable. There is no venue for \textit{here are thirteen things that do not work and the reasons}, no citation economy that rewards it, and no obvious way to present it at a conference. That is a fact about publishing rather than about the knowledge, and I want to state it plainly rather than pretend the incentives are aligned: \textbf{the better this method works, the more of its output takes a form the field does not reward.} + +Which raises a question I cannot answer from one project. If AI collaboration makes elimination cheap --- and it does; most of these thirteen cost an afternoon each --- then a great deal of the total knowledge produced will be of a kind we currently have no way to share. Either the practice changes to accommodate it, or the knowledge stays in repositories that nobody reads. + +\subsection{That this report exists at all} + +These notes were produced under the method they describe, by the collaboration they analyse, including their own claims-verification pass. The recursion is not decoration. It is either the strongest available evidence for the method or a closed loop, and I owe the reader a position on which. + +The honest answer is: \textbf{both, in different parts, and the boundary is checkable.} + +The parts that are evidence are the ones where the method's mechanisms fired against this document. The first claims pass ran on the technical report and moved eleven of its eighteen substantive claims. The repeated-figure check fired on the template's own front page. A specification about this work was rejected on a false premise about which surface held which file. Those are not testimonials; they are recorded events with commits attached, and a reader can go and look. + +The part that is a closed loop is the argument. \S6's finding --- that a rigorous project will not use its own informal lane --- is my reading of my own behaviour, produced in conversation with the collaborator whose behaviour is also under examination. No mechanism checked it. It survived because it explained something that had happened twice and because I recognised it, which is exactly the standard \S4.1 says is insufficient. + +So: trust the events, weigh the arguments. That is the same instruction the method gives about any of its own claims, and applying it here is not modesty but consistency. + +\subsection{What I would want checked} + +Three things a second project would settle that this one could not. + +\textbf{Whether the failures are the method's or mine.} One researcher. Every failure in \S6 is consistent with \textit{this is what happens to a rigorous project} and equally consistent with \textit{this is what happens when this particular person is given a rigorous framework}. I lean toward the first because informal exploration went undone in two consecutive projects --- in the first because there was no lane, in the second although there was --- but two is not many. + +\textbf{Whether the mechanisms transfer or were shaped by the domain.} Pre-registration with a falsifier works cleanly when an experiment produces a number that can be compared against a threshold. Whether it works for research whose outputs are qualitative, or exploratory, or where the interesting result is a construction rather than a measurement, I do not know. I would guess the falsifier survives and the threshold does not. + +\textbf{Whether the record is readable by anyone but its author.} Nine thousand lines of ledger, written by one collaborator, read by one human, in five working days. A colleague opening it cold is the test that has not been run, and until it is, \textit{the repository is the memory} is a claim about a memory of one. + +\subsection{How did it feel} + +As stated at the beginning, this is a document written in collaboration between a researcher and an AI entity. The curious reader may be challenged to imagine how this collaboration actually took place---and perhaps even to identify who wrote which parts. + +This subsection, starting with its very title, is the only one for which our readership can be certain that it was written by a human---because a machine does not have feelings, right? + +So, let us rephrase the question that cries out for an answer: \textbf{``How did it feel to partner with an AI?''} (Which, by the way, I prefer to call an \textit{entity}, not an \textit{agent}.) + +Unfortunately, it is indescribable. Like jazz, you have to experience it to know. It is certainly different, and it feels new. It is here to stay, but each one of us, as humans, must find the answer in our own inner way. + +On the other hand, for endeavors such as the one described in this document, we can say that the collaboration is a kind of \textit{contract} whose terms the two parties must learn to negotiate in order for things to work out. It is also a moving target, because everything is changing at a rapid pace and requires us to learn as we go. + +But since, as a researcher, I am naturally drawn to discovery as well as to the unknown, I can assure you that \textbf{``It Feels Good!''} + +\subsection{What I would tell a student starting now} + +For a scientist who is also a mentor and educator, this is perhaps the question of a lifetime. It has two inseparable sides: one seen from the perspective of the teacher, and the other from that of the student. Together, they reveal an intrinsic interdependence of purpose, learning, and aspiration. + +The present moment is changing not only how we learn, but also what is worth learning. The emphasis is gradually moving away from mechanical procedures and the memorization of rules---the kinds of tasks that machines can now perform remarkably well and with increasing ease. What becomes more important, then, are basic principles, fundamentals, genuine understanding, intuition, judgment, and creativity. And, of course, talent helps. + +In that spirit, the advice I would give is simple: ``Discover the things you love and the things you are good at. Find where they meet, and pursue that path.'' + +Perhaps education, in this new context, is less about preparing someone to follow a predefined path and more about helping them acquire the confidence and understanding needed to discover one of their own. + +\subsection{Is this a good way to spend a career?} + +To answer this question, we may first need to reconsider what we mean by a \textit{career}. In this new context, perhaps the traditional idea of a career---a more or less predefined trajectory through a profession---will no longer be the most useful one. + +A career may increasingly become something that is continuously invented: a sequence of questions, discoveries, collaborations, changes of direction, and new beginnings. + +So, to close briefly, I would return to the advice of the previous subsection: discover what you care about, discover what you can contribute, and invent your own career around the intersection of the two. + +If you can do that, perhaps the question of whether it is a good way to spend a career becomes less important. + +It may simply become a good way to spend a lifetime. diff --git a/report/sections/09-appendix-availability.tex b/report/sections/09-appendix-availability.tex new file mode 100644 index 0000000..baf0e48 --- /dev/null +++ b/report/sections/09-appendix-availability.tex @@ -0,0 +1,12 @@ +\section{Availability} + +Four public repositories. The first two are the predecessors these notes refer to; the third is the project the examples come from; the fourth is the template \S7 describes. + +\begin{itemize} +\item \textbf{\texttt{visgraf/bioeye}} --- a thin vertical slice through the framework, written first. It closed the accumulation loop and not the perception--action loop. +\item \textbf{\texttt{visgraf/active-stereo}} --- a full implementation with verified geometry and a decision record. The loop never ran. Both the failure that motivated the low-ceremony lane and the constitution clause that talked the reviewer out of verifying are in this repository's history. +\item \textbf{\texttt{visgraf/bio-3d-vision}} --- the project. Thirteen experiments, each with its pre-registration, runner, raw results, verdicts and findings; two ledgers; a technical report with its claims-verification pass; and a forward-only history in which every correction appears as an amendment. +\item \textbf{\texttt{visgraf/math-ai-method}} --- the template, instantiable. +\end{itemize} + +Every claim these notes make about what happened can be checked in the third. That is the point of having kept it. From 1d2f4929f723876dde4f9ee2661d57a4ed51985b Mon Sep 17 00:00:00 2001 From: Luiz Velho Date: Tue, 8 Sep 2026 20:07:08 -0300 Subject: [PATCH 04/11] Close od-004: parse the ledgers in CI, with no new dependency PART D of four. A `ledgers` job in report.yml loads docs/state.yaml and docs/inherited-measurements.yaml and fails if either does not parse or does not come back a mapping. Two files, one command, no install step. RUBY, NOT PYTHON, AND THAT IS THE WHOLE REASON THIS WAS OPEN. YAML is not in Python's standard library and PyYAML is not guaranteed on the runner, so the obvious check would have needed a `pip install` -- the dependency question od-004 was waiting on an answer to. Ruby ships Psych in its standard library and is preinstalled on ubuntu-latest. The smallest dependency that works is none. `YAML.load` rather than `safe_load_file`, which is Ruby 3.x only. The ledgers contain no anchors or aliases -- checked -- so the portable form is both safe and version-independent. A SEPARATE JOB, not a step in `build`: a one-character YAML error should not be reported from behind a 44-second TeX install, and the record should still be checked when the document fails to compile. IT WAS WATCHED FAILING BEFORE IT WAS COMMITTED. An unclosed flow sequence appended to each ledger in turn produced exit 1 naming the file, line and column. The command was EXTRACTED FROM THE WORKFLOW FILE and run, not hand-typed, so what was tested is what CI will run -- the quoting inside the YAML block is part of what could break. A check nobody has seen fail is a check nobody knows can fail. Scope recorded on the entry: it tests that the files parse. It does not test that ids are unique, that a superseded entry names its successor, or that a cited id resolves. Co-Authored-By: Claude Opus 5 (1M context) --- .github/workflows/report.yml | 51 ++++++++++++++++++++++++++++++++---- docs/state.yaml | 29 +++++++++++++++++++- 2 files changed, 74 insertions(+), 6 deletions(-) diff --git a/.github/workflows/report.yml b/.github/workflows/report.yml index ec4df59..466a8c1 100644 --- a/.github/workflows/report.yml +++ b/.github/workflows/report.yml @@ -5,11 +5,10 @@ # could not even reach its first step, and a workflow that is red for reasons # unrelated to the report is one people learn to override. # -# WHAT THAT COSTS, said here rather than left to be discovered: this is now the -# whole automated surface. Nothing checks that docs/state.yaml and -# docs/inherited-measurements.yaml are parseable YAML, and they are the record. -# See od-004, which is open and blocked only on the question of adding a -# dependency. +# TWO JOBS NOW: `ledgers` parses the record, `build` compiles the document. +# od-004 closed -- the gap that used to be recorded here, that nothing checked +# whether docs/state.yaml and docs/inherited-measurements.yaml still parse, is +# closed by the first job below. # # Triggers do not fire on literally every push, so a branch with an open pull # request is not built twice for the same commit. Every change in this repository @@ -32,6 +31,48 @@ concurrency: cancel-in-progress: true jobs: + # The record, checked first because it is cheap and because a ledger that no + # longer parses is a worse failure than a document that no longer builds. + # + # A SEPARATE JOB, NOT A STEP IN `build`: it must not wait behind a 44-second + # TeX install to report a one-character YAML error, and it must still run when + # the document build fails. + # + # NO DEPENDENCY, WHICH IS WHY IT IS THIS AND NOT PYTHON. YAML is not in + # Python's standard library and PyYAML is not guaranteed on the runner, so the + # obvious version of this check would have to `pip install` something -- and + # CLAUDE.md's working agreement says never add a dependency without asking. + # Ruby ships Psych in its standard library and is preinstalled on + # ubuntu-latest, so this costs one `ruby -ryaml` and no install step at all. + # `YAML.load` rather than `safe_load_file`, which is Ruby 3.x only: these are + # our own files, they contain no anchors or aliases, and the portable form + # works on every Psych version. + # + # IT WAS TESTED BY BREAKING A LEDGER ON PURPOSE before it was committed -- + # appending an unclosed flow sequence to docs/state.yaml made it exit 1 and + # name the file, line and column. A check nobody has watched fail is a check + # nobody knows can. + ledgers: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + + - name: Ledgers parse + run: | + ruby -ryaml -e ' + ok = true + ["docs/state.yaml", "docs/inherited-measurements.yaml"].each do |f| + begin + d = YAML.load(File.read(f)) + raise "top level is #{d.class}, expected a mapping" unless d.is_a?(Hash) + puts "ok #{f} (#{d.keys.size} top-level keys)" + rescue => e + warn "FAIL #{f}: #{e.message.split("\n").first}" + ok = false + end + end + exit(ok ? 0 : 1)' + build: runs-on: ubuntu-latest diff --git a/docs/state.yaml b/docs/state.yaml index 84c786d..a5481d5 100644 --- a/docs/state.yaml +++ b/docs/state.yaml @@ -547,7 +547,34 @@ open_decisions: - id: od-004 altitude: infra - status: open + status: RESOLVED + resolved_at: 2026-09-08 + resolution: >- + CLOSED WITHOUT ADDING A DEPENDENCY, which was the only thing blocking it. + A `ledgers` job in .github/workflows/report.yml parses both files with + `ruby -ryaml` and fails the build if either does not load or does not come + back a mapping. + WHY RUBY AND NOT PYTHON, since the obvious choice was Python: YAML is not + in Python's standard library, PyYAML is not guaranteed on the runner, and + the check would therefore have needed a `pip install` — the ask this entry + was waiting on. Ruby ships Psych in its standard library and is + preinstalled on ubuntu-latest, so the check costs one command and no + install step. `YAML.load` rather than `safe_load_file`, which is Ruby 3.x + only; the ledgers contain no anchors or aliases, so the portable form is + both safe and version-independent. + A SEPARATE JOB RATHER THAN A STEP IN `build`, so a one-character YAML error + is not reported from behind a 44-second TeX install, and so it still runs + when the document fails to compile. + IT WAS WATCHED FAILING BEFORE IT WAS COMMITTED. An unclosed flow sequence + appended to each ledger in turn made it exit 1 and name the file, line and + column; the command was extracted from the workflow file itself and run, + rather than a hand-typed approximation of it. A check nobody has seen fail + is a check nobody knows can fail, which is CLAUDE.md's own standard and + §4.3's subject. + WHAT IT DOES NOT CHECK, so nobody reads it as more coverage than it is: it + tests that the files PARSE, not that any id is unique, that a `superseded` + entry names its successor, or that a cited id resolves. Those are real and + would each need a rule written down first. title: Nothing verifies that the two ledgers are parseable YAML. the_question: >- docs/state.yaml and docs/inherited-measurements.yaml are the record, they From a37f276aa1b2c6ff981e2208908223295bf6b85b Mon Sep 17 00:00:00 2001 From: Luiz Velho Date: Tue, 8 Sep 2026 20:10:39 -0300 Subject: [PATCH 05/11] Fix the ledger check: YAML.parse, not YAML.load The check committed in the previous commit FAILED ON ITS FIRST CI RUN, with "Tried to load unspecified class: Date". It had passed locally. CAUSE. `YAML.load` is unsafe-by-default in Psych 3 (Ruby 2.6, this machine) and SAFE-by-default in Psych 4 (Ruby 3.x, ubuntu-latest), where it refuses to instantiate Date. Both ledgers carry unquoted dates like 2026-09-08, which YAML resolves to a Date. FIX. Not permitting the class -- `YAML.parse`, which returns the node tree and constructs nothing. It behaves identically on every Psych version, and it is a more exact test of what od-004 asks: not "does this load into Ruby objects" but "is this parseable". The mapping-root assertion moves to Psych::Nodes::Mapping. Re-verified on three axes, not one: good ledgers pass; a broken one exits 1 naming file and line; a non-mapping root is rejected; an unquoted date parses without loading a class. WHY THIS IS RECORDED ON od-004 RATHER THAN QUIETLY FIXED. The previous commit claimed "it was watched failing before it was committed", and that claim was true and insufficient -- it was watched failing on the one axis its author thought of, and never run anywhere but the author's machine. That is the exact shape .gitignore already warns about in this repository: something that passes locally and fails everywhere else because the author's environment is not the runner's. "I watched it fail" is not the same claim as "I watched it fail where it runs." Co-Authored-By: Claude Opus 5 (1M context) --- .github/workflows/report.yml | 29 +++++++++++++++++++---------- docs/state.yaml | 29 +++++++++++++++++++++++------ 2 files changed, 42 insertions(+), 16 deletions(-) diff --git a/.github/workflows/report.yml b/.github/workflows/report.yml index 466a8c1..bfd9105 100644 --- a/.github/workflows/report.yml +++ b/.github/workflows/report.yml @@ -44,14 +44,22 @@ jobs: # CLAUDE.md's working agreement says never add a dependency without asking. # Ruby ships Psych in its standard library and is preinstalled on # ubuntu-latest, so this costs one `ruby -ryaml` and no install step at all. - # `YAML.load` rather than `safe_load_file`, which is Ruby 3.x only: these are - # our own files, they contain no anchors or aliases, and the portable form - # works on every Psych version. + # `YAML.parse`, NOT `YAML.load`, AND THE REASON IS A FAILURE THIS CHECK ALREADY + # HAD. The first version used YAML.load. It passed locally on Ruby 2.6 (Psych 3, + # where load is unsafe-by-default) and FAILED on ubuntu-latest with + # "Tried to load unspecified class: Date" -- Psych 4 made YAML.load + # safe-by-default, and the ledgers carry unquoted dates like `2026-09-08`, + # which YAML resolves to a Date. The fix is not to permit the class; it is to + # stop instantiating classes at all. YAML.parse returns the node tree and + # constructs nothing, which is exactly what "is this file parseable" asks, and + # it behaves the same on every Psych version. # - # IT WAS TESTED BY BREAKING A LEDGER ON PURPOSE before it was committed -- - # appending an unclosed flow sequence to docs/state.yaml made it exit 1 and - # name the file, line and column. A check nobody has watched fail is a check - # nobody knows can. + # THE LESSON IS THE ONE THIS REPOSITORY IS ABOUT, so it is recorded rather than + # quietly fixed: the check WAS watched failing before it was committed -- a + # broken ledger made it exit 1 -- but only on the axis its author thought of. + # It was never run anywhere but the author's machine, so the environment + # difference was invisible until CI. A check tested on one axis is a check + # tested on one axis. ledgers: runs-on: ubuntu-latest steps: @@ -63,9 +71,10 @@ jobs: ok = true ["docs/state.yaml", "docs/inherited-measurements.yaml"].each do |f| begin - d = YAML.load(File.read(f)) - raise "top level is #{d.class}, expected a mapping" unless d.is_a?(Hash) - puts "ok #{f} (#{d.keys.size} top-level keys)" + doc = YAML.parse(File.read(f)) + raise "empty document" if doc.nil? + raise "root is #{doc.root.class}, expected a mapping" unless doc.root.is_a?(Psych::Nodes::Mapping) + puts "ok #{f} (#{doc.root.children.length / 2} top-level keys)" rescue => e warn "FAIL #{f}: #{e.message.split("\n").first}" ok = false diff --git a/docs/state.yaml b/docs/state.yaml index a5481d5..d00b99c 100644 --- a/docs/state.yaml +++ b/docs/state.yaml @@ -565,12 +565,29 @@ open_decisions: A SEPARATE JOB RATHER THAN A STEP IN `build`, so a one-character YAML error is not reported from behind a 44-second TeX install, and so it still runs when the document fails to compile. - IT WAS WATCHED FAILING BEFORE IT WAS COMMITTED. An unclosed flow sequence - appended to each ledger in turn made it exit 1 and name the file, line and - column; the command was extracted from the workflow file itself and run, - rather than a hand-typed approximation of it. A check nobody has seen fail - is a check nobody knows can fail, which is CLAUDE.md's own standard and - §4.3's subject. + IT WAS WATCHED FAILING BEFORE IT WAS COMMITTED — ON ONE AXIS, WHICH WAS + NOT ENOUGH, AND THE FIRST VERSION SHIPPED BROKEN. An unclosed flow + sequence appended to each ledger made it exit 1 and name the file, line + and column, and the command was extracted from the workflow file rather + than hand-typed. It still failed on the first CI run: + `Tried to load unspecified class: Date`. + THE CAUSE: `YAML.load` is unsafe-by-default in Psych 3 (Ruby 2.6, the + author's machine) and SAFE-by-default in Psych 4 (Ruby 3.x, ubuntu-latest), + where it refuses to instantiate Date — and these ledgers carry unquoted + dates like 2026-09-08, which YAML resolves to a Date. + THE FIX IS NOT TO PERMIT THE CLASS. It is `YAML.parse`, which returns the + node tree and constructs nothing, behaves identically on every Psych + version, and is a more exact test of what this entry actually asks: not + "does this file load into Ruby objects" but "is this file parseable". + Re-verified on three axes — good ledgers pass, a broken one exits 1 naming + file and line, a non-mapping root is rejected, and an unquoted date parses. + WHY THE FAILURE IS RECORDED RATHER THAN QUIETLY FIXED: it is this + repository's own subject, and it is the shape .gitignore already warns + about — something that passes locally and fails everywhere else because + the author's environment is not the runner's. The check was tested on the + axis its author thought of. A check tested on one axis is a check tested + on one axis, and "I watched it fail" is not the same claim as "I watched + it fail where it runs". WHAT IT DOES NOT CHECK, so nobody reads it as more coverage than it is: it tests that the files PARSE, not that any id is unique, that a `superseded` entry names its successor, or that a cited id resolves. Those are real and From fa309cd4c0a2907466d99e5c0989928868ed4a7b Mon Sep 17 00:00:00 2001 From: Luiz Velho Date: Tue, 8 Sep 2026 20:17:40 -0300 Subject: [PATCH 06/11] =?UTF-8?q?PART=20B:=20=C2=A76,=20"The=20method=20de?= =?UTF-8?q?stroys=20its=20own=20best=20evidence"?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Inserted verbatim between "The devil's advocate was agreed and never written down" and "And now the one that matters". Not edited. ITS FIVE CHECKABLE CLAIMS WERE VERIFIED BEFORE INSERTION, all confirmed, and recorded as A1-A5 in report/claims-verified.md. New prose making factual claims is what that document exists to catch, and it does not get an exemption for being new. THE FLAGGED STOP CONDITION DID NOT FIRE. The specification warned that the section asserts no specification text survives anywhere in bio-3d-vision at 5f8e39f, and said to stop if that failed. IT WOULD HAVE FAILED -- three fragments survive, quoted inside artifacts that were being committed anyway. The supplied text does not say that. It says the specification "is the only artifact in the loop that is never committed", which is the true and weaker claim, and which re-checks clean: zero tracked files are specifications and no filename matches spec, task, brief, instruction, relay or prompt. The distinction is load-bearing and the section gets it right. Recorded as A3 so nobody re-derives the stronger version later. A5 -- "five of the six unsupported claims" -- checks out under the selector the sentence states, and DISAGREES WITH gap-004's "four", which used a stricter one. Both are right. Recorded as two counts with their selectors rather than resolved by picking one, which is exactly what mn-001 and mn-002 required and for the same reason. The odd one out under either selector is claim 38: not an event but a proportion of a document that still exists, so it is unsupported and MEASURABLE -- the only one of the six a successor could close by going to look. Typeset with the rest: sections regenerated, transit re-verified lossless (12,184 words in, 12,184 out, zero differing blocks), 22 pages, zero overfull or underfull boxes. The section lands as §6.3 and does not disturb §6's opening claim that "the last one is the serious one" -- "And now the one that matters" is still last. Co-Authored-By: Claude Opus 5 (1M context) --- docs/inherited-measurements.yaml | 18 +++++++- report/claims-verified.md | 60 ++++++++++++++++++++++++- report/draft.md | 62 ++++++++++++++++++++++++++ report/main.tex | 4 ++ report/sections/06-where-it-failed.tex | 24 ++++++++++ 5 files changed, 165 insertions(+), 3 deletions(-) diff --git a/docs/inherited-measurements.yaml b/docs/inherited-measurements.yaml index c7bb484..90d71aa 100644 --- a/docs/inherited-measurements.yaml +++ b/docs/inherited-measurements.yaml @@ -621,8 +621,22 @@ gaps: - id: gap-004 statement: >- The method has no channel for a specification that was correctly rejected - before work was spent. Six claims in the pass are UNSUPPORTED and four of - them are unrecordable BY CONSTRUCTION, not merely unrecorded. + before work was spent. Six claims in the pass are UNSUPPORTED and most of + them are unrecordable rather than merely unrecorded. + TWO COUNTS ARE IN CIRCULATION AND BOTH ARE RIGHT, which is the mn-001 / + mn-002 situation again and is recorded the same way rather than resolved + by picking one. FOUR are unrecordable BY CONSTRUCTION — the workflow + guarantees no artifact: claims 17 and 20 live in specifications, which are + never committed; 12 is a rate over conversational turns; 25 is a cleared + session's search. FIVE "describe an event the workflow guarantees will + leave no trace", which is the selector §6's subsection uses, and it admits + claim 45 as well: the afternoons a foreclosure cost were events, and + nothing in the method logs duration. + THE ODD ONE OUT UNDER EITHER SELECTOR IS CLAIM 38 — "about a third of the + original was domain-specific". It is not an event at all; it is a + proportion of a document that still exists, so it is unsupported but + MEASURABLE, and it is the only one of the six a successor could close by + going and looking. why_it_is_load_bearing: >- It bounds §4.2 and §5 of the draft, which rest on "three specifications were rejected on false assumptions" (claim 17) and call the rejections diff --git a/report/claims-verified.md b/report/claims-verified.md index f39f930..ad3031e 100644 --- a/report/claims-verified.md +++ b/report/claims-verified.md @@ -241,9 +241,22 @@ pattern is a property of first-order reports rather than of prose in general. **Six claims are UNSUPPORTED — not contradicted, unestablishable.** The specification predicted what this would mean: §4.5's failure occurring in the document that describes it. It is that, and it is sharper than that, because -**the unsupported claims are not scattered. Every one of them is about an event +**the unsupported claims are not scattered. Five of the six are about an event the workflow guarantees will leave no artifact.** +> **Two counts, both right, recorded rather than reconciled away.** *Four* are +> unrecordable **by construction** — the workflow guarantees no artifact +> (claims 17 and 20 live in specifications, which are never committed; 12 is a +> rate over conversational turns; 25 is a cleared session's search). *Five* +> "describe an event the workflow guarantees will leave no trace", the selector +> §6's *The method destroys its own best evidence* uses, which also admits claim +> 45: the afternoons a foreclosure cost were events, and nothing logs duration. +> **Claim 38 is the odd one out under either selector** — not an event at all, +> but a proportion of a document that still exists, so it is unsupported and +> *measurable*, and the only one of the six a successor could close by looking. +> This is `mn-001`/`mn-002` again: two numbers, two selectors, neither wrong. +> See `gap-004`. + | claim | what it asserts | why no record exists | |---|---|---| | 12 | false claims "at a rate of roughly one per day" | a rate over conversations; nothing counts them | @@ -549,6 +562,51 @@ does not yet reach, the rejections. Recorded as `od-006`. --- +# Addendum — §6, *The method destroys its own best evidence* + +**Added after the first pass and checked before it was inserted**, because new +prose making factual claims is what this document exists to catch. Verified at +the same pinned SHAs. Five checkable claims, all CONFIRMED — so the tally in +`mn-004` is unchanged and these are numbered separately as A1–A5. + +### A1. "§4.2 describes three specifications rejected on false assumptions about repository state… and calls this *the clearest case in the project of a rule paying for itself*" +**CONFIRMED.** `report/draft.md` §4.2: "Three specifications were rejected on +false assumptions, each correctly, each before any work was spent… it is the +clearest case in the project of a rule paying for itself." Quoted accurately. + +### A2. "That claim came back from the verification pass **unsupported**" +**CONFIRMED.** Claim 17 of this document, UNSUPPORTED. + +### A3. "It is the only artifact in the loop that is never committed." +**CONFIRMED, and this is the claim the specification flagged as the stop +condition.** Re-checked at `instance` @`5f8e39f`: no committed file is a +specification, and no tracked filename matches *spec*, *task*, *brief*, +*instruction*, *relay* or *prompt* — zero hits. +- **Note what this sentence does NOT say.** An earlier draft of this section was + reported to assert that *no specification text survives anywhere*, which is + **false**: three fragments survive, quoted inside artifacts that were being + committed anyway (`exp007/preregistration.md:38`, `exp009/preregistration.md:19`, + `exp007/findings.md:46`). The supplied text says "never committed", which is + the true and weaker claim. **The distinction is load-bearing and the section + gets it right.** + +### A4. "One rejection survives in the record, and only incidentally: it reshaped a decision that was taken anyway… That fragment is not even a stale-state rejection" +**CONFIRMED on all three parts.** `instance: docs/state.yaml` — the `fc-012` +entry reads "THAT VERSION WAS REJECTED BECAUSE IT MAKES AZIMUTH AND VERGENCE +INERT, and the rejection is the reason this entry exists in the form it does." +`fc-012` was taken (rectifying in the camera), so the decision stands. And the +rejection was on a geometry defect, not on a false assumption about repository +state — so it is not of §4.2's class. + +### A5. "Five of the six unsupported claims in this document have that shape." +**CONFIRMED under the selector the sentence states** — "describes an event the +workflow guarantees will leave no trace" admits 12, 17, 20, 25 and 45, and +excludes 38, which is a proportion of a document rather than an event. See the +two-count note under falsifier 3: a stricter selector gives four, and both are +recorded rather than reconciled away. + +--- + # Resolutions ## What resolved diff --git a/report/draft.md b/report/draft.md index 1c2969b..e1eafb0 100644 --- a/report/draft.md +++ b/report/draft.md @@ -26,6 +26,10 @@ > `gap-004` and `od-006`. Read §4.2's rejection count and §4.5's cold-session > search as recollection, not record. > +> §6's *The method destroys its own best evidence* was written after that pass +> and its five checkable claims were verified before insertion — all confirmed, +> recorded as A1–A5. +> > Sections 8.6, 8.7 and 8.8 are the first author's, written in the first person. > They make no checkable factual claims and are out of scope for that pass. @@ -945,6 +949,64 @@ the failure of a method going unquestioned was itself never subjected to the method. If the sessions had been cleared, it would have vanished without trace, and nobody would have known to miss it. +## The method destroys its own best evidence + +The three failures above are ordinary: rules that arrived after the thing they +were built for. This one is different in kind, because it cannot be fixed +retroactively at all. + +§4.2 describes three specifications rejected on false assumptions about +repository state, each caught before any work was spent, and calls this *the +clearest case in the project of a rule paying for itself.* That claim came back +from the verification pass **unsupported** — not contradicted, but not +establishable from any record. + +The reason is structural. A specification is written on one surface, relayed by +a human, and executed on another. **It is the only artifact in the loop that is +never committed.** The work it produces lands in a branch; the specification +itself lives in a chat window and a paste buffer. So a specification that was +*executed* leaves a commit behind, and a specification that was *rejected* +leaves nothing at all — no branch, no diff, no entry, because the whole point of +the rejection is that no work was done. + +Five of the six unsupported claims in this document have that shape. Each +describes an event the workflow guarantees will leave no trace. One rejection +survives in the record, and only incidentally: it reshaped a decision that was +taken anyway, so the decision's entry carries a fragment of it. That fragment is +not even a stale-state rejection, which is the class §4.2's claim was about. + +The finding is worse than a gap in the record. + +> **The method logs what work produced and has no channel for work it correctly +> prevented.** Its most valuable events are the ones where a rule stopped +> something before it existed — and those are precisely the events that leave +> nothing behind. + +This inverts a claim made twice elsewhere in these notes. §5 argues that the +costs and the catches are the same list read twice, and that a wrong belief +costs everything downstream of it. Both are true. But the catches that produce +an artifact are the ones where something *was* built and then corrected, and the +catches that produce nothing are the ones where the rule worked earliest and +cheapest. **The record is therefore biased toward the expensive saves and blind +to the cheap ones** — which means it systematically understates the value of the +mechanisms that fire first. + +And there is no repairing this project's record. The three rejections happened; +the specifications that were rejected are gone; nothing can be reconstructed +that would not be reconstruction. This is the only failure in these notes with +no remedy available to the project that suffered it. + +For a successor the fix is trivial and should have been obvious: **commit the +specification alongside the branch it produces, and commit a rejected one too, +with the assumption that was false.** A rejected specification is a small file +recording that a rule fired, what it caught, and what it cost — which is exactly +the evidence this section is unable to supply. + +That it was not obvious is worth sitting with. The method's foundational rule is +that every handoff is a versioned artifact and never chat context. The +specification **is the handoff.** It is the one thing the rule was most clearly +about, and it was the one thing the rule was never applied to. + ## And now the one that matters Two components of the method were never used. Not underused. **Zero times.** diff --git a/report/main.tex b/report/main.tex index 33b9e8f..0c863eb 100644 --- a/report/main.tex +++ b/report/main.tex @@ -133,6 +133,10 @@ \texttt{gap-004} and \texttt{od-006}. Read \S4.2's rejection count and \S4.5's cold-session search as recollection, not record. +\S6's \textit{The method destroys its own best evidence} was written after that +pass and its five checkable claims were verified before insertion --- all +confirmed, recorded as A1--A5. + Sections 8.6, 8.7 and 8.8 are the first author's, written in the first person. They make no checkable factual claims and are out of scope for that pass. \end{quote} diff --git a/report/sections/06-where-it-failed.tex b/report/sections/06-where-it-failed.tex index e88d00a..c7164cc 100644 --- a/report/sections/06-where-it-failed.tex +++ b/report/sections/06-where-it-failed.tex @@ -22,6 +22,30 @@ \subsection{The devil's advocate was agreed and never written down} By the method's own foundational rule --- every handoff is a versioned artifact, never chat context --- that means it did not exist. A mechanism designed to catch the failure of a method going unquestioned was itself never subjected to the method. If the sessions had been cleared, it would have vanished without trace, and nobody would have known to miss it. +\subsection{The method destroys its own best evidence} + +The three failures above are ordinary: rules that arrived after the thing they were built for. This one is different in kind, because it cannot be fixed retroactively at all. + +\S4.2 describes three specifications rejected on false assumptions about repository state, each caught before any work was spent, and calls this \textit{the clearest case in the project of a rule paying for itself.} That claim came back from the verification pass \textbf{unsupported} --- not contradicted, but not establishable from any record. + +The reason is structural. A specification is written on one surface, relayed by a human, and executed on another. \textbf{It is the only artifact in the loop that is never committed.} The work it produces lands in a branch; the specification itself lives in a chat window and a paste buffer. So a specification that was \textit{executed} leaves a commit behind, and a specification that was \textit{rejected} leaves nothing at all --- no branch, no diff, no entry, because the whole point of the rejection is that no work was done. + +Five of the six unsupported claims in this document have that shape. Each describes an event the workflow guarantees will leave no trace. One rejection survives in the record, and only incidentally: it reshaped a decision that was taken anyway, so the decision's entry carries a fragment of it. That fragment is not even a stale-state rejection, which is the class \S4.2's claim was about. + +The finding is worse than a gap in the record. + +\begin{quote} +\textbf{The method logs what work produced and has no channel for work it correctly prevented.} Its most valuable events are the ones where a rule stopped something before it existed --- and those are precisely the events that leave nothing behind. +\end{quote} + +This inverts a claim made twice elsewhere in these notes. \S5 argues that the costs and the catches are the same list read twice, and that a wrong belief costs everything downstream of it. Both are true. But the catches that produce an artifact are the ones where something \textit{was} built and then corrected, and the catches that produce nothing are the ones where the rule worked earliest and cheapest. \textbf{The record is therefore biased toward the expensive saves and blind to the cheap ones} --- which means it systematically understates the value of the mechanisms that fire first. + +And there is no repairing this project's record. The three rejections happened; the specifications that were rejected are gone; nothing can be reconstructed that would not be reconstruction. This is the only failure in these notes with no remedy available to the project that suffered it. + +For a successor the fix is trivial and should have been obvious: \textbf{commit the specification alongside the branch it produces, and commit a rejected one too, with the assumption that was false.} A rejected specification is a small file recording that a rule fired, what it caught, and what it cost --- which is exactly the evidence this section is unable to supply. + +That it was not obvious is worth sitting with. The method's foundational rule is that every handoff is a versioned artifact and never chat context. The specification \textbf{is the handoff.} It is the one thing the rule was most clearly about, and it was the one thing the rule was never applied to. + \subsection{And now the one that matters} Two components of the method were never used. Not underused. \textbf{Zero times.} From e80c49fcbe39e84ad4069090431466fcc2af01c4 Mon Sep 17 00:00:00 2001 From: Luiz Velho Date: Tue, 8 Sep 2026 23:17:53 -0300 Subject: [PATCH 07/11] PART B, revised: the corrected section, and two defects it surfaced Section replaced in report/draft.md and regenerated into report/sections/06-where-it-failed.tex, so both stay committed and the translation stays diffable. 23 pages, zero overfull or underfull boxes, transit re-verified lossless: 12,292 words in, 12,292 out, zero differing blocks. VERIFIED AT SOURCE: seven checkable claims, A1-A7 in claims-verified.md. Five CONFIRMED, one CORRECTED, one UNDER-QUALIFIED. The two that did not pass: A6 -- "THREE passages in the instance quote specification text" is FOUR, and one of the three named is not one of them. exp007/findings.md:46 records what the specification did NOT state, which is an absence, not a quotation. Meanwhile exp007/findings.md:123 -- "The specification said 18 steps for A, A', D and E", under a heading reading "A deviation from the specification, and why" -- is the clearest instance of the section's own thesis and is not among the three. A7 -- "Every one is text that an artifact was arguing with" is UNDER-QUALIFIED. Three of the four depart from the clause they quote; exp007/findings.md:32 AGREES with it: "The specification asked for this to be visible rather than inferred, and it is." THE CONCLUSION IS UNAFFECTED, WHICH IS WHY THE CORRECTION IS CHEAP. The load-bearing condition is the next sentence -- the survival mechanism is parasitic on the work happening -- and it holds for all four. Agreement or disagreement, the fragment exists because the specification was EXECUTED and produced an artifact. A rejected specification produces no artifact either way. "Arguing with" is too narrow a mechanism for a conclusion that only needs "executed". Both proposed replacements are in the addendum; the prose is unedited, per the standing practice. A6 IS MY ERROR PROPAGATING, and it is named as such. The search behind the previous round returned FOUR hits and my report said THREE; findings.md:123 was in the output and never opened, and the section was written from that summary. docs/spec-defects.md gains entry 6: a verification search written from the expected finding rather than from the claim. It records BOTH instances -- Chat's grep for REPOSITORY STATE ASSUMED, which could only have found a committed specification and never a quoted one, and mine, which was better aimed and still read from the expectation rather than from the output. Two stations, same error, three iterations apart. The check is two clauses: search for what would falsify the claim, and open every hit before reporting a count. AND THE od-003 CHECK ITSELF HAD THE SAME DEFECT. It was case-sensitive: it saw "three passages" and missed "Three passages". Measured over the typeset sources, 252 hits against 301 -- missing 49, or 16%, every sentence-initial figure. Fixed with -i and a tolower in the grouping, in the documented command and on od-003. It had been run three times and reported as passing. Found by hand, not by the check. Co-Authored-By: Claude Opus 5 (1M context) --- docs/inherited-measurements.yaml | 18 ++++ docs/spec-defects.md | 66 ++++++++++++- docs/state.yaml | 20 +++- report/claims-verified.md | 128 +++++++++++++++++-------- report/draft.md | 36 ++++--- report/sections/06-where-it-failed.tex | 8 +- 6 files changed, 219 insertions(+), 57 deletions(-) diff --git a/docs/inherited-measurements.yaml b/docs/inherited-measurements.yaml index 90d71aa..613168a 100644 --- a/docs/inherited-measurements.yaml +++ b/docs/inherited-measurements.yaml @@ -645,6 +645,24 @@ gaps: commit: docs/workflow.md states that a specification is the only artifact of the loop that leaves no trace, and that no specification was ever committed in the instance. + COUNT CORRECTED 2026-09-08 (second correction to this entry). This entry + said THREE fragments survive. It is FOUR passages that report specification + content — exp007/preregistration.md:38, exp009/preregistration.md:19, + exp007/findings.md:123 ("The specification said 18 steps for A, A', D and + E", under a heading reading "A deviation from the specification, and why"), + and exp007/findings.md:32. The third item originally listed here, + exp007/findings.md:46, does NOT qualify: it records what the specification + did NOT state, which is an absence, not a quotation. + HOW THE UNDERCOUNT HAPPENED, since it is the reusable part: the search + returned four hits and the summary said three. findings.md:123 was in the + output and was never opened. Recorded as docs/spec-defects.md §6. + AND THE "ARGUING WITH" CHARACTERISATION IS TOO NARROW. Three of the four + depart from the clause they quote; exp007/findings.md:32 AGREES with it — + "The specification asked for this to be visible rather than inferred, and + it is". The conclusion is unaffected and this is why the correction is + cheap: the load-bearing condition is that the specification was EXECUTED + and produced an artifact, which holds for all four. Agreement or + disagreement, a rejected specification produces no artifact either way. PRECISION ADDED 2026-09-08, correcting this entry's own overstatement. "No specification was ever COMMITTED" is true — no specification file exists in the instance at 5f8e39f, and no filename matches spec, task, diff --git a/docs/spec-defects.md b/docs/spec-defects.md index 497ec60..ef5934d 100644 --- a/docs/spec-defects.md +++ b/docs/spec-defects.md @@ -186,6 +186,63 @@ assembly, and the two checks are both mechanical two-minute reads. --- +## 6. A verification search written from the expected finding, not from the claim + +**The defect.** A claim was to be checked: *no specification text survives +anywhere in the instance.* The search run against it grepped for +`REPOSITORY STATE ASSUMED` and `reject and report if` — **phrases a committed +specification would contain.** Finding none, it reported the claim confirmed. + +But those phrases could only ever have found a *whole specification sitting in a +file*. The claim was about **text**, and the way specification text actually +reaches a record is as a **quotation inside some other artifact** — a +pre-registration repeating a clause in order to depart from it, a findings file +naming the instruction it deviated from. A search for a committed specification +is structurally incapable of finding a quoted one. It was not a weak search; it +was a search for the wrong object, and it returned exactly the answer it had +been built to return. + +**What it cost.** The claim shipped in a draft section, was caught on a +re-check, and the section had to be rewritten around the weaker and true claim — +*no specification was ever committed.* One round trip, and the corrected version +is better than the original, because where the fragments survive turned out to +be the interesting part. + +**It then happened a second time, in the correction.** The re-check searched for +`specification (said|says|reads|stated)` and returned **four** hits. The report +said **three** — one hit was in the output and was never opened, and it was the +best example of the very thing the section argues. The replacement text was +written from that summary and inherited the wrong count and a false universal +("every one is text an artifact was arguing with"; one of them agrees). **The +second search was better-aimed than the first and was still read from the +expectation rather than from the output.** + +**Why it is hard to see when writing.** You search for what you expect to find, +because that is the only image you have of what the thing looks like. A search +built from the expected answer returns the expected answer whether or not the +answer is true, and — this is the part that makes it dangerous — **it returns it +with the authority of a mechanical check.** "I grepped the tree" sounds like +evidence. It is evidence only about the pattern you chose. + +**The check.** Two clauses, both cheap: + +- **Search for what would falsify the claim, not for what would confirm it.** If + the claim is *no X survives*, the search must be for anything that could + possibly be X — including X quoted, paraphrased, referenced, or embedded in + something else — not for X in the form you imagine it taking. +- **Open every hit before you report a count.** A grep that returns four and a + report that says three is not a mechanical check; it is a mechanical check + followed by an unmechanical reading. This is `## 3` — count the set, not the + ceiling — occurring one level up: **the ceiling of your attention rather than + the ceiling of an identifier.** + +**Where this one came from is part of the entry**, as with `## 3`: the first +instance was written by Chat, the second by Code, in a repository whose subject +is claims that are well-formed, plausible and wrong. Both stations made the same +error inside three iterations of each other. + +--- + ## The general shape Entries 1 and 2 are the same defect wearing different clothes: **the @@ -198,7 +255,7 @@ A specification needs both. They fail differently: a missing falsifier produces a result nobody can interpret, and a missing finish condition produces work nobody can find. -**Entries 3, 4 and 5 are a second family, and the file is more useful for +**Entries 3, 4, 5 and 6 are a second family, and the file is more useful for separating them.** They are not about what "done" looks like; each states a finish condition perfectly well. They are defects of **assembly** — a figure copied from the wrong place, a clause contradicted by another clause, a @@ -208,3 +265,10 @@ between two things. Every check in this family is therefore mechanical: count th set, diff the setup against the task, grep for your own placeholders. If a proposed check for a defect in this family can only be described as "be careful", it is not a check. + +**Entry 6 is the family's sharpest case**, because the defective check *was* +mechanical. A grep is not made trustworthy by being a grep: it inherits every +assumption in the pattern, and then hands its output to a reader who already +believes the answer. The two clauses that survive contact with that — search for +the falsifier, open every hit — are the same two moves the repeated-figure check +in `report/claims-verified.md` makes, which is not a coincidence. diff --git a/docs/state.yaml b/docs/state.yaml index d00b99c..503b029 100644 --- a/docs/state.yaml +++ b/docs/state.yaml @@ -522,9 +522,23 @@ open_decisions: eighteen-claims figure (draft.md:391) that a per-passage reading missed and that the two amended locations left inconsistent, and it found that "thirteen" carries three different referents and "eighteen" two. - ITS LIMIT, recorded so it is not over-trusted: the word list is enumerated, - so a figure spelled as a word not on the list is missed. Extending it is a - per-document maintenance cost, not a one-time fix. + CASE-SENSITIVITY BUG, FOUND AND FIXED 2026-09-08, AFTER THIS ENTRY WAS + CLOSED. The shipped version was case-sensitive: it saw "three passages" and + missed "Three passages". Measured over the typeset sources, 252 hits + against 301 case-insensitive — MISSING 49, or 16%, and the missed class was + every sentence-initial figure, which in prose is a large class. Fixed with + `-i` on the grep AND `tolower` in the awk grouping; without the second, + "Three" and "three" count as two different figures and the check reports + false distinctness instead of false absence. + HOW IT WAS FOUND: by hand, noticing that a figure known to be in §6 was not + in the output. NOT by the check. A check that silently drops a sixth of its + input is worse than no check, because it is reported as a clean run — which + is docs/spec-defects.md §6 occurring inside the tool built to catch §6's + defect. The first version of this check had been run three times and + reported as passing before anyone noticed. + ITS REMAINING LIMIT, recorded so it is not over-trusted: the word list is + enumerated, so a figure spelled as a word not on the list is missed. + Extending it is a per-document maintenance cost, not a one-time fix. title: >- The repeated-claim grep in report/claims-verified.md does not work on this repository's prose. diff --git a/report/claims-verified.md b/report/claims-verified.md index ad3031e..d0574d4 100644 --- a/report/claims-verified.md +++ b/report/claims-verified.md @@ -564,46 +564,82 @@ does not yet reach, the rejections. Recorded as `od-006`. # Addendum — §6, *The method destroys its own best evidence* -**Added after the first pass and checked before it was inserted**, because new -prose making factual claims is what this document exists to catch. Verified at -the same pinned SHAs. Five checkable claims, all CONFIRMED — so the tally in -`mn-004` is unchanged and these are numbered separately as A1–A5. +**Checked at the same pinned SHAs, on the revised text.** Seven checkable +claims: **five CONFIRMED, one CORRECTED, one UNDER-QUALIFIED.** Numbered A1–A7 +and kept out of `mn-004`'s tally, which covers §§1–8.5 as they stood at the +first pass. -### A1. "§4.2 describes three specifications rejected on false assumptions about repository state… and calls this *the clearest case in the project of a rule paying for itself*" -**CONFIRMED.** `report/draft.md` §4.2: "Three specifications were rejected on -false assumptions, each correctly, each before any work was spent… it is the -clearest case in the project of a rule paying for itself." Quoted accurately. +### A1. "§4.2 describes three specifications rejected… *the clearest case in the project of a rule paying for itself*" +**CONFIRMED.** Quoted accurately from §4.2. ### A2. "That claim came back from the verification pass **unsupported**" -**CONFIRMED.** Claim 17 of this document, UNSUPPORTED. - -### A3. "It is the only artifact in the loop that is never committed." -**CONFIRMED, and this is the claim the specification flagged as the stop -condition.** Re-checked at `instance` @`5f8e39f`: no committed file is a -specification, and no tracked filename matches *spec*, *task*, *brief*, -*instruction*, *relay* or *prompt* — zero hits. -- **Note what this sentence does NOT say.** An earlier draft of this section was - reported to assert that *no specification text survives anywhere*, which is - **false**: three fragments survive, quoted inside artifacts that were being - committed anyway (`exp007/preregistration.md:38`, `exp009/preregistration.md:19`, - `exp007/findings.md:46`). The supplied text says "never committed", which is - the true and weaker claim. **The distinction is load-bearing and the section - gets it right.** - -### A4. "One rejection survives in the record, and only incidentally: it reshaped a decision that was taken anyway… That fragment is not even a stale-state rejection" -**CONFIRMED on all three parts.** `instance: docs/state.yaml` — the `fc-012` -entry reads "THAT VERSION WAS REJECTED BECAUSE IT MAKES AZIMUTH AND VERGENCE -INERT, and the rejection is the reason this entry exists in the form it does." -`fc-012` was taken (rectifying in the camera), so the decision stands. And the -rejection was on a geometry defect, not on a false assumption about repository -state — so it is not of §4.2's class. - -### A5. "Five of the six unsupported claims in this document have that shape." -**CONFIRMED under the selector the sentence states** — "describes an event the -workflow guarantees will leave no trace" admits 12, 17, 20, 25 and 45, and -excludes 38, which is a proportion of a document rather than an event. See the -two-count note under falsifier 3: a stricter selector gives four, and both are -recorded rather than reconciled away. +**CONFIRMED.** Claim 17, UNSUPPORTED. + +### A3. "**No specification was ever committed.** No file in the instance is one, and none is named as one." +**CONFIRMED, and this is the sentence the previous round got wrong.** Re-checked +at `instance` @`5f8e39f`: no tracked file is a specification, and no filename +matches *spec*, *task*, *brief*, *instruction*, *relay* or *prompt* — zero hits. +The earlier form, *"no specification text survives anywhere"*, was false. This +form is true. + +### A4. "One rejection survives… it reshaped a decision that was taken anyway… not even a stale-state rejection" +**CONFIRMED on all three parts.** `fc-012`'s entry: "THAT VERSION WAS REJECTED +BECAUSE IT MAKES AZIMUTH AND VERGENCE INERT, and the rejection is the reason +this entry exists in the form it does." The decision was taken; the rejection +was on a geometry defect, not a repository-state assumption. + +### A5. "Five of the six unsupported claims… describe events the workflow guarantees will leave no trace" +**CONFIRMED under the selector the sentence states** — admits 12, 17, 20, 25, 45; +excludes 38, which is a proportion of a document rather than an event. A stricter +selector ("unrecordable by construction") gives four. Both recorded; see +`gap-004`. + +### A6. "**Three** passages in the instance quote specification text" +**CORRECTED → four passages report specification content, and one of the three named is not one of them.** + +| passage | reports spec content? | what it does | +|---|---|---| +| `exp007/preregistration.md:38` | **yes** | quotes it — the specification says "six arms, one stimulus" | +| `exp009/preregistration.md:19` | **yes** | reports the clause, then argues it is wrong | +| `exp007/findings.md:123` | **yes** | "The specification said 18 steps for A, A′, D and E", under a heading reading *"A deviation from the specification, and why"* | +| `exp007/findings.md:32` | **yes** | "The specification asked for this to be visible rather than inferred, and it is" | +| `exp007/findings.md:46` | **no** | "the correct reading, which neither exp005 nor this specification stated" — **records an ABSENCE in the specification.** The blockquote after it is the findings file's own reading, not the specification's words. | + +- **The third named fragment is the one that does not qualify**, and + `exp007/findings.md:123` — omitted — is the clearest instance of the section's + own thesis: an artifact departing from a specification clause and quoting it to + say so. +- **This is my error propagating, and it is worth naming as such.** The + search behind the previous round returned *four* hits and the report said + *three*: `findings.md:123` was in the output and never opened. The section was + then written from that summary. See `docs/spec-defects.md` §6. + +**Proposed:** "Four passages in the instance report specification text — a +pre-registration quoting a clause before departing from it, another quoting an +instruction and then arguing it is wrong, a findings file recording a deviation +and the clause it deviates from, and one recording that the specification asked +for something that was duly delivered." + +### A7. "**Every one is text that an artifact was arguing with.**" +**UNDER-QUALIFIED — true of three of the four, and the fourth is a counterexample.** +- Arguing: `exp007/preregistration.md:38` (declares a design change), + `exp009/preregistration.md:19` ("The second half is wrong"), + `exp007/findings.md:123` ("A deviation from the specification, and why"). +- **Not arguing:** `exp007/findings.md:32` — "The specification asked for this to + be visible rather than inferred, **and it is**." An artifact *agreeing* with a + specification and carrying its content into the record. + +**THE SECTION'S CONCLUSION SURVIVES INTACT, AND THIS IS THE IMPORTANT PART.** The +load-bearing sentence is the next one — *the survival mechanism is parasitic on +the work happening* — and `findings.md:32` satisfies it completely: agreement or +disagreement, the fragment is there because the specification was **executed** +and produced an artifact. A rejected specification produces no artifact either +way. **"Arguing with" is too narrow a mechanism for a conclusion that only needs +"executed".** Narrowing it costs the argument nothing and removes a false +universal. + +**Proposed:** "Every one is text that an artifact had to reckon with — mostly to +depart from it, once to confirm it." --- @@ -673,11 +709,25 @@ also matches only numerals, and this document spells its figures out. # 3. -H so the field offsets hold for one file as well as many; # 4. drop bare decimals -- in markdown they are section numbers, not claims. for f in "$@"; do - grep -onHE '\b([0-9]+(\.[0-9]+)?(×|x|:1|%|-fold)?|zero|one|two|three|four|five|six|seven|eight|nine|ten|eleven|twelve|thirteen|fourteen|eighteen|twenty-two|forty-one|eighty-seven|thousand)\b' "$f" \ + grep -onHEi '\b([0-9]+(\.[0-9]+)?(×|x|:1|%|-fold)?|zero|one|two|three|four|five|six|seven|eight|nine|ten|eleven|twelve|thirteen|fourteen|eighteen|twenty-two|forty-one|eighty-seven|thousand)\b' "$f" \ | grep -vE ':[0-9]+\.[0-9]+$' -done | awk -F: '{n[$3]=n[$3]" "$2} END {for (v in n) {c=split(n[v],a," "); if (c>1) printf "%3d %-12s lines:%s\n", c, v, n[v]}}' | sort -rn +done | awk -F: '{v=tolower($3); n[v]=n[v]" "$2} END {for (v in n) {c=split(n[v],a," "); if (c>1) printf "%3d %-12s lines:%s\n", c, v, n[v]}}' | sort -rn ``` +> **`-i`, AND THE GROUPING LOWERCASED TOO — ADDED AFTER THE CHECK WAS ALREADY +> SHIPPED AS `od-003`'s FIX.** The first version was case-sensitive, so it saw +> "three passages" and missed "Three passages". Measured over these files: +> **252 hits case-sensitive against 301 case-insensitive — it was missing 49, or +> 16%, and the missed class was every sentence-initial figure.** Prose puts +> figures at the start of sentences constantly. +> +> It was found by hand: a figure known to be in §6 did not appear in the output. +> **A check that silently drops a sixth of its input is worse than no check, +> because it is reported as a clean run** — which is `docs/spec-defects.md` §6 +> in the tool built to prevent §6's defect. Both halves matter: `-i` on the grep +> finds the hits, and `tolower` in the awk stops "Three" and "three" from being +> counted as two different figures. + **It was validated against a known answer before being trusted**, which the shipped one never was: it had to surface the `mn-001`/`mn-002` pair, and it does — "eighteen" at seven locations and "twenty-two" at two. diff --git a/report/draft.md b/report/draft.md index e1eafb0..3de6913 100644 --- a/report/draft.md +++ b/report/draft.md @@ -962,18 +962,30 @@ from the verification pass **unsupported** — not contradicted, but not establishable from any record. The reason is structural. A specification is written on one surface, relayed by -a human, and executed on another. **It is the only artifact in the loop that is -never committed.** The work it produces lands in a branch; the specification -itself lives in a chat window and a paste buffer. So a specification that was -*executed* leaves a commit behind, and a specification that was *rejected* -leaves nothing at all — no branch, no diff, no entry, because the whole point of -the rejection is that no work was done. - -Five of the six unsupported claims in this document have that shape. Each -describes an event the workflow guarantees will leave no trace. One rejection -survives in the record, and only incidentally: it reshaped a decision that was -taken anyway, so the decision's entry carries a fragment of it. That fragment is -not even a stale-state rejection, which is the class §4.2's claim was about. +a human, and executed on another. **No specification was ever committed.** No +file in the instance is one, and none is named as one. The work a specification +produces lands in a branch; the specification itself lives in a chat window and +a paste buffer. + +Fragments do survive, and where they survive is the finding. Three passages in +the instance quote specification text — a pre-registration repeating a clause +before narrowing it, another quoting an instruction and then arguing it is +wrong, a findings file recording a reading *"which neither exp005 nor this +specification stated."* **Every one is text that an artifact was arguing with.** + +That is the whole mechanism. Specification text reaches the record only where a +pre-registration or a findings file had to disagree with it in order to justify +doing something else — which requires the specification to have been **executed** +and to have produced an artifact in the first place. A specification rejected +before work begins produces no artifact, so nothing is ever in a position to +quote it. **The survival mechanism is parasitic on the work happening, which is +exactly why it cannot preserve a rejection.** + +Five of the six unsupported claims in this document describe events the workflow +guarantees will leave no trace. One rejection survives in the record, and only +incidentally: it reshaped a decision that was taken anyway, so the decision's +entry carries a fragment of it. That fragment is not even a stale-state +rejection, which is the class §4.2's claim was about. The finding is worse than a gap in the record. diff --git a/report/sections/06-where-it-failed.tex b/report/sections/06-where-it-failed.tex index c7164cc..02b23e2 100644 --- a/report/sections/06-where-it-failed.tex +++ b/report/sections/06-where-it-failed.tex @@ -28,9 +28,13 @@ \subsection{The method destroys its own best evidence} \S4.2 describes three specifications rejected on false assumptions about repository state, each caught before any work was spent, and calls this \textit{the clearest case in the project of a rule paying for itself.} That claim came back from the verification pass \textbf{unsupported} --- not contradicted, but not establishable from any record. -The reason is structural. A specification is written on one surface, relayed by a human, and executed on another. \textbf{It is the only artifact in the loop that is never committed.} The work it produces lands in a branch; the specification itself lives in a chat window and a paste buffer. So a specification that was \textit{executed} leaves a commit behind, and a specification that was \textit{rejected} leaves nothing at all --- no branch, no diff, no entry, because the whole point of the rejection is that no work was done. +The reason is structural. A specification is written on one surface, relayed by a human, and executed on another. \textbf{No specification was ever committed.} No file in the instance is one, and none is named as one. The work a specification produces lands in a branch; the specification itself lives in a chat window and a paste buffer. -Five of the six unsupported claims in this document have that shape. Each describes an event the workflow guarantees will leave no trace. One rejection survives in the record, and only incidentally: it reshaped a decision that was taken anyway, so the decision's entry carries a fragment of it. That fragment is not even a stale-state rejection, which is the class \S4.2's claim was about. +Fragments do survive, and where they survive is the finding. Three passages in the instance quote specification text --- a pre-registration repeating a clause before narrowing it, another quoting an instruction and then arguing it is wrong, a findings file recording a reading \textit{"which neither exp005 nor this specification stated."} \textbf{Every one is text that an artifact was arguing with.} + +That is the whole mechanism. Specification text reaches the record only where a pre-registration or a findings file had to disagree with it in order to justify doing something else --- which requires the specification to have been \textbf{executed} and to have produced an artifact in the first place. A specification rejected before work begins produces no artifact, so nothing is ever in a position to quote it. \textbf{The survival mechanism is parasitic on the work happening, which is exactly why it cannot preserve a rejection.} + +Five of the six unsupported claims in this document describe events the workflow guarantees will leave no trace. One rejection survives in the record, and only incidentally: it reshaped a decision that was taken anyway, so the decision's entry carries a fragment of it. That fragment is not even a stale-state rejection, which is the class \S4.2's claim was about. The finding is worse than a gap in the record. From 068a285988b8a80de6ece9f2298bd9489c64ea54 Mon Sep 17 00:00:00 2001 From: Luiz Velho Date: Tue, 8 Sep 2026 23:30:58 -0300 Subject: [PATCH 08/11] Convert straight double quotes in the transit, and guard against them MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two straight double quotes reached the typeset output, both inside verbatim quotations: the active-stereo constitution passage in §2 and the findings-file phrase in §6. LaTeX renders " as two RIGHT-facing marks, so the opening quote of each pointed the wrong way. Markdown renders both forms identically, which is why the defect is invisible at the source and visible only after the transit -- which is what the draft/typeset separation exists to surface, so the practice is now recorded as having fired rather than argued for. FIXED IN THE CONVERTER, NOT IN THE .TEX. The section files are generated from report/draft.md; a hand-edit would have been reverted by the next regeneration, silently, and the document would have been correct exactly until someone re-ran the conversion. Straight pairs now become `` and '' before output. AND THE CONVERTER NOW FAILS RATHER THAN WARNS. It refuses to write output containing a straight double quote, an unconverted curly quote, an em or en dash or a section mark. Tested by removing one closing quote from the draft: the conversion exits non-zero and names the file and the count. The first attempt at that test was worthless -- it produced a BALANCED pair, which the converter correctly handles -- and was redone with a genuinely unpaired quote. FULL AUDIT OF report/sections/, reported in claims-verified.md. Straight quotes: zero. Apostrophes: 49, all correct -- LaTeX renders ' as a right single quote, which is what an apostrophe is. Five more on section references (\S4.1's). Opening single quotes, ellipses, stray backticks, unconverted unicode: none. One straight quote remains in main.tex, inside a LaTeX comment, never rendered -- reported rather than changed, because changing it would suggest it mattered. CHECKED BECAUSE IT WOULD HAVE BEEN SILENT: the quote conversion also runs over code-span contents, so a " inside a `...` span would have been turned into typographic marks inside a literal \texttt. There are none in this draft. That is a property of this draft, not of the converter. 23 pages, zero boxes, transit re-verified lossless at 12,292 words. od-007 opened, because this fix exposed a gap it did not create: the converter is NOT IN THE REPOSITORY. The regeneration a reader is told to perform cannot be performed by them, and the guard that now protects this class of defect protects nobody else. The obvious fix reopens fc-002 and fc-003 by putting Python back, so the three options and their costs are recorded rather than one being taken. Co-Authored-By: Claude Opus 5 (1M context) --- docs/state.yaml | 33 +++++++++++++++++ report/claims-verified.md | 51 ++++++++++++++++++++++++++ report/sections/02-the-setting.tex | 2 +- report/sections/06-where-it-failed.tex | 2 +- 4 files changed, 86 insertions(+), 2 deletions(-) diff --git a/docs/state.yaml b/docs/state.yaml index 503b029..2f53f78 100644 --- a/docs/state.yaml +++ b/docs/state.yaml @@ -645,6 +645,39 @@ open_decisions: a record's authority, which is claim 25's defect. depends_on: [gap-004] + - id: od-007 + altitude: infra + status: open + title: >- + The markdown-to-LaTeX converter is not in the repository, and it now + carries a guard that is load-bearing. + the_question: >- + report/sections/*.tex are GENERATED from report/draft.md. The script that + generates them has never been committed — it lives outside the tree — so + the regeneration a reader is told to perform cannot be performed, and the + diff between draft and typeset output that report/main.tex calls the place + a mangled number shows cannot be reproduced by anyone else. + IT MATTERS MORE SINCE 2026-09-08. The converter now refuses to emit a + straight double quote, an unconverted curly quote, dash or section mark — + the guard added after two straight quotes reached the built PDF. That + guard is the only thing standing between this document and a repeat, and + it is not in the repository. + what_would_decide_it: >- + A judgement the maintainer has to make, because the obvious fix reopens a + closed decision. Committing the script means putting Python back in a + repository that deleted it deliberately (fc-002, fc-003), which would drag + pyproject.toml, the lint and type gates, and the .reference/ import guard + back with it — fc-002's scope says exactly that. Three options, none free: + (1) commit the script alone, exempt from tooling, and say so — cheapest, + and it makes "no executable code here" false; + (2) commit it and accept the tooling back, honouring fc-002's scope; + (3) leave it out and delete the claim that the .tex is regenerable, + treating the typeset files as hand-maintained from here on — which + costs the diffability the whole draft/typeset practice exists for. + DECIDE BEFORE THE NEXT EDIT TO THE PROSE, because (3) becomes true by + default the moment someone edits a .tex directly. + depends_on: [] + - id: od-005 altitude: impl status: RESOLVED diff --git a/report/claims-verified.md b/report/claims-verified.md index d0574d4..8153bc8 100644 --- a/report/claims-verified.md +++ b/report/claims-verified.md @@ -745,6 +745,57 @@ and that is a maintenance cost, not a one-time fix. --- +# The transit, and the defect it surfaced + +**This is what the draft/typeset separation is for**, and it has now caught +something, so the practice is recorded as having fired rather than argued for. + +**Straight double quotes survived into the typeset output**, in two places, both +inside verbatim quotations — the passage from `active-stereo`'s constitution in +§2, and the findings-file phrase in §6. LaTeX renders `"` as **two right-facing +marks**, so the *opening* quote of each pointed the wrong way. In Markdown the +two forms render identically, which is exactly why the defect is invisible at the +source and visible only after the transit. + +**The cause was a converter that handled curly quotes and not straight ones.** +The draft mostly uses `“ ”`, which were mapped from the first version; the two +straight pairs were the exceptions, and nothing looked for them. + +**The fix is in the converter, not in the `.tex`.** The section files are +generated from `report/draft.md`; editing them by hand would have been reverted +by the next regeneration, silently, and the document would have been correct +exactly until someone re-ran the conversion. Straight pairs are now converted to +`` `` ``…`` '' `` before output. + +**And the converter now fails rather than warns.** It refuses to write output +containing a straight double quote, an unconverted curly quote, an em or en dash, +or a section mark — the guard was tested by removing one closing quote from the +draft, and the conversion exits non-zero naming the file and the count. *This +defect was found by eye, in a built PDF, which is the most expensive place to +find it; it cannot ship that way again.* + +## The rest of the audit, reported in full + +| class | found | verdict | +|---|---|---| +| straight `"` in `sections/` | **0** after the fix | enforced by the guard | +| apostrophes (`word's`, `experiments'`) | 49 | **correct as-is** — LaTeX renders `'` as a right single quote, which is what an apostrophe is | +| possessives on section refs (`\S4.1's`) | 5 | correct; flagged only because a digit precedes them | +| opening single quotes | 0 | none to fix | +| ellipses (`...` or `…`) | 0 | none present | +| stray backticks | 0 | every `` ` `` is half of a `` `` `` pair | +| unconverted `— – § “ ”` | 0 | clean | +| straight `"` in `main.tex` | 1 | **inside a LaTeX comment**, never rendered — reported rather than "fixed", because changing it would suggest it mattered | +| quotes inside `\texttt{}` | 0 | checked, because the conversion runs on code-span contents too and would have corrupted a literal | + +**The last row is the one worth keeping.** The quote conversion also runs over +code-span contents, so a `"` inside a `` ` ` `` span would have been silently +turned into typographic marks in a literal. There are none in this draft — but +that is a property of this draft, not of the converter, and a future one could +trip it. + +--- + # What this pass did not do - **No prose was written and no `.tex` file was touched.** The two amendments in `9b58d1a` were made before this pass, under the specification's explicit exception, and are recorded as claim 14's partial resolution. diff --git a/report/sections/02-the-setting.tex b/report/sections/02-the-setting.tex index 9bd5ca0..ebc2c2a 100644 --- a/report/sections/02-the-setting.tex +++ b/report/sections/02-the-setting.tex @@ -34,7 +34,7 @@ \subsection{The four-station loop} Two properties of that circuit matter more than its shape. -\textbf{Chat never edits the repository, but it does hold the files.} The clone is a separate checkout, outside the working tree, which Chat reads and runs and never commits to. This is the reviewer's seat: it can check everything and change nothing. The predecessor project's constitution had explicitly ruled it out --- \textit{Chat holds excerpts by construction, so "Chat verifies Code's numbers" would rot into a formality} --- and that premise was simply false. The seat existed and the workflow had talked the reviewer out of occupying it. +\textbf{Chat never edits the repository, but it does hold the files.} The clone is a separate checkout, outside the working tree, which Chat reads and runs and never commits to. This is the reviewer's seat: it can check everything and change nothing. The predecessor project's constitution had explicitly ruled it out --- \textit{Chat holds excerpts by construction, so ``Chat verifies Code's numbers'' would rot into a formality} --- and that premise was simply false. The seat existed and the workflow had talked the reviewer out of occupying it. \textbf{Escalation is expected, not exceptional.} It is how the loop discovers that a specification was mis-aimed, which happens often and should. A stretch of tasks with no escalations means either the specifications are unusually good or the reviewer is not looking hard enough, and the second is more likely. diff --git a/report/sections/06-where-it-failed.tex b/report/sections/06-where-it-failed.tex index 02b23e2..265badd 100644 --- a/report/sections/06-where-it-failed.tex +++ b/report/sections/06-where-it-failed.tex @@ -30,7 +30,7 @@ \subsection{The method destroys its own best evidence} The reason is structural. A specification is written on one surface, relayed by a human, and executed on another. \textbf{No specification was ever committed.} No file in the instance is one, and none is named as one. The work a specification produces lands in a branch; the specification itself lives in a chat window and a paste buffer. -Fragments do survive, and where they survive is the finding. Three passages in the instance quote specification text --- a pre-registration repeating a clause before narrowing it, another quoting an instruction and then arguing it is wrong, a findings file recording a reading \textit{"which neither exp005 nor this specification stated."} \textbf{Every one is text that an artifact was arguing with.} +Fragments do survive, and where they survive is the finding. Three passages in the instance quote specification text --- a pre-registration repeating a clause before narrowing it, another quoting an instruction and then arguing it is wrong, a findings file recording a reading \textit{``which neither exp005 nor this specification stated.''} \textbf{Every one is text that an artifact was arguing with.} That is the whole mechanism. Specification text reaches the record only where a pre-registration or a findings file had to disagree with it in order to justify doing something else --- which requires the specification to have been \textbf{executed} and to have produced an artifact in the first place. A specification rejected before work begins produces no artifact, so nothing is ever in a position to quote it. \textbf{The survival mechanism is parasitic on the work happening, which is exactly why it cannot preserve a rejection.} From 3deb87dacf2b61d94544fc633bad9432b6605dab Mon Sep 17 00:00:00 2001 From: Luiz Velho Date: Wed, 9 Sep 2026 00:05:08 -0300 Subject: [PATCH 09/11] Commit the converter and take the tooling back with it (od-007, option 2) tools/md2tex.py generates report/sections/*.tex from report/draft.md. It was outside the repository, which made report/main.tex's claim that the .tex is regenerable true and unverifiable at the same time. od-007 closed. fc-002 AND fc-003 ARE ANNOTATED, NOT REVERSED, per CLAUDE.md's rule. What came back is the tooling fc-002's scope said would come back with pyproject.toml: ruff, ruff format, mypy and pytest, as a `code` job in CI. Taking the code and leaving the gates would have been taking the convenient half of a decision. WHAT DID NOT COME BACK. No [build-system], no hatchling, no package -- there is still no library, and a build backend for a package that does not exist is scaffolding that gets believed. No dependencies: the converter is standard library only. src/ and experiments/ stay closed. THE ORDER fc-003 PRESCRIBED WAS FOLLOWED. Its scope says "restore the [pinned-clone] guard first, before the others", and that guard is the first test in tests/test_scaffold.py for that reason and no other. IT FIRED IMMEDIATELY, on the docstrings of the two files that restored it, which named the directory in prose. The prose was rewritten and the guard left crude. Loosening a check to admit its first true positive is how a check stops being one. THE SECOND GUARD IS WHAT EARNS THE REOPENING. test_typeset_sections_match_the_draft regenerates the sections into a temporary directory and compares bytes, so main.tex's claim that the .tex is generated from the draft can now fail. Both guards were watched failing before being committed -- one by appending a comment to a .tex, one by adding a real read of the pinned clones -- and the sync test writes only to a temp directory, so a failing test can never repair the thing it is checking. fc-002's SCOPE NAMED THE WRONG TRIGGER, and that is the finding worth keeping. It said the entry stops holding "the moment one generated figure is wanted". No figure is wanted; a converter arrived. fc-003 said "the moment any .py file lands" and covered this exactly. The difference is that fc-002's scope described the USE it was picturing and fc-003's described the CONDITION. A scope clause that names an instance will be silent on the case that actually occurs. COSTS PAID, recorded because they are most of the cost. The script was scratch-quality -- semicolons, percent formatting -- and had to be rewritten to pass the gates it was arriving under; its output is verified byte-identical before and after, and again after ruff format. Five statements elsewhere became false and are corrected in this commit: CLAUDE.md's Commands section and its definition-of-done clause 2, .gitignore's note that the guard was gone, README.md's "no Python", and report/figures/README.md's restore instructions. Unchanged: 23 pages, zero boxes, and `make -C report` still needs no Python -- the .tex files are committed and `make -C report sections` is a separate step that never runs during a build. Co-Authored-By: Claude Opus 5 (1M context) --- .github/workflows/report.yml | 56 ++++++- .gitignore | 16 +- CLAUDE.md | 57 +++++--- README.md | 7 +- docs/state.yaml | 81 +++++++++- pyproject.toml | 53 +++++++ report/Makefile | 13 +- report/figures/README.md | 12 +- tests/test_scaffold.py | 109 ++++++++++++++ tools/md2tex.py | 276 +++++++++++++++++++++++++++++++++++ 10 files changed, 646 insertions(+), 34 deletions(-) create mode 100644 pyproject.toml create mode 100644 tests/test_scaffold.py create mode 100644 tools/md2tex.py diff --git a/.github/workflows/report.yml b/.github/workflows/report.yml index bfd9105..1b66926 100644 --- a/.github/workflows/report.yml +++ b/.github/workflows/report.yml @@ -5,7 +5,16 @@ # could not even reach its first step, and a workflow that is red for reasons # unrelated to the report is one people learn to override. # -# TWO JOBS NOW: `ledgers` parses the record, `build` compiles the document. +# THREE JOBS NOW: `ledgers` parses the record, `code` gates the one Python file +# in the repository, `build` compiles the document. +# +# THE `code` JOB EXISTS BECAUSE fc-002's SCOPE SAID IT WOULD. tools/md2tex.py +# generates report/sections/*.tex from report/draft.md; it was outside the +# repository, which made main.tex's claim that the .tex is regenerable true and +# unverifiable at once (od-007). Committing it reopened fc-002 and fc-003, and +# fc-002's scope is explicit that pyproject.toml comes back with the gates +# attached. Restoring the code without the gates would have been taking the +# convenient half of a decision and leaving the part that cost something. # od-004 closed -- the gap that used to be recorded here, that nothing checked # whether docs/state.yaml and docs/inherited-measurements.yaml still parse, is # closed by the first job below. @@ -82,6 +91,51 @@ jobs: end exit(ok ? 0 : 1)' + # The one Python file, and the two guards that make its claims testable. + # + # PYTEST IS THE POINT, and the other three are the price fc-002's scope set. + # test_scaffold.py regenerates the sections into a temporary directory and + # compares bytes, so "report/sections/*.tex is generated from report/draft.md" + # -- asserted in main.tex's header since the document was first typeset -- is + # now a claim that can fail. It was verified failing before it was committed, + # by appending a comment to one .tex. + # + # ONE INTERPRETER, NOT A MATRIX. The template ran 3.12 and 3.13 because its + # pyproject.toml declared `requires-python = ">=3.12"` and something had to + # test the declaration. This one declares the same, and the honest note is + # that a single stdlib script is not evidence for the floor -- the matrix is + # not restored because there is nothing here it would discriminate. + code: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + + - uses: actions/setup-python@v5 + with: + python-version: "3.13" + cache: pip + + # No project dependencies: tools/md2tex.py is standard library only, and + # pyproject.toml says so. Only the dev extra is installed. + - name: Install + run: | + python -m pip install --upgrade pip + pip install ".[dev]" + + - name: Lint + run: ruff check tools tests + + - name: Format check + run: ruff format --check tools tests + + - name: Types + run: mypy + + # `-rs` reports skips: a silent skip is a gap in the evidence that looks + # like a pass. + - name: Tests + run: pytest -rs + build: runs-on: ubuntu-latest diff --git a/.gitignore b/.gitignore index b0761c2..0dc40d8 100644 --- a/.gitignore +++ b/.gitignore @@ -4,12 +4,16 @@ # claim about `visgraf/bio-3d-vision` or its predecessors, and "verifiable at # source" means a checkout at the SHA pinned in docs/inherited-measurements.yaml. # -# THE ENFORCEMENT THAT USED TO SIT BESIDE THIS IS GONE, and that is worth saying -# rather than leaving to be discovered. The template guarded this directory with -# a test asserting that no code reads from it. This repository has no code, so -# the guard had nothing to guard and was deleted with the rest of the suite. The -# convention survives on nobody's authority but this comment. If a figure script -# ever lands here, restore the guard with it -- see report/figures/README.md. +# THE ENFORCEMENT IS BACK, and this comment is no longer the only thing holding +# the convention up. It was: the template's guard was deleted with the rest of +# the suite because there was no code to guard, and this paragraph said so. When +# tools/md2tex.py landed (od-007), fc-003's scope required the guard to be +# restored FIRST, before any other -- and it was. tests/test_scaffold.py now +# fails if anything under tools/ or tests/ so much as names this directory. +# +# IT FIRED IMMEDIATELY, on the docstrings of the two files that restored it, +# which mentioned the path in prose. The prose was rewritten rather than the +# guard loosened. It is crude on purpose. .reference/ # LaTeX build artifacts are ignored in report/.gitignore, beside the build that diff --git a/CLAUDE.md b/CLAUDE.md index f41bc8d..0c2d06a 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -186,15 +186,19 @@ A task is done when all of these hold: > *Why this clause is first:* the predecessor's five conditions were all about > artifact quality, so a perfectly-tested component could be added to a loop > that had never run — and was. -2. **Every claim it adds to the prose is traceable.** There is no test suite - here and clause 2 cannot be about one. Its replacement is the standard the - draft is held to: a new factual sentence names which repository it is true - of, and any figure in it has an entry in - `docs/inherited-measurements.yaml` — written at the same time, not later. - > *Why the substitution and not a deletion:* clause 2 existed so that new - > behaviour could not arrive unchecked. New prose is the only thing that - > arrives here, and the unchecked version of it is this project's entire - > failure mode. +2. **Every claim it adds to the prose is traceable, and new behaviour has a + test.** Both halves are live now. The prose half is the one that matters + almost always: a new factual sentence names which repository it is true of, + and any figure in it has an entry in `docs/inherited-measurements.yaml`, + written at the same time and not later. The code half applies to + `tools/` and `tests/`, which exist again as of `od-007`. + > *Why this clause was rewritten twice:* it originally read "new behaviour has + > a test, and the suite passes", which was inapplicable when there was no + > suite; it was replaced by the prose standard alone; and the code half came + > back when `tools/md2tex.py` did. **The prose half was never the substitute + > for the code half — it is the more important of the two here**, because new + > prose is what mostly arrives and the unchecked version of it is this + > project's entire failure mode. 3. Conventions are stated in `## Domain invariants` above, because there is no code to state them at. See the note there. 4. `docs/state.yaml` is updated if any of it changed. @@ -238,8 +242,8 @@ the source is the finding and the prose is the bug. ## Working agreement -- **Branch.** `docs/…`, `report/…`, `fix/…`. Never commit to `main`. (`exp/…` - and `spike/…` are dropped: this repository has neither.) +- **Branch.** `docs/…`, `report/…`, `feat/…`, `fix/…`. Never commit to `main`. + (`exp/…` and `spike/…` stay dropped: this repository has neither.) - **Never commit or push unless asked.** Never force-push or rewrite history. > *Why the asking is explicit:* a specification that names a branch but never > asks for a commit leaves the work sitting uncommitted and costs a round @@ -271,12 +275,29 @@ method, with its provenance. ```bash make -C report # build the report PDF (needs a TeX installation) +make -C report sections # regenerate report/sections/*.tex from draft.md (needs Python) +pytest -q # the two scaffold guards +ruff check tools tests # lint +ruff format tools tests # format +mypy # types ``` -**That is the whole list, and it is one line.** No suite, no linter, no type -checker — there is no code for them to run against, and `pyproject.toml` was -deleted rather than kept as an empty promise. The build is the only automated -check this repository has, and `.github/workflows/report.yml` is the only -workflow. Everything else that could go wrong here goes wrong in prose, and the -only instrument for that is `report/claims-verified.md`, run by a person against -a pinned checkout. +**This list was one line until `tools/md2tex.py` landed, and the change is worth +naming rather than absorbing.** The converter generates `report/sections/*.tex` +from `report/draft.md`. It lived outside the repository, which made +`report/main.tex`'s claim that the `.tex` is regenerable true and unverifiable at +the same time — `od-007`. Committing it reopened `fc-002` and `fc-003`, and +`fc-002`'s scope is explicit that `pyproject.toml` returns with the gates +attached. Taking the code without the gates would have been taking the convenient +half of a decision. + +**What is still true:** there is no library and no experiment. `pyproject.toml` +carries tool configuration and no build backend. The PDF build needs no Python — +the `.tex` files are committed, so `make -C report` compiles from a clean +checkout with TeX alone, and `make -C report sections` is a separate step that +never runs during a build. + +**And the prose instrument is unchanged and still the important one.** The gates +catch nothing that a claims pass catches. `report/claims-verified.md`, run by a +person against a pinned checkout, is where this repository's characteristic +failure is caught. diff --git a/README.md b/README.md index 322fb8a..aba1bf8 100644 --- a/README.md +++ b/README.md @@ -68,7 +68,12 @@ done. Read the draft accordingly. Instantiated from the template [`visgraf/math-ai-method`](https://github.com/visgraf/math-ai-method) at commit `81a638e`, then adapted to a document-only shape: no library, no experiments, no -test suite, no Python. What was deleted and what was kept is recorded with its +test suite, no Python. **Three of those four still hold.** `tools/md2tex.py` +generates `report/sections/*.tex` from the draft; it was outside the repository, +which made the claim that the `.tex` is regenerable true and unverifiable at once +(`od-007`). Committing it brought back `pyproject.toml` (tool configuration only, +no build backend), two scaffold guards, and the lint, type and test gates — +because `fc-002`'s scope said it would. What was deleted and what was kept is recorded with its reasoning in the `foreclosures` block of [`docs/state.yaml`](docs/state.yaml) — `fc-001` through `fc-006` — rather than left to be inferred from an absence. diff --git a/docs/state.yaml b/docs/state.yaml index 2f53f78..8f62482 100644 --- a/docs/state.yaml +++ b/docs/state.yaml @@ -354,6 +354,19 @@ runs: packages on a GitHub runner, first run — which is a measurement of the APT INSTALL, not of the compile, and was made against the template's package set rather than this one. Treat it as a prior. + - call: make -C report sections + does: >- + Regenerates report/sections/*.tex from report/draft.md via + tools/md2tex.py. NEEDS PYTHON, which the PDF build deliberately does not: + the .tex files are committed so a clean checkout compiles with TeX alone. + Never runs during a build. + cost: Sub-second; nine files. + - call: pytest -q + does: >- + Two scaffold guards — nothing under tools/ or tests/ reads the pinned + clones, and report/sections/ is byte-identical to what the converter + produces from the draft. + cost: Sub-second. - call: the claims-verification pass does: >- Not a command. A person, reading report/draft.md against cloned @@ -647,7 +660,30 @@ open_decisions: - id: od-007 altitude: infra - status: open + status: RESOLVED + resolved_at: 2026-09-09 + resolution: >- + OPTION 2 TAKEN, on the maintainer's decision: the converter is committed at + tools/md2tex.py and the tooling came back with it, honouring fc-002's + scope rather than taking the code and leaving the price. pyproject.toml is + tool configuration only — no build backend, no package — because there is + still no library. Both foreclosures are ANNOTATED, not reversed. + WHAT THE REOPENING BOUGHT, which is more than the converter being readable: + tests/test_scaffold.py::test_typeset_sections_match_the_draft regenerates + report/sections/ into a temporary directory and compares bytes. The claim + in report/main.tex's header that the .tex is generated from the draft has + been asserted since the document was first typeset and was untestable + until now. Option 3 — dropping the claim and hand-maintaining the .tex — + would have cost exactly the diffability the draft/typeset practice exists + for, and it was the option that would have become true by default. + COSTS PAID, recorded so the trade is legible: the script had to be + rewritten to pass the gates it was arriving under (it was scratch-quality — + semicolons, percent formatting), and its output was verified byte-identical + afterwards; CLAUDE.md's Commands section, its definition-of-done clause 2, + .gitignore's note that the guard was gone, README.md's "no Python", and + report/figures/README.md's restore instructions were all made false by this + change and were corrected in the same commit. A decision that invalidates + five statements elsewhere is one whose cost is mostly in the prose. title: >- The markdown-to-LaTeX converter is not in the repository, and it now carries a guard that is load-bearing. @@ -822,6 +858,29 @@ foreclosures: contingency; see fc-005. cost_to_reopen: Low. Restore from `template` and add the plotting dependency, having asked. decided_in: The instantiation commit. + annotated_2026_09_09: >- + REOPENED BY ITS OWN TERMS, BUT NOT BY THE TRIGGER IT NAMED, and the closure + is annotated rather than reversed per CLAUDE.md. pyproject.toml is back — + as TOOL CONFIGURATION ONLY, with no [build-system], no hatchling and no + package, because there is still no library. The half this entry was really + about is still closed. + THE SCOPE CLAUSE NAMED THE WRONG TRIGGER, and that is the finding worth + keeping. It said this entry "stops holding the moment one generated figure + is wanted". No figure is wanted and report/figures/ is still empty. What + arrived was tools/md2tex.py, the converter that generates + report/sections/*.tex — a use nobody had imagined when the scope was + written. Compare fc-003, whose scope said "the moment any .py file lands" + and covered this exactly. THE DIFFERENCE IS SPECIFICITY: fc-002's scope + described the use it happened to be picturing, fc-003's described the + CONDITION. A scope clause that names an instance instead of a condition + will be silent on the case that actually occurs, and will look wrong when + it is not. + What came back with it, per this entry's own instruction that the gates + return with the file: ruff, ruff format, mypy and pytest, as the `code` job + in .github/workflows/report.yml. Taking the code without the gates would + have been taking the convenient half of a decision. NO DEPENDENCIES were + added; the converter is standard library only, and this entry's requirement + to ask before adding one is untouched and still binding. - id: fc-003 altitude: fw @@ -853,6 +912,26 @@ foreclosures: Moderate for src/ and experiments/, but reopening those means this is no longer a document-only repository, which is a different decision. decided_in: The instantiation commit. + annotated_2026_09_09: >- + THE CONDITION THIS SCOPE NAMED HAS OCCURRED, AND THE ORDER IT PRESCRIBED + WAS FOLLOWED. tools/md2tex.py landed (od-007), so "stops holding the moment + any .py file lands" fired, and the pinned-clone guard was restored FIRST, + as the clause required — it is the first test in tests/test_scaffold.py and + it is first for that reason and no other. + IT CAUGHT SOMETHING ON ITS FIRST RUN: the docstrings of the two files that + restored it named the directory in prose, and the guard is a crude line + scan that cannot tell prose from a code path. The prose was rewritten and + the guard was left crude. Loosening a check to admit its first true + positive is how a check stops being one. + WHAT REMAINS CLOSED: src/ and experiments/. There is still no library and + no experiment, so the package-imports guard and the + every-experiment-has-a-preregistration guard were NOT restored — each would + have passed by scanning an empty set. A SECOND GUARD WAS ADDED INSTEAD, and + it is the one that earns the reopening: test_typeset_sections_match_the_draft + regenerates the sections into a temporary directory and compares bytes, so + report/main.tex's standing claim that the .tex is generated from the draft + is now a claim that can fail. It was watched failing before it was + committed, by appending a comment to one .tex file. - id: fc-004 altitude: fw diff --git a/pyproject.toml b/pyproject.toml new file mode 100644 index 0000000..4c8e512 --- /dev/null +++ b/pyproject.toml @@ -0,0 +1,53 @@ +# TOOL CONFIGURATION ONLY. NO PACKAGE, NO BUILD BACKEND. +# +# This file was deleted at instantiation (fc-002) and is restored to honour that +# entry's own scope clause: bringing Python back brings the gates back with it. +# What is NOT restored is the half fc-002 was really about -- `[build-system]`, +# hatchling, `[project] name/version`, and `packages = ["src/research"]`. There +# is no library here and there is not going to be one; a build backend for a +# package that does not exist is the kind of scaffolding that gets believed. +# +# WHAT ACTUALLY LANDED, and why the tooling followed it: tools/md2tex.py, the +# converter that generates report/sections/*.tex from report/draft.md. It was +# outside the repository, which made report/main.tex's claim that the .tex is +# regenerable true and unverifiable at the same time (od-007). +# +# NOTE THAT fc-002's SCOPE NAMED THE WRONG TRIGGER. It said the entry stops +# holding "the moment one generated figure is wanted". No figure is wanted; a +# converter arrived instead. fc-003's scope named the right one -- "the moment +# any .py file lands" -- and is generic where fc-002's was specific to the use +# it happened to be imagining. Both entries are annotated in docs/state.yaml. +# +# NO DEPENDENCIES, and the converter is written to keep it that way: standard +# library only, no markdown or LaTeX package. Adding one needs asking +# (CLAUDE.md's working agreement), and this is the file where the asking would +# become visible in a diff. + +requires-python = ">=3.12" + +[project.optional-dependencies] +dev = [ + "pytest>=8.0", + "mypy>=1.11", + "ruff>=0.6", +] + +[tool.pytest.ini_options] +testpaths = ["tests"] +addopts = "-q --strict-markers" +markers = [ + "infrastructure: pins the scaffold, not any claim about the subject matter", +] + +[tool.ruff] +line-length = 100 +src = ["tools", "tests"] + +[tool.ruff.lint] +select = ["E", "F", "I", "UP", "B", "SIM"] + +[tool.mypy] +python_version = "3.12" +files = ["tools", "tests"] +strict = true +warn_unreachable = true diff --git a/report/Makefile b/report/Makefile index cd999fa..6d9a8b8 100644 --- a/report/Makefile +++ b/report/Makefile @@ -14,7 +14,7 @@ LATEXMK := latexmk -.PHONY: all figures clean distclean +.PHONY: all sections figures clean distclean all: main.pdf @@ -24,6 +24,17 @@ all: main.pdf main.pdf: main.tex $(wildcard sections/*.tex) $(wildcard figures/*.pdf) refs.bib $(LATEXMK) -pdf main.tex +# Regenerates report/sections/*.tex from report/draft.md. NEEDS PYTHON; the PDF +# build above does not, and must not -- the .tex files are committed so that a +# clean checkout compiles with a TeX installation alone. Run this after editing +# the draft, then commit both, which is what keeps the translation diffable. +# +# tests/test_scaffold.py regenerates into a temporary directory and compares +# bytes, so a .tex that has drifted from the draft fails CI rather than being +# silently overwritten the next time someone runs this. +sections: + python3 ../tools/md2tex.py + # Regenerates every committed figure, then you commit the PDFs it changes: a # figure and the claim it supports move through the same pull request, which is # what makes a provenance appendix true rather than decorative. diff --git a/report/figures/README.md b/report/figures/README.md index 5175234..770604b 100644 --- a/report/figures/README.md +++ b/report/figures/README.md @@ -13,12 +13,12 @@ practice this repository performs.** ## If a figure is ever added, these come back with it -**Restore `pyproject.toml` first** (from the template at `81a638e`), having asked -before adding the plotting dependency — `CLAUDE.md`'s working agreement, and the -scope clause of `fc-002`. Restore the `.reference/` import guard from -`tests/test_scaffold.py` with it, per `fc-003`: a figure script is the first -executable code here, and it is the first thing that could read a pinned checkout -from code instead of by eye. +**`pyproject.toml` and the guards are already back** — restored when +`tools/md2tex.py` landed (`od-007`), not for a figure. So the two steps this +section used to prescribe are done, and what remains for a figure is narrower: +**ask before adding the plotting dependency**, per `CLAUDE.md`'s working +agreement and `fc-002`'s scope. `pyproject.toml` is currently dependency-free and +that is a property worth not losing by accident. **Commit the generated PDF.** That is what lets `.github/workflows/report.yml` build with a TeX installation alone and no language runtime, which is the diff --git a/tests/test_scaffold.py b/tests/test_scaffold.py new file mode 100644 index 0000000..58d7023 --- /dev/null +++ b/tests/test_scaffold.py @@ -0,0 +1,109 @@ +"""Scaffold guards. Infrastructure -- these pin the repository's shape, not any +claim about the subject matter. + +RESTORED WITH tools/md2tex.py, in the order fc-003's scope required: "stops +holding the moment any .py file lands: restore the [pinned-clone] guard first, +before the others." That guard is first in this file for that reason and for no +other. The bracket is a substitution -- fc-003 names the directory literally and +this file may not, because the guard below scans this file for that string. + +WHAT IS NOT CARRIED BACK, and why, so nobody restores it out of habit: + +* the spikes/ import ban -- there is no spikes/ (fc-004); +* the package-imports guard -- there is no package (fc-003 stands for src/); +* the every-experiment-has-a-preregistration guard -- there are no experiments. + +Restoring a guard whose subject does not exist would make it pass by scanning an +empty set, which is the shape of a check that cannot fail. Two guards have a +subject here, and both are below. +""" + +import pathlib +import subprocess +import sys +import tempfile + +import pytest + +pytestmark = pytest.mark.infrastructure + +REPO = pathlib.Path(__file__).resolve().parents[1] + +# Built by concatenation rather than written literally, so the scan below can +# cover *this* file too instead of having to exempt its own needle. +REFERENCE_DIR = "." + "reference" + +# Directories whose code must stay free of it. There is no src/ and no +# experiments/ in this repository; tools/ and tests/ are where code lives. +GOVERNED = ("tools", "tests") + + +def _python_files(*roots: str) -> list[pathlib.Path]: + out: list[pathlib.Path] = [] + for root in roots: + base = REPO / root + if base.is_dir(): + out.extend(sorted(base.rglob("*.py"))) + return out + + +def test_nothing_reaches_into_the_reference_checkouts() -> None: + """The references are read by people, never by code. + + They are pinned clones of other repositories, they are git-ignored, and they + are absent in CI. Any code path that reads one passes locally and fails + everywhere else, which is the worst available failure mode -- and this + repository has now had that failure once for real, in the ledger check that + passed on Psych 3 and failed on Psych 4 (od-004). The guard is cheap and the + failure it prevents is not. + + THIS IS THE GUARD fc-003 NAMED AS THE ONE TO RESTORE FIRST. Every claim in + report/claims-verified.md is resolved against those clones by a person + reading them; the moment code reads one instead, the verification becomes + unreproducible for anyone who does not have them on disk. + """ + offenders: list[str] = [] + for path in _python_files(*GOVERNED): + for lineno, line in enumerate(path.read_text(encoding="utf-8").splitlines(), start=1): + if REFERENCE_DIR in line and "REFERENCE_DIR" not in line: + offenders.append(f"{path.relative_to(REPO)}:{lineno}") + assert not offenders, "code must not reference the pinned clones:\n" + "\n".join(offenders) + + +def test_typeset_sections_match_the_draft() -> None: + """report/sections/*.tex are generated from report/draft.md, and this proves it. + + THE CLAIM THIS TESTS WAS PREVIOUSLY A COMMENT. report/main.tex says the .tex + is generated from the draft and that the diff between them is where a mangled + figure shows. Nothing checked it, so an edit made directly to a .tex would + have silently broken the correspondence the whole draft/typeset practice + depends on -- and the next regeneration would have thrown that edit away + without telling anyone. + + It regenerates into a temporary directory and compares bytes. It does not + write to report/sections/, so a failing test never "fixes" itself. + """ + with tempfile.TemporaryDirectory() as tmp: + result = subprocess.run( + [sys.executable, str(REPO / "tools" / "md2tex.py"), "--out", tmp], + capture_output=True, + text=True, + ) + assert result.returncode == 0, f"conversion failed:\n{result.stdout}\n{result.stderr}" + + generated = sorted(pathlib.Path(tmp).glob("*.tex")) + committed = sorted((REPO / "report" / "sections").glob("*.tex")) + assert [p.name for p in generated] == [p.name for p in committed], ( + "the set of section files differs from what the converter produces" + ) + + stale = [ + g.name + for g, c in zip(generated, committed, strict=True) + if g.read_bytes() != c.read_bytes() + ] + assert not stale, ( + "report/sections/ is out of date with report/draft.md: " + + ", ".join(stale) + + "\nRun `make -C report sections` and commit the result." + ) diff --git a/tools/md2tex.py b/tools/md2tex.py new file mode 100644 index 0000000..49c45a8 --- /dev/null +++ b/tools/md2tex.py @@ -0,0 +1,276 @@ +"""Convert report/draft.md into report/sections/*.tex. + +WHY THIS FILE IS IN THE REPOSITORY, AND WHAT IT COST TO PUT IT HERE. +report/main.tex says the .tex files are generated from the draft and that the +diff between the two is where a mangled figure shows. That was true and +unverifiable at the same time: the converter lived outside the tree, so the +regeneration a reader is told to perform could not be performed by them. +Recorded as od-007 and closed by committing this. + +IT REOPENED TWO FORECLOSURES TO GET HERE. fc-003 anticipated exactly this -- +"stops holding the moment any .py file lands: restore the [pinned-clone] guard +first, before the others" -- and that order was followed. (The bracket is a +substitution: fc-003 names the directory literally, and this file may not, because +the guard it is quoting scans this file for exactly that string. The guard is +crude on purpose and prose bends around it, not the other way round.) fc-002 did not: its +scope named a GENERATED FIGURE as the trigger that would bring pyproject.toml +back, and the actual trigger was a converter. Both entries are annotated in +docs/state.yaml rather than reversed. + +RUN IT: make -C report sections + +IT IS NOT PART OF THE PDF BUILD, deliberately. report/sections/*.tex are +COMMITTED, so `make -C report` compiles from a clean checkout with a TeX +installation alone and no language runtime -- the property +.github/workflows/report.yml depends on. This script regenerates them; it never +runs during a build. Same arrangement report/figures/README.md prescribes for +figures, and for the same reason. + +DETERMINISTIC AND ENFORCED. tests/test_scaffold.py regenerates into a temporary +directory and asserts the result is byte-identical to what is committed, so "the +.tex is generated from the draft" is a tested claim rather than a comment. Edit +the draft, re-run this, commit both. + +STANDARD LIBRARY ONLY. No markdown or LaTeX dependency, so pyproject.toml can +stay dependency-free. +""" + +from __future__ import annotations + +import pathlib +import re +import sys + +REPO = pathlib.Path(__file__).resolve().parents[1] + +SPECIALS = { + "\\": r"\textbackslash{}", + "&": r"\&", + "%": r"\%", + "$": r"\$", + "#": r"\#", + "_": r"\_", + "{": r"\{", + "}": r"\}", + "~": r"\textasciitilde{}", + "^": r"\textasciicircum{}", +} + +SECTION_FILES = ( + "01-what-this-is", + "02-the-setting", + "03-the-project", + "04-how-it-goes-wrong", + "05-what-it-cost", + "06-where-it-failed", + "07-the-template", + "08-reflection", + "09-appendix-availability", +) + +# Characters that must never reach the output. Each one renders wrongly or not at +# all, and each has been seen doing so: the straight double quote shipped into a +# built PDF as two right-facing marks, inside a verbatim quotation. +FORBIDDEN = ( + ('"', "straight double quote (LaTeX renders it as two RIGHT marks)"), + ("\u201c", "unconverted left curly quote"), + ("\u201d", "unconverted right curly quote"), + ("\u2014", "unconverted em dash"), + ("\u2013", "unconverted en dash"), + ("\u00a7", "unconverted section mark"), +) + +HEADING_NUMBER = re.compile(r"^(?:Appendix\s+[A-Z]|\d+(?:\.\d+)?)\.?\s+") +HEADING = re.compile(r"^(#{1,3})\s+(.*)$") +BULLET = re.compile(r"^\s*[-*]\s+") +LEVELS = {1: "section", 2: "subsection", 3: "subsubsection"} + + +def escape(text: str) -> str: + return "".join(SPECIALS.get(char, char) for char in text) + + +def typography(text: str) -> str: + """Unicode and straight quotes to their LaTeX forms. + + Straight double quotes are paired here. LaTeX renders `"` as two RIGHT-facing + marks, so an opening straight quote points the wrong way -- and it does so + inside verbatim quotations, which is where it is most visible and least + excusable. Curly quotes were handled from the first version and straight ones + were not, because Markdown renders both identically and the exceptions were + invisible at the source. Pairing here leaves only a genuinely unbalanced + quote for the guard in `main` to catch. + """ + text = ( + text.replace("\u2014", "---") + .replace("\u2013", "--") + .replace("\u00a7", r"\S") + .replace("\u201c", "``") + .replace("\u201d", "''") + ) + return re.sub(r'"([^"]*)"', r"``\1''", text) + + +def inline(text: str) -> str: + """Inline markup. Code spans are stashed first, so nothing rewrites inside them.""" + code: list[str] = [] + + def stash(match: re.Match[str]) -> str: + code.append(match.group(1)) + return f"\0{len(code) - 1}\0" + + text = re.sub(r"`([^`]+)`", stash, text) + text = escape(text) + text = re.sub(r"\*\*(.+?)\*\*", r"\\textbf{\1}", text, flags=re.S) + text = re.sub(r"(? tuple[list[str], int]: + index = start + block: list[str] = [] + while index < len(lines) and ( + lines[index].startswith(">") + or ( + lines[index].strip() == "" + and index + 1 < len(lines) + and lines[index + 1].startswith(">") + ) + ): + block.append(re.sub(r"^>\s?", "", lines[index])) + index += 1 + + out = [r"\begin{quote}"] + para: list[str] = [] + for line in block: + if line.strip() == "": + if para: + out.append(inline(" ".join(para))) + out.append("") + para = [] + else: + para.append(line.strip()) + if para: + out.append(inline(" ".join(para))) + out.append(r"\end{quote}") + out.append("") + return out, index + + +def _itemize(lines: list[str], start: int) -> tuple[list[str], int]: + index = start + items: list[str] = [] + while index < len(lines): + if BULLET.match(lines[index]): + items.append(BULLET.sub("", lines[index]).strip()) + index += 1 + elif lines[index].startswith(" ") and lines[index].strip() and items: + items[-1] += " " + lines[index].strip() + index += 1 + else: + break + + out = [r"\begin{itemize}"] + out.extend(rf"\item {inline(item)}" for item in items) + out.append(r"\end{itemize}") + out.append("") + return out, index + + +def convert(markdown: str) -> str: + out: list[str] = [] + lines = markdown.split("\n") + index = 0 + while index < len(lines): + line = lines[index] + + if line.strip() == "---": + index += 1 + continue + + heading = HEADING.match(line) + if heading: + title = HEADING_NUMBER.sub("", heading.group(2).strip()) + out.append(rf"\{LEVELS[len(heading.group(1))]}{{{inline(title)}}}") + out.append("") + index += 1 + continue + + if line.startswith(">"): + block, index = _quote(lines, index) + out.extend(block) + continue + + if BULLET.match(line): + block, index = _itemize(lines, index) + out.extend(block) + continue + + if line.strip() == "": + out.append("") + index += 1 + continue + + para: list[str] = [] + while ( + index < len(lines) + and lines[index].strip() != "" + and not lines[index].startswith(">") + and not HEADING.match(lines[index]) + and lines[index].strip() != "---" + and not BULLET.match(lines[index]) + ): + para.append(lines[index].strip()) + index += 1 + out.append(inline(" ".join(para))) + out.append("") + + return re.sub(r"\n{3,}", "\n\n", "\n".join(out)).strip() + "\n" + + +def main(argv: list[str]) -> int: + source = (REPO / "report" / "draft.md").read_text(encoding="utf-8") + body = source[source.index("\n# 1.") :] + parts = [part for part in re.split(r"\n(?=# )", body) if part.strip()] + + if "--out" in argv: + outdir = pathlib.Path(argv[argv.index("--out") + 1]) + outdir.mkdir(parents=True, exist_ok=True) + else: + outdir = REPO / "report" / "sections" + + if len(parts) != len(SECTION_FILES): + headings = [part.split("\n")[0] for part in parts] + print( + f"expected {len(SECTION_FILES)} top-level sections, got {len(parts)}: {headings}", + file=sys.stderr, + ) + return 1 + + problems: list[str] = [] + for name, part in zip(SECTION_FILES, parts, strict=True): + rendered = convert(part) + # THE GUARD. This class of defect shipped once and was found by eye in a + # built PDF, which is the most expensive place to find it. An + # untranslated character is now a hard failure of the conversion. + for char, why in FORBIDDEN: + if char in rendered: + problems.append(f"{name}.tex: {rendered.count(char)} x {why}") + (outdir / f"{name}.tex").write_text(rendered, encoding="utf-8") + print(f"{name + '.tex':<32} {len(rendered):5d} bytes <- {part.split(chr(10))[0][:52]}") + + if problems: + print("CONVERSION FAILED, untranslated characters left in the output:", file=sys.stderr) + for problem in problems: + print(f" {problem}", file=sys.stderr) + return 1 + + print("guard: no straight quotes, curly quotes, dashes or section marks survived") + return 0 + + +if __name__ == "__main__": + raise SystemExit(main(sys.argv[1:])) From de575e4c8df0a34f47ada3fc9a27b93862884cdc Mon Sep 17 00:00:00 2001 From: Luiz Velho Date: Wed, 9 Sep 2026 00:08:25 -0300 Subject: [PATCH 10/11] Fix the code job: PEP 735 dependency groups, not a package install The `code` job added in the previous commit FAILED ON ITS FIRST CI RUN. It ran `pip install ".[dev]"`, which asks pip to build this directory as a package. There is no build backend by design, so setuptools was guessed at and rejected pyproject.toml: "`project` must contain ['version'] properties". IT PASSED LOCALLY, BECAUSE THE INSTALL STEP NEVER RAN THERE -- the tools were already in the environment. That is od-004's failure again, three commits later: local success meant nothing because the author's machine had already done the step CI has to do from scratch. THE FIX IS NOT A VERSION AND A BUILD BACKEND. That would manufacture the package pyproject.toml exists to avoid. [dependency-groups] (PEP 735) is the table for development tooling belonging to something that is not a package, and `pip install --group dev` never builds anything. Versions stay in pyproject.toml rather than the workflow, so local and runner cannot drift. THEN RE-RAN THE GATES AGAINST THE VERSIONS CI ACTUALLY INSTALLS -- ruff 0.16.6, mypy 2.3.1, pytest 9.1.1, all newer than the local ones -- rather than against the local ones again. All four pass. Checking locally alone would have made the same mistake a third time. Recorded on od-007. Co-Authored-By: Claude Opus 5 (1M context) --- .github/workflows/report.yml | 15 ++++++++++++--- docs/state.yaml | 20 ++++++++++++++++++++ pyproject.toml | 18 ++++++++++++++++-- 3 files changed, 48 insertions(+), 5 deletions(-) diff --git a/.github/workflows/report.yml b/.github/workflows/report.yml index 1b66926..03f275d 100644 --- a/.github/workflows/report.yml +++ b/.github/workflows/report.yml @@ -115,12 +115,21 @@ jobs: python-version: "3.13" cache: pip - # No project dependencies: tools/md2tex.py is standard library only, and - # pyproject.toml says so. Only the dev extra is installed. + # `--group dev`, NOT `pip install ".[dev]"`, AND THE FIRST VERSION FAILED + # HERE. Installing the directory asks pip to BUILD IT AS A PACKAGE; there + # is no build backend on purpose, so setuptools was guessed at and rejected + # pyproject.toml with "`project` must contain ['version'] properties". It + # passed locally because the tools were already installed and the install + # step never ran. PEP 735 dependency groups install tooling without + # building anything, which is exactly this case. `--upgrade pip` first, + # because --group needs pip 25.1 or newer. + # + # No project dependencies to install: tools/md2tex.py is standard library + # only, and pyproject.toml says so. - name: Install run: | python -m pip install --upgrade pip - pip install ".[dev]" + pip install --group dev - name: Lint run: ruff check tools tests diff --git a/docs/state.yaml b/docs/state.yaml index 8f62482..990d2b4 100644 --- a/docs/state.yaml +++ b/docs/state.yaml @@ -676,6 +676,26 @@ open_decisions: until now. Option 3 — dropping the claim and hand-maintaining the .tex — would have cost exactly the diffability the draft/typeset practice exists for, and it was the option that would have become true by default. + AND THE FIRST ATTEMPT AT THE GATES FAILED IN CI, which is worth the entry + because it is the same failure shape as od-004's. The `code` job ran + `pip install ".[dev]"`, which asks pip to BUILD THIS DIRECTORY AS A + PACKAGE. There is no build backend by design, so setuptools was guessed at + and rejected pyproject.toml: "`project` must contain ['version'] + properties". It passed locally because the tools were already installed and + the install step never ran there — LOCAL SUCCESS AGAIN MEANT NOTHING, + exactly as with the Psych 3/Psych 4 split in od-004, and for the same + reason: the author's machine had already done the step CI has to do from + scratch. + THE FIX IS NOT TO ADD A VERSION AND A BUILD BACKEND, which would + manufacture the package this file exists to avoid. It is + `[dependency-groups]` (PEP 735) with `pip install --group dev`, the table + for development tooling belonging to something that is not a package. The + versions stay in pyproject.toml rather than the workflow, so local and + runner cannot drift. + THE GATES WERE THEN RE-RUN AGAINST THE VERSIONS CI ACTUALLY INSTALLS — + ruff 0.16.6, mypy 2.3.1, pytest 9.1.1, all newer than the local ones — and + all four pass. Checking against the local versions alone would have + repeated the mistake a third time. COSTS PAID, recorded so the trade is legible: the script had to be rewritten to pass the gates it was arriving under (it was scratch-quality — semicolons, percent formatting), and its output was verified byte-identical diff --git a/pyproject.toml b/pyproject.toml index 4c8e512..e7b4187 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -23,9 +23,23 @@ # (CLAUDE.md's working agreement), and this is the file where the asking would # become visible in a diff. -requires-python = ">=3.12" +# DEPENDENCY GROUPS (PEP 735), NOT `[project.optional-dependencies]`, AND THE +# REASON IS THE FAILURE THAT PUT THEM HERE. The first version used an optional +# dependency and CI installed it with `pip install ".[dev]"` -- which asks pip to +# BUILD THIS DIRECTORY AS A PACKAGE. There is no build backend, deliberately, so +# setuptools was guessed at and rejected the file: "`project` must contain +# ['version'] properties". It passed locally, where the tools were already +# installed and nothing ever ran the install step. +# +# The fix is not to add a version and a build backend -- that would manufacture +# the package this file exists to avoid. `[dependency-groups]` is the table for +# exactly this case: development tooling for something that is not a package. +# Installed with `pip install --group dev`, which never builds anything. +# +# The versions stay HERE rather than in the workflow, so a local environment and +# the runner install the same set and neither can drift ahead of the other. -[project.optional-dependencies] +[dependency-groups] dev = [ "pytest>=8.0", "mypy>=1.11", From d8c70524f464f992d080ae956a6b2a258f8a3de3 Mon Sep 17 00:00:00 2001 From: Luiz Velho Date: Wed, 9 Sep 2026 00:19:49 -0300 Subject: [PATCH 11/11] Close A6 and A7, and the restatement neither verdict named A6 -- "THREE passages in the instance quote specification text" -> FOUR, with the list rebuilt. exp007/findings.md:46 is dropped: it records what the specification did NOT state, which is an absence and not a quotation. exp007/findings.md:123 and :32 are added, the first being the clearest instance of the section's own thesis. All four re-read at 5f8e39f before the sentence was written, and each characterisation checked against its text. A7 -- "Every one is text that an artifact was arguing with" -> "had to reckon with -- mostly to depart from it, once to confirm it." exp007/findings.md:32 agrees with its specification rather than arguing: "The specification asked for this to be visible rather than inferred, and it is." AND A THIRD SENTENCE, WHICH HAD NO VERDICT OF ITS OWN. The paragraph after A7 restated the same overreach in different words -- "reaches the record only where a pre-registration or a findings file had to DISAGREE with it in order to justify doing something else". A7's verdict named one sentence; the defect was in two, sixty words apart, in the same subsection. Corrected to "had to ACCOUNT FOR it -- usually to justify departing from it, once to note that it was met", which keeps the mechanism and drops the false universal. THE MECHANICAL CHECK COULD NOT HAVE FOUND THAT ONE. It repeats an IDEA, not a number, and every word in the restatement differs. Recorded in claims-verified.md as a practice: when a characterisation is corrected, read forward for its restatements, exactly as a corrected figure is grepped for its other occurrences. The section's conclusion is unchanged. The load-bearing condition was always that the specification had to be EXECUTED and produce an artifact, and that holds for all four passages; "arguing with" was a narrower mechanism than the conclusion needed. ALSO FIXED, AND IT WAS MY SLIP: the draft status block and main.tex still said the section's "five checkable claims" were "all confirmed, recorded as A1-A5". That was written for the first version of the section and never updated when the revision took it to seven claims with two non-confirmed. Both now say A1-A7, five confirmed, one corrected, one under-qualified, both since closed. gap-004 already carried the corrected count of four, so the ledger and the prose now agree. Before this change they did not, and the ledger was right. 23 pages, zero boxes, transit lossless, all gates green. Co-Authored-By: Claude Opus 5 (1M context) --- report/claims-verified.md | 41 ++++++++++++++++++++++++++ report/draft.md | 28 ++++++++++-------- report/main.tex | 5 ++-- report/sections/06-where-it-failed.tex | 4 +-- 4 files changed, 62 insertions(+), 16 deletions(-) diff --git a/report/claims-verified.md b/report/claims-verified.md index 8153bc8..1cb2642 100644 --- a/report/claims-verified.md +++ b/report/claims-verified.md @@ -569,6 +569,12 @@ claims: **five CONFIRMED, one CORRECTED, one UNDER-QUALIFIED.** Numbered A1–A7 and kept out of `mn-004`'s tally, which covers §§1–8.5 as they stood at the first pass. +> **BOTH NON-CONFIRMED CLAIMS ARE NOW CLOSED IN THE PROSE** (A6, A7), together +> with a third sentence neither verdict named — see *Resolutions*. The section +> stands as verified throughout. The verdicts below are kept as written, with the +> superseded wording quoted, because the disagreement between what was drafted +> and what the record supports is the information. + ### A1. "§4.2 describes three specifications rejected… *the clearest case in the project of a rule paying for itself*" **CONFIRMED.** Quoted accurately from §4.2. @@ -620,6 +626,10 @@ instruction and then arguing it is wrong, a findings file recording a deviation and the clause it deviates from, and one recording that the specification asked for something that was duly delivered." +**APPLIED 2026-09-09**, verbatim. All four passages were re-read at +`5f8e39f` before the sentence was written, and each characterisation checked +against its text. + ### A7. "**Every one is text that an artifact was arguing with.**" **UNDER-QUALIFIED — true of three of the four, and the fourth is a counterexample.** - Arguing: `exp007/preregistration.md:38` (declares a design change), @@ -641,6 +651,17 @@ universal. **Proposed:** "Every one is text that an artifact had to reckon with — mostly to depart from it, once to confirm it." +**APPLIED 2026-09-09**, verbatim — **and it was not enough on its own.** The +paragraph that follows restated the same overreach in different words: +*"reaches the record only where a pre-registration or a findings file had to +**disagree** with it in order to justify doing something else."* A7's verdict +named one sentence and the defect was in two. Corrected to "had to **account +for** it — usually to justify departing from it, once to note that it was met", +which keeps the mechanism and drops the false universal. **This is the +repeated-claim rule applying to a characterisation rather than a figure**, and +nothing in the mechanical check would have caught it: it repeats an idea, not a +number. + --- # Resolutions @@ -680,6 +701,26 @@ wholesale approval. They touched **20 passages**, because five claims recur: For 41 the second option — record the four gaps in the template and cite them — is not available from this repository. +### §6's addendum: A6 and A7, closed 2026-09-09 + +| claim | was | resolution | +|---|---|---| +| **A6** | "**Three** passages in the instance quote specification text", listing one that records an *absence* rather than a quotation | Applied. **Four** passages, re-read at `5f8e39f` and re-characterised individually. `exp007/findings.md:46` dropped from the list; `exp007/findings.md:123` and `:32` added. | +| **A7** | "Every one is text that an artifact was arguing with" | Applied. "…had to **reckon with** — mostly to depart from it, once to confirm it." | +| **(unnumbered)** | "reaches the record only where a pre-registration or a findings file had to **disagree** with it" | **Corrected in the same change, and it had no verdict of its own.** The next paragraph restated A7's overreach in different words. Now "had to **account for** it — usually to justify departing from it, once to note that it was met". | + +**The third row is the one worth keeping.** A7's verdict named one sentence; the +defect was in two, sixty words apart, in the same subsection. **The mechanical +repeated-figure check could not have found it** — it repeats an *idea*, not a +number, and every word in the restatement differs. The only thing that finds this +class is applying a correction and then re-reading the paragraph around it for +the same claim in other words. That is now the practice: **when a +characterisation is corrected, read forward for its restatements**, exactly as a +corrected figure is grepped for its other occurrences. + +`gap-004` already carried the corrected count of four, so the ledger and the +prose now agree; before this change they did not, and the ledger was right. + ### Still standing: the six UNSUPPORTED claims **12, 17, 20, 25, 38, 45 are unchanged in the prose.** They carry no proposed diff --git a/report/draft.md b/report/draft.md index 3de6913..f7bab7c 100644 --- a/report/draft.md +++ b/report/draft.md @@ -27,8 +27,9 @@ > search as recollection, not record. > > §6's *The method destroys its own best evidence* was written after that pass -> and its five checkable claims were verified before insertion — all confirmed, -> recorded as A1–A5. +> and checked separately: **seven claims, A1–A7 — five confirmed, one corrected, +> one under-qualified.** Both non-confirmed claims have since been corrected in +> the prose, so the section now stands as verified throughout. > > Sections 8.6, 8.7 and 8.8 are the first author's, written in the first person. > They make no checkable factual claims and are out of scope for that pass. @@ -967,16 +968,19 @@ file in the instance is one, and none is named as one. The work a specification produces lands in a branch; the specification itself lives in a chat window and a paste buffer. -Fragments do survive, and where they survive is the finding. Three passages in -the instance quote specification text — a pre-registration repeating a clause -before narrowing it, another quoting an instruction and then arguing it is -wrong, a findings file recording a reading *"which neither exp005 nor this -specification stated."* **Every one is text that an artifact was arguing with.** - -That is the whole mechanism. Specification text reaches the record only where a -pre-registration or a findings file had to disagree with it in order to justify -doing something else — which requires the specification to have been **executed** -and to have produced an artifact in the first place. A specification rejected +Fragments do survive, and where they survive is the finding. Four passages in +the instance report specification text — a pre-registration quoting a clause +before departing from it, another quoting an instruction and then arguing it is +wrong, a findings file recording a deviation and the clause it deviates from, +and one recording that the specification asked for something that was duly +delivered. **Every one is text that an artifact had to reckon with — mostly to +depart from it, once to confirm it.** + +That is the whole mechanism, and the fourth passage is what makes it precise. +Specification text reaches the record only where a pre-registration or a findings +file had to account for it — usually to justify departing from it, once to note +that it was met — which in either case requires the specification to have been +**executed** and to have produced an artifact in the first place. A specification rejected before work begins produces no artifact, so nothing is ever in a position to quote it. **The survival mechanism is parasitic on the work happening, which is exactly why it cannot preserve a rejection.** diff --git a/report/main.tex b/report/main.tex index 0c863eb..65ff0ad 100644 --- a/report/main.tex +++ b/report/main.tex @@ -134,8 +134,9 @@ cold-session search as recollection, not record. \S6's \textit{The method destroys its own best evidence} was written after that -pass and its five checkable claims were verified before insertion --- all -confirmed, recorded as A1--A5. +pass and checked separately: \textbf{seven claims, A1--A7 --- five confirmed, one +corrected, one under-qualified.} Both non-confirmed claims have since been +corrected in the prose, so the section now stands as verified throughout. Sections 8.6, 8.7 and 8.8 are the first author's, written in the first person. They make no checkable factual claims and are out of scope for that pass. diff --git a/report/sections/06-where-it-failed.tex b/report/sections/06-where-it-failed.tex index 265badd..65104c4 100644 --- a/report/sections/06-where-it-failed.tex +++ b/report/sections/06-where-it-failed.tex @@ -30,9 +30,9 @@ \subsection{The method destroys its own best evidence} The reason is structural. A specification is written on one surface, relayed by a human, and executed on another. \textbf{No specification was ever committed.} No file in the instance is one, and none is named as one. The work a specification produces lands in a branch; the specification itself lives in a chat window and a paste buffer. -Fragments do survive, and where they survive is the finding. Three passages in the instance quote specification text --- a pre-registration repeating a clause before narrowing it, another quoting an instruction and then arguing it is wrong, a findings file recording a reading \textit{``which neither exp005 nor this specification stated.''} \textbf{Every one is text that an artifact was arguing with.} +Fragments do survive, and where they survive is the finding. Four passages in the instance report specification text --- a pre-registration quoting a clause before departing from it, another quoting an instruction and then arguing it is wrong, a findings file recording a deviation and the clause it deviates from, and one recording that the specification asked for something that was duly delivered. \textbf{Every one is text that an artifact had to reckon with --- mostly to depart from it, once to confirm it.} -That is the whole mechanism. Specification text reaches the record only where a pre-registration or a findings file had to disagree with it in order to justify doing something else --- which requires the specification to have been \textbf{executed} and to have produced an artifact in the first place. A specification rejected before work begins produces no artifact, so nothing is ever in a position to quote it. \textbf{The survival mechanism is parasitic on the work happening, which is exactly why it cannot preserve a rejection.} +That is the whole mechanism, and the fourth passage is what makes it precise. Specification text reaches the record only where a pre-registration or a findings file had to account for it --- usually to justify departing from it, once to note that it was met --- which in either case requires the specification to have been \textbf{executed} and to have produced an artifact in the first place. A specification rejected before work begins produces no artifact, so nothing is ever in a position to quote it. \textbf{The survival mechanism is parasitic on the work happening, which is exactly why it cannot preserve a rejection.} Five of the six unsupported claims in this document describe events the workflow guarantees will leave no trace. One rejection survives in the record, and only incidentally: it reshaped a decision that was taken anyway, so the decision's entry carries a fragment of it. That fragment is not even a stale-state rejection, which is the class \S4.2's claim was about.