Skip to content

Three of the six README line-number citations are outside the only gate that checks them, and all three had drifted #960

Open

Description

@tannevaled

Three of the six README line-number citations are outside the only gate that checks them — and all three had drifted

TestReadmeStubbedHelpersMatchTheShim derives the stubbed-helper list from spec_helper.rb and checks the line numbers in the conformance README's table. It is a good gate and it works: when #843's change inserted 26 lines into the shim, it went red for exactly the three citations it covers, and those three were correct beforehand.

The population is the problem. Six citations into spec_helper.rb / run.sh exist across the two READMEs, and three are outside the gate. Measured while fixing #843:

citation where in the gate's population state before #843's change
output_to_fd → spec_helper.rb:280 conformance README table yes correct
ruby_exe → spec_helper.rb:904 conformance README table yes correct
ruby_cmd → spec_helper.rb:905 conformance README table yes correct
ruby_exe/ruby_cmd → spec_helper.rb:882-883 README.md, "The shim is not MSpec" bullet no ⛔ stale by 22 lines — they were at :904-905
ComplainMatcher → spec_helper.rb:204-249 conformance README, complain section no ⛔ stale on its end — the class ended at :236
the swept trees → run.sh:106 README.md, "Only language/ and core/ are swept" no unverifiable by hand; find was at :106, correct

Three in the gate: three correct. Three outside it: two already wrong. That is not a coincidence to argue about — it is what an unchecked citation does over time.

⛔ The blind spot is documented at the exact spot that created it

readme_conformance_prose_test.go:231-232:

// A bare helper name only: `spec_helper.rb:882-883` and the like do not match,
// so a citation is never mistaken for a claim.
helperName := regexp.MustCompile("`([a-z_][a-z0-9_]*[?!]?)`")

The exclusion is right for what that loop does — it is deciding which helpers the bullet calls stubbed, and a line citation is not such a claim. The gap is that nothing else then looks at the citation as a citation. The very string chosen as the example of what to skip, spec_helper.rb:882-883, was already wrong when that comment was written.

What a gate would check

A line number alone is not checkable; a line number plus the name beside it is. Every one of the six citations sits next to the thing it points at, so the check is derivable with no line number typed into the test:

  • read the citation's number out of the README (so the README stays the source of the claim),
  • assert the cited line of the cited file contains the named symbol (so the file stays the source of truth),
  • for a range, assert the first line opens the named construct and the last closes it.

Worked examples of each shape: `output_to_fd` (`:306`) → line 306 must contain def output_to_fd; `ruby_exe` and `ruby_cmd` (`spec_helper.rb:930-931`) → 930 contains def ruby_exe, 931 contains def ruby_cmd; `ComplainMatcher` at `spec_helper.rb:230-262` → 230 contains class ComplainMatcher and 262 is its closing end; `run.sh:118` → line 118 contains the find over language and core.

That also catches the shape no current test can: a citation whose number is plausible — pointing at some other line of the same file — which is exactly how :882-883 survived.

Not in scope

The :280-style shorthand (a bare :N whose file comes from the previous mention in the same sentence) needs the parser to carry the last-named file forward. It is one of the six, so it has to be handled, but it is the only fiddly part.

Refs #913 — the merged pull request that built this gate, deriving the conformance prose from the table and the shim. Nothing here says it was wrong: it chose the table as its population, and the table is exactly what it keeps honest. This is the complement it did not claim.

Refs #843 — which corrected all six citations to their measured values (run.sh:106→:118, output_to_fd :280→:306, ruby_exe :904→:930, ruby_cmd :905→:931, ComplainMatcher :204-249→:230-262) and stays closed; this is the gate that would have caught the two that were wrong before it.

🤖 Generated with Claude Code

Prior art: priorart "README line-number citations into spec_helper.rb drift unchecked by any gate" at 2026-11-20 08:32 CET reported prior art; checked: priorart on README line-number citation drift: 1153 entries, 10 branches. #913 (merged) is the pull request that BUILT this gate and chose the table as its population; #955 (merged) is the same family for the spec-file counts. No open entry covers the three citations outside that population, which is what this files.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions