Repository navigation
The release checklist runs three baselines against four rule-bearing ledgers, so expected_since_2.1.0.toml is audited by nothing #503
Description
Rationale
The release checklist in root AGENTS.md opens with:
Run the differential gate at EVERY baseline that has a ledger with rules, and
confirm each exits 0.
and then gives three copy-pasteable commands against four rule-bearing
ledgers. DEFAULT_BASELINE = "2.2.0", so the three cover 1.4.0, 2.0.0 and 2.2.0.
--baseline 2.1.0 is reached by no line.
Measured 2026-09-03: all four ledgers carry rules — 75 / 66 / 59 / 1 [[change]]
entries at 1.4.0 / 2.0.0 / 2.1.0 / 2.2.0. So expected_since_2.1.0.toml and its
59 rules are audited by nothing a releaser executes.
What that ledger misses
Three checks are per-ledger and run only inside compare.py, so a baseline
nobody runs gets none of them:
- dormancy — a rule that has stopped explaining anything, or a
dormant
declaration that has stopped being true; - over-declaration (No ledger rule's reach is bounded: a name_regex can claim most of the corpus #452) — a rule whose declared
fieldsexceed the union
of the diffs it explains; - the recorded-shape checks (Measured claims in the differential comments go stale unnoticed (a reviewer read "751 names" as fact; the corpus holds 1116) #497) —
MOVED SHAPE, the departed-name
refusal, and theNOT CHECKEDnote.recorded = _RECORDED_DIFFS.get(ledger.name, {})
is strictly per-ledger.
The last one is why this is being filed now rather than left as a comment. #497
added a check whose whole purpose is that a recorded value nobody re-measures
agrees with itself forever, and at 2.1.0 nobody re-measures it.
Why it is latent today, and what arms it
expected_since_2.1.0.toml's _RECORDED_DIFFS section is empty, so the shape
check has nothing to miss there right now. #501 is open on whether the 2.x
sections stay empty — it records six contested diffs across the two 2.x ledgers
with no pins, one of them at 2.1.0. The first 2.1.0 row to land would be checked
by nothing in the documented process.
The dormancy and over-declaration gaps are live now, not latent.
Options
- Add the fourth command. One line in the checklist. It is the obvious fix
and it costs a sub-second run — measured, a whole gate run is ~0.35s at 1.4.0
and ~0.57s at 2.2.0, and all four back to back are ~2s. - Make the checklist derive the list rather than enumerate it — loop the
expected_since_*.tomlglob, so a fifth ledger at the next release is covered
without anyone remembering. Slightly more to paste; cannot go stale. - Leave it and say so. If 2.1.0 is deliberately out of the release audit,
the "EVERY baseline that has a ledger with rules" line should say so instead,
since it currently promises what the list does not do.
Option 2 is the one that matches what the codebase does elsewhere — the corpus
glob in compare.py is deliberately a glob rather than a list, with the comment
"a corpus that has to be asked for by name is one that stops being run".
Recompute
grep -c '^\[\[change\]\]' tools/differential/expected_since_*.toml
grep -n 'baseline' AGENTS.md
python -c "import re;print(re.search(r'DEFAULT_BASELINE = \"([^\"]+)\"',open('tools/differential/compare.py').read()).group(1))"
Provenance
Found by a design-docs-reviewer pass on #502 (PR for #497). AGENTS.md
carries a note recording the gap; this issue is what gives it an owner, and the
note cites it.
Activity
docs(agents): the unreachable ledger gets an owner (#503)
review round: PR #504 findings across the tool, its tests and the che…
docs(agents): the release checklist derives its baselines from the le…
docs: review round on the watched-shapes arc and the checklist record