Skip to content

Make a ledger rule that explains nothing say why (#372) - #373

Merged
derek73 merged 16 commits into
masterfrom
claude/issue-372-dormant-rules
Aug 12, 2026
Merged

Make a ledger rule that explains nothing say why (#372)#373
derek73 merged 16 commits into
masterfrom
claude/issue-372-dormant-rules

Conversation

@derek73

@derek73 derek73 commented Aug 12, 2026

Copy link
Copy Markdown
Owner

Closes part of #372: a rule that stops explaining anything is invisible. Reverting a fix leaves its rule inert and the run exits 0 — demonstrated in the issue, where reverting #370's fix failed 5 unit tests but produced no ledger or differential failure at all.

[[change]] rules gain an optional dormant = "<reason>". Two tiers fail without it:

  • statictests/v2/test_ledger_guards.py fails in CI when a rule's name_regex reaches no corpus name. No baseline wheel needed, so it is cheap and early.
  • dynamiccompare.py fails the run when a rule explained no diff. Always on, not behind a flag: a check nobody passes a flag to is a check that does not exist.

Both directions fail — a rule declaring dormant that explains a diff is a false statement in the ledger, and is reported too.

When a rule explains nothing the harness says which kind, because the three have three different fixes: reverted (matched no diffing name), shadowed by <issue> (an earlier rule claimed every diff it would have), or refused by a [[never]] exclusion.

The two rules that legitimately explain nothing today — fix(suffix-delimiter-rendering) and the ambiguous-surname-acronym data change — now carry their reason.

Measured

Gates unmoved: 108 intentional / 0 unexplained at 1.4.0, 1 / 0 at 2.1.0, both exit 0. Suite 3214.

Every guard was demonstrated failing by mutation, on its own assert:

Mutation Result
Drop a dormant declaration static guard fails, naming the rule
Point a live rule's regex at nothing harness exits 1, reports reverted
Declare a firing rule dormant harness exits 1, reports no longer dormant
Drop the dormancy terms from main()'s return new wiring tests fail
Delete both report loops new wiring tests fail

The last two exist because review found the wiring commit had no regression coverage: reverting the exit-code fold left all 116 tests green. dormant_rules() was well unit-tested as a pure function while the composition was not — the gap _run_main's own docstring was written about.

dormant_rules() is pure over (rules, explained, diffing) so pytest exercises all three diagnoses without a wheel. It sorts internally: main() classifies against _sorted_rules, and a diagnosis computed against a different order would name a rule that did not actually win.

One pre-existing test fixture had a permanently shadowed rule, which the new check correctly flagged; it is declared dormant so the test keeps pinning sort order alone.

Deliberately out of scope

#372 stays open for these:

  • Ambiguity reporting. Measured: 28% of claimed (name × role) pairs already have two or more matching rules — 432 of them fix(comma-family) under fix(suffix-routing) alone. A report firing on 28% of what it inspects is wallpaper. The shadowed diagnosis above delivers the slice that costs something.
  • A specificity floor for fields-only rules. There is exactly one such rule in any ledger, naming 3 of 7 roles. The check would fire on nothing, and nothing would reveal it was vacuous.
  • Narrowing fix(suffix-routing) / fix(comma-family). Needs this detector first, to have a success criterion at all: "did narrowing wake a shadowed rule?" is answerable, "does this rule describe that diff?" is judgement.

Known gap, documented

expected_since_2.0.0.toml is never run by the release checklist, so its rules get static coverage only — and the static tier exempts on the mere presence of dormant without checking the reason is still true. A wrong dormant is caught in the ledgers the checklist runs and is unauditable in the ones it does not. Recorded in tools/differential/README.md; closing it means widening the checklist or sweeping every ledger.

🤖 Generated with Claude Code

derek73 and others added 9 commits August 12, 2026 02:29
Adds dormant_rules(), a pure function taking rules, the set of issues
that explained a diff, the diffing (name, fields) pairs, and optional
exclusions. It distinguishes three reasons a rule can explain nothing
(reverted, shadowed, excluded), reports rules that should have declared
`dormant` but did not, and reports declared-dormant rules that woke up.

Sorts rules internally with _sorted_rules before classifying against
them, so the shadower it names matches what main()'s own classify()
call would pick regardless of the order the caller passes; and among
multiple rules claiming a dormant rule's matched names, reports the
most frequent claimant (Counter) rather than the alphabetically first
one, so the diagnosis points at the rule actually doing the shadowing.

Nothing calls it yet -- wiring into main() is a later task.
Wires dormant_rules into main(): collect every diffing name and its
diff fields, ask which declared-dormant rules woke up and which
undeclared rules explained nothing, and fold both into the exit code
alongside unexplained diffs. A rule that stops matching real output is
as broken a contract as a diff nothing explains.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Wiring dormant_rules into main() (previous commit) makes
test_main_sorts_a_name_regex_rule_ahead_of_a_fields_only_one's
'broad' rule -- permanently shadowed by 'specific' in that one-name
fixture, by construction -- get reported as EXPLAINED NOTHING, which
broke its `"broad" not in out` assertion. Declaring it dormant with a
reason keeps the test pinning sort order alone rather than sort order
and the new dormancy report at once.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
897944d wired dormant_rules into main() but nothing exercised it
end-to-end: dropping the dormancy terms from the return, or deleting
the EXPLAINED NOTHING / NO LONGER DORMANT print loops, left every test
in this file green. Add two _run_main scenarios covering both
directions -- a rule that explains nothing, and a declared-dormant
rule that wakes up -- each failing the run and naming the rule.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@derek73 derek73 self-assigned this Aug 12, 2026
@codecov

codecov Bot commented Aug 12, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.51%. Comparing base (e6a9085) to head (44d57ae).
⚠️ Report is 17 commits behind head on master.

Additional details and impacted files
@@           Coverage Diff           @@
##           master     #373   +/-   ##
=======================================
  Coverage   98.51%   98.51%           
=======================================
  Files          44       44           
  Lines        2895     2895           
=======================================
  Hits         2852     2852           
  Misses         43       43           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

derek73 and others added 7 commits August 12, 2026 15:10
The duplicate check tested isinstance twice in one loop, once to decide
whether to compare and again to decide whether to record. Guard and
continue instead.

The skip is not a silent pass: a non-string `issue` is rejected by the
per-rule loop immediately below, with a message that names the rule
number. Deciding it here would only mean saying it worse.

No behavior change -- verified both paths: a duplicate still exits with
"two rules sharing the issue", and issue=5 still exits with "rule #1 has
no string 'issue'".

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@derek73
derek73 merged commit e0d2123 into master Aug 12, 2026
9 checks passed
@derek73
derek73 deleted the claude/issue-372-dormant-rules branch August 12, 2026 22:21
derek73 added a commit that referenced this pull request Aug 24, 2026
The review round found the first draft shipped the inverse of the bug it
fixed. It let ANY piece open an entry, as the tail block always had --
safe there, because assign routes every tail piece to SUFFIX, which is
what `tail` means, and wrong off it, where a title piece routes to TITLE.

Two failures, one cause, neither visible to the gates that passed. The
`joined` tag is role-BLIND and the facade heals it for every role:

    "Smith, Rev. Dr."     title_list  ['Rev.','Dr.'] -> ['Rev. Dr.']
    "Smith Jr., Mr. Jr."  suffix      'Jr., Jr.'     -> 'Jr. Jr.'

The second glues a suffix backward across a comma the writer typed --
exactly what #429 exists to stop. The differential compares strings and
cannot see the first; the case table asserts the title STRING, which is
space-joined either way, and could not see it either.

Two joins that had been one, separated: WITHIN a piece the tag renders a
merged piece as one unit whatever role it holds; BETWEEN pieces it
continues an entry, and only a piece rendering into the same run may do
that. Sticky across a piece that is not in the entry, so an interleaved
title does not split its run ("Smith, MD Dr. PhD" -> 'MD PhD'); a
delimiter core still closes it.

Eight case rows and a facade test for the list views, which is the only
surface that shows the title collapse. Both regression guards verified
against a mutation copy -- they fail with the old condition restored.

Prose corrections, all measured by the reviewers:

- The round-trip claim was false AND backwards: str() of a fixed parse
  is a no-comma string, which re-parses with the comma back. master was
  the str-stable one. Struck from the release log and the case note.
- "one-word family comma" is not the condition -- there is no word-count
  gate, so "John Smith, Jr. III" moves too (1.4.0's reading), as does a
  title-led "Smith, Dr. MD PhD". Scope restated as it reads.
- The delimiter parity is #206 (021823e, "Apply suffix_delimiter only at
  suffix-consumption sites"), NOT #191, the German/Dutch vocabulary PR.
  Three code comments carried the error; corrected with it.
- The dormant-rule tell is #373's, and #426 the precedent for dropping a
  shadowed rule -- neither #424 entry mentions it.
- "boundary example" in the entry and both ledgers: the example FIRES,
  which is why the annotation came off.
- "filed rather than folded in" claimed an issue that does not exist.

C1 gains `_group.py` in `implemented:`, with the verbatim citation the
equality guard requires -- the whole-run half of the rule renders here.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant