feat(tools): backlog-lint — gating consistency checks for the issue backlog

[jcschaff]

Nine mechanical consistency checks over the open backlog and
project board #1, wired to a weekly Action.

Follow-on from #1996 (the backlog triage docs), but independent of it — this branches from
master and does not depend on that PR merging.

What it checks

Facts only. Every rule is decidable from metadata alone; nothing here decides whether an issue
matters.

id	Fails when
not-on-board	An open issue is on no project board
priority-formula	Priority ≠ Importance + Simplicity
scored-not-ranked	Importance set, Priority never computed
half-scored	Simplicity set, Importance not
done-but-open	Board says Done, issue still open
stale-active	Active and untouched 30+ days
shipped-release-label	Carries a label naming a shipped release
many-assignees	3+ assignees, so nobody owns it
queued-thin-body	Queued with a body under 200 chars

It gates, it does not report

A scheduled job that posts a report gets ignored — this repo already learned that with the BMDB
nightly, which was changed to fail on changed results after the report version went unread.

So findings are compared against a committed baseline (tools/backlog-lint/baseline.json) and the
run fails only on findings not in it. Accepting a new violation means regenerating the
baseline, reading the diff, and committing it — a deliberate act, not a silent drift.

The baseline shrinking is grooming getting done. The baseline growing is someone deciding to
accept something.

Verified before committing, not assumed

• Reproduces the hand counts exactly — 55 unboarded, 53 half-scored, 4 done-but-open, 15
  stale-active, 45 shipped-release labels, 33 many-assignees, 28 thin Queued bodies. 233
  findings total, which is now the baseline.
• priority-formula and scored-not-ranked report 0, independently confirming the five board
  score corrections landed.
• Negative control: un-accepting two baseline entries makes it exit 1 and name exactly those
  two; restoring makes it exit 0 again.
• Failure paths exit non-zero rather than passing silently — bad token → 1, missing token → 2,
  and the workflow fails with an explicit message when the secret is absent.

Before this can run: one secret

GH_PROJECT_TOKEN — a PAT with read:project scope. GITHUB_TOKEN cannot read
organization ProjectV2 boards, so there is no way around this. Until the secret exists the
workflow fails loudly on purpose, rather than reporting a false pass.

Add project scope too only if you want to use the fix_board toggle.

Scheduling

Mondays 08:00 UTC, plus workflow_dispatch (with strict and fix_board toggles), plus PRs
touching tools/backlog-lint/** so the tool is exercised by its own changes.

What it deliberately does not do

• Detect duplicates. I built and tested this, and it does not work well enough to ship:
  TF-IDF similarity over title+body found only 5 of 12 hand-identified duplicate pairs, missed
  the two that matter most (#1008/#1199 ranked 579th, #1604/#1606 ranked 9377th),
  and its highest-scoring hit was a deliberate split (#1046/#1069). Lexical similarity finds
  sibling issues in a subsystem, which is a different thing.
• Close anything or write issue bodies. Generating repro steps for a four-year-old empty issue
  produces confident prose that is really invention. A blank body is honestly blank.
• Assign Importance. That is a judgment about users and belongs to people.

The single write it can perform is adding unboarded issues to the board, opt-in behind
--fix-board / the fix_board input, off by default.

A note on the scoring model

tools/backlog-lint/README.md documents Priority = Importance + Simplicity, which existed
nowhere but in the board data (exact on 56 of 57 ranked issues). Worth considering making
Priority a computed column — hand-entry is what let #1495 drift out of sync.

🤖 Generated with Claude Code

https://claude.ai/code/session_018kr8SbzXtwW3gMVUgMfDDt