ci(dev): deterministic gates for doc/tree drift, test coverage, and a shared toolchain

[potiuk]

Four commits, each a step further into the same problem: things that should
track each other, don't, and nothing notices.

1. check-doc-sync — deterministic gates for doc/tree drift

#1159, #1162 and #1163 fixed documentation that had drifted from the tree. Every
instance was found by reading. This makes five of them mechanical, as one prek
hook:

Check	Guards	What it would have caught
Spec index	every tools/spec-loop/specs/*.md listed in both overview.md and README.md	10 specs absent from both, one added four PRs earlier
Family counts	README.md family table vs live family: frontmatter	security 12→15, repo-health 6→7
Mode counts	docs/modes.md Modes at a glance vs live mode: frontmatter	Triage 34→35, Drafting 9→11
Catalogue totals	bare "N skills" in docs/setup/marketplaces.md	71→74, three occurrences
Dev-script docs	every script in tools/dev/ named in its README	3 undocumented scripts, on first run

The validator's existing modes-doc rule looks like it covers the mode counts.
It does not — _read_mode_table() parses the mode and status columns only and
never reads the count, so those numbers had no guard at all.

2. tools/dev under the workspace checks, with tests

The scripts in tools/dev guard everyone else's changes and were the one Python
tree nothing checked: the workspace checks iterate [tool.uv.workspace] members
and tools/dev was not one, so ruff, mypy and pytest all skipped it.

20 tests for check-doc-sync.py, written around the red case — each builds a
miniature repo in tmp_path, asserts the drift is reported with both numbers,
then that the corrected tree is silent. A gate that cannot fail is worse than no
gate: it reads as evidence while measuring nothing. Also covered: a spec listed
in only one index still counts as missing, a README row naming no family is not
read as a family with a wrong count, a mode nothing declares must say 0 rather
than be skipped, every stale total on a page is reported rather than just the
first, and the totals check stays inside its allowlist.

Bringing the tree under the checks surfaced four lint findings in pre-existing
scripts. Three are fixed; RUF003 is ignored with a reason — the repo's prose
style uses en dashes throughout, and rewriting comments to hyphens to satisfy a
homoglyph check would make them worse. Every script was re-run afterwards to
confirm behaviour is unchanged.

3. Shared dev toolchain via magpie-dev

Every member repeated the same three pins under an instruction in the root
pyproject to "keep the version bounds here and in every member in lockstep".
They were not in lockstep:

mypy    >=2.1.0, >=2.3.0, >=1.10   (three floors)
pytest  >=9.1.1, >=8.0             (two)
ruff    >=0.16.1, >=0.16.2         (two)

tools/skill-reconciler-diff declared only pytest, and tools/fossil had no
dev group at all despite being ruff-checked — its checks ran against whatever
was in the environment. The duplication was the defect; the lockstep instruction
was the workaround.

tools/dev becomes magpie-dev, a metadata-only wheel whose dependencies are
the toolchain. Every other member's dev group becomes ["magpie-dev"], with the
workspace source declared once at the root. Extras are preserved
(tools/gmail/oauth-draft keeps mcp). Bumping a version is now one edit.

Each member's environment stays self-contained — the checks still run
uv run --directory <member> --project ., so nothing leaks in from the root.
Only the declaration is shared.

4. Verify every member's tests actually run

Both the CI pytest matrix and the workspace sweep are driven by the presence of
[tool.pytest.ini_options]. A project can therefore carry a full tests/
directory and be executed by nothing: the job is simply never emitted, so
there is no red tick for anyone to notice. Nothing checked this.

check-workspace-members.py — which already owns "the members list decides which
projects get hooks and CI matrix entries" — now also reports three shapes:

Shape	Consequence
tests, no [tool.pytest.ini_options]	the tests never run
[tool.pytest.ini_options], no tests	the CI job runs and collects nothing, so its green tick proves nothing
neither	no tests at all

[tool.magpie.checks] skip = ["pytest"] is the declared exemption — the same
opt-out the sweep and the matrix already honour, so a project that should not be
tested says so in its own file rather than being a silent absence from a matrix
nobody reads.

All 33 members pass today, so this locks in the current state rather than
fixing a present gap — which is the best moment to add it.

Ten tests cover it, including the one false negative that would make the whole
check worthless: a test file vendored inside a member's .venv or
site-packages must not make an untested member look tested.

Verification

• prek run --all-files passes (exit 0)
• Whole battery re-run after deleting every member .venv, not against a
  warm cache: ruff 30/30, ruff-format 30/30, mypy 27/27, pytest 33/33
• Each of the five doc-sync checks verified to fail on its specific defect
  and pass once restored
• Each of the three tests-actually-run shapes verified by hand against a real
  member, and the declared opt-out confirmed to silence all three
• Every tools/dev script re-run after reformatting to confirm unchanged
  behaviour
• Check discovery confirmed section-based ([tool.ruff], [tool.mypy],
  [tool.pytest.ini_options]), not dev-group-based — so giving a member more
  tools does not silently enrol it in more checks

Notes for reviewers

One real behaviour change: tools/bitbucket moves from mypy>=1.10 /
pytest>=8.0 to the current floors. It passes the fresh sweep, but that is a
version bump, not a no-op refactor.

Deliberate limits on the totals check: it reads an allowlist rather than
sweeping the repo. Plenty of docs legitimately count a subset ("Nine skills cover
the staged path"), and a greedy scan would flag those forever.

.last-sync staleness is not a hook. It would fail nearly every PR, so it
would be noise rather than a gate; the human rule in AGENTS.md is its home.

🤖 Generated with Claude Code

https://claude.ai/code/session_01So3JRGXrbqSGrohtZuHWKg