feat: calibrate.py — audit saturation meter + CI guardrail

[RasputinKaiser]

What

Adds scripts/calibrate.py, a saturation meter that makes "the audit is not saturated" a measurable, monitored property, and wires its selftest into CI.

python3 scripts/calibrate.py [roots...] scores a set of plugin SOURCE roots (default: the same local inventory portfolio.py sweeps; never read-only cache/ installs) and reports the score DISTRIBUTION:

• Overall stats: N, min, max, mean, median, stdev of total (auto) scores.
• Text histogram (0–100) and band distribution across the frozen bands (Exceptional 92–100 / Strong 82–91 / Solid 68–81 / Needs work 50–67 / Poor <50).
• Per-dimension max-out (saturation) rate — the key signal: % of plugins where a dimension hits its rubric ceiling.
• --check: exits 1 if the scoring looks re-saturated — any dimension maxes out for more than --max-maxout-frac (default 0.60) of plugins, or total-score stdev falls below --min-stdev (default 5.0). Prints which check failed.
• --json / --md output modes; robust to a plugin that fails to score (skip + note).
• selftest subcommand: hermetic temp-dir fixtures spanning the quality range; asserts the distribution stats and that --check passes on a healthy spread and fails on a degenerate all-max spread.

Relies only on score.py's stable schema (total.auto/total.max, dimensions[k].auto/.max) and reuses portfolio.py's source discovery. Stdlib-only.

CI

Adds a Calibrate selftest (saturation meter) step to .github/workflows/ci.yml. Existing steps (including the score.py . --min floor) are unchanged.

Baseline reproduction

Over the current (unrecalibrated) scorer against ~/plugins/* (13 plugins): hooks_health maxes out for 92% of plugins, skill_quality 0%, total-score stdev 6.78 — reproducing the measured saturation, and --check correctly fails naming hooks_health as the culprit.

🤖 Generated with Claude Code