Skip to content

ci: run the three spec guards that nothing was running - #3427

Merged
gHashTag merged 1 commit into
masterfrom
ci/spec-guards
Sep 7, 2026
Merged

ci: run the three spec guards that nothing was running#3427
gHashTag merged 1 commit into
masterfrom
ci/spec-guards

Conversation

@gHashTag

@gHashTag gHashTag commented Sep 7, 2026

Copy link
Copy Markdown
Owner

Closes #3426

Three checkers were written over the last passes, each after a defect it would have caught, and each has since caught a live one. All three ran only when someone typed them.

tool what it caught
check_seal_currency.py 116 seals describing output the compiler had stopped producing — spec_hash stale for zero of them, so every existing check was green #3415
check_ring_spec_drift.py one of ten ring/spec pairs had drifted #3422
ring_spec_differential.py 126 of 1190 cases disagreed behind 16 of 16 identical signatures #3420, #3424

Cost — measured, and it was never a decision

I deferred this three times as "a cost decision for the owner":

step time
seal currency, full scan 12 s
ring/spec drift, full scan 0 s
differential, per CONVERGED pair ~1 s
all three self-checks ~1 s

On top of a cargo build --release -p t27c the repository already pays for elsewhere. That is the third time this session I called something a decision without measuring it first.

Four deliberate choices

  1. Self-checks run first, in their own steps. A checker whose own negative control fails cannot be believed when it then reports zero; sharing a step would let a green scan hide a broken instrument.
  2. The pair list is derived, not hard-coded. --converged prints it, because a list in the YAML goes stale the first time a pair converges. An empty list fails the step rather than looping zero times and reporting success — verified with a control that exits 1.
  3. Not a required context, and it must not become one without you. A paths:-filtered workflow that is required hangs every PR it does not match, forever, on a check that never posts — measured earlier at 15 of 40 merged PRs for cli-tri. A push: trigger is included so master has a baseline at all; a paths-filtered workflow without one runs zero times on the default branch.
  4. while read, not mapfile. mapfile is bash 4+ and macOS ships bash 3.2, so that step could not have been run locally before pushing it. A construct only the runner can execute is one nobody tested. The loop, its empty-list guard, and the final-line-without-a-trailing-newline case were each run verbatim under bash 3.2:
cases 936  agree 936  disagree 0      covered 16 of 16
cases  25  agree  25  disagree 0      covered  4 of 4
CONVERGED pairs run: 2

The display name Spec Guards is unique across all workflows — checked, because two workflows sharing a display name is how cli-tri became unnameable in the ruleset.

Closes #3426

Three checkers were written over the last passes, each after a defect it would
have caught, and each has since caught a live one:

  check_seal_currency.py     116 seals described output the compiler had
                             stopped producing, and spec_hash was stale for
                             ZERO of them, so every existing coverage and
                             staleness check was green (#3415)
  check_ring_spec_drift.py   one of ten ring/spec pairs had drifted (#3422)
  ring_spec_differential.py  126 of 1190 cases disagreed behind 16 of 16
                             identical signatures (#3420, #3424)

All three ran only when someone typed them.

I deferred this three times as a cost decision for the owner. Measured, it is
not one: 12s for the seal scan, 0s for the drift scan, ~1s per differential
pair, ~1s for all three self-checks, on top of a compiler build the repository
already pays for elsewhere. That is the third time this session I called
something a decision without measuring it first.

Four things the workflow does deliberately:

  * Self-checks run FIRST, in their own steps. A checker whose own negative
    control fails cannot be believed when it then reports zero, and sharing a
    step would let a green scan hide a broken instrument.
  * The CONVERGED pair list is derived from `check_ring_spec_drift.py
    --converged`, not written into the YAML, because a hard-coded list goes
    stale the first time a pair converges. An empty list FAILS the step rather
    than looping zero times and reporting success.
  * It is not a required context and must not become one without the owner: a
    paths-filtered workflow that is required hangs every pull request it does
    not match on a check that never posts. A `push:` trigger is included so
    master has a baseline at all.
  * `while read`, not `mapfile`. mapfile is bash 4+ and macOS ships bash 3.2,
    so that step could not have been run locally before pushing it -- a
    construct only the runner can execute is one nobody tested. The loop, the
    empty-list guard and the final-line-without-a-trailing-newline case were
    each run verbatim under bash 3.2.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@gHashTag
gHashTag enabled auto-merge (squash) September 7, 2026 22:29
@github-actions

github-actions Bot commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

📓 NotebookLM Notebook linked to this PR

This notebook contains session context, decisions, and artifacts for this work.

@github-actions

github-actions Bot commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

PR Dashboard

Generated at: 2026-09-07 22:30:03 UTC

Summary

Status Count
Total Open PRs 16
PRs with Failing Checks 13
PRs with All Checks Green 3
READY 2
FAILING 13
PENDING 0
NO CHECKS YET 0

These columns do not partition: 2 + 13 + 0 + 0 = 15, and there are 16 open PRs. A PR is being counted twice or not at all.

Seal Status

  • ⚠️ STALE -- sha256(compiler.rs)=1d58d30ea13e != manifest seal=87e5cbd3ad94.
    The committed NMSE numbers were certified against an older compiler.rs.
    Run scripts/reseal-check.sh locally for the two-step reseal command (advisory; not a merge gate).

@gHashTag
gHashTag merged commit 7e5444b into master Sep 7, 2026
26 of 29 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Three spec guards exist, have each caught a live defect, and nothing runs them

1 participant