Skip to content

fix(ci): untrusted-input checks the corpus figure against its anchor, not anywhere in the doc (Closes #5799) - #5801

Merged
gHashTag merged 1 commit into
masterfrom
fix/untrusted-input-corpus-count
Oct 3, 2026
Merged

fix(ci): untrusted-input checks the corpus figure against its anchor, not anywhere in the doc (Closes #5799)#5801
gHashTag merged 1 commit into
masterfrom
fix/untrusted-input-corpus-count

Conversation

@gHashTag

@gHashTag gHashTag commented Oct 3, 2026

Copy link
Copy Markdown
Owner

Closes #5799

What untrusted-input required, and why that was wrong

Step "Every re-taken proposition is still anchored" runs scripts/ci/test_retaken_propositions_still_match.py. For the corpus it checked str(walked) in doc: today's count of .t27 files under specs/ (outside scratch) had to appear anywhere in the 27k-line docs/theory/IGLA-FORMAL-RESULTS.md. The intent (#3086) was "the re-takes must not quote a corpus that has moved". In practice:

  • Red on every spec PR, for adding a spec. On master 188884e89 it fails: 1166 appears nowhere -- the re-takes quote a corpus that has moved again. The re-take says 1146, anchored at 4f65684d.
  • Green by coincidence. At 1156 master passed only because T444 enumerates "1156 alphabets".

Mechanism

The figure lives in one generated, marked block inside the 4f65684d re-take:

<!-- corpus-count anchor=4f65684dffa799dab4e1068ce7c6416be5c8cca7 -->1146<!-- /corpus-count -->
  • Written by python3 scripts/ci/test_retaken_propositions_still_match.py --write (--anchor <rev> moves it). The hand copies of 1146 in the heading and prose are gone.
  • Checked as the relation it states: the block equals the count in the tree of the anchor commit, using the walker's rule. CI's checkout is depth 1, so the script fetches the anchor by SHA. If it can't, the check fails; it never skips.
  • The block must also match the commit in its RE-TAKEN AT heading and the re-take's total row. Moving the anchor without re-taking t27c impl-status is red.
  • Nothing outside the marker is read. A planted control (3 of 7 paths) pins the counting rule.

Who updates it: nobody, per spec PR. I chose the stated relation over "every spec PR regenerates a live count":

  • Master took 15 specs on 2026-10-03, with five spec merges inside 90 s.
  • Two branches that each regenerate N to N+1 make the identical edit, so git merges them cleanly into a master that is wrong by one. Neither PR goes red.
  • docs(igla): re-take the corpus figures on master, 1146 specs (Refs #5497) #5597 set that policy, and two days later master was 20 specs past the figure.
  • An anchored figure is also this document's own rule.

Controls, from output

C1 block 1147 (wrong)                                new rc=1
   FAILED  block 4f65684dff: states the corpus of its anchor
   - ... the block says '1147', the tree at 4f65684dff... has 1146 specs. Regenerate it -- do not type it: python3 scripts/ci/test_retaken_propositions_still_match.py --write
C2 block 1147 + "1146" and "1166" planted elsewhere  OLD check rc=0 ("ok  the corpus size is stated somewhere")
                                                     new rc=1 (same FAILED line as C1)
C3 correct block                                     new rc=0  "ok: every re-take is anchored, and each corpus figure is its anchor's tree."
C0 old check, same correct doc, master tree          OLD rc=1  "1166 appears nowhere"
C4 planted extra spec (walked 1167)                  new rc=0; OLD rc=1 "1167 appears nowhere"
C5 heading moved to 188884e8, block not              new rc=1  "one of them was moved without the other"
C6 marker removed, bare 1146 left                    new rc=1  "no <!-- corpus-count anchor=... --> block"
C7 block 9999, then --write                          "4f65684dff '9999' -> 4f65684dff 1146", rc=0, doc byte-identical
C8 --write --anchor HEAD                             "-> 188884e89d 1166", rc=1: heading mismatch + "impl-status table says total ['1146'], the anchor has 1166"

CI-shaped run: in a fresh git clone --depth 1 https://github.com/gHashTag/t27, the anchor was absent (Not a valid object name). The script fetched it in 9 s (.git 60M to 62M), counted specs at 4f65684dff: 1146, and returned rc=0.

Siblings unchanged and passing locally: check_untrusted_shell_interp, check_untrusted_javascript_interp, check_pr_branch_filters, test_catalog_table_matches_the_gate, test_status_tables_name_paths_that_exist.

Not done here

  • test_catalog_table_matches_the_gate.py has the same substring shape (**{records}** anywhere in the doc). It is green today and only moves with the catalog, so it is not fixed here.
  • spec-guards (ring-096 vs specs/numeric/formats.t27) is a pre-existing red on master and is left alone.

🤖 Generated with Claude Code

@gHashTag @claude
fix(ci): the corpus figure is checked against its anchor, not found a…
5e572fa 
…nywhere in the doc

Untrusted Input Gate's re-take guard required today's spec count to appear
anywhere in docs/theory/IGLA-FORMAL-RESULTS.md (`str(walked) in doc`). It was
red on master (1166 appears nowhere), red on every PR that adds a spec merely
for adding it, and green at 1156 only because T444 enumerates 1156 alphabets.

The figure is now one generated, marked block:

  <!-- corpus-count anchor=<40-hex> -->1146<!-- /corpus-count -->

written by `test_retaken_propositions_still_match.py --write`, and the check
is the relation it states: the block equals the .t27 count (outside any
scratch dir) in the tree of the anchor commit, fetched by SHA when the
depth-1 CI checkout lacks it. The block must also match its RE-TAKEN AT
heading and the re-take's total row. No number outside the marker is read.

A live count regenerated by every spec PR was rejected: five spec merges
landed inside 90 s on 2026-10-03, and two branches that each write N+1 merge
cleanly into a wrong master.

Closes #5799

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Oct 3, 2026

Copy link
Copy Markdown
Contributor

PR Dashboard

Generated at: 2026-10-03 19:11:19 UTC

Summary

Status Count
Total Open PRs 50
PRs with Failing Checks 41
PRs with All Checks Green 9
READY 7
FAILING 41
PENDING 0
NO CHECKS YET 0

These columns do not partition: 7 + 41 + 0 + 0 = 48, and there are 50 open PRs. A PR is being counted twice or not at all.

Seal Status

  • ⚠️ STALE -- sha256(compiler.rs)=b45a356c2eb6 != manifest seal=87e5cbd3ad94.
    The committed NMSE numbers were certified against an older compiler.rs.
    Run scripts/reseal-check.sh locally for the two-step reseal command (advisory; not a merge gate).

@github-actions

github-actions Bot commented Oct 3, 2026

Copy link
Copy Markdown
Contributor

📓 NotebookLM Notebook linked to this PR

This notebook contains session context, decisions, and artifacts for this work.

@gHashTag
gHashTag merged commit 17f0986 into master Oct 3, 2026
33 of 34 checks passed
This was referenced Oct 3, 2026
Merged
Closed
Merged
gHashTag added a commit that referenced this pull request Oct 4, 2026
@gHashTag
fix(ci): catalog-table check reads one anchored block, not any **N** (#…
c8afc32 
…5891)

P17's catalog figure is now one generated block, <!-- catalog-count anchor=<40-hex> -->N<!-- /catalog-count -->. It is checked against the CATALOG: lines in specs/numeric/formats_catalog.t27 at the anchor, the RE-TAKEN AT heading and the mandatory-field row. Nothing outside the marker is read.
The marker, fetch-by-SHA and --write move into scripts/ci/anchored_count.py, which both gates share; the #5801 corpus gate's output is byte-identical on the doc.
--self-check runs four controls in CI: correct block gives 0; wrong block, wrong block with the number planted elsewhere, and marker removed each give 1.

Closes #5881
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

untrusted-input: corpus count is a substring anywhere in a 27k-line doc -- red on every spec PR, green by coincidence

1 participant

@gHashTag