Skip to content

[scratch, do not merge] CI of the results-PR tests fix with 6.7.5 certified - #200

Closed
MaxGhenis wants to merge 6 commits into
mainfrom
w6/scratch-tests-fix-with-675
Closed

MaxGhenis wants to merge 6 commits into
mainfrom
w6/scratch-tests-fix-with-675

Conversation

@MaxGhenis

@MaxGhenis MaxGhenis commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

Evidence only; it will be closed once CI has run and is never merged.

This branch is #196's head, plus:

ci.yml takes #196's version, which reads the matrix from the index. CI here shows whether the whole suite passes once a variant newer than 6.5.0 is certified, which local runs can't check during the disk pause.

🤖 Generated with Claude Code

MaxGhenis and others added 5 commits October 19, 2026 11:35
The results pull request no longer edits ci.yml. A replay-variants job
lists the certified variants of data/uk/replay_variants.json in index
order (.github/scripts/uk_replay_engine_matrix.py, the standard library
under the runner's python3 -I, from a sparse checkout of that script and
the index), and replay-engine's matrix is that list. The lister refuses
the whole matrix on a record it cannot use.

The builder drops add_matrix_variant, its only yaml import and ci.yml
from EDITED; apply needs no package in either job; scope refuses any path
under .github/. The App token drops permission-workflows, the build job's
CONTROL drops pyyaml, and propose explains a refused rebuild over a main
that changed a workflow. Finding 4/6 of both reviews of #193.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The refused-rebuild hint now also names a workflow change that main made
and then reverted: it checks every commit from the old tip, not just the
tips. Variant ids are capped at 120 characters in the lister and the
builder alike, so two ids joined by "__" (a release diff's directory) fit
one 255-byte path component and the matrix output stays far under
GitHub's 1 MB per job.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Default path-limited history prunes a merge to a parent its paths match,
so a workflow change and its revert made on a side branch and merged
after main moved on were invisible to the hint. --full-history shows
them; the test's stand-in for GitHub reads full history too. Answers the
review of 34fba89.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The first automated results PR (#195, 6.7.5) failed 30 tests in this file
that named 6.5.0 as the newest certified variant or the last committed
one: the end-to-end harness certified a synthetic 6.6.0, now refused as
older than 6.7.5; the NEWEST/COMMITTED round trip stripped 6.5.0's lines;
the paragraph-placement test expected 6.5.0's closing sentence last.

Those tests now read the committed index: NEWEST_ID is
qualify.latest_certified, the COMMITTED round trip uses the index's last
two ids, the paragraph must sit directly before the subsection's next
heading, the synthetic release is one minor version above the newest
certified, and the older-version refusal names the live newest version.
PE641 and PE650 stay where a test reproduces the hand-built #188 and #190.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Commit the receiver's sealed qualification of
pe6.7.5_uk2.130.1_policyengine-uk-data-1.58.1 from replay-state request
bc2090402dea079fad906cada6ca93378c7421b7c598ab800787820074b8e6aa
(receipt
c390b7db49fd2cfde3ac0de048bfbdba12dca08a332a11ed808f4828e90b6070,
receipt commit 1291853): its results
and release diff and requirements freeze byte for byte, the variant
record, the CI matrix entry, the tests that name the newest certified
variant and a RELEASE_REPLAY.md paragraph from the receipt's counts.

Replay-Request: bc2090402dea079fad906cada6ca93378c7421b7c598ab800787820074b8e6aa
@vercel

vercel Bot commented Oct 19, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
policyengine-scorecard Ready Ready Preview Oct 19, 2026 3:44pm UTC

Request Review

The discovery floor cases were written when 6.5.0 was the newest
certified variant. #193 pinned the index they pass to candidates(), but
discover() reads the committed index through variants.load_document(), so
25 cases failed on the first automated results PR (#195, 6.7.5): with
6.7.5 certified, the floor moved and 6.5.1 and 6.5.2 were no longer
discoverable. An autouse fixture now hands discover() the same floor
index; a test that needs another index still patches it itself.

Checked against #195's index (6.7.5 appended): the old file fails 25
tests, this one passes all 127.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis

Copy link
Copy Markdown
Contributor Author

Closing as planned: this was evidence only. The changes it combined have landed separately. #199 and #196 merged, #201 merged as fdcdea29, and #202 is rebased onto main and running CI. The 6.7.5 results commit is superseded by the 6.9.0 results rebuild, per the stack review's recommendation (~/reviews/budget-coordinator/scorecard-results-stack/review-out-opus.md).

@MaxGhenis MaxGhenis closed this Oct 21, 2026
@MaxGhenis
MaxGhenis deleted the w6/scratch-tests-fix-with-675 branch October 21, 2026 23:35

This branch was successfully deployed

1 active deployment
Preview — e8d1c43e Deployed Oct 19, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant