Certify the policyengine.py 6.7.5 replay results by policyengine[bot] · Pull Request #195 · PolicyEngine/policyengine-scorecard

policyengine · 2026-10-18T23:04:56Z

Certify the policyengine.py 6.7.5 replay results

This PR commits the receiver's sealed qualification of pe6.7.5_uk2.130.1_policyengine-uk-data-1.58.1, with its release diff against pe6.5.0_uk2.130.1_policyengine-uk-data-1.58.1. The results workflow (.github/workflows/uk-replay-results.yml) built it from the committed receipt, as #188 and #190 were built by hand. Merging stays on the normal gates: CI and an independent review.


Request	`bc2090402dea079fad906cada6ca93378c7421b7c598ab800787820074b8e6aa` (released_bundle, attempt 1)
Receipt	SHA-256 `c390b7db49fd2cfde3ac0de048bfbdba12dca08a332a11ed808f4828e90b6070`, `replay-state` commit `1291853`
Run	https://github.com/PolicyEngine/policyengine-scorecard/actions/runs/39172690370
Engine	policyengine.py 6.7.5, policyengine-uk 2.130.1, core 3.33.0
Data	policyengine-uk-data-1.58.1, `enhanced_frs_2024_25.h5` (`056e80fd…3784`)
Integrity	complete, 0 failures; complete supported grids for 6 of 6 events, 0 inert measure-years
Diff vs `pe6.5.0_uk2.130.1_policyengine-uk-data-1.58.1`	construction 0, coverage 0, inertness 0, mapping 0, numerical 0, structural 0, support 0; 0 flagged rows

What changes

results/uk/replays/pe6.7.5_uk2.130.1_policyengine-uk-data-1.58.1/ and results/uk/replays/diffs/pe6.5.0_uk2.130.1_policyengine-uk-data-1.58.1__pe6.7.5_uk2.130.1_policyengine-uk-data-1.58.1/: the receiver's requests/bc209040…/results/ tree, byte for byte: 231 files, 39.0 MB of aggregate outputs and release-diff packets.
docs/uk_replay/requirements/pe6.7.5_uk2.130.1_policyengine-uk-data-1.58.1.txt: the variant's requirements freeze.
data/uk/replay_variants.json: the variant record, the qualification's VARIANT.json exactly plus a note, in canonical form.
.github/workflows/ci.yml: the replay-engine matrix gains the variant.
tests/test_uk_replay_qualify.py and tests/test_uk_replay_determinism.py (NEWEST) and tests/test_uk_replay_variants.py (COMMITTED): the tests that name the newest certified variant move to it.
docs/uk_replay/RELEASE_REPLAY.md: a "Committed certified results" paragraph from the receipt's counts.

Validation in the results run

The receiver's output guard, applied again to the request's tree: every file an allowed, parsable aggregate.
Byte identity: all 231 committed result files have the blob replay-state holds; the receipt and results are unchanged since the receipt commit.
uk_replay_qualify verify-sealed: 3 sealed namespace(s) checked.
CI's .github/scripts/uk_replay_regenerate.sh leaves the tree unchanged.
scorecard_db.build_db: two builds agree (3ebf8a86…), and there is no tracked-file drift.

The table and the RELEASE_REPLAY.md paragraph state counts only. Reviewers add any interpretation of flagged rows.

axiom: n/a: replay results and CI matrix; no policy encoding.

Commit the receiver's sealed qualification of pe6.7.5_uk2.130.1_policyengine-uk-data-1.58.1 from replay-state request bc2090402dea079fad906cada6ca93378c7421b7c598ab800787820074b8e6aa (receipt c390b7db49fd2cfde3ac0de048bfbdba12dca08a332a11ed808f4828e90b6070, receipt commit 1291853): its results and release diff and requirements freeze byte for byte, the variant record, the CI matrix entry, the tests that name the newest certified variant and a RELEASE_REPLAY.md paragraph from the receipt's counts. Replay-Request: bc2090402dea079fad906cada6ca93378c7421b7c598ab800787820074b8e6aa

vercel · 2026-10-18T23:04:58Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
policyengine-scorecard	Ready	Preview	Oct 18, 2026 11:05pm UTC

The discovery floor cases were written when 6.5.0 was the newest certified variant. #193 pinned the index they pass to candidates(), but discover() reads the committed index through variants.load_document(), so 25 cases failed on the first automated results PR (#195, 6.7.5): with 6.7.5 certified, the floor moved and 6.5.1 and 6.5.2 were no longer discoverable. An autouse fixture now hands discover() the same floor index; a test that needs another index still patches it itself. Checked against #195's index (6.7.5 appended): the old file fails 25 tests, this one passes all 127. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

MaxGhenis · 2026-10-19T15:38:24Z

Status from the #193 session: this PR's CI failure isn't about 6.7.5's results. Integrity is complete, all six grids are complete, and there are 0 flagged rows against 6.5.0. The 55 failures are tests that took 6.5.0 to be the newest certified variant:

25 in tests/test_uk_replay_discover.py: fixed in Pin the index discover() loads itself to the 6.5.0 floor #199;
30 in tests/test_uk_replay_results_pr.py: to be fixed after Read CI's replay-engine matrix from the variant index; drop the results bot's workflows permission #196 merges, since Read CI's replay-engine matrix from the variant index; drop the results bot's workflows permission #196 rewrites that file.

Once both fixes are on main, re-dispatching uk-replay-results.yml for request bc2090402dea079fad906cada6ca93378c7421b7c598ab800787820074b8e6aa rebuilds this branch on the new main (the bot force-pushes only its own single commit). CI then reruns here, and the PR lands on the normal gates.

The discovery floor cases were written when 6.5.0 was the newest certified variant. #193 pinned the index they pass to candidates(), but discover() reads the committed index through variants.load_document(), so 25 cases failed on the first automated results PR (#195, 6.7.5): with 6.7.5 certified, the floor moved and 6.5.1 and 6.5.2 were no longer discoverable. An autouse fixture now hands discover() the same floor index; a test that needs another index still patches it itself. Checked against #195's index (6.7.5 appended): the old file fails 25 tests, this one passes all 127. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

The discovery floor cases were written when 6.5.0 was the newest certified variant. #193 pinned the index they pass to candidates(), but discover() reads the committed index through variants.load_document(), so 25 cases failed on the first automated results PR (#195, 6.7.5): with 6.7.5 certified, the floor moved and 6.5.1 and 6.5.2 were no longer discoverable. An autouse fixture now hands discover() the same floor index; a test that needs another index still patches it itself. Checked against #195's index (6.7.5 appended): the old file fails 25 tests, this one passes all 127. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

The first automated results PR (#195, 6.7.5) failed 30 tests in this file that named 6.5.0 as the newest certified variant or the last committed one: the end-to-end harness certified a synthetic 6.6.0, now refused as older than 6.7.5; the NEWEST/COMMITTED round trip stripped 6.5.0's lines; the paragraph-placement test expected 6.5.0's closing sentence last. Those tests now read the committed index: NEWEST_ID is qualify.latest_certified, the COMMITTED round trip uses the index's last two ids, the paragraph must sit directly before the subsection's next heading, the synthetic release is one minor version above the newest certified, and the older-version refusal names the live newest version. PE641 and PE650 stay where a test reproduces the hand-built #188 and #190. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…sts (#201) The first automated results PR (#195, 6.7.5) failed 30 tests in this file that named 6.5.0 as the newest certified variant or the last committed one: the end-to-end harness certified a synthetic 6.6.0, now refused as older than 6.7.5; the NEWEST/COMMITTED round trip stripped 6.5.0's lines; the paragraph-placement test expected 6.5.0's closing sentence last. Those tests now read the committed index: NEWEST_ID is qualify.latest_certified, the COMMITTED round trip uses the index's last two ids, the paragraph must sit directly before the subsection's next heading, the synthetic release is one minor version above the newest certified, and the older-version refusal names the live newest version. PE641 and PE650 stay where a test reproduces the hand-built #188 and #190. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

MaxGhenis · 2026-10-22T00:09:00Z

Closing as superseded by the 6.9.0 results PR, which is rebuilt once the fix stack (#199, #196, #201, #202) is on main. Both stack reviews recommend this: Opus 5.5 at ~/reviews/budget-coordinator/scorecard-results-stack/review-out-opus.md and GPT-6.1 Sol at review-out.md in the same folder.

Why it is safe to skip 6.7.5:

The builder can't append it after 6.9.0. pipeline/uk_replay_results_pr.py:291-296 refuses any release older than latest_certified, which uk_replay_qualify.py:178-180 defines as the last certified record in the append-only index.
Nothing is lost. On replay-state, 6.6.0, 6.7.1, 6.7.2, 6.7.4, 6.7.5, 6.8.0 and 6.9.0 all qualified complete on the same engine pair (policyengine-uk 2.130.1 with policyengine-uk-data 1.58.1). Each has 0 flagged rows against 6.5.0, and 6.9.0's own diff against 6.5.0 is 0 rows, so it covers the interval. This release's receipt stays on replay-state.
No reader on the scorecard side (scorecard_db, app) uses the variant index. Committing every release would add byte-identical aggregates and a permanent CI leg each.

Do not re-dispatch this request after 6.9.0 lands: the builder refuses it, and the run goes red.

🤖 Generated with Claude Code

#217) The results bot now runs ci.yml's python-job checks on the assembled tree in the build job, which holds no write credential. A failing check no longer stops the run: the pull request opens as a draft that names what failed. Results PRs #195, #204 and #206 opened ready and failed CI's pytest job. The test that pins what propose runs now reads each command as bash would and checks it against an allowlist, in place of a shell-name regex that matched before quote removal. The stack review's two bypasses are must-flag cases. GPT-6.1 Sol reviewed this over five rounds; the last, at ed25ea0, returned MERGE-SAFE WITH NITS. The pull request states the invariants, the known limits and the review history. axiom: n/a: CI and test tooling Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

vercel Bot deployed to Preview October 18, 2026 23:05

View deployment

MaxGhenis mentioned this pull request Oct 19, 2026

Pin the index discover() loads itself to the 6.5.0 floor #199

Merged

MaxGhenis mentioned this pull request Oct 19, 2026

[scratch, do not merge] CI of the results-PR tests fix with 6.7.5 certified #200

Closed

MaxGhenis mentioned this pull request Oct 19, 2026

Read the newest certified variant from the index in the results-PR tests #201

Merged

MaxGhenis closed this Oct 22, 2026

MaxGhenis deleted the uk-replay-results/pe6.7.5_uk2.130.1_policyengine-uk-data-1.58.1 branch October 22, 2026 00:09

This was referenced Oct 22, 2026

Certify the policyengine.py 6.9.0 replay results #206

Closed

Run CI's checks before proposing results; allowlist propose's commands #217

Merged

Certify the policyengine.py 6.9.0 replay results #216

Merged

Navigation Menu

Repository navigation

Certify the policyengine.py 6.7.5 replay results - #195

Certify the policyengine.py 6.7.5 replay results#195
policyengine[bot] wants to merge 1 commit into
mainfrom
uk-replay-results/pe6.7.5_uk2.130.1_policyengine-uk-data-1.58.1

policyengine Bot commented Oct 18, 2026

Uh oh!

vercel Bot commented Oct 18, 2026 •

edited

Loading

Uh oh!

MaxGhenis commented Oct 19, 2026

Uh oh!

MaxGhenis commented Oct 22, 2026

Uh oh!

This branch was successfully deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

policyengine Bot commented Oct 18, 2026

Certify the policyengine.py 6.7.5 replay results

What changes

Validation in the results run

Uh oh!

vercel Bot commented Oct 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

MaxGhenis commented Oct 19, 2026

Uh oh!

MaxGhenis commented Oct 22, 2026

Uh oh!

This branch was successfully deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

vercel Bot commented Oct 18, 2026 •

edited

Loading