[MaxGhenis]

This PR fixes the six non-blocking findings from the approving hard-tier review of #193, which reviewed head 78e080c6 and merged as 2c89c529. The review is at w6-lanes/reviews/review_193_delta2.md. The job split, the token permissions and the CI matrix mechanism are unchanged.

Changes

1. The tip check runs on the guarded request (.github/scripts/uk_replay_results_pr.sh, rebuild()).
   • Before, rebuild re-checked the request against main's tip from its own git archive of the request. It did that before assemble had run the output guard, so it read unguarded JSON, through any symlink. A missing request also failed there with a raw git error.
   • The tip check now runs after assemble and the tree comparison, on $STAGE/$REQUEST, the copy the guard has already checked. The separate extraction is gone.
   • The order pin names it, and it requires that rebuild contains no git archive or tar and exactly one builder check, on the staged copy.
   • Two behavioural tests move main so the tip check would run. A state commit without the request now ends with "has no request". A state commit whose STATUS.json is a symlink is refused by the guard before anything reads it.
2. A held variant is compared by identity, not field by field (pipeline/uk_replay_results_pr.py, check()).
   • A request whose variant id main already holds is skipped only when the variant_id and identity_sha256 agree and main's record is certified. The rest of the record is not compared. That covers the note, and anything a later change to the index (a schema migration) adds or reshapes.
   • Because the skip rests on the digest, VARIANT.json's identity_sha256 must now be the digest of its own identity.
   • Tests run in both directions, in strict and stdlib modes, with Hypothesis properties. The end-to-end skip test also migrates main's record after the merge.
3. The matrix tests use the bytes Certify the policyengine.py 6.4.1 replay results from qualification attempt 3 #188 and Certify the policyengine.py 6.5.0 replay results, the first with the processed-value check #190 actually edited.
   • The old strict == lean test held by construction, so it is replaced. tests/fixtures/uk_replay/results_pr/ holds e796ffb5^'s and f4164f20^'s ci.yml, pinned by git blob id, because CI's checkout is shallow.
   • With yaml unimportable, the stdlib edit of Certify the policyengine.py 6.5.0 replay results, the first with the processed-value check #190's input is Certify the policyengine.py 6.5.0 replay results, the first with the processed-value check #190's output blob 29f7a1b8 exactly. Of Certify the policyengine.py 6.4.1 replay results from qualification attempt 3 #188's input, it is Certify the policyengine.py 6.4.1 replay results from qualification attempt 3 #188's matrix hunk alone.
   • The same edit is also checked under python3 -I -S.
   • A property checks that the list check alone refuses any other matrix.
   • The propose pin no longer skips every ::error:: line. It sets aside only the text of a message string with no command substitution. It also requires that propose has no inline comment on a command line.
4. The control-environment pin follows imports transitively.
   • ImportScan (in the test file) follows what runs from the builder's main and from the verify-sealed branch of uk_replay_qualify's main. That is each reached module's top level, the functions and classes reached code names (directly or through module aliases), and import statements only where reached code holds them.
   • The reached third-party set must be exactly CONTROL's three packages, and no release package may be reached.
   • A synthetic package tests the scanner itself.
   • The test uv stub now emulates uv run --with …: it refuses every installed package except the specs and what they require. So the end-to-end build jobs run the builder and verify-sealed with CONTROL's packages alone. Before, they ran with numpy and everything else the suite installs.
5. GITHUB_SHA is set as a real dispatch sets it.
   • In the end-to-end harness, every run is a dispatch: GITHUB_SHA is main as dispatched, and both jobs check that commit out.
   • New tests:
     • A run dispatched before main moved proposes on the newer main.
     • A consistent older-base build, whose tree matches because job 1 really built on that base, passes with its own run's GITHUB_SHA. A later dispatch's GITHUB_SHA refuses it, and nothing else does.
6. Nit: the orphan # scoped to comment line is removed and the comment is rewrapped.

docs/uk_replay/RELEASE_REPLAY.md ("The results pull request") and the workflow comment now describe the new order and the identity comparison.

Invariants

• A held id: a request whose variant id main holds is skipped exactly when the id and the verified identity_sha256 agree and main's record is certified. No other change to main's record affects the decision. Property tests cover any non-identity change (skipped) and any identity leaf (refused).
• The matrix edit: on any ci.yml, the stdlib edit succeeds only when the matrix lists exactly the certified variants before the new one (property test). Differential tests on the real inputs compare it with Certify the policyengine.py 6.5.0 replay results, the first with the processed-value check #190's committed bytes and Certify the policyengine.py 6.4.1 replay results from qualification attempt 3 #188's matrix hunk.
• The guard comes first: job 2 reads request JSON only from the copy the output guard checked. This is pinned statically and behaviourally.
• CONTROL's packages: what the strict paths can import equals CONTROL's packages. The static scan and the runtime emulation check this independently: the end-to-end build jobs, plus verify-sealed on every committed namespace.

Verification

• Fleet ops' disk hold is on, so I ran no local pytest. bash -n, shellcheck and actionlint are clean, and ruff format --check passes.
• I called the static and fixture tests directly with the shared venv's interpreter, and they pass. That covers the workflow, builder and order pins, both fixture matrix tests, the -I -S test, the wrong-matrix property, the control scan, the synthetic scan, the uv emulation probe, and verify-sealed under CONTROL on every committed namespace.
• I checked each new pin by breaking the code it guards, and each one failed: the tip check moved back before the guard, the tip check reading its own extraction, a builder call on an ::error:: line, a builder call after a #, and the stdlib list check removed.
• I also checked the scanner the same way. It catches an import added in validate_record, at the interface module's top level, behind a new verify_sealed → compute_uk_event call, and in a builder helper. It correctly ignores one in qualify's corpus branch.
• CI runs the collision and end-to-end tests.

Review

• Hard-tier review of c5c17db7: APPROVE (w6-lanes/reviews/review_194.md), with three P3 limitations in the new pins.
• 3ddb3311 answers them:
  • the propose pin's message must be a whole quoted word;
  • ImportScan resolves names by Python scope and to every binding a name may have, treats a module used as a value as fully reachable, and flags aliased dynamic imports;
  • the uv emulation evaluates markers and purges modules loaded before it.
• Delta review of 3ddb3311: APPROVE (w6-lanes/reviews/review_194b.md).
  • CI run 39252322886 on that head: 2,265 passed, 15 skipped.
  • Locally, the results-PR test file passes 84/84 (-n 4, after the disk hold was lifted).
• Non-blocking notes the delta review left, for a follow-up after Read CI's replay-engine matrix from the variant index; drop the results bot's workflows permission #196 (which rewrites these tests):
  • The propose pin should track quote state across the whole body; its message regex can still be fooled by a closing quote.
  • ImportScan should bind a function's global imports and __main__-block imports module-wide, and add exec/eval to its attribute list.
  • The scan's branch selection, decorator registries and module __getattr__ are not modelled. None of these patterns occurs on the real entry points.

axiom: n/a: CI and test changes to the results-PR builder; no policy encoding.

🤖 Generated with Claude Code

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
policyengine-scorecard	Ready	Preview	Oct 19, 2026 12:29pm UTC