UK data certification: which checks must come before a release, and can they run in parallel? #722

@MaxGhenis, for you and the UK hub.

Why

The Budget dashboard can only re-pin to a policyengine.py release, and a release waits for the certification evidence. For #720 (the 2.151.0 pair), that took 12 hours from a ready head to the 6.11.1 release:

• 26 Oct 13:29Z: head 99a3e48b4 ready, with certify-data, snapshots, the strict check and the UC floor already run on the PR.
• 14:11Z: the hub took the run; the host was over its load limit, so the run waited at its first gate.
• 15:44Z: pre-flight, producer gates, the national hash check, the replay, certification and budget inputs done. The reform contract started.
• 20:13Z: the reform contract was still running. Reform responsiveness, the attribution report and the evidence index followed.
• 22:48Z: evidence done (8 h 37 min after the hub took it).
• 23:07Z coordinator decision, 23:19Z independent review, 27 Oct 00:06Z merge, 01:31Z release.

The next pair is due tonight. PolicyEngine/microcosm#1411 builds it on b71ce793, the head of PolicyEngine/policyengine-uk#2695 (2.162.12 plus PolicyEngine/policyengine-uk#2695). It is to be certified with the first policyengine-uk release that carries PolicyEngine/policyengine-uk#2695. A run like #720's would again put certification on the Budget's critical path (PolicyEngine/uk-autumn-budget-dashboard-2026#262 lists what the re-pin needs).

Where the time goes

From the hub's posts on #715 (a133) and #720:

Each step runs alone under Fleet ops' memory caps, and the evidence binds to the PR's head commit, so a push means running it all again.

Questions

1. Which steps must come before the merge? For a same-line re-cut, which steps gate promotion, and which are evidence the coordinator reads? The 2×2 attribution is reported as "computed; moves disclosed" rather than as a pass or fail, and the scorecard replay was unavailable in both runs. Could steps like these follow the merge?
2. Could the evidence bind to the pair, not the head commit? That is, to the model wheel SHA-256, the core version, the data SHA-256s and the harness commit. Then a push that touches no calculation path (docs, tests, changelog) wouldn't restart the run.
3. Could independent steps run at the same time, on separate machines? The reform contract, the responsiveness run and the four 2×2 cells don't read each other's results. The certify workflow already has a qualify job on CERTIFY_RUNNER_X64, with a 330-minute timeout.
4. Could the two longest steps be split? For example, the reform contract by route or by case family, and reform responsiveness with one baseline shared by its ten reforms.
5. Could the run start when each cut is tagged, before the certification PR exists? Budget inputs, producer gates and the hash checks need only the cut and the engine.
6. Could tonight's pair reuse evidence across identical code? The cuts are built on b71ce793 and record 2.162.12. Suppose the release that carries Settle property allowance elections without the charge where it cannot change them policyengine-uk#2695 has the same package code: git diff b71ce793 <tag> -- policyengine_uk ':!policyengine_uk/tests' shows only the version and changelog. Can evidence computed on b71ce793 then stand for it, so the hub starts before Settle property allowance elections without the charge where it cannot change them policyengine-uk#2695 is released? If not, which steps must run again on the release wheel?
7. How long does the local-area audit take? With Allow a UK local-area release on an accepted model mismatch #699, routing the local-area cut by an accepted mismatch needs audited input-surface evidence (regional_input_surface.py). Can it run alongside the national steps?
8. Two small fixes. Could the suite run tests/test_uk_budget_reform_contract.py in a process of its own, so one stalled worker can't hold the run for two hours? And could the harness retry Hugging Face fetches itself? The national hash check timed out once on Certify the Microcosm UK pair on policyengine-uk 2.151.0 (a6299504 national, c9df4b86 local area) #720.

This asks to drop no check. It asks for a list of what each step establishes and which must precede a release, so the producer and the dashboard can plan around it, and for independent steps to run side by side where they can.