[juaristi22]

Takes over #1348 and #1350 as one PR, as Max asked. It carries all 31 of #1348's commits and #1350's one commit, with their authorship kept, so it supersedes both.

What it carries

• Let anyone build, sign and certify a Microcosm UK cut #1348 (Max Ghenis, with the UK hub's rehearsal commits). Its description remains the reference for the design. It adds:
  • the signing-key registry microcosm.data.gate_signing_keys;
  • tools/run_uk_cut.py and tools/uk_cut_inputs.json;
  • tools/modal_uk_cut.py and tools/modal_rebuild_uk_chronicle_artifact.py;
  • tools/certify_uk_cut_in_policyengine_py.py;
  • docs/uk-cut-anyone.md.
• Register a team UK gate signing key (hold: needs María's agreement) #1350, cherry-picked as 252c168: the team key uk-gate-team-2026 (dd31e325…).

What changed since

María's review of #1348 (25 Oct)

1. The accepted-mismatch route certifies with policyengine.py's audited evidence (c550496). Following María's ruling on policyengine.py#699, its scripts/audit_regional_input_surface.py replaces this tool's own comparison.
   • The tool refuses a base without the audit, and downloads every regional microdata H5 at its manifest pin, checked by sha256.
   • In the worktree it installs the target model and core and runs the audit. Evidence that isn't build-relative (schema 2), or that drops a stored column, is refused.
   • It passes --regional-input-surface-evidence to certify-data, and the PR body lists what area runs default.
   • --audited-by and --regional-limitation are now required with a mismatch, and --allow-dropped-columns is gone.
   • policyengine.py commands no longer inherit the caller's UV_PROJECT_ENVIRONMENT.
2. A resumed cut is bound to its signer (810b2ec).
   • Preflight reads attestation.signing_key_sha256 from the reports of finished steps. It refuses any other key and names the signer.
   • The receipt takes its fingerprint from those reports.
   • Once the cut is certified, a signer that has since been retired can still finish it.
3. The one-writer check (c28cd37):
   • matches certify-uk-<id> anywhere in a remote branch's name or an open PR's head, so certify-uk-a133f37a-national is caught;
   • lists up to 1000 open PRs;
   • refuses when a listing can't be read.
4. all end to end: Vahid's run; see Verification.
5. UK_GATE_SIGNING_KEY_ENV is imported, not copied (fc288e1).
   • This covers the runner, the Modal wrapper and the local-release preflight.
   • The Modal wrapper can't import microcosm, so it loads the stdlib-only registry file by path: from the checkout locally, and from a copy the image carries in the container.
6. No machine-specific defaults (97bd378, ee91651).
   • The host-reservation check is opt-in (--reservation-tool with --reservation-caller), and a named tool that is missing fails.
   • The certifier has no load limit unless --max-load1 sets one. Its limits pass as keyword arguments through resource_guard, Shell and model_surface.
   • --pepy-repo is now required.
   • The doc no longer points at one machine's runbook.
7. Credits (61e20cb). Certification commits and PR bodies credit only the people the operator names, through --co-author and --pr-body-footer.
8. UC-floor receipts (ad73a94). A malformed receipt now fails operationally instead of crashing. A Hypothesis test over arbitrary JSON covers it.

Also

• Pins (96a9314). NTS 2002-2024 and LCFS 2024-25 are pinned at María's uploads (revisions 559b69e9 and 11051052), with their archive digests checked. Every hosted archive must now carry both pins. The doc notes that the LCFS zip is a repack of the extracted files.
• Team key (34daa6d). Its custodian is Max Ghenis. held_in names his agent store and says agent sessions on his build Mac may read it. María's key 66bf0e29… stays active.
• main merged (ef82ec8, at 5810fce, policyengine-uk 2.162.3), with no pin or fixture to regenerate. The Chronicle artifact's Hub path now follows chronicle_feed.json (b769292). It reads chronicle-uk/<source commit>/ at main and checks it against the feed's digests, so a feed re-pin no longer breaks the runner.
• Chronicle 272c987 hosted (7a97104). The artifact main pins is now on the Hub (commit 5d28b8f0), uploaded unchanged from the build UK: impute alcohol, tobacco and gambling consumption so policyengine-uk's excise duties stop scoring zero #1359 pinned. The runner's fetch --only chronicle_uk fetched and verified both files from the derived path.
• Orrery (5e1650a). The national step passes --no-publish-orrery, as the hand-run launchers do.
• Logbook (c212517). The release-cut certification's Logbook row now chains on the national build's own row, as the launchers chain it, instead of sharing the operator's predecessor.
• Doc fix (1e54fa8) repairs a doc table that one of these edits garbled.

Since Vahid's round 1 (at 6c6039941)

1. The accepted-mismatch route: already done in c550496, which landed after his read.
2. HMAC limits (9cddb43). The doc's § Signing keys says what the signature does not prove: only a key holder can verify, a retired key can still mint reports, and revocation is local. The Ed25519 follow-up is microcosm#1406.
3. Key recipes (833ea03). generate --keychain-service NAME stores the key through security -i on stdin, reads it back and prints only the fingerprint, so fingerprint.txt is no longer needed. Plain generate refuses to print a key to a terminal. The doc and this description now pass the key to one command at a time instead of exporting it.
4. Key scope (c3fe8f8). The runner gives the key only to spine, national, certify, assemble and publication. The certifier withholds it from every policyengine.py command, since policyengine.py reads no Microcosm key.
5. Modal (89f513c, 9cddb43). Planning is plain Python and sends nothing. modal run attaches credentials only with MICROCOSM_UK_CUT_MODAL_RUN=1 … --run. The doc says what a run puts on Modal: the ephemeral secret, the volume (readable across the workspace), and the key's custody.
6. Gated, not private (9cddb43). The doc now says that policyengine-uk-data-private is gated, with a public file list, and that approving access is what admits a builder.

Tests

On this head, 850 tests pass locally and none fail. They cover:

• the registry;
• the runner and the Modal wrapper (new stub-Modal tests);
• both certify suites;
• the build CLI and the national role;
• the calibration run and release certification;
• the local preflight;
• the gate battery and contract pins.

Lint and format are clean. The full suite runs in CI.

Verification

The last check before merging is the runner's first real run, which Vahid Ahmadi makes from his laptop with his own key.

What it needs:

• HF read access to the input repositories, and write access to policyengine/populace-uk-private and policyengine/populace-uk-staging;
• cover under the UKDS licences for the inputs;
• about 60 GB of free disk;
• AC power.

git fetch origin && git switch --detach origin/claude/uk-cut-anyone-takeover-974500
uv sync --all-packages --locked --extra uk
# Once: store a fresh key in your login keychain (it is never printed) and
# send only the fingerprint this prints, to be registered here.
uv run --no-sync python -m microcosm.data.gate_signing_keys generate --keychain-service microcosm-uk-gate-signing-key
# Then, with HF_TOKEN exported, give the key to one command at a time:
MICROCOSM_UK_TERMINAL_GATE_SIGNING_KEY="$(security find-generic-password -a "$USER" -s microcosm-uk-gate-signing-key -w)" \
  uv run --no-sync python -m microcosm.data.gate_signing_keys check
MICROCOSM_UK_TERMINAL_GATE_SIGNING_KEY="$(security find-generic-password -a "$USER" -s microcosm-uk-gate-signing-key -w)" \
  uv run --no-sync python tools/run_uk_cut.py all --label verify-<date> \
  --allow-off-main "María Juaristi: verification run before merging this PR"
uv run --no-sync python tools/run_uk_cut.py status --label verify-<date>

Leave out --publish. Report back which steps passed and how long each took, from status and the steps in cut_receipt.json, but no figures. Outputs stay on the laptop.

What's left

• policyengine.py#699. The accepted-mismatch route needs Run the UK test suite and a smoke-rung build in CI #699 merged; it now has María's ruling at f14690ee. Until then a base without the audit is refused before anything is downloaded.
• Vahid's key, which is registered here once he sends its fingerprint.
• Vahid's run. María then merges, with a merge commit.

The Modal wrappers have still never run on Modal; a separate plan replaces them.

Supersedes #1348 and #1350.

axiom: n/a: build infrastructure (no policy encoding)

🤖 Generated with Claude Code

[juaristi22]

@vahid-ahmadi, could you review this PR? It takes over #1348 and #1350, and the description lists what changed since María's review.

Before your verification run we also need your own signing key. The registry gives each builder their own key, so nobody shares a secret. Here's how to make one:

1. Check out this branch and sync:

   git fetch origin && git switch --detach origin/claude/uk-cut-anyone-takeover-974500
   uv sync --all-packages --locked --extra uk

2. Generate the key straight into your keychain, without printing it. The fingerprint goes to fingerprint.txt:

   KEY="$(uv run --no-sync python -m microcosm.data.gate_signing_keys generate 2> fingerprint.txt)" \
     && security add-generic-password -a "$USER" -s microcosm-uk-gate-signing-key -w "$KEY" \
     && unset KEY && cat fingerprint.txt

3. Reply here with only the fingerprint (the key_sha256 … line). Never post or commit the key itself.

I'll then register it as uk-gate-vahid-ahmadi-2026, with you as custodian and "held in his laptop's keychain", and push.

Once that's in, you can run the recipe under Verification in the description from your laptop. Export the key with:

export MICROCOSM_UK_TERMINAL_GATE_SIGNING_KEY="$(security find-generic-password -a "$USER" -s microcosm-uk-gate-signing-key -w)"
uv run --no-sync python -m microcosm.data.gate_signing_keys check   # ok: uk-gate-vahid-ahmadi-2026, active

The run needs:

• HF read access to the input repositories;
• write access to policyengine/populace-uk-private and policyengine/populace-uk-staging;
• about 60 GB of free disk, and AC power.

Run it without --publish. Please report only which steps passed and how long they took, not figures from the data.

🤖 Generated with Claude Code

[vahid-ahmadi]

@juaristi22 round 1, at 6c6039941.

Verdict: the registry and the runner are sound and the takeover is faithful, but don't merge yet: one must (the accepted-mismatch route) and five shoulds, mostly about where the signing key and the UKDS inputs travel.

What holds up

• Key strength and storage. generate uses secrets.token_bytes(32) (gate_signing_keys.py:266), every decoder insists on exactly 32 bytes (:165), and the repo holds only SHA-256 fingerprints (:96-137). With a 256-bit random key, publishing the fingerprint gives nothing away.
• Unknown, retired and revoked keys. I checked it in-process against the committed registry, outside pytest so the root conftest didn't apply. An unknown key is refused for both sign and verify. Retired verifies but can't sign, and revoked can do neither (:68-70, :179-207). The strict-registry tests cover every signer: battery, certifier, national role, re-signer and local preflight (test_uk_gate_signing_key_registry.py:104-225).
• Constant-time compare. Every verifier uses hmac.compare_digest (release_certification.py:719; contract.py:2676, :3117, :3335, :4866).
• No secret in output. check and every refusal print the fingerprint only, never the key (gate_signing_keys.py:192-206, :242-255). The key and HF_TOKEN stay out of argv. Both runner and certifier log commands (run_uk_cut.py:1313, :1330), and neither command line carries a secret.
• Resume binding. It does what your finding 2 asked (run_uk_cut.py:1025-1075).
• Outputs stay private. --publish goes to populace-uk-private with --tag-only (run_uk_cut.py:62), and telemetry goes to populace-uk-staging. Without a token, the HF metadata API returns 401 for both and for microcosm-uk-build-inputs, so all three are private. Before it commits, the certifier stages only the generated certification files (certify_uk_cut_in_policyengine_py.py:1266-1268).

Findings

1. must: the accepted-mismatch route still can't certify (certify_uk_cut_in_policyengine_py.py:889-893, :1101-1104). The certifier still forwards only --accept-regional-model-mismatch, with no --regional-input-surface-evidence. So it is still your round-0 finding 1. The PR body lists it as pending. Before merge, either wire in the evidence or make the flag refuse at parse time. As it stands, an operator only finds out after the worktree, the sync and the column audit have run.

2. should: HMAC means nobody outside the key holder can verify a certificate. The registry only checks the key in the verifier's own environment (contract.py:1110, :1128). A "certified" cut is therefore one key holder vouching for their own reports. policyengine.py and users downstream are trusting HF write access, not the signature. The design does hold together for one-holder-end-to-end. Three limits should be written down, though:

   • Retirement can't be checked. retired_on is only metadata (gate_signing_keys.py:93, read nowhere else). Whoever holds a retired key can still mint reports that verify.
   • Revocation is local. It only binds checkouts and wheels that include the revoking commit.
   • Verifying needs the secret. Anyone who verifies must hold the secret, and so can also sign.

   Two asks: say this plainly in docs/uk-cut-anyone.md § Signing keys, and open a follow-up issue for Ed25519. With Ed25519 the registry would store public keys and anyone could verify.

3. should: the key recipes expose the key.

   • Your comment's recipe. It runs security add-generic-password ... -w "$KEY". security is an external binary, so the key sits in its argv, and any user on the machine can read it with ps while the command runs. Pipe it to security -i with the shell's built-in printf instead, which never shows up in the process list:

     printf 'add-generic-password -U -a %s -s microcosm-uk-gate-signing-key -w %s\n' "$USER" "$KEY" | security -i

   • The doc and the CLI. Step 1 in the doc (docs/uk-cut-anyone.md:246-247) prints the key to the terminal, where it stays in scrollback and in agent transcripts. Consider having generate (gate_signing_keys.py:269) refuse to write the key to a TTY. Another option is a --keychain-service NAME that stores the key itself through security -i on stdin.
   • The export. export MICROCOSM_UK_TERMINAL_GATE_SIGNING_KEY=... leaves the key in the whole shell. Prefer setting it per command.

4. should: the signing key reaches processes that don't need it.

   • The certifier passes {**os.environ} to every command it runs (certify_uk_cut_in_policyengine_py.py:467). That includes uv pip install and policyengine.py's test suite (:1225-1238), which is third-party code that doesn't verify anything.
   • The runner also lets every step and the pytest checks inherit it (run_uk_cut.py:1243, :1332).

   Pass MICROCOSM_UK_TERMINAL_GATE_SIGNING_KEY only to the steps that sign or verify. That may include certify-data, if it verifies the gate reports, and I haven't checked which steps do. Strip it from the rest.

5. should: the Modal path sends the key and an HF write token to Modal, maybe even in "plan only" mode (modal_uk_cut.py:82-90, :119).

   • Plan only. _secrets() runs at import, and modal run creates the ephemeral app, with its secrets, before main runs. So I believe the "plan only (free)" call (:13-15) also uploads both values and builds the image. I couldn't confirm this without running Modal. If it's right, only attach the secrets behind --run, for example with cut.with_options(secrets=[...]) inside main.
   • Wording. "Never stored as Modal secrets" (:17-19, doc :121) undersells it. Secret.from_dict still sends the values to Modal.
   • Shared volume. The microcosm-uk-cuts volume (:108) keeps UKDS inputs and outputs, and anyone in the Modal workspace can read it.
   • Custody. A key used on Modal is no longer only "held in" one machine as the registry says. The doc already holds this back until the data owners agree, which is right. The registry's held_in should say so if it ever happens.

6. should: the input repository is gated, not private. policyengine/policyengine-uk-data-private is a model repo. Read without a token, the HF API reports private: false, gated: "manual". Anonymous downloads return 401, so nothing is public, but the file list is (FRS, LCFS, NTS, WAS, SPI and ETB zips). Access then depends on whoever approves gate requests. The doc says it "admits only covered users" (docs/uk-cut-anyone.md:23). Who approves requests, and do they check UKDS cover? If nobody checks, make the repo private like the other three. This PR now points every builder at it for the NTS and LCFS zips.

7. nit: the team key is readable by any agent session on Max's Mac (gate_signing_keys.py:124-128). Your custodian request on Register a team UK gate signing key (hold: needs María's agreement) #1350 is met: it now names Max (34daa6d). Still, that wording means any agent session there can mint certified cuts. Worth narrowing.

8. nit: the recipe writes fingerprint.txt into the checkout (2> fingerprint.txt). It's not secret, but it is an untracked file in the repo. It also catches any uv warning on stderr.

Takeover and earlier reviews

• Let anyone build, sign and certify a Microcosm UK cut #1348. Its head 300ce55ab is an ancestor of this head, and all 31 commits are carried with their authors. Thirteen are by Budget rehearsal <budget-rehearsal@localhost>, as they were on Let anyone build, sign and certify a Microcosm UK cut #1348. That's a nit: the identity doesn't trace to any person.
• Register a team UK gate signing key (hold: needs María's agreement) #1350. fe4cf02d0 was cherry-picked as 252c168f5, with Max as author. git range-diff shows only the added cherry-pick trailer.
• Max's reviews. There are none on Let anyone build, sign and certify a Microcosm UK cut #1348 or Register a team UK gate signing key (hold: needs María's agreement) #1350. The only reviews are yours (25 Oct).
• Your 8 findings. I spot-checked 3 (certify_…:492-503), 5 (the variable is now imported), 6 (run_uk_cut.py:1152-1163, and max_load1 defaults to None at certify_…:180) and 7 (:871-876). All four are fixed. 1 is still open (finding 1 above).

CI

engine-uk (3.14) hit the job's 30-minute limit while running UK engine tests (.github/workflows/test.yml:118; annotation "exceeded the maximum execution time of 30m0s"). No test failed. main has the same problem: today's 18:52 run on main (40381631550) timed out on engine-uk (3.13) in the same way. This PR adds no engine-uk tests, since all its new tests are engine-free. Re-run it, and raise the limit in a separate PR. engine-free and engine-us were still pending when I looked.

What I ran

• Tests. In a fresh worktree and uv venv, with the key and HF variables unset, I ran the 6 new test files: registry (data and build), runner, Modal stub, both certify suites. 174 passed.
• Registry probe. An in-process check of unknown, active, retired and revoked keys against the real registry, using a throwaway random key that was never printed or stored.
• HF visibility. Anonymous HF metadata checks on the four repos.
• CI. Job annotations, plus main's engine-uk timings.

What I didn't read or run

• No key was generated, no keychain was touched, and I didn't do the Verification run.
• No Modal runs and no builds.
• I didn't read the bodies of test_run_uk_cut.py or the two certify test files, uk_cut_inputs.json beyond the repo fields, or most of run_uk_cut.py's step logic.

Vahid will generate his signing key himself and post only the fingerprint here separately.

🤖 Generated with Claude Code

[juaristi22]

@vahid-ahmadi thanks for the review. Here's round 1 point by point; the head is now 1de1e0670.

1. must: the accepted-mismatch route. This was fixed in c550496, which landed just after your read at 6c6039941.
   • The certifier now calls policyengine.py#699's own scripts/audit_regional_input_surface.py, which now measures drops against the build model (f14690ee). It refuses a base without that script before downloading anything, and refuses evidence that isn't schema 2 or that drops a column.
   • It then passes --regional-input-surface-evidence to certify-data.
   • --audited-by and --regional-limitation are required at parse time.
   • The route still needs Run the UK test suite and a smoke-rung build in CI #699 merged.
2. HMAC. Agreed. § Signing keys now says what the signature does not prove: verifying needs the secret, a retired key's holder can still mint reports that verify, and revocation is local (9cddb43). The Ed25519 follow-up is Sign UK gate reports with Ed25519 so anyone can verify a cut #1406.
3. The key recipes. generate --keychain-service NAME stores the key itself, through security -i on stdin, so the key is in no argv. It reads the key back to confirm, then prints only the fingerprint (833ea03). Plain generate now refuses to print a key to a terminal. The doc and the PR description pass the key to one command at a time, with no export. If you haven't generated your key yet, the new command is in the description under Verification.
4. Key scope (c3fe8f8).
   • The runner gives the key only to the steps that sign or verify: spine, national, certify, assemble and publication. The dry run, the score, the stage-1 comparison and the preflight pytest runs get none.
   • policyengine.py never reads a Microcosm key (I grepped main and Run the UK test suite and a smoke-rung build in CI #699's head), so the certifier withholds it from every policyengine.py command, along with the caller's UV_PROJECT_ENVIRONMENT and VIRTUAL_ENV.
5. Modal (89f513c, 9cddb43). Your reading of modal run was right.
   • Planning is now plain python tools/modal_uk_cut.py --commit … --label …, which contacts no Modal service.
   • The launcher attaches credentials only with MICROCOSM_UK_CUT_MODAL_RUN=1, and the entrypoint refuses anything but that plus --run.
   • The docstring and doc now say what Modal holds: the ephemeral secret for the run, and the volume, which anyone in the workspace can read, so only a workspace limited to licensed users may hold it. A key used there must say so in its held_in.
6. Gated, not private. Confirmed: anonymously, the repo reports private: false, gated: "manual" with its file list visible, while the other three return 401. The doc now says exactly that (9cddb43). Who approves its access requests, and whether it should go private like the others, is with the repository owners.
7. The team key's readers. held_in records Max's setup as he described it on Let anyone build, sign and certify a Microcosm UK cut #1348. Narrowing it is Max's call as custodian, and the entry will follow what he decides.
8. fingerprint.txt. Gone. With --keychain-service, only the fingerprint is printed, on stdout.

On the takeover. The 13 Budget rehearsal commits are kept unchanged on purpose: rewriting them would change the SHAs that #1348 and the rehearsal cite, and Max is their committer.

CI. The run on the new head is in progress. I'll re-run any engine-uk job that hits the 30-minute limit, since main shows the same timeouts. Raising the limit belongs in its own PR.

The touched suites pass locally: registry, runner, Modal stub, both certify suites and the local preflight.

🤖 Generated with Claude Code

[juaristi22]

@vahid-ahmadi, a follow-up on round 1. The head is now 74721a944.

• Items 6 and 7 are decided: both stay as they are.
  • policyengine/policyengine-uk-data-private stays gated. The doc says what that means: its file list is public, and builders are admitted by approving their access request.
  • The team key's held_in keeps Max's setup, readable by agent sessions on his build Mac.
• policyengine.py#699 after your review there (f0870aa7). certify-data now re-reads both models, the target in process and the build model through the pinned uv probe. The certifier already installs the target before certify-data, so its flow doesn't change.
  • Let anyone build, sign and certify a Microcosm UK cut #1405 now refuses, at parse time, a mismatch reason, auditor or limitation with no printable non-space character, which Run the UK test suite and a smoke-rung build in CI #699 refuses too (74721a9).
  • The doc notes that certify-data also needs uv and PyPI.

The route still waits on #699 merging. Your fingerprint is all that's left before the verification run.

🤖 Generated with Claude Code

[vahid-ahmadi]

@juaristi22 round 2, at 74721a944.

Verdict: the must is fixed, and so are four of the five shoulds. The fifth, the gated repo, is now documented and left to the repo owners. Two new shoulds are small. One licence question is for Vahid and Max, not for this PR. The route still waits on policyengine.py#699 merging, and CI was still running when I looked.

Round 1, item by item

1. must, the accepted-mismatch route: fixed (c550496, 74721a9).
   • Reason, --audited-by and at least one --regional-limitation are required at parse time, and each needs a printable non-space character (tools/certify_uk_cut_in_policyengine_py.py:881, :980-995).
   • The certifier downloads the regional H5s at their manifest pins and checks each sha256. It deletes any earlier evidence file (:1182), runs Run the UK test suite and a smoke-rung build in CI #699's own audit (:1200), and refuses evidence that is not schema 2, is for other model versions, or drops a column (:712-733). Only then does it pass --regional-input-surface-evidence to certify-data (:1204-1207).
   • On our Run the UK test suite and a smoke-rung build in CI #699 finding. On Run the UK test suite and a smoke-rung build in CI #699 I found that certify-data checked the evidence only against itself, so a hand-edited file acted as an override (comment). This route doesn't inherit that, because it makes the evidence itself in the same run from hash-checked files instead of taking a file. Run the UK test suite and a smoke-rung build in CI #699's 1a93f7e70 also says certify-data now re-reads both models. I haven't re-verified that at f0870aa79. One part stays: the stored-column list rests on the auditor's word, bound to the H5 sha256, and a hand run of bundle.py certify-data still takes any evidence file. That belongs to Run the UK test suite and a smoke-rung build in CI #699, not here.
2. should, HMAC limits: done. § Signing keys says verifying needs the secret, a retired key can still mint, and revocation is local (docs/uk-cut-anyone.md:287-302). Ed25519 is Sign UK gate reports with Ed25519 so anyone can verify a cut #1406, which is open.
3. should, the key recipes: done. See the next section. The doc passes the key to one command only (docs/uk-cut-anyone.md:32-35).
4. should, key scope: done.
   • The runner gives the key only to spine, national, certify, assemble, publish-preflight and publish (tools/run_uk_cut.py:69-88, :1338). I checked these names against the real runner.run calls.
   • The certifier withholds the key, VIRTUAL_ENV and UV_PROJECT_ENVIRONMENT from every policyengine.py command (certify_uk_cut_in_policyengine_py.py:430-438, :489).
5. should, Modal: done for modal_uk_cut.py. With no opt-in, _secrets() returns nothing (tools/modal_uk_cut.py:95). Planning is plain Python, and main refuses unless both the opt-in and --run are set (:209). The doc now names the ephemeral secret and the volume's readers (docs/uk-cut-anyone.md:126-136). There's one gap, in the other Modal tool: see new finding B.
6. should, gated not private: the doc is now accurate, but the repo decision is open (docs/uk-cut-anyone.md:159-163). See the next section but one.
7. nit, team key readers. Left as Max's call. Fine.
8. nit, fingerprint.txt. Gone.

Key storage (833ea03)

I read this code and didn't run it. No key was generated, and no keychain was touched.

• argv. The key goes only to security -i on stdin (gate_signing_keys.py:286-287). The read-back command line holds only the account and service (:293), and the account and service names are checked as plain (:270, :281-283). Your test checks that no argv holds the key.
• Environment. generate puts the key into no environment variable.
• Logs and output.
  • Both security calls capture their output, and nothing prints it.
  • On failure, it prints the CalledProcessError text (:313), which holds the argv and exit code but not the stdin input.
  • A read-back mismatch prints only the service name (:298-301).
  • On success, only the fingerprint is printed (:316).
  • Plain generate refuses to print to a terminal (:318).
• Files. No temp file is written.

So the key never appears in argv, in an environment dump, in a log or in a file. One problem:

A. should: -U silently replaces an existing key (gate_signing_keys.py:287). The doc's own rotation steps trigger this. Step 1 uses the fixed service name microcosm-uk-gate-signing-key (docs/uk-cut-anyone.md:271-272), and step 4 says a retired key's cuts "still verify, but only for someone who holds that key" (:278-280). Anyone who rotates by following step 1 overwrites the old key with no backup, so after that nobody holds it. Its cuts then can't be verified, and step 4's promise no longer holds. Running it twice by mistake does the same to an active key. Please drop -U and refuse if the item already exists, or print a clear "already stored" refusal.

One fact for the doc, not a defect: an item that security adds without -T can be read back, with no prompt, by any process running as that user that calls security find-generic-password -w. That is how the doc's recipe reads it. Like Max's agent-secret, the login keychain shields the key from other users, not from the user's own agent sessions.

Gated, not private: for Vahid and Max

These are the facts. Read anonymously, policyengine/policyengine-uk-data-private reports private: false, gated: "manual". Its file names (the FRS, LCFS, NTS, WAS, SPI and ETB zips) are public. The files download only after a person approves an access request. On a gated repo, Hugging Face shows the owners the requester's details, but it doesn't check UKDS registration or licence cover. That check exists only if the approver does it by hand.

Whether that is enough for UKDS End User Licence data is a licence question. Vahid and Max should decide it with whoever holds the UKDS licences, not this PR. Two points to weigh: public file names that show which UKDS studies PolicyEngine holds, and access that depends on a manual approval step with no written check. The doc now describes the setup accurately, so nothing here blocks the PR.

New findings

B. should: modal_rebuild_uk_chronicle_artifact.py still sends HF_TOKEN to Modal in "plan (free)" mode (tools/modal_rebuild_uk_chronicle_artifact.py:17-19, :26-28, :74-80, :133). This is round 1 item 5 again, in the tool 89f513c didn't touch. _secrets() attaches HF_TOKEN whenever it is set. The doc's main recipe has the builder export HF_TOKEN (docs/uk-cut-anyone.md:31). So modal run tools/modal_rebuild_uk_chronicle_artifact.py sends a token with write access to the private repos as soon as it plans. The Chronicle data is public, but the token isn't. Its docstring and the doc (:217) still say "never stored as a Modal secret". Use the same opt-in as modal_uk_cut.py.

C. nit: the opt-in variable alone attaches the secrets, without --run. MICROCOSM_UK_CUT_MODAL_RUN=1 modal run … --label x (no --run) sends both secrets and only then refuses (modal_uk_cut.py:95 and :209). The doc says the variable is what attaches them, so this is just a sharp edge. It would help if the refusal said "credentials were already sent".

D. nit: secrets still reach third-party code in two places.

• The certifier passes HF_TOKEN and HUGGING_FACE_TOKEN, write tokens, to policyengine.py's uv pip install, uv sync and test suite (certify_uk_cut_in_policyengine_py.py:489). The H5 downloads run in the certifier's own process (:625), so the token may only be needed by certify-data. I haven't checked which policyengine.py steps read it.
• The Modal container passes its whole environment, signing key included, to uv sync before the runner scopes it (modal_uk_cut.py:141).

E. nit: the Modal refusal says export … before --run (modal_uk_cut.py:218), which goes against the doc's one-command-only advice.

CI

Run 40404384382 on 74721a944: lint, wheels, select-countries and integration-uk pass. engine-free, engine-us and engine-uk (3.13, 3.14) were still pending when I posted, so I can't say they are green. Your plan to re-run any engine-uk job that times out is fine.

What I ran

• Tests. In a fresh worktree and uv venv, with the key and HF variables unset, I ran the six touched test files: both registry suites, runner, Modal stub and both certify suites. 186 passed. I deselected 8 tests whose names mention generate, keychain or read_back, so that nothing created a key, even a throwaway one. The generate and keychain tests I read but didn't run.
• Reading. I read 833ea03, c3fe8f8 and 89f513c in full, plus the certifier's audit path, both Modal tools and the doc sections cited above.
• Not done. No key generation, no keychain access, no Verification run, no Modal runs and no builds. I didn't re-run Run the UK test suite and a smoke-rung build in CI #699's tests at f0870aa79.

Not approving here. Vahid will post his fingerprint separately.

🤖 Generated with Claude Code

[juaristi22]

@vahid-ahmadi, round 2 is addressed; the head is now 7eb54e8cb.

• A, -U overwrote an existing key (c2c9d0a).
  • generate --keychain-service now checks for an existing item first. If one exists, it refuses and leaves it untouched; otherwise it adds without -U.
  • The doc's rotation step stores each new key under a new service name, so a retired key stays held and its cuts stay verifiable.
  • The doc also states your fact: the login keychain shields a key from other users, not from the user's own processes or agent sessions.
  • If you already have an item under microcosm-uk-gate-signing-key, the command refuses rather than replacing it. In that case, pass another --keychain-service name.
• B, the Chronicle rebuild sent HF_TOKEN when planning (cb39f2c).
  • Planning is now plain python tools/modal_rebuild_uk_chronicle_artifact.py, which contacts no Modal service.
  • The token is attached only with MICROCOSM_UK_CHRONICLE_MODAL_UPLOAD=1, a separate opt-in from the cut's, so consenting to one tool doesn't consent to the other. A rebuild without --upload sends no credential.
  • The docstring and the doc no longer say "never stored as a Modal secret".
• C, the opt-in without --run. The refusal now says the credentials were already sent and nothing was built (cb39f2c).
• D, tokens reaching third-party code.
  • The certifier's uv sync, uv lock and uv pip install steps now get no Hugging Face token (7eb54e8). policyengine.py's audit, certify-data, UC floor and tests keep it, since they read the private cut repositories, and policyengine.py's dataset tests need it.
  • In the Modal container, uv sync now runs without HF_TOKEN or the signing key (cb39f2c).
• E, export … before --run. The refusal now asks for the credentials to be given to that one command (cb39f2c).
• The gated repo. Noted as a licence question for you and Max, outside this PR. It stays as it is for now.

Locally, the touched suites pass: both registry suites, runner, the Modal stub for both tools, both certify suites, and the local preflight. CI is running on the new head, and I'll hold further pushes until your fingerprint, so it can finish.

For the verification run, these are in place:

• the inputs, each on the Hub at its pin, including Chronicle 272c987 (runner-fetched and verified today) and the NTS and LCFS zips;
• the branch, pushed;
• the recipe under Verification in the description.

Two things to check on your side first:

• that your HF account is approved for the gated policyengine-uk-data-private and can read microcosm-uk-build-inputs, which run_uk_cut.py fetch will show;
• your fingerprint, which I'll register as uk-gate-vahid-ahmadi-2026 as soon as you post it.

🤖 Generated with Claude Code

[vahid-ahmadi]

@juaristi22 my signing key is generated and stored in my login keychain, under service microcosm-uk-gate-signing-key, using generate --keychain-service at 7eb54e8cb. Only the fingerprint below has left the keychain:

key_sha256 e9ab40bddffa31e67c3f7c4a5fba2fe3d5f8a71d3200f92ecbc6592b877a7eb1

Please register it as uk-gate-vahid-ahmadi-2026, custodian Vahid Ahmadi, held in his laptop's keychain.

🤖 Generated with Claude Code

[juaristi22]

@vahid-ahmadi, your key is registered as uk-gate-vahid-ahmadi-2026 (e9ab40bd…, active, held in your laptop's login keychain), in 928874f. You're clear to start the verification run from the head of this branch.

I checked the inputs on the Hub without downloading anything. Every pinned archive is at its revision, and the Hub's own sha256 for it equals the pin. That covers:

• FRS, SPI, WAS and ETB at a2039519;
• NTS at 559b69e9 and LCFS at 11051052;
• the enhanced FRS at a9e52499;
• Chronicle 272c987.

The HMRC workbook is still served by gov.uk at its pinned size. Access to the gated policyengine-uk-data-private has been granted to you. fetch is the first step, so it will tell you at once if anything is missing.

The recipe is under Verification in the description:

• run_uk_cut.py all --label verify-<date> --allow-off-main "María Juaristi: verification run before merging #1405";
• give the key to that one command;
• leave out --publish.

Please report only which steps passed and how long each took, from status and the receipt's steps, and no figures from the data.

CI restarted on the new head. If an engine-uk job hits the 30-minute limit, I'll re-run it.

🤖 Generated with Claude Code

[juaristi22]

@vahid-ahmadi, here is how to run the verification build from your laptop. It is the runner's first real end-to-end run, and the last check before #1405 merges.

What to expect

• What it builds. One Microcosm UK national cut, built from scratch at this branch's head:
  1. it fetches every input by its pin;
  2. preflight;
  3. spine (about 15 min);
  4. national dry run, then the national calibration (about 1 h);
  5. release-cut certification (about 25 min);
  6. assembly;
  7. the publisher's preflight.
• Total: about 2.5–3 h.
• What leaves your laptop. Nothing is published or promoted. Like every cut, the national step stages its bundle to the private policyengine/populace-uk-private and its telemetry to the private policyengine/populace-uk-staging.
• Your laptop needs:
  • 60 GB free disk, which is preflight's floor (the cut itself uses about 30 GB);
  • AC power, with the lid open for the whole run.
• Credentials:
  • your Hugging Face login, with read access to the gated policyengine-uk-data-private (granted) and to microcosm-uk-build-inputs, and write access to populace-uk-private and populace-uk-staging;
  • your signing key in the login keychain (done).
• Outputs go to ~/microcosm-uk-work, outside any repository. They derive from UKDS-licensed data, so keep them there: don't upload, share or commit them.

Steps

1. Check out this branch, clean, and sync it.

git clone https://github.com/PolicyEngine/microcosm.git ~/microcosm-verify && cd ~/microcosm-verify
git fetch origin claude/uk-cut-anyone-takeover-974500
git switch --detach FETCH_HEAD
git log -1 --format=%h        # expect 928874f14 or later
uv sync --all-packages --locked --extra uk

2. Check the key with check. It should print "ok": true and uk-gate-vahid-ahmadi-2026.

MICROCOSM_UK_TERMINAL_GATE_SIGNING_KEY="$(security find-generic-password -a "$USER" -s microcosm-uk-gate-signing-key -w)" \
  uv run --no-sync python -m microcosm.data.gate_signing_keys check

3. Fetch the inputs. This downloads about 9 GB and verifies every file against its pin. Every line should say ok. A 401 or 403 means a repository access is missing.

HF_TOKEN="$(cat ~/.cache/huggingface/token)" \
  uv run --no-sync python tools/run_uk_cut.py fetch --work-dir ~/microcosm-uk-work

4. Run the cut.

• The credentials are given to this one command and not exported. They go in as shell prefixes rather than through env, so they appear in no process list.
• caffeinate stops the Mac sleeping while it runs.
• Leave out --publish.

HF_TOKEN="$(cat ~/.cache/huggingface/token)" \
MICROCOSM_UK_TERMINAL_GATE_SIGNING_KEY="$(security find-generic-password -a "$USER" -s microcosm-uk-gate-signing-key -w)" \
  caffeinate -ims uv run --no-sync python tools/run_uk_cut.py all \
    --label verify-1405 --work-dir ~/microcosm-uk-work \
    --allow-off-main "María Juaristi: verification run before merging #1405"

all starts with preflight. If preflight refuses, it says why, and nothing has been built.

5. If a step fails, fix the cause and run the same command again. It resumes at the failed step without repeating finished ones, and never overwrites a finished output.

• An unfinished spine or national attempt is kept for inspection. Add --retry-spine or --retry-national to move it aside and start that step again.
• If staging hits Hugging Face's commit-rate limit, wait a few minutes and re-run with --retry-national.

6. Report back. When it finishes, it prints the assembled cut's tag and the receipt path. Post here only this:

uv run --no-sync python tools/run_uk_cut.py status --label verify-1405 --work-dir ~/microcosm-uk-work
python3 -c "import json,sys; r=json.load(open(sys.argv[1])); [print(k, s.get('status'), s.get('seconds')) for k, s in r['steps'].items()]" \
  ~/microcosm-uk-work/cuts/verify-1405-$(git rev-parse --short=9 HEAD)/cut_receipt.json

That is which steps passed and how long each took. Please post no figures from the data: no gate values, totals or file contents. If something fails, post the step name and the runner's REFUSED: line only. Don't paste from logs/ or the tools' output, since those can contain figures from the data.

🤖 Generated with Claude Code

[juaristi22]

@MaxGhenis, could you run the verification build for this PR on your build Mac? Vahid's laptop is too loaded to take a 3-hour run. This is the runner's first real end-to-end cut and the last check before #1405 merges. It will also be the first cut signed with the team key, uk-gate-team-2026, which agent sessions on your Mac can read, so it tests that key too.

What to expect

• Where it runs. On your build Mac, not through tools/modal_uk_cut.py. The Modal wrapper builds only from commits on main, and processing UKDS data on Modal still needs the data owners' OK.
• What it builds. One Microcosm UK national cut, from scratch, at this branch's head (17e8147cd, which includes main as of today, policyengine-uk 2.162.8):
  1. it fetches every input by its pin;
  2. preflight;
  3. spine;
  4. national dry run, then the national calibration;
  5. release-cut certification;
  6. assembly;
  7. the publisher's preflight.
• Total: about 2.5–3 h.
• What leaves your Mac. Nothing is published or promoted. Like every cut, the national step stages its bundle to the private populace-uk-private and its telemetry to the private populace-uk-staging.
• The inputs are ready. I checked each against the Hub's own sha256: FRS, SPI, WAS and ETB at a2039519, NTS at 559b69e9, LCFS at 11051052, the enhanced FRS at a9e52499, and Chronicle 272c987. The HMRC workbook is still served by gov.uk at its pinned size.
• Your Mac needs:
  • 60 GB free disk, preflight's floor;
  • AC power, with the lid open for the whole run;
  • a Hugging Face token that reads the gated policyengine-uk-data-private and microcosm-uk-build-inputs, and writes populace-uk-private and populace-uk-staging.
• Outputs go to ~/microcosm-uk-work, outside any repository. They derive from UKDS-licensed data, so keep them there.

Steps

1. Check out this branch, clean, and sync it.

git clone https://github.com/PolicyEngine/microcosm.git ~/microcosm-verify && cd ~/microcosm-verify
git fetch origin claude/uk-cut-anyone-takeover-974500
git switch --detach FETCH_HEAD
git log -1 --format=%h        # expect 17e8147cd or later
uv sync --all-packages --locked --extra uk

2. Check the team key with check. It should print "ok": true and uk-gate-team-2026.

MICROCOSM_UK_TERMINAL_GATE_SIGNING_KEY="$(agent-secret get microcosm/uk-gate-signing-key-team maxghenis)" \
  uv run --no-sync python -m microcosm.data.gate_signing_keys check

3. Fetch the inputs. This downloads about 9 GB and verifies each file against its pin; every line should say ok.

HF_TOKEN="$(agent-secret get HUGGING_FACE_TOKEN_MAX maxghenis)" \
  uv run --no-sync python tools/run_uk_cut.py fetch --work-dir ~/microcosm-uk-work

4. Run the cut.

• The credentials go to this one command only, as shell prefixes and not through env, so they appear in no process list.
• Leave out --publish.

HF_TOKEN="$(agent-secret get HUGGING_FACE_TOKEN_MAX maxghenis)" \
MICROCOSM_UK_TERMINAL_GATE_SIGNING_KEY="$(agent-secret get microcosm/uk-gate-signing-key-team maxghenis)" \
  caffeinate -ims uv run --no-sync python tools/run_uk_cut.py all \
    --label verify-1405 --work-dir ~/microcosm-uk-work \
    --allow-off-main "María Juaristi: verification run before merging #1405"

5. If a step fails, fix the cause and run the same command again. It resumes at the failed step without repeating finished ones.

• --retry-spine or --retry-national moves an unfinished attempt aside and starts that step again.
• A resumed cut accepts only the key that signed its earlier steps, so keep using the team key throughout.

6. Report back here, with only which steps passed and how long each took:

uv run --no-sync python tools/run_uk_cut.py status --label verify-1405 --work-dir ~/microcosm-uk-work
python3 -c "import json,sys; r=json.load(open(sys.argv[1])); [print(k, s.get('status'), s.get('seconds')) for k, s in r['steps'].items()]" \
  ~/microcosm-uk-work/cuts/verify-1405-$(git rev-parse --short=9 HEAD)/cut_receipt.json

Please post no figures from the data. If something fails, post the step name and the runner's REFUSED: line only. Don't paste from logs/ or the tools' output, which can contain figures.

🤖 Generated with Claude Code

[vahid-ahmadi]

@juaristi22 verification run at 928874f14, label verify-1405, on my laptop with my key (uk-gate-vahid-ahmadi-2026), no --publish. Steps and times only, no figures from the data.

Step	Result	Time (from log start and end)
fetch	passed: 10 of 10 inputs verified against their pins	about 10 min (first fetch, about 9 GB)
preflight	passed	about 23 min (11:28 to 11:51)
spine	done	31 min
national dry run	done	11 min
national	done	49 min
score	passed	(part of the certify window)
certify	refused	14 min, then stopped
assemble, publish preflight	not reached

The refusal:

REFUSED: certify exited 1; see .../cuts/verify-1405-928874f14/logs/certify.log
GateBatteryBlockedError: Gate battery blocked at phase 'terminal'
  - [uk_degenerate_release_surface] household.beer_draught_share: persisted release column is constant ...; populate it with signal, drop it, or record a reviewed exclusion.
  - [uk_degenerate_release_surface] household.cider_draught_share: persisted release column is constant ...; populate it with signal, drop it, or record a reviewed exclusion.

I've left the constant values out. Both columns look like they come from the alcohol inputs in #1381, which is in this branch through the main merge. If so, a cut from main may hit the same gate, not just this PR. cut_receipt.json wasn't written, because the run stopped before assembly. The cut directory is kept, so after a fix I can re-run the same command and it will resume at certify.

🤖 Generated with Claude Code

[vahid-ahmadi]

@juaristi22 a summary of what the verification run did, to go with the step times above.

Built on my laptop (about 12.5 GB, in ~/microcosm-uk-work/cuts/verify-1405-928874f14/, staying there and not shared):

• the spine (10 GB);
• the national candidate, with its build record and evidence (2.5 GB).

What left the laptop: only what the national step stages for every cut.

• The bundle went to the private policyengine/populace-uk-private, under staged/uk-frs-calibration-attempt-20261027T123319Z-… (about 200 MB).
• Telemetry went to the private policyengine/populace-uk-staging.
• Nothing was published or promoted: no --publish and no release tag. The cut failed certification, so it can't be used as a release. Delete the staged attempt if you'd rather not keep it.

What it shows:

• The runner works end to end up to certification on a second builder's machine, with that builder's own registered key.
• The terminal gate then caught a real problem: household.beer_draught_share and household.cider_draught_share are constant across every household.
• The cut directory is kept, so once that's fixed I can re-run the same command and it resumes at certify.

🤖 Generated with Claude Code