Skip to content

Add a certified pair registry so the £100k childcare lane can run on policyengine.py 6.11.0's Microcosm pair - #229

Merged
MaxGhenis merged 5 commits into
mainfrom
uk/childcare-100k-microcosm-6cba5d71
Oct 26, 2026
Merged

Add a certified pair registry so the £100k childcare lane can run on policyengine.py 6.11.0's Microcosm pair#229
MaxGhenis merged 5 commits into
mainfrom
uk/childcare-100k-microcosm-6cba5d71

Conversation

@MaxGhenis

@MaxGhenis MaxGhenis commented Oct 26, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

First half of #228: lets the uk-childcare-100k lane run on policyengine.py 6.11.0's certified Microcosm pair. That pair is policyengine-uk 2.132.6 with build microcosm-uk-2024-25-national-20261019T083323Z-6cba5d71, certified in PolicyEngine/policyengine.py#685. The run itself, the notes checked at 2.132.6 and the README update follow in a second PR. The run waits for a gap in the host bookings long enough to finish it.

Changes

  • Certified pair registry. data/uk/certified_pairs.json and pipeline/uk_certified_pairs.py record a policyengine.py release's certified bundle in the replay variant's identity shape, under the id pair-<variant slug>.
    • A pair passes the same identity checks as a variant. uk_replay_variants.validate_identity is split out of validate_record, and the round-1 review found the same 32 checks before and after the split.
    • The pair preflight is verify_installed followed by compute_uk_event.materialized_preflight, which is the second half of variant_preflight.
    • A pair is never a replay variant: it has no layout and no result root, the event runner and the experiments code refuse its id, and nothing writes replay_variants.json.
    • The pair is pinned by a requirements file of 82 pins. The policyengine 6.11.0 wheel sha256 it cites matches PyPI.
  • Compute and ingest resolve either kind of record (pairs.get_record, pairs.preflight, and ingest's run_records).
    • Ingest refuses a repeated id in either file or across them, a pair id without pair-, a variant id with it, a registry file of another kind or schema version, and an identity its digest does not seal.
  • Each child's own cap on funded hours (max_free_entitlement_hours_used) is now a recorded dataset input.
    • The full-use diagnostic needs it at its default of 30, shown by either the run's record or the pair's verified statement.
    • The 19 June pair's verified entry states that populace_uk_2023.h5 has no such column, read from the HDF metadata.
  • Zero-weight guard: year_block refuses a world whose benefit units weigh nothing in total.
  • Docs: docs/uk_replay/RELEASE_REPLAY.md gains a "Certified pairs" section, and ARCHITECTURE.md and the two lanes' annotation schemas name the new file.

Data impact

No number changes. The database content hash moves from e5497fd5 to 5fbff9ec for two rows only: the CBP 2031-32 rows' pe_missing reason now names both registry files. The re-exported feeds carry that text and the export date.

Invariants (tested)

  • Every committed pair validates under pair-<its slug>, and no id appears in both files.
  • Changing any identity value (string, integer or null) without resealing the digest is refused, in the pair's form and in the variant's form. This is a Hypothesis property test.
  • Given a variant's own fields, a pair passes validate_record (differential).
  • Ingest and the pipeline resolve every id to the same record (differential).
  • Ingest's copy of the identity digest equals uk_replay_variants.identity_sha256 on any identity of strings, integers, floats (NaN and infinity included) and nulls: both give the same digest, or both refuse (differential property test).
  • Eight malformed registries are refused by both ingest and the pipeline; an earlier version of this PR's ingest reader accepted each of them.
  • The preflight routes each kind its own way, and it refuses a changed or unprefixed pair view before verify_installed.

Review

GPT-6.1 Sol, three rounds:

  • Round 1, at 44174a17: AGREE, with two minor findings. Both are fixed in 1f2ef028.
  • Round 2, at 58e286fb: AGREE, with two nits. Both are fixed in 466df5af.
  • Round 3, at 466df5af: AGREE, no findings. It ran the shipped digest differential on 10 seeds (10 passed) and an allow_nan=True copy (10 failed).

Tests

At 466df5af the full suite has 2,842 passed and 18 skipped, run under a 4 GB memory cap with 2 workers.

  • One earlier full run had a single failure in tests/test_uk_replay_results_pr.py::test_job_2_guards_the_request_before_deciding_on_mains_tip.
  • That test passed alone, passed in three parallel runs of its file, and passed in a second full run.
  • git status after the suite shows no drift.

Refs #228

axiom: n/a: microsim/data lane

🤖 Generated with Claude Code

@MaxGhenis @claude
Let the childcare lane run on policyengine.py 6.11.0's certified pair
1024f06 
Adds a certified pair registry (data/uk/certified_pairs.json,
pipeline/uk_certified_pairs.py) for policyengine.py releases the lane
runs on before the release-replay pipeline qualifies them. A pair is
recorded in the replay variant's identity shape under the id
pair-<variant slug> and passes the same identity checks
(uk_replay_variants.validate_identity, split out of validate_record),
but it is never a replay variant: no layout, no result root, and
nothing writes the variant index.

The first pair is policyengine.py 6.11.0 with policyengine-uk 2.132.6
and Microcosm build microcosm-uk-2024-25-national-20261019T083323Z-6cba5d71,
pinned by a requirements file whose 82 pins the local venv matches.

The lane now reads each child's own cap on funded hours
(max_free_entitlement_hours_used) as a dataset input, records it in
the artifact, and treats it as part of the full-use evidence. The
19 June pair's verified entry states that the cap is not in its
dataset. year_block refuses a world whose benefit units weigh nothing.

Refs #228. No numbers change: the run on the new pair is still to come.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@vercel

vercel Bot commented Oct 26, 2026 •
edited
Loading

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
policyengine-scorecard Ready Ready Preview Oct 26, 2026 4:57pm UTC

Request Review

@MaxGhenis @claude
Document certified pairs alongside replay variants
44174a1 
RELEASE_REPLAY.md gains a section on data/uk/certified_pairs.json: the
identity checks a pair shares with a variant, why no event replay runs
on one, and why a pair stays once a variant arrives. ARCHITECTURE.md and
the two lanes' annotation schemas name the new file where they named the
variant index.

Refs #228.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis @claude
Refuse a stripped pair id and a registry ingest would read differently
1f2ef02 
Round-1 review of #229 (GPT-6.1 Sol, AGREE with two minor findings):

- pairs.preflight refuses a pair view whose id has lost its pair-
  prefix, before verify_installed; one once reached the materialised
  path under a variant's id.
- ingest's run_records refuses a repeated id in or across the two
  files, a pair id without pair- or a variant id with it, a pairs file of
  another kind, and an identity its digest does not seal. It used to keep
  the last of two rows and let a pair overwrite a variant, where the
  pipeline refused or resolved the variant. scorecard_db does not import
  pipeline/, so the digest rule is copied, and a differential property
  test pins it to uk_replay_variants.identity_sha256.

The tests also cover what the review noted they did not: null identity
leaves in the property test, refusal in a variant's form as well as a
pair's, and whole-record agreement between ingest and the pipeline. Each
of the seven registry mutations was accepted by the previous reader.

Refs #228.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis @claude
Re-export the feeds: two CBP rows cite the certified pairs file
58e286f 
The 2031-32 CBP rows' pe_missing reason names both files a certified
pair can come from (data/uk/replay_variants.json and
data/uk/certified_pairs.json). That text, and the export date, are the
only changes; the database content hash moves from e5497fd5 to
5fbff9ec for those two rows alone. No number changes.

Refs #228.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis MaxGhenis changed the title Run the £100k childcare lane on policyengine.py 6.11.0's certified Microcosm pair Add a certified pair registry so the £100k childcare lane can run on policyengine.py 6.11.0's Microcosm pair Oct 26, 2026
@MaxGhenis @claude
Close the round-2 review nits on the pair registry
466df5a 
GPT-6.1 Sol round 2 of #229 (AGREE, two nits):

- run_records refuses a variants file whose schema_version is not 1, as
  the pipeline's loader does; the registry-mutation test gains the case.
- The digest differential draws floats, NaN and infinity included, and
  compares the two rules' digest or refusal; an allow_nan=True copy,
  which the round-2 mutants showed surviving, now fails it.

Also sets deadline=None on test_no_local_data_manifest_survives_isolation,
as its neighbours have: the review's first run under host load failed it
on Hypothesis's 200 ms deadline (319 ms), and it passed alone.

Refs #228.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis
MaxGhenis marked this pull request as ready for review October 26, 2026 17:28
@MaxGhenis
MaxGhenis merged commit 3a20920 into main Oct 26, 2026
13 checks passed
@MaxGhenis
MaxGhenis deleted the uk/childcare-100k-microcosm-6cba5d71 branch October 26, 2026 17:37

This branch was successfully deployed

1 active deployment
Preview — 466df5af Deployed Oct 26, 2026 by vercel[bot]
View deployment
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@MaxGhenis