Repository navigation
Bind APS employment by SOC 2020 major group by region and sub-major group by sex, on the chronicle#411 feed (#1490) - #1520
Conversation
Chronicle main 6475380 (PolicyEngine/chronicle#411 on top of 08962f1) brings the UK facts of PolicyEngine/chronicle#407, which microcosm#1490's occupation targets read: the ONS Annual Population Survey's people in employment by SOC 2020 major and sub-major group (Nomis APS218, three twelve-month periods from April 2024 to March 2026) and ASHE Table 2's gross weekly and annual pay by occupation (73,039 rows; 1,585,295 in all). The merge tree differs from the pull request head's, so the artifact was built from the merge commit itself. The rows move to chronicle.consumer_fact.v5 (PolicyEngine/chronicle#377, one label per dimension value). Microcosm declares the v5 row contract beside v1 to v4 in chronicle_epoch.py: the per-row id is carried, never gated on, and labels never enter a fact key. A row-by-row diff of the two artifacts finds no row dropped and no value moved; 183,848 rows change display labels only. No compiled value moves on either surface: both reference files and both compile-parity receipts are byte-identical, which, since a receipt lists every compiled target unless it equals the frozen fixture exactly, means every resolved target value is unchanged. The national membership moves its feed label, the local membership its feed label and the 1,263 DESNZ constituency keys (their derived keys hash the feed identity), the vendored resources and the DESNZ constituency resource carry the same rows under the new feed identity, the census restates the pin, and the H2 fixture moves the parameters of the 19 stages that bind vendored resources. The high-value home stock's feed inputs are unchanged, so it is not rebuilt. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The UK contract binds nine calibration targets from the ONS Annual Population Survey (Nomis APS218, January to December 2025, through PolicyEngine/chronicle#411): people in employment in each SOC 2020 major group, all persons and all employment types, each over the twelve region-tier areas (108 references, family ons_occupation, category ons.employment). María's ruling of 4 November 2026 chose the majors by region; sub-major groups are not targets, since the sub-major draw takes them from the same APS records. Each target counts the people whose soc2020_major_group is that group and whose employment_status is full-time or part-time employed or self-employed, summed to households, as ons.public_sector_employment does. Out-of-work adults keep their last job's group and are not counted; the APS also counts unpaid family workers. The selector picks the estimate concept by its dimensions; the confidence intervals are concepts of their own and are not read. The twelve areas of each group sum to the APS's UK figure within its rounding to the nearest hundred. The national references gain the 108 rows and nothing else moves: the national compile-parity receipt adds 108 ledger-only rows, every other entry unchanged, and the local surface cannot move (the local generator keeps only targets with local geography levels). employment_status and soc2020_major_group join the reviewed plain-column filters. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Arm results for this PR. Both arms ran locally on 5 November, one after the other on a quiet machine. Each built a full spine and ran a national calibration, with the same review date (2026-11-04) and the re-pinned ledger (64753803). Nothing was uploaded.
The two spines hold the same data, and every existing row's support in the calibration matrix is identical. So the 108 references are the only difference between the arms. The bound cells Each of the 108 cells rests on at least 30 survey households. The gaps to the APS, control → head:
Occupation cells the targets don't bind The same APS release gives the 26 sub-major groups, and splits every major group by sex and by employee or self-employed. None of these cells is bound, so they show whether the targets improve occupation beyond the cells they fix. The median gap, control → head, with the design weights in brackets:
What the targets don't fix is the mix inside a group. They set each group's total in each area, and the split below that comes from the frame's records and the sub-major draw. Where the split is off, moving the total can push a part further from the APS. On the head, these UK cells sit outside the APS's confidence interval (control in brackets):
The other rows
Time
🤖 Generated with Claude Code |
…ex (#1490) The UK contract binds 52 more calibration targets from the ONS Annual Population Survey (Nomis APS218, January to December 2025, through PolicyEngine/chronicle#411): people in employment in each of the 26 SOC 2020 sub-major groups, for women and for men, at UK level (family ons_occupation, category ons.employment). María's ruling of 5 November 2026: with only the major groups bound, the sub-major draw left 16 of the 26 UK sub-major groups, and the mix of women and men inside many of them, outside the APS's confidence intervals on the calibrated frame. The sub-major groups by region are not bound: their intervals are wide, and in an offline cross-validation of the national solve, binding them did not improve the cells held out. Each target counts the people whose soc2020_sub_major_group is that group, whose gender is that sex and whose employment_status is full-time or part-time employed or self-employed, summed to households. Women and men together fix each sub-major group's UK total, which nests in its major group's regional rows, so the totals are not bound separately. Nomis rounds each cell to the nearest hundred, so each pair adds to the all-persons figure within 100. soc2020_sub_major_group joins the reviewed plain-column filters. The national references gain the 52 rows and nothing else moves: the national compile-parity receipt adds 52 ledger-only rows, every other entry unchanged, and the local surface cannot move. The changelog fragment now covers the major and the sub-major rows. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#1510 merged into main first with the same chronicle.consumer_fact.v5 declaration this branch's re-pin made in chronicle_epoch.py, and with checks that refuse a feed whose pin or rows name an undeclared row contract. The merge keeps #1510's chronicle_epoch.py and test_chronicle_epoch.py. This branch's pin and rows both name v5, so those checks pass on it; main's other changes leave the regenerated references and receipts as they are. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Review of bde6d3f. I read the 61 new contract entries in Should
Nit
Checked and fine
CI at bde6d3f: all 11 checks pass (lint, engine-free 3.13/3.14, engine-uk and engine-us 3.13/3.14, integration-uk, wheels, installed-emitter, select-countries). The major-group rows look ready. The sub-major-by-sex rows need the support numbers and probably a trim of the thinnest cells before this leaves draft. 🤖 Generated with Claude Code |
…e occupation rows select (#1490) Vahid's review of bde6d3f. The two UK rows by sex for community and civil enforcement (63) are the thinnest of the 52: their APS intervals are 60% (women) and 29% (men) of the estimates, and few households on the frame carry the group. So 63 now binds only through its major group's regional rows. Every other row rests on at least 40 distinct FRS households; women in skilled metal and electrical trades (52) and in skilled construction (53), whose intervals sit above 20%, rest on at least 45 each and stay. A new engine-free test pins what every ons.occupation row selects: the APS estimate concept, all employment types, calendar 2025, the four in-work statuses, one sex per UK row, only major groups over the region tier, and the groups bound. It also checks on the committed receipt that each major group's rows by sex fit inside its regional rows. National references -2 and the national compile-parity receipt -2 ledger-only rows; nothing else moves. The sub-major groups stay unbound by region: too many of their area cells rest on few households (María, 5 November 2026). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Thanks, Vahid. Pushed 1. Thin UK cells by sex. The household support of the 52 rows by sex, on the sub-major arm's frame (distinct FRS households):
Sub-major groups by region. We also looked into binding the sub-major groups over the region tier, but those cells are too thin: 61 of the 300 area cells rest on fewer than 30 distinct FRS households, and 48 have APS intervals over 20% of the estimate. Binding them would stretch a few households' weights to fit noise, so they stay unbound. We can look at this again once the spine pools several FRS years and, if it does, carries more clones. 2. The arm for the sub-major head (
3. Test.
It also makes your nesting check permanent: on the committed receipt, each major group's rows by sex must fit inside its regional rows, and must make up all of it, within Nomis's rounding, wherever every sub-major group binds. 🤖 Generated with Claude Code |
|
Published for inspection: the sex-split build is on
🤖 Generated with Claude Code |
|
Round 2 at 9de8ba6. I read the one new commit ( Earlier items
New Nit
Checked and fine
CI at 9de8ba6: all 11 checks pass. Verdict: all three round 1 items are done. Item 4 is a small tightening, and the head's own measurement of group 63 is the only open point before this leaves draft. 🤖 Generated with Claude Code |
|
Round 3 (post-merge), on the published cut Earlier items, on
Measurement on this cut The stored weights sit at the dataset year: the release divides the solved weights by the engine's 2025 population growth (0.33%,
So the cut reproduces your arm for Sub-major groups by region (not bound) APS218 gives a numeric estimate and interval for 301 of the 312 cells. The 11 without are all group 63. Against those 301:
Checked and fine
Verdict: the published cut fits the 160 rows it binds, within 0.41%. The two round 2 points stay open on 🤖 Generated with Claude Code |
Follows #1511 (merged), which carried the occupation stages of #1490 (from #1512). This PR adds the calibration targets those stages are fitted to. It now targets
main, which is merged in; the only conflict was the v5 row declaration #1510 landed first (see section 1).Summary
The UK contract binds the ONS Annual Population Survey's people in employment by SOC 2020 occupation, from Nomis APS218 for January to December 2025, which reaches Microcosm through PolicyEngine/chronicle#411. There are 158 references:
Community and civil enforcement (63) has no rows of its own and binds only through its major group's regional rows. The sub-major groups are not bound by region. Both points are covered in section 4.
The APS facts are only in Chronicle after PolicyEngine/chronicle#411, so the PR first re-pins the UK feed. It has four commits: the re-pin, the major groups, the sub-major groups by sex, and the fixes from the review.
mainis merged in between the third and the fourth.1. Re-pin the UK feed onto the merge of chronicle#411
The feed moves from
08962f1to6475380(PolicyEngine/chronicle#411 on top of08962f1), 1,585,295 rows. The merge's tree differs from the PR head's, so the artifact was built at the merge commit itself.chronicle.consumer_fact.v5(Give each dimension value one Chronicle-declared label, keep the package's own wording beside it (consumer_fact.v5) chronicle#377, one label per dimension value).chronicle_epoch.py. Declare Chronicle consumer_fact.v5 and refuse feeds on undeclared row contracts #1510 merged intomainfirst with the same declaration, plus checks that refuse a feed whose pin or rows name an undeclared row contract. The merge ofmainkeeps Declare Chronicle consumer_fact.v5 and refuse feeds on undeclared row contracts #1510'schronicle_epoch.pyand its test.The runbook entry is in
docs/uk-chronicle-feed-repin.md, and the fragment ischangelog.d/1490-uk-feed-repin-6475380.changed.md.2. Bind the APS's people in employment by SOC 2020 major group
ons.occupation.<group>_by_region, familyons_occupation, categoryons.employment. Each is a two-level target that fans out over the region tier.soc2020_major_groupis the group and whoseemployment_statusis full-time or part-time employed or self-employed, summed to households, asons.public_sector_employmentdoes.employment_statusandsoc2020_major_groupjoin the reviewed plain-column filters.3. Bind the APS's people in employment by SOC 2020 sub-major group and sex
ons.occupation.<female|male>_<sub-major group>, familyons_occupation, categoryons.employment. Each is a UK-level row, like the ONS population rows by sex.soc2020_sub_major_groupis the group, whosegenderis the sex and whoseemployment_statusis in work, summed to households.soc2020_sub_major_groupjoins the reviewed plain-column filters.4. Fixes from the review
test_uk_occupation_targets.pypins it:The fragment
changelog.d/1490-uk-occupation-targets.added.mdcovers sections 2 to 4.Testing
test_h2_uk_spine_paritypasses.main: the 15 UK and shared test files this PR ormaintouches, Declare Chronicle consumer_fact.v5 and refuse feeds on undeclared row contracts #1510's row-contract tests among them: 1,093 tests, all passing; the 7 skips need the feed.df0323743, the 52 rows by sex) against the major-group arm, both with review date 2026-11-04:--dry-run, review date 2026-11-05) passes. It compilesuk.full.national_targetsfrom the pinned feed with the build's own loader, including the reconciliation, cross-grain coverage and subset checks.main's merge, which adds checks only. The sub-major commit's build is published for inspection (comment below).🤖 Generated with Claude Code