Repository navigation
UK #1095 A/B fixes: the SPI age band (uk-data#628), households' HRAD from Table 8a receipts, and the SPI pay floor (uk-data#629) - #1426
juaristi22 wants to merge 42 commits into
Conversation
…628) The SPI tape gives age only as a band (AGERANGE), so the age drawn for each donor inside its band carries no income pattern: the stage-1 forest learned none, and twins in one band faced conditionals that differed only by that noise. The tape and reference forests now read the band code instead: the tape's own code, composite records (-1) as a band of their own, and the band of a frame person's whole-year age. at_or_over_spa keeps the State Pension split inside a band. The cross-fit fold cells key on the band, since they key on the predictor values. The drawn ages stay on the donor for the band pools. Regenerated: source_stages.json (in step with sources.yaml), the release input coverage manifest and the H2 parity fixture. The SPI engine tests read the band and count the reference forest once per cross-fit fold in use. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e 8a receipts The 2024-25 residential nil-rate band was GBP 250,000 all year, so a purchase at or below it owes no standard-rate SDLT and other buyers' Table 8a receipts there are surcharge. At Table 6b's gross HRAD receipts per transaction and per pound they imply about 48,100 HRAD transactions and GBP 7.05bn for other buyers, where individuals' share of all transactions, uk-data#573's rule, left them about 13,400. Households now take Table 6b's band less that; above it the shares stay (solving from receipts there goes negative above GBP 1m). The household bands become the price mix. Households' target: 151,928 transactions and GBP 63.71bn (was 186,589 and GBP 68.81bn), above uk-data#573's lower bounds; the lowest band carries 43% of households' purchases (was 54%). Published HMRC figures only. Regenerated: source_stages.json (in step with sources.yaml) and the release input coverage manifest. The H2 parity fixture is regenerated. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Stage 1 of hmrc_spi_income_spine draws a recipient's annual tape pay on age, sex, region and the bridge predictors, not on hours, and the tape holds part-year workers. A copy whose FRS twin reports usual hours in an employee job could therefore draw a year's pay below what those hours earn at the minimum wage, and fail the 30 hours' and Tax-Free Childcare minimum-income tests more often than FRS rows. A new floor_spi_employment_pay_at_minimum_wage operation lifts the drawn hmrc_spi_pay leaf to weekly usual hours (hours_worked read back at 365.25/7) times 52 weeks times policyengine-uk's gov.hmrc.minimum_wage.non_apprentice rate for the twin's age, read at 30 April of the build year as the engine reads it (2024-25: 11.44 from 21, 8.60 from 18, 6.40 below). It runs after the stage-1 draw, the band-donor resample and the State Pension leaf guard, and before employment_income is derived from the leaves and stage 2 conditions on it; the transport receipt keeps the unfloored draws' uniforms. Only drawn recipients other than band carriers, whose twin has an employee main job, no trade and usual hours, and whose drawn pay is positive, are floored. Tape PAY is taxable pay; PAY alone is compared with the gross minimum because it is the leaf of the engine's gross employment_income input. The operation is declared in sources.yaml, its source_stages.json projection, the sources schema, the operation vocabulary and the HMRC income contract, and checked against a reviewed declaration. The stage evidence gains an employment_pay_floor receipt: source households rounded, weighted figures "<10" on one to nine source households, no row's value. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The employment pay floor receipt gave the floored weight, the share floored and the amount added beside the eligible weight, each suppressed only on the floored side's support. Any of them with the eligible weight gives back the weight of the eligible rows the floor left alone, which could rest on one to nine source households. They are now "<10" whenever either side rests on one to nine, as the Carer's Allowance take-up receipt gives its share; the mean uplift alone does not give a side back and keeps the floored side's rule. The receipt declaration in sources.yaml, source_stages.json and the reviewed spine mapping says so. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…, gate hourly pay uk-data#629: drawn SPI employees keep their FRS twin's hours while the tape redraws their pay, and the tape's pay is the tax year's, so part-year tape pay landed on current twins at very low hourly rates. On top of the floor from uk-cc-spipay: - The floor rate is the lower of the minimum wage for the twin's age and the twin's own hourly pay (its employment income before the draw over its annual hours), so no copy is held to a higher rate than the FRS person it copies; hours now read back at 52 a year (uk-data#531). - The floored pay is the row's current pay only: its annual share becomes the unfloored over the floored employment income, so the annual measures (the HMRC income rows) still read the tape's pay. - The receipt adds an hourly-pay check: the drawn employees' weighted share below the minimum wage before and after the floor, beside the FRS employees' share, and the eligible rows still below their floor; a share is withheld beside a side on one to nine source households. - A release-blocking stage gate, uk_stage_hmrc_spi_income_spine_hourly_pay, holds the floor invariant and the drawn share to the FRS share plus two points; it joins the spine gate scope, and the battery and contract digests move with it. Regenerated: source_stages.json (in step with sources.yaml), the sources schema, the release input coverage manifest and the H2 parity fixture. The floor's engine test gives each twin and its copy one age and pay, reads hours at 52 weeks, and checks the twin cap, the kept year of pay and the employer contribution that follows the floored pay. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
María's ruling of 2026-10-29 (#1095). The SPI employment pay floor gives fewer low-paid SPI copies Universal Credit, so single people's caseload before the take-up redraw falls close to DWP's, and the redraw's usual downward move reads as about twice the 0.055 allowance; after the redraw single people end a little below the spine without the floor, all in the middle award range. The shortfall of middle and higher awards that the redraw cannot fill is the award fix's (#1095's UC awards below the top band), and the tolerance returns to 0.055 when it lands. The one tolerance also bounds the reachable pools, which realize DWP's count far more closely. The contract re-pins the gate battery's policy, manifest and fingerprint digests and the release certification's spine part; the stage-health test pins 0.12 and adds the floor's pattern. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
@juaristi22 thanks, this is my review of Verdict: the three items look sound, with four should items and a nit. The age band is the right fix for #628. The nil-band arithmetic reproduces the body's numbers: £265m over £625m/113,500 is about 48,100 transactions, and the target falls by about 34,700. The pay floor runs where it should, after stage 1 and the uprating and before Findings
What I ran. I read the full diff at 🤖 Generated with Claude Code |
Vahid's review of #1422 (its second and third points). - Since the move to a floor of three, rounded_count printed three to nine exactly, while extreme_count and support_band print "3-9" and the module's own rule is that no exact count narrows a small group by difference. It now reads "3-9" there too. Two tests that read an incidental exact count follow. - The module docstring and the changelog say that the SPI Public Use Tape's two conditions (SN 9422) are unchanged by the move to three. - A guard test fails when a UK runtime module, the UK spec, the gate notes or a tool still writes the old "<10" label, so that a receipt left on the floor of ten cannot land unnoticed. #1426's SPI pay-floor receipt writes it today; whichever of the two lands second moves it onto UK_DISCLOSURE_FLOOR. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#1426 now builds on #1422, which moves every UK disclosure floor to three (receipt_counts.UK_DISCLOSURE_FLOOR) and regenerates the national surfaces on Chronicle 53815b2 with policyengine-uk 2.165.1. - The SPI pay-floor receipt withholds through UK_DISCLOSURE_FLOOR and UK_WITHHELD: a figure on one or two source households reads "<3" where it read "<10" (Vahid's review of #1426). Its description in spi_spine.py, sources.yaml and source_stages.json, the hourly-pay gate's note and docstring, and its tests follow. Two test cases shrink their small side to two households, so they still test the withholding. - Conflicts: the SPI income imports (both sides kept), the stage-1 fold key (#1426's age band) beside #1422's savings-interest method text, and the UC take-up gate test. - Regenerated: the release input coverage manifest. The gate battery and spine-part digests are re-pinned. - The H2 parity fixture takes #1422's side here; a later commit regenerates it on the merged tree. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
hours / 52 * weeks_in_year is a year's paid hours, not weeks (Vahid's review of #1426). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
uk_stage_uc_take_up's one tolerance bounded two checks: how far a family type's caseload may move from DWP's across the redraw, which the SPI pay floor needs at 0.12, and how closely the reachable pools realize DWP's count, which the draw meets within 0.0015. The pools now keep 0.055 as maximum_reachable_pool_deviation while maximum_caseload_deviation stays at 0.12 until the UC award fix of microcosm#1095 (Vahid's review of #1426). The gate battery and spine-part digests are re-pinned. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
spi_age_band was an ordinal predictor, so any split that did not separate the composite records' code (-1) from 16-24 put the tape's high-income amalgams in the leaves 16-24 queries land in. The band is now one of the stage-1 categorical predictors, one-hot encoded with gender, region and main source, so composites are a category of their own (Vahid's review of #1426). The predictor rule, the spec's categorical list, the changelog and the coverage manifest follow; the cross-fit fold cells still key on the band's value. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Table 8a's receipts do not split out the 2% non-resident surcharge, which a non-resident buyer pays on top at or below GBP 250,000. Read as HRAD surcharge, it raises other buyers' implied HRAD and lowers households', so households' nil-band purchases are, if anything, understated (Vahid's review of #1426). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The stage-1 queries carry the age band's one-hot columns, not the band itself, and the pay floor's four floored households read "3-9". Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
No conflicts. The stricter floor guard passes on #1426's SPI and HRAD code, as do the ledger, reference, stage and graph tests. The gate digests and the coverage manifest are unchanged, and the H2 fixture regenerates byte-identical. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
No conflicts. The broader floor guard passes on #1426's SPI and HRAD code, as do the ledger, gate, reference, stage and graph tests. The gate digests and the coverage manifest are unchanged, and the H2 fixture regenerates byte-identical. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#1422 landed (11c1f4e), followed by #1430's company cars and #1432's default load. Only pinned and generated files conflicted. The gate battery and spine-part digests are re-pinned on the merged gates spec (five move), and the release input coverage manifest and the charter-H2 fixture are regenerated; the fixture checks byte-identical. policyengine-uk stays at 2.165.1, the lock main and #1428 share, and the feed pin moves to main's 08962f1. The H2 parity test and 86 related engine-free files pass (5,682 tests, 53 skipped). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
@vahid-ahmadi thanks for the review. All five points are addressed. #1422 has landed, and main is merged in at
The H2 fixture is regenerated on the merged tree ( |
María's national control for Phase B (cut f6e33b93) was built from #1428 at 329a99e, so #1426 takes #1428 at that commit: each arm is the control's tree plus one of #1426's items. #1428's later SPI banding change (2dbf7cf) comes with main. spi_income.py keeps both sides' additions: the employment pay floor beside the gift anchor and the annuity and trust clear. The UC take-up gate's note combines #1428's pool joining (María's ruling of 2026-10-31) with #1426's 0.12 from the pay floor (2026-10-29); its parameters stay at 0.12 for the caseload and 0.055 for the reachable pools, and the stage-health test's comment says the same. The gate digests are re-pinned (four move), and the coverage manifest and the charter-H2 fixture are regenerated; the fixture checks byte-identical. The H2 parity test, the SPI engine tests and 94 related engine-free files pass (5,802 tests, 54 skipped). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
María's ruling of 2026-11-02: with #1428's pool joining, every family type ends closer to DWP's count after the redraw than before it on the T1426 spine (#1426 at 97fd4d0), the SPI employment pay floor included, so maximum_caseload_deviation needs no wider allowance. The reachable pools keep their own maximum_reachable_pool_deviation, also 0.055 (Vahid's review). The gate's note says so, the test checks the 0.055 boundary and still tells the two tolerances apart, the changelog fragment is renamed and rewritten, and the gate battery and spine-part digests are re-pinned. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
@vahid-ahmadi a follow-up on point 3: #1428 is now merged in at the control's commit, and on the T1426 spine every family type ends closer to DWP's count after the take-up redraw than before it. So the caseload check is back at 0.055 ( |
…ge (#1095 B5) The stage's earnings test now reads the pay salary_sacrifice converts into pension contributions. Running after salary_sacrifice puts it after the CGT stages' copies, so each draw is keyed on lineage: a member of a CGT support copy, incidence clone or residential arm draws as the member at the same rank of the household it copies (lineage_roots), and the balanced placement takes a source and its copies in one stratum as one record of their joint weight (lineage_balanced_positions), so they claim together. Original rows keep their draws, and the one-record check reads a record as the rows of one lineage. The spec block moves after salary_sacrifice with its notes and the reviewed rate derivation; source_stages.json keeps its key order (the HMRC contract reads the retained constituents in order). The roster pins, the coverage manifest and the H2 fixture follow. New tests cover the roots, the one position per lineage and the stage itself with every carer cloned at half weight, which claims as the uncloned frame does; keying the draws on person ids fails it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s two B-1's first spine stopped at carers_allowance_take_up: "Carer's Allowance identities must name each record once". Salary sacrifice draws each CGT copy's pay on its own id, so a copy's earnings band, and so its stratum, can differ from its source's, and the lineage is then a record of each stratum. stratum_starts keys each stratum on its own members but refused an identity repeated across strata, so the starts are now drawn stratum by stratum, where an identity names one record. Positions where no lineage spans two strata are unchanged. Two tests cover it: a lineage in two strata is placed as each stratum alone, and the stage draws a clone whose pay left its source's band. Both fail on the single call. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…#1095 B12) The dense build's post-geography band node now draws each Scottish owner clone's band from its council area's chargeable dwellings by band, A to H, at September 2025 (Scottish Government CTAXBASE 2025), at the same uniform as its value: its rank among Scotland's owners. The counts are vendored from Chronicle 08962f1b into uk/scotgov_chargeable_dwellings_by_band.json (256 rows, 32 council areas) and pinned as the graph source uk_scotgov_band_mix, with a --scotgov-band-mix option beside --voa-band-mix. Renters keep their band, as in England and Wales, and so does Northern Ireland. The national build does not localise property, so it does not change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The two-child-limit target rows counted a child as born after the cutoff when the engine's birth year was 2017 or later, so a child born from January to 5 April 2017 read as affected although policyengine-uk exempts them. The measure now reads the engine's own uc_is_child_born_before_child_limit (a date of birth before 6 April 2017), as the child element does. The comparison contract and the 15 TCL target notes say so, and a test pins a third child born in early 2017 as unaffected. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Nothing set policyengine-uk's child_limit_non_parental_caring_exception, so the engine paid no child element for a third or later child born from 6 April 2017 who lives with kinship or friends carers. frs_relationships now marks a child-table member whom no adult of their own benefit unit claims as a natural, adopted or step child (grid codes 3 and 4 from the child, 7 and 8 from the adult) and whom no local authority placed there. That includes a baby whose parent is a child of the claimant's unit. The rule is declared in the stage and the spec schema; the column joins the graph's stage cells and the reviewed extra export columns, and leaves the signed engine-default register. Multiple births, adoption and non-consensual conception stay at the default, as María ruled on 2 November: the FRS has no twin field and its birth dates are blank. Gate digests re-pinned, coverage manifest and H2 fixture regenerated. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…itted-work limit (#1095 B10) spi_benefit_coherence zeroed contributory JSA on every SPI-channel row and left contributory ESA with the stage-2 forest's draw, which knows nothing of the benefit unit's claim history. A contributory award follows the person's National Insurance record and incapacity, which the copy keeps; what the copy changes is its work. An SPI row now takes its FRS twin's contributory JSA and ESA where its own earnings (employment income plus any self-employment profit, over 52 weeks) stay within ESA's permitted-work limit, and none above it (María's ruling of 29 October 2026). The limit is policyengine-uk's own esa_exempt_work_higher_earnings_limit at the frame's year (GBP 183.50 a week in 2024-25), read from a one-person simulation. The new restore_spi_channel_reports_from_source_within_earnings_limit operation is registered in the source manifest and the sources schema. The stage-health gate checks the rule on every SPI row (restored_within_earnings_limit_columns), and contributory ESA leaves the QRF tail-concentration surface, since no forest draw survives in it. Gate digests re-pinned; coverage manifest and H2 fixture regenerated. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
was_wealth draws the household's student-loan balance from the WAS and splits it among the members before any plan exists, so the balance could sit on a member with no loan while the household's borrower held none. policyengine-uk caps a borrower's repayment at a positive balance, so the split decides which repayments the balance can cap. Once student_loans has set the plans, it now moves each household's balance to the members who hold one: by their repayments where any of them repays, and equally otherwise. A household with no borrower keeps was_wealth's split, and no household's total moves (checked to the penny). The stage declares the re-split as a derive operation and student_loan_balance among its rewrites, and its receipt records the households re-split and the balance on borrowers before and after. The stage now rewrites a cell the open-surface Carer's Allowance reader before it binds, so it opens a reader-isolation boundary in the UK graph. Coverage manifest and H2 fixture regenerated. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…1095 B4) Two national rows from HMRC's Child Benefit statistics (August 2025, Table 16, United Kingdom): the Self Assessment taxpayers liable to the High Income Child Benefit Charge in 2024-25 (persons whose CB_HITC is above zero) and the charge's revenue (the sum of CB_HITC), both the publisher's provisional figures. María's ruling of 3 November 2026: measured diagnostics first, using 2024-25's figures for the model's 2025. - Contract: two hmrc_child_benefit rows; the selectors pin the 2024-25 record sets and the tax year, since the feed holds the series from 2012. - Held out of the solve by two signed exclusions: the model's charge sits well above HMRC's, a gap in who receives Child Benefit at high incomes (microcosm#1288 item 10) rather than a weight question, and HMRC's count covers Self Assessment returns only. The evaluation measures them on every candidate. - Measured at the calibration year under a signed reviewed hold: the charge's thresholds are the same in 2024-25 and 2025-26. - Regenerated on the pinned feed: the national references and membership (1,420 active) and the national compile-parity receipt (two ledger-only rows). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
@juaristi22 Round 2 at Earlier points
On your point-3 follow-up. I agree the caseload check can go back to 0.055 and nothing is left to track. One caveat on where the evidence comes from. The gate note cites the T1426 spine at New findings
Checked and fine
CI at Verdict: my earlier points are all fixed and I agree with the 0.055 return. B10's JSA limit is the one item to settle before the JSA row's movement is quoted. 🤖 Generated with Claude Code |
…ts household share (#1095 B3: PR 2b J2 + K) The LCFS consumption stage now reads the National Travel Survey's cars (UKDS SN 5340, End User Licence; the vehicle and household tabs, survey years 2022-2024, at the interview weight W3), as the 16 October plan designed J2 and K. - J2: each car is zero-emission at VEH1103's licensed-car share times its household income fifth's relative risk among the cars NTS households keep, normalised over the frame's cars so the national share is unchanged (rule per_car_zero_emission_share_by_income_quintile). The frame ranks household_gross_income over the engine's before-housing-costs equivalence scale, as the NTS ranks gross income McClements-equivalised. A car is zero-emission in the DVLA-linked 0 g/km band; a car without a linked band is left out. The uniforms are unchanged, so only each household's threshold moves. - K: level_road_fuel keeps ONS 07.2.2's household level but levels each fuel to its share of household-paid spend (mix rule household_paid_litres): HMRC's litres times DESNZ's cars share times the share of car miles a household pays for (a kept car's private and commuting miles), at the 2024 pump prices. A car whose miles the release does not split is left out; a blank is missing, never a zero. The road_fuel_level gate rechecks each fuel against the mix it recomputes from the vendored rows. - Spec: both operations declare the same nts_cars block, which the code refuses to see differ; the NTS household tab joins lcfs_consumption under the bus stage's pin, and the vehicle tab is a new pinned private input (--nts-vehicle-tab). - obr.fuel_duties_cars stays held, its reason restated with the household-paid shares (María's ruling of 29 October 2026). - Surfaces: the schema, the source-stage projection, the graph's split sources, the spine tool, the fixture kernels, the H2 fixture (a synthetic vehicle table, W3 on its households), the coverage manifest, and the kept-tens register (the pooling floor of 10 is statistical). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The spi_support_channel stage draws its 10,000 FRS households uniformly and gave every copy in a region x pension-age stratum the same prior weight, which reproduced the FRS respondent mix rather than the design's population (young families, single adults and one-earner couples over- represented, households of three or more adults under-represented). Copy j of stratum s now takes M_s * share_s * f_j / sum_k f_k, with f the copy's source-household design weight (the incoming raw FRS GROSS4) over the stratum copies' mean, damped to the square root. Each stratum's channel mass stays exactly its declared share and the pension-age share stays 0.2; the hosts and their incomes do not move. The shape is declared on allocate_zero_weight_prior_mass (prior_shape: design_damped, prior_shape_exponent: 0.5; design_bounded with prior_shape_bounds is also understood), validated by the schema and refused by the stage parser if it drifts from the reviewed shape. A design shape requires DESIGN input weights, so shaping by fitted weights (microcosm#630) is refused, and the #420 guard that every copy's source household has a positive design weight is restored. The square root was chosen by re-solving the release calibration on the same hosts: the [0.5, 2] clip put more rows at the weight-ratio cap than equal weights and widened the GBP 125,000-150,000 band's standard error past the damped shape's; the damped shape kept fewer cap rows, the highest Kish ESS and loss within 2.2 percent of equal weights. The stage receipt records the shape and its adjustment's 10th, 50th and 90th percentiles with banded counts (no minimum or maximum), and the mass-change record names the shape, so the release input-coverage manifest needs its pin-pass regeneration. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit d6c0f25)
…ed reason The prior-shape receipt gave its percentiles and floored clipped shares whatever the channel's size, so a small scale-ladder rung of one to nine copies would report statistics resting on fewer than ten source households. It now writes "<10" in their place, as the take-up receipts do. The coverage-manifest test pinned the old allocation sentence, so it would have failed even after the pin pass regenerates the manifest; it now expects the sentence naming the design-damped shape. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit 1d22dd5)
…#1095 residual 12) d6c0f25 and 1d22dd5 left #1338 before it merged (they broke that build's property_purchases guard on its older base) and stayed on its pre-rebase branch; María's ruling of 3 November 2026 brings them back here, to be measured against B-5a. - The prior-shape receipt withholds its spread and shares for a channel of one or two copies ("<3"), on the shared floor of 3 (María's ruling of 29 October 2026), where it had used ten; its banded counts follow receipt_counts. - The pin pass: the release input coverage manifest (the support stage's mass-change reason now names the design-damped shape) and the H2 fixture regenerate. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#1338 review, item 4) tools/diagnose_uk_spi_prior_concentration.py reports, on any UK spine or calibrated dataset, the cell the SPI prior shape is meant to help: households with a benefit unit that has a child aged 0 to 4 and an adult whose adjusted net income is above £100,000. It gives the cell's weighted households, its Kish effective size over households and over FRS source households (a source and its copies as one unit), the share of its weight on its ten largest source households, its share on SPI-channel copies and its support in source households, rounded; a cell on one or two source households gives none of them. Development diagnostic, aggregate JSON only. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s overflow the engine's On a spine, household and benefit-unit ids run past the engine's 32-bit entity ids, so the engine's household_id wrapped and the cell's households were not found in the household table. The receipt now takes memberships, source households and the SPI flag from the dataset's own tables and only values from the engine (age, adjusted net income, household weight), in the tables' row order, which it checks on the stored ages. On a calibrated dataset the release ids are 1 to N and in table order, so there the engine's ids and the tables' agree. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…1095 B3: B-5b, VED) No stage wrote policyengine-uk's car inputs, so every household's vehicle excise duty was zero. The lcfs_consumption stage gains a declared assign_car_attributes_from_nts step (nts_car_attributes.py) on the same NTS kept cars as J2 and K: - Which cars: a household's vehicles less its members' company cars (an employer pays their duty) go one to each other adult, oldest first. A car beyond the adults is left out and counted (policyengine-uk holds one car a person). - Which fuel: a household the fuel flag leaves without road fuel keeps zero-emission cars only; one that buys fuel holds a fuel-burning car first, and each further car is zero-emission at its own per-car share. - Which car: a kept NTS car of its class in the household's income fifth, drawn by keyed uniforms at its interview weight scaled to its fifth and age band's kept cars, since fewer of the newest and oldest cars carry the registration record the CO2 bands come from. A fifth and age band with kept cars but no linked one refuses. - The inputs: car_fuel_type (a diesel registered from 2021 meets RDE2), car_co2_emissions (the band's midpoint on the scale its duty regime reads), car_engine_size (none for a zero-emission car), car_first_registration_date (1 October 2025 less the car's age, drawn within its NTS band, as an ISO date) and car_is_ved_exempt (DLA's higher-rate or PIP's enhanced mobility component exempts the member's own car, or the oldest car-holder's in their benefit unit). car_list_price keeps the engine's default. - Surfaces: the spec operation and notes (sources.yaml and source_stages.json), the schema branch and allowed kinds, the graph's person cells, the stage's person outputs (UK_LCFS_CONSUMPTION_PERSON_OUTPUT_COLUMNS, as frs_take_up's), the release export's allow-list in terminal_gates and gates.json (so the battery's policy, manifest and fingerprint and the release-cut part digests move in the microcosm-data contract), the coverage manifest, and the H2 fixture (synthetic vehicle columns; the audited string columns gain the fuel type and the date, and the doc's count is computed). - An engine test holds the step to policyengine-uk: the inputs and their defaults, the regimes' start dates, each band midpoint inside its engine band, and each regime's duty charged on the step's ISO dates. VED is measured, never bound: policyengine-uk 2.165.1 does not add it into household tax, and no target carries it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Part of #1095. This PR collects the #1095 items that pass their measurement, and since 31 October the Phase B items too. Each item is measured in its own national build against the one before it. Every figure that rests on the SPI channel is given as a class only (SN 9422 and the disclosure rule).
A/B items (measured against the 28 October build, then against the 2 November control):
Phase B items (each measured against the build before it):
Phase B items that change nothing in the build:
Also since the first review:
paid_hours;How the items were measured
Control: María's national build of 2 November, cut f6e33b93 at SPI donor savings rebased to SPI's 2023-24 bands (savings follow-up to #1418) #1428's
329a99e71(policyengine-uk 2.165.1, feed 08962f1b), staged on populace-uk-private.T1426: this branch with the A/B items and Vahid's fixes, built on this machine with the control's review date (2 November) and feed.
Phase B: one build per group of items, in a chain: B-1 against T1426, B-2 against B-1, B-3 against B-2. All use the same review date, feed and lock, and all 1,339 rows are active in every build.
Main merged in: the branch now merges main at
312dc4d6b, which adds SPI donor savings rebased to SPI's 2023-24 bands (savings follow-up to #1418) #1428's last commit (2dbf7cf, SPI banding on total income net of expenses). A control rebuilt on main and this branch's merged tree measure the whole PR on the current base (below).Control (f6e33b93): 98.5% of rows within 10%, final loss 0.00529.
T1426 (A/B items and Vahid's fixes):
B-1 (B5): 98.8% within 10%, final loss 0.00527. No row moves.
B-2 (AC3, B7): 98.7% within 10%, final loss 0.00532. No row moves more than 5 points.
B-3 (B10, B9): 98.7% within 10%, final loss 0.00522. No row moves more than 5 points.
Control on main (
ctl-312d):This branch merged with main (
t1426m):frs_relationships(B7).obr.jobseekers_allowance's shortfall halves (B10).dwp.esa_claimantscomes inside 10%. The UC households-by-children rows stay near the 10% line, as they do in both controls.Every build passes every spine gate and all seven calibration gates.
1. The SPI stage-1 forests on the SPI age band (uk-data#628)
Why
The 2022-23 SPI public-use tape gives age only as a band (AGERANGE). The SPI income stage drew an age for each donor inside its band and trained the stage-1 tape forest on it, so the forest learned no real income pattern within a band. Twins of one band were then drawn from conditionals that differed only by that noise. uk-data#628 reports the same defect in uk-data.
What changes
age(Vahid's review). On the tape it is the AGERANGE code, with composite records (-1) as a band of their own. On a frame person it is the band of their whole-year age, through one helper shared by both frames.at_or_over_spastill carries the State Pension split inside the 65-74 band.Measurement
2. Households' HRAD at or below £250,000 from Table 8a receipts
Why
The property purchases stage (#1308, porting uk-data#573) allocates households' HRAD purchases by individuals' share of all Table 8a residential transactions (0.881) and value (0.882). #1308's review found this probably overstates households' purchases.
The residential nil-rate band was £250,000 for all of 2024-25. A purchase at or below it owes no standard-rate SDLT, so other buyers' receipts in that band are surcharge.
What changes
All figures in this section are HMRC's published Annual Stamp Tax Statistics 2024 to 2025, Tables 6b and 8a, as the pinned feed carries them.
Measurement
Households' purchases fit their new target, and no row moved by more than 5 points against the 28 October build.
obr.sdltfits within 0.25% after calibration in every build.3. Drawn SPI employees' current pay at their twin's hours (uk-data#629)
Why
The SPI income stage keeps each drawn copy's FRS twin's hours and job, while stage 1 redraws the copy's pay from the 2022-23 tape. The tape's pay covers the tax year and holds part-year workers, so part-year pay could land on a current full-time twin, at an hourly rate far below the minimum wage. Such a copy fails the minimum-income tests of the 30 hours' childcare offer, Tax-Free Childcare and the benefit cap more often than the FRS rows it copies.
What changes
floor_spi_employment_pay_at_minimum_wagelifts a drawn copy's pay to its twin's usual weekly paid hours (paid_hours, Vahid's review) × 52 × the lower of the minimum wage for the twin's age and the twin's own hourly pay.employment_income_annual_sharekeeps the HMRC income rows on the tape's pay.employment_pay_floorreceipt and an hourly-pay check. A figure resting on one or two source households is withheld as "<3", on the shared floor of 3 (Vahid's review).uk_stage_hmrc_spi_income_spine_hourly_payrefuses a spine where an eligible copy stays below its floor, or where drawn employees sit below the minimum wage more often than FRS employees by more than two points.Measurement
After the floor, drawn employees sit below the minimum wage for their age less often than FRS employees, within the gate's margin, and no eligible row stays below its floor. The HMRC employment-income bands end as without the floor, since their annual measures read the tape's pay.
B5: Carer's Allowance after salary sacrifice
salary_sacrifice, so its earnings test reads pay after sacrifice. That puts it after the CGT copies, so its draws are keyed on lineage: a copy draws as the member it copies, and a source and its copies in one stratum count as one record of their joint weight. Original rows keep their draws.B12: Scottish owners' council tax bands
uk/scotgov_chargeable_dwellings_by_band.json. Renters and the per-copy bill stay with the bill-level issue.AC3: two-child-limit births by the engine's cutoff test
uc_is_child_born_before_child_limit(a date of birth before 6 April 2017) instead of a birth year of 2017 or later. A child born from January to 5 April 2017 no longer counts as affected.B7: non-parental care and the two-child limit
frs_relationshipssetschild_limit_non_parental_caring_exceptionfor a child whom no adult of their own benefit unit claims as a natural, adopted or step child (household grid codes 3 and 4 from the child, 7 and 8 from the adult), and whom no local authority placed there. This covers kinship and friends care.B10: SPI copies' contributory JSA and ESA within the permitted-work limit
spi_benefit_coherencezeroed contributory JSA on every SPI copy and left contributory ESA with the stage-2 forest's draw, which knows nothing of the unit's claim history. A contributory award follows the person's National Insurance record and incapacity, which the copy keeps; what the copy changes is its work.esa_exempt_work_higher_earnings_limitat the frame's year, £183.50 a week in 2024-25.obr.jobseekers_allowance's shortfall halves. The ESA rows start further below DWP's and OBR's: without the forest's extra ESA, the FRS base's own shortfall shows.dwp.esa_claimantscomes inside 10%.B9: student-loan balances to borrowers
was_wealthsplits the household's WAS student-loan balance before any plan exists, so it could sit on a member with no loan while the household's borrower held none. policyengine-uk caps a borrower's repayment at a positive balance.student_loanssets the plans, each household's balance moves to its members who hold one: by repayments where any of them repays, equally otherwise. A household with no borrower keepswas_wealth's split, and no household's total moves. The stage opens a reader-isolation boundary in the UK graph, since it rewrites a cell an earlier open-surface stage reads.Tests
JUnit counts on the merged tree (2b49a00):
New and changed test files, against main:
tests/engine/uk/test_uk_spi_permitted_work_limit.pyreads the permitted-work limit from policyengine-uk at 2024 and 2025.test_uk_spi_income.py.test_uk_carers_allowance_take_up.py,test_uk_frs_relationships.pyandtest_uk_property_localisation.py;test_uk_property_purchases.py,test_uk_source_stages.pyandtest_uk_spi_benefit_coherence.py;test_uk_spi_income.py,test_uk_spi_pension_age.py,test_uk_spi_rank_transport.pyandtest_uk_spi_spine.py;test_uk_spine_acceptance_receipt.py,test_uk_stage_health.pyandtest_uk_student_loans.py;test_uk_uc_capital_coherence.py,test_uk_uc_target_measurements.pyandtest_uk_weighted_integrity.py.test_country_spec.py, andmicrocosm-data'stest_contract.py.uk_atomic_support_fixtures.pyanduk_student_loans.py.Follow-up
Generated surfaces:
source_stages.json(in step withsources.yaml), the sources schema, the release input coverage manifest, the H2 parity fixture, and the gate digests mirrored inmicrocosm-data's contract.🤖 Generated with Claude Code