Skip to content

UK #1095 A/B fixes: the SPI age band (uk-data#628), households' HRAD from Table 8a receipts, and the SPI pay floor (uk-data#629) - #1426

Draft
juaristi22 wants to merge 42 commits into
mainfrom
uk-1095-spi-age-band-hrad
Draft

juaristi22 wants to merge 42 commits into
mainfrom
uk-1095-spi-age-band-hrad

Conversation

@juaristi22

@juaristi22 juaristi22 commented Oct 29, 2026 •

Copy link
Copy Markdown
Collaborator

Part of #1095. This PR collects the #1095 items that pass their measurement, and since 31 October the Phase B items too. Each item is measured in its own national build against the one before it. Every figure that rests on the SPI channel is given as a class only (SN 9422 and the disclosure rule).

A/B items (measured against the 28 October build, then against the 2 November control):

  1. The SPI stage-1 forests train and query on the SPI age band (uk-data#628).
  2. Households' additional-dwelling (HRAD) purchases at or below £250,000 are allocated from HMRC's Table 8a receipts.
  3. Drawn SPI employees' current pay is floored at what their twin's hours earn, and the tape's pay stays their year of pay (uk-data#629).

Phase B items (each measured against the build before it):

  • B5: Carer's Allowance claims are drawn after salary sacrifice, keyed on lineage.
  • B12: Scottish owners' council tax bands are drawn from their council area's band mix.
  • AC3: the two-child-limit rows date births by the engine's cutoff test.
  • B7: children in non-parental care are marked for the two-child limit's exception.
  • B10: SPI copies take their twin's contributory JSA and ESA within the permitted-work limit.
  • B9: each household's student-loan balance moves to its borrowers.

Phase B items that change nothing in the build:

Also since the first review:

  • Vahid's five points of 29 October, each in its own commit (reply):
    • the one-hot SPI age band;
    • the pay floor's receipt on the floor of 3;
    • paid_hours;
    • the UC gate's pools;
    • the HRAD docstring.
  • The UC take-up gate is back at 0.055 for each family type, by María's ruling of 2 November (follow-up).

How the items were measured

  • Control: María's national build of 2 November, cut f6e33b93 at SPI donor savings rebased to SPI's 2023-24 bands (savings follow-up to #1418) #1428's 329a99e71 (policyengine-uk 2.165.1, feed 08962f1b), staged on populace-uk-private.

  • T1426: this branch with the A/B items and Vahid's fixes, built on this machine with the control's review date (2 November) and feed.

    • Its 17 stages before the SPI income stage give the same evidence and records as the control's. So the two machines' inputs agree.
  • Phase B: one build per group of items, in a chain: B-1 against T1426, B-2 against B-1, B-3 against B-2. All use the same review date, feed and lock, and all 1,339 rows are active in every build.

  • Main merged in: the branch now merges main at 312dc4d6b, which adds SPI donor savings rebased to SPI's 2023-24 bands (savings follow-up to #1418) #1428's last commit (2dbf7cf, SPI banding on total income net of expenses). A control rebuilt on main and this branch's merged tree measure the whole PR on the current base (below).

  • Control (f6e33b93): 98.5% of rows within 10%, final loss 0.00529.

  • T1426 (A/B items and Vahid's fixes):

    • 98.8% within 10%, final loss 0.00527.
    • 21 rows move more than 5 points, all HMRC SPI regional and income-band rows. That is reseed-sized: a placebo that only reseeded the SPI stage moved 22.
  • B-1 (B5): 98.8% within 10%, final loss 0.00527. No row moves.

  • B-2 (AC3, B7): 98.7% within 10%, final loss 0.00532. No row moves more than 5 points.

  • B-3 (B10, B9): 98.7% within 10%, final loss 0.00522. No row moves more than 5 points.

  • Control on main (ctl-312d):

    • 98.7% within 10%, final loss 0.00539.
    • Its 17 stages before the SPI income stage give the same evidence and records as the control's; from that stage on it differs, as main's SPI banding commit should.
    • Against the control, 10 rows move more than 5 points, all HMRC income-band and regional rows.
  • This branch merged with main (t1426m):

    • 98.7% within 10%, final loss 0.00520.
    • Against the control on main, by stage:
      • Its stages before the SPI income stage give the same evidence and records, except frs_relationships (B7).
      • From the SPI income stage on, every stage's evidence differs, as this branch's SPI items should.
      • The stage order differs only where B5 moves the Carer's Allowance stage.
    • Against the control on main, by row:
      • 14 rows move more than 5 points, all HMRC income-band and regional rows. That is fewer than the 22 a reseed-only placebo moved.
      • Before calibration, obr.jobseekers_allowance's shortfall halves (B10).
      • After calibration, dwp.esa_claimants comes inside 10%. The UC households-by-children rows stay near the 10% line, as they do in both controls.

Every build passes every spine gate and all seven calibration gates.

1. The SPI stage-1 forests on the SPI age band (uk-data#628)

Why

The 2022-23 SPI public-use tape gives age only as a band (AGERANGE). The SPI income stage drew an age for each donor inside its band and trained the stage-1 tape forest on it, so the forest learned no real income pattern within a band. Twins of one band were then drawn from conditionals that differed only by that noise. uk-data#628 reports the same defect in uk-data.

What changes

  • The stage-1 tape and reference forests read the SPI age band, one-hot, instead of age (Vahid's review). On the tape it is the AGERANGE code, with composite records (-1) as a band of their own. On a frame person it is the band of their whole-year age, through one helper shared by both frames.
  • at_or_over_spa still carries the State Pension split inside the 65-74 band.
  • The cross-fit fold cells key on the band.
  • The drawn donor ages stay on the donor for their other readers: the band pools' State Pension side and the property components' tape seed.

Measurement

  • Against the 28 October build, the item moved no more rows than a draw-seed placebo, and the UC rows ended closer to DWP's.
  • The copies' pay now rises with age inside the youngest bands nearly as FRS employees' does, and falls with age over 55 as theirs does. The placebo changed none of this.

2. Households' HRAD at or below £250,000 from Table 8a receipts

Why

The property purchases stage (#1308, porting uk-data#573) allocates households' HRAD purchases by individuals' share of all Table 8a residential transactions (0.881) and value (0.882). #1308's review found this probably overstates households' purchases.

The residential nil-rate band was £250,000 for all of 2024-25. A purchase at or below it owes no standard-rate SDLT, so other buyers' receipts in that band are surcharge.

What changes

  • The nil-rate band from receipts: households' HRAD at or below £250,000 is Table 6b's band less other buyers' HRAD implied by their Table 8a receipts. The stage refuses an implied figure outside Table 6b's band or above other buyers' own transactions there.
  • Above the band, uk-data#573's shares stay. Solving for other buyers' HRAD from receipts goes negative above £1m. The module docstring says the shares probably still overstate households' purchases. Table 8a doesn't split out the 2% non-resident surcharge, so the nil-band figure, if anything, understates them (Vahid's review).
  • Targets: households' 2024-25 HRAD target falls from 186,589 transactions and £68.81bn to 151,928 and £63.71bn.

All figures in this section are HMRC's published Annual Stamp Tax Statistics 2024 to 2025, Tables 6b and 8a, as the pinned feed carries them.

Measurement

Households' purchases fit their new target, and no row moved by more than 5 points against the 28 October build. obr.sdlt fits within 0.25% after calibration in every build.

3. Drawn SPI employees' current pay at their twin's hours (uk-data#629)

Why

The SPI income stage keeps each drawn copy's FRS twin's hours and job, while stage 1 redraws the copy's pay from the 2022-23 tape. The tape's pay covers the tax year and holds part-year workers, so part-year pay could land on a current full-time twin, at an hourly rate far below the minimum wage. Such a copy fails the minimum-income tests of the 30 hours' childcare offer, Tax-Free Childcare and the benefit cap more often than the FRS rows it copies.

What changes

  • The floor: floor_spi_employment_pay_at_minimum_wage lifts a drawn copy's pay to its twin's usual weekly paid hours (paid_hours, Vahid's review) × 52 × the lower of the minimum wage for the twin's age and the twin's own hourly pay.
  • Who it touches: only drawn recipients other than reserved band carriers, whose twin has an employee main job, no trade and usual hours, and whose drawn pay is positive. FRS rows are never modified.
  • The tape's year of pay: the floored pay is current pay only. employment_income_annual_share keeps the HMRC income rows on the tape's pay.
  • Receipt: an employment_pay_floor receipt and an hourly-pay check. A figure resting on one or two source households is withheld as "<3", on the shared floor of 3 (Vahid's review).
  • Gate: uk_stage_hmrc_spi_income_spine_hourly_pay refuses a spine where an eligible copy stays below its floor, or where drawn employees sit below the minimum wage more often than FRS employees by more than two points.
  • The UC take-up gate is at 0.055 again for each family type and for the reachable pools (María's ruling of 2 November). With SPI donor savings rebased to SPI's 2023-24 bands (savings follow-up to #1418) #1428's pool joining, every family type ends closer to DWP's count after the redraw.

Measurement

After the floor, drawn employees sit below the minimum wage for their age less often than FRS employees, within the gate's margin, and no eligible row stays below its floor. The HMRC employment-income bands end as without the floor, since their annual measures read the tape's pay.

B5: Carer's Allowance after salary sacrifice

  • What changes: the Carer's Allowance take-up stage runs after salary_sacrifice, so its earnings test reads pay after sacrifice. That puts it after the CGT copies, so its draws are keyed on lineage: a copy draws as the member it copies, and a source and its copies in one stratum count as one record of their joint weight. Original rows keep their draws.
  • Measurement (B-1 against T1426):
    • Only the Carer's Allowance stage differs, and every calibration row is unchanged.
    • The change turns off the claim flag only on CGT copies that salary sacrifice puts over the earnings limit, and the engine already paid them nothing.
    • So it makes the flag consistent with what the engine pays.

B12: Scottish owners' council tax bands

  • What changes: Scottish owners' bands are drawn, at the same uniform, from their council area's chargeable-dwellings mix (Scottish Government, September 2025, A to H), vendored from the pinned feed as uk/scotgov_chargeable_dwellings_by_band.json. Renters and the per-copy bill stay with the bill-level issue.
  • Measurement: it runs only in dense builds, so the national builds here can't measure it. It is measured in the next dense build.

AC3: two-child-limit births by the engine's cutoff test

  • What changes: the 15 two-child-limit rows read the engine's uc_is_child_born_before_child_limit (a date of birth before 6 April 2017) instead of a birth year of 2017 or later. A child born from January to 5 April 2017 no longer counts as affected.
  • Measurement (B-2): the seven rows whose measure reads the cutoff start 1% to 4% lower, and all 15 still fit within 0.5%.

B7: non-parental care and the two-child limit

  • What changes: frs_relationships sets child_limit_non_parental_caring_exception for a child whom no adult of their own benefit unit claims as a natural, adopted or step child (household grid codes 3 and 4 from the child, 7 and 8 from the adult), and whom no local authority placed there. This covers kinship and friends care.
    • Multiple births stay at the engine's default: the FRS has no twin field, and its birth dates are blank (María's ruling of 2 November).
    • The flag leaves the signed engine-default register.
  • Measurement (B-2): inert in this build. The flag reaches the calibrated dataset, but no flagged child is a third or later child born after the cutoff in a paid UC claim, so no row moves.

B10: SPI copies' contributory JSA and ESA within the permitted-work limit

  • Why: spi_benefit_coherence zeroed contributory JSA on every SPI copy and left contributory ESA with the stage-2 forest's draw, which knows nothing of the unit's claim history. A contributory award follows the person's National Insurance record and incapacity, which the copy keeps; what the copy changes is its work.
  • What changes:
    • An SPI copy takes its FRS twin's contributory JSA and ESA where its own earnings stay within ESA's permitted-work limit, and none above it (María's ruling of 29 October). The earnings are employment income plus any self-employment profit, over 52 weeks.
    • The limit is policyengine-uk's own esa_exempt_work_higher_earnings_limit at the frame's year, £183.50 a week in 2024-25.
    • The new operation is in the source manifest and the sources schema.
    • The stage gate checks the rule on every copy, and contributory ESA leaves the QRF tail-concentration surface, since no forest draw survives in it.
  • Measurement (B-3 against B-2):
    • On the copies, contributory ESA falls (the forest had given them more than their twins report) and contributory JSA returns at a small level.
    • Before calibration, obr.jobseekers_allowance's shortfall halves. The ESA rows start further below DWP's and OBR's: without the forest's extra ESA, the FRS base's own shortfall shows.
    • All of them fit after calibration, and dwp.esa_claimants comes inside 10%.

B9: student-loan balances to borrowers

  • Why: was_wealth splits the household's WAS student-loan balance before any plan exists, so it could sit on a member with no loan while the household's borrower held none. policyengine-uk caps a borrower's repayment at a positive balance.
  • What changes: once student_loans sets the plans, each household's balance moves to its members who hold one: by repayments where any of them repays, equally otherwise. A household with no borrower keeps was_wealth's split, and no household's total moves. The stage opens a reader-isolation boundary in the UK graph, since it rewrites a cell an earlier open-surface stage reads.
  • Measurement (B-3): the SLC repayment rows' starting estimates move by under 1%, and they all fit after calibration. Their large starting overshoot is not the balance's: the WAS balance is a small part of SLC's loan book, so few repayments are capped.

Tests

JUnit counts on the merged tree (2b49a00):

  • Engine-free: 351 files in 30-file chunks, 13,226 tests, 125 skipped.
    • Two tests fail, both only on this machine: one expects the UK extra to be absent, and the Orrery reconstruction test meets a local Node error.
  • Engine tier: 101 tests, none failing:
    • H2 parity and the full graph admission;
    • SPI income and the permitted-work limit;
    • the UK graph, its terminal gates and the full target graph;
    • release input coverage and the concept mapping.
  • Generated surfaces:
    • The H2 parity fixture regenerates byte-identically.
    • The release input coverage manifest, the CI test plan and the spec-engine coverage are current.

New and changed test files, against main:

  • New: tests/engine/uk/test_uk_spi_permitted_work_limit.py reads the permitted-work limit from policyengine-uk at 2024 and 2025.
  • Engine tier: test_uk_spi_income.py.
  • Engine-free, UK:
    • test_uk_carers_allowance_take_up.py, test_uk_frs_relationships.py and test_uk_property_localisation.py;
    • test_uk_property_purchases.py, test_uk_source_stages.py and test_uk_spi_benefit_coherence.py;
    • test_uk_spi_income.py, test_uk_spi_pension_age.py, test_uk_spi_rank_transport.py and test_uk_spi_spine.py;
    • test_uk_spine_acceptance_receipt.py, test_uk_stage_health.py and test_uk_student_loans.py;
    • test_uk_uc_capital_coherence.py, test_uk_uc_target_measurements.py and test_uk_weighted_integrity.py.
  • Engine-free, shared: test_country_spec.py, and microcosm-data's test_contract.py.
  • Test support: uk_atomic_support_fixtures.py and uk_student_loans.py.

Follow-up

  • B4, the HICBC liability target: HMRC's count of Self Assessment taxpayers with a liability, and the charge's revenue, are already in the pinned feed (Child Benefit statistics, August 2025, Table 16). The target's definition is María's call.
  • Phase B builds still to run: B-4 (the gambling stake level), B-5 (PR 2b, the NTS vehicle file) and B-6 (copy ids re-keyed). The target-side items B11, B14 and B15 follow them.
  • NS4 (benefit-cap support): still on a thin base. The repair options are an SPI-channel rule for copies' cap state (UK SPI channel after #840: residuals, post-merge measurements and stale pointers #1288) or a support stage.
  • uk-data#573 still allocates every band by the shares. Raising this there is María's call.

Generated surfaces: source_stages.json (in step with sources.yaml), the sources schema, the release input coverage manifest, the H2 parity fixture, and the gate digests mirrored in microcosm-data's contract.

🤖 Generated with Claude Code

juaristi22 and others added 6 commits October 29, 2026 19:35
…628)

The SPI tape gives age only as a band (AGERANGE), so the age drawn for each
donor inside its band carries no income pattern: the stage-1 forest learned
none, and twins in one band faced conditionals that differed only by that
noise. The tape and reference forests now read the band code instead: the
tape's own code, composite records (-1) as a band of their own, and the band
of a frame person's whole-year age. at_or_over_spa keeps the State Pension
split inside a band. The cross-fit fold cells key on the band, since they key
on the predictor values. The drawn ages stay on the donor for the band pools.

Regenerated: source_stages.json (in step with sources.yaml), the release
input coverage manifest and the H2 parity fixture. The SPI engine tests read
the band and count the reference forest once per cross-fit fold in use.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e 8a receipts

The 2024-25 residential nil-rate band was GBP 250,000 all year, so a purchase
at or below it owes no standard-rate SDLT and other buyers' Table 8a receipts
there are surcharge. At Table 6b's gross HRAD receipts per transaction and per
pound they imply about 48,100 HRAD transactions and GBP 7.05bn for other
buyers, where individuals' share of all transactions, uk-data#573's rule, left
them about 13,400. Households now take Table 6b's band less that; above it the
shares stay (solving from receipts there goes negative above GBP 1m). The
household bands become the price mix.

Households' target: 151,928 transactions and GBP 63.71bn (was 186,589 and
GBP 68.81bn), above uk-data#573's lower bounds; the lowest band carries 43% of
households' purchases (was 54%). Published HMRC figures only.

Regenerated: source_stages.json (in step with sources.yaml) and the release
input coverage manifest. The H2 parity fixture is regenerated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Stage 1 of hmrc_spi_income_spine draws a recipient's annual tape pay on
age, sex, region and the bridge predictors, not on hours, and the tape
holds part-year workers. A copy whose FRS twin reports usual hours in an
employee job could therefore draw a year's pay below what those hours earn
at the minimum wage, and fail the 30 hours' and Tax-Free Childcare
minimum-income tests more often than FRS rows.

A new floor_spi_employment_pay_at_minimum_wage operation lifts the drawn
hmrc_spi_pay leaf to weekly usual hours (hours_worked read back at
365.25/7) times 52 weeks times policyengine-uk's
gov.hmrc.minimum_wage.non_apprentice rate for the twin's age, read at 30
April of the build year as the engine reads it (2024-25: 11.44 from 21,
8.60 from 18, 6.40 below). It runs after the stage-1 draw, the band-donor
resample and the State Pension leaf guard, and before employment_income
is derived from the leaves and stage 2 conditions on it; the transport
receipt keeps the unfloored draws' uniforms. Only drawn recipients other
than band carriers, whose twin has an employee main job, no trade and
usual hours, and whose drawn pay is positive, are floored. Tape PAY is
taxable pay; PAY alone is compared with the gross minimum because it is
the leaf of the engine's gross employment_income input.

The operation is declared in sources.yaml, its source_stages.json
projection, the sources schema, the operation vocabulary and the HMRC
income contract, and checked against a reviewed declaration. The stage
evidence gains an employment_pay_floor receipt: source households
rounded, weighted figures "<10" on one to nine source households, no
row's value.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The employment pay floor receipt gave the floored weight, the share
floored and the amount added beside the eligible weight, each suppressed
only on the floored side's support. Any of them with the eligible weight
gives back the weight of the eligible rows the floor left alone, which
could rest on one to nine source households. They are now "<10" whenever
either side rests on one to nine, as the Carer's Allowance take-up
receipt gives its share; the mean uplift alone does not give a side back
and keeps the floored side's rule. The receipt declaration in
sources.yaml, source_stages.json and the reviewed spine mapping says so.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…, gate hourly pay

uk-data#629: drawn SPI employees keep their FRS twin's hours while the tape
redraws their pay, and the tape's pay is the tax year's, so part-year tape
pay landed on current twins at very low hourly rates. On top of the floor
from uk-cc-spipay:

- The floor rate is the lower of the minimum wage for the twin's age and the
  twin's own hourly pay (its employment income before the draw over its
  annual hours), so no copy is held to a higher rate than the FRS person it
  copies; hours now read back at 52 a year (uk-data#531).
- The floored pay is the row's current pay only: its annual share becomes
  the unfloored over the floored employment income, so the annual measures
  (the HMRC income rows) still read the tape's pay.
- The receipt adds an hourly-pay check: the drawn employees' weighted share
  below the minimum wage before and after the floor, beside the FRS
  employees' share, and the eligible rows still below their floor; a share
  is withheld beside a side on one to nine source households.
- A release-blocking stage gate, uk_stage_hmrc_spi_income_spine_hourly_pay,
  holds the floor invariant and the drawn share to the FRS share plus two
  points; it joins the spine gate scope, and the battery and contract
  digests move with it.

Regenerated: source_stages.json (in step with sources.yaml), the sources
schema, the release input coverage manifest and the H2 parity fixture. The
floor's engine test gives each twin and its copy one age and pay, reads hours
at 52 weeks, and checks the twin cap, the kept year of pay and the employer
contribution that follows the floored pay.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
María's ruling of 2026-10-29 (#1095). The SPI employment pay floor gives
fewer low-paid SPI copies Universal Credit, so single people's caseload
before the take-up redraw falls close to DWP's, and the redraw's usual
downward move reads as about twice the 0.055 allowance; after the redraw
single people end a little below the spine without the floor, all in the
middle award range. The shortfall of middle and higher awards that the
redraw cannot fill is the award fix's (#1095's UC awards below the top
band), and the tolerance returns to 0.055 when it lands. The one tolerance
also bounds the reachable pools, which realize DWP's count far more closely.

The contract re-pins the gate battery's policy, manifest and fingerprint
digests and the release certification's spine part; the stage-health test
pins 0.12 and adds the floor's pattern.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@juaristi22 juaristi22 changed the title UK #1095 A/B fixes: the SPI age band (uk-data#628) and households' HRAD from Table 8a receipts UK #1095 A/B fixes: the SPI age band (uk-data#628), households' HRAD from Table 8a receipts, and the SPI pay floor (uk-data#629) Oct 29, 2026
@vahid-ahmadi

Copy link
Copy Markdown
Contributor

@juaristi22 thanks, this is my review of 457af0274.

Verdict: the three items look sound, with four should items and a nit. The age band is the right fix for #628. The nil-band arithmetic reproduces the body's numbers: £265m over £625m/113,500 is about 48,100 transactions, and the target falls by about 34,700. The pay floor runs where it should, after stage 1 and the uprating and before employment_income, stage 2 and pension coherence, with the twin cap and the kept year of pay. Nothing here blocks the A/B arms.

Findings

  1. should: composites (-1) sit next to 16-24 on an ordinal predictor. spi_age_band is numeric, not one of SPI_STAGE1_CATEGORICAL_PREDICTORS (spi_income.py:2252, 2623). So any split above -1 that doesn't separate -1 from 1 puts composite records in the leaves 16-24 queries land in. Before, composites drew an age across 16-70, so they spread over every band. Now they can only mix with the youngest one, and composites are the tape's high-income amalgams. Please either keep composites out of the stage-1 training set, one-hot the band, or check on the arm that 16-24 draws' top tail is not higher than on main. A line in the predictor rule saying which would do.
  2. should: the new floor receipt hardcodes ten. spi_income.py:936, 940, 951-952 and 961 use < 10 and "<10", and the gate docstring says "fewer than ten" (stage_health.py:1425). UK: take up #1095's actionable items on policyengine-uk 2.165.1 #1422 moves every UK floor onto receipt_counts.UK_DISCLOSURE_FLOOR. Whichever lands second should move these onto the constant and the helpers (small_cell_share, rounded_count), or this receipt will be the only one still on ten. The gate's "a withheld share is not judged" branch then follows too.
  3. should: one tolerance loosens two checks. uk_stage_uc_take_up goes from 0.055 to 0.12 (gates.json:633). That widens the "no further from DWP than before the redraw" leg, which the floor needs. But the same number bounds the reachable pools, which the notes say realize DWP's count within 0.0015. A second parameter for the pools, left at 0.055 or tighter, would keep that check meaningful while the other is wide. Please also name the issue that tracks the return to 0.055, so the revert isn't left to memory.
  4. should (a question): does Table 8a's nil-band receipts figure for other buyers include the 2% non-resident surcharge? If it does, part of those receipts is not HRAD surcharge. Then other buyers' implied HRAD is too high and households' too low (property_purchases.py:473). The direction is the safe one, and it may be small. One sentence in the module docstring saying which way it goes would do.
  5. nit: paid_weeks (spi_income.py:893) is annual paid hours (hours / 52 * 52), not weeks. paid_hours would read better.

What I ran. I read the full diff at 457af0274 and checked the HRAD figures by hand. I also checked that the SPI leaf reconciliation runs on the donor tape, before the floor, so the floor can't break it. CI is green at this head. I didn't run tests locally or build anything, and I didn't check the private A/B figures.

🤖 Generated with Claude Code

juaristi22 added a commit that referenced this pull request Oct 31, 2026
Vahid's review of #1422 (its second and third points).

- Since the move to a floor of three, rounded_count printed three to nine
  exactly, while extreme_count and support_band print "3-9" and the
  module's own rule is that no exact count narrows a small group by
  difference. It now reads "3-9" there too. Two tests that read an
  incidental exact count follow.
- The module docstring and the changelog say that the SPI Public Use
  Tape's two conditions (SN 9422) are unchanged by the move to three.
- A guard test fails when a UK runtime module, the UK spec, the gate notes
  or a tool still writes the old "<10" label, so that a receipt left on
  the floor of ten cannot land unnoticed. #1426's SPI pay-floor receipt
  writes it today; whichever of the two lands second moves it onto
  UK_DISCLOSURE_FLOOR.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
juaristi22 and others added 15 commits October 31, 2026 11:22
#1426 now builds on #1422, which moves every UK disclosure floor to three
(receipt_counts.UK_DISCLOSURE_FLOOR) and regenerates the national surfaces
on Chronicle 53815b2 with policyengine-uk 2.165.1.

- The SPI pay-floor receipt withholds through UK_DISCLOSURE_FLOOR and
  UK_WITHHELD: a figure on one or two source households reads "<3" where it
  read "<10" (Vahid's review of #1426). Its description in spi_spine.py,
  sources.yaml and source_stages.json, the hourly-pay gate's note and
  docstring, and its tests follow. Two test cases shrink their small side to
  two households, so they still test the withholding.
- Conflicts: the SPI income imports (both sides kept), the stage-1 fold key
  (#1426's age band) beside #1422's savings-interest method text, and the UC
  take-up gate test.
- Regenerated: the release input coverage manifest. The gate battery and
  spine-part digests are re-pinned.
- The H2 parity fixture takes #1422's side here; a later commit regenerates
  it on the merged tree.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
hours / 52 * weeks_in_year is a year's paid hours, not weeks (Vahid's
review of #1426).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
uk_stage_uc_take_up's one tolerance bounded two checks: how far a family
type's caseload may move from DWP's across the redraw, which the SPI pay
floor needs at 0.12, and how closely the reachable pools realize DWP's
count, which the draw meets within 0.0015. The pools now keep 0.055 as
maximum_reachable_pool_deviation while maximum_caseload_deviation stays at
0.12 until the UC award fix of microcosm#1095 (Vahid's review of #1426).
The gate battery and spine-part digests are re-pinned.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
spi_age_band was an ordinal predictor, so any split that did not separate
the composite records' code (-1) from 16-24 put the tape's high-income
amalgams in the leaves 16-24 queries land in. The band is now one of the
stage-1 categorical predictors, one-hot encoded with gender, region and main
source, so composites are a category of their own (Vahid's review of #1426).
The predictor rule, the spec's categorical list, the changelog and the
coverage manifest follow; the cross-fit fold cells still key on the band's
value.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Table 8a's receipts do not split out the 2% non-resident surcharge, which a
non-resident buyer pays on top at or below GBP 250,000. Read as HRAD
surcharge, it raises other buyers' implied HRAD and lowers households', so
households' nil-band purchases are, if anything, understated (Vahid's review
of #1426).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Only the gate digests conflict; they are re-pinned on the merged gates.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
tools/graph_uk_spine_fixture.py on #1426's items over #1422's branch, after
the review fixes; --check regenerates it byte-identically.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The stage-1 queries carry the age band's one-hot columns, not the band
itself, and the pay floor's four floored households read "3-9".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Only the gate digests conflict; they are re-pinned on the merged gates.
#1422's new floor-of-ten guard passes on #1426's SPI pay floor and age
band code.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The gate digests and the coverage manifest's source digest conflict; they
are re-pinned and regenerated on the merged gates and spec. #1422's
broader floor-of-ten guard passes on #1426's SPI code.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The fixture conflicts, as #1426's SPI stages differ from #1422's; it is
regenerated on the merged tree, --check reproduces it byte-identically, and
the H2 parity test passes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
No conflicts. The exact-count floor guard passes on #1426's SPI and HRAD
code, as do the stage, ledger and reference tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
juaristi22 and others added 2 commits November 1, 2026 15:48
No conflicts. The stricter floor guard passes on #1426's SPI and HRAD
code, as do the ledger, reference, stage and graph tests. The gate
digests and the coverage manifest are unchanged, and the H2 fixture
regenerates byte-identical.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
No conflicts. The broader floor guard passes on #1426's SPI and HRAD
code, as do the ledger, gate, reference, stage and graph tests. The
gate digests and the coverage manifest are unchanged, and the H2
fixture regenerates byte-identical.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#1422 landed (11c1f4e), followed by #1430's company cars and #1432's
default load. Only pinned and generated files conflicted. The gate
battery and spine-part digests are re-pinned on the merged gates spec
(five move), and the release input coverage manifest and the charter-H2
fixture are regenerated; the fixture checks byte-identical.
policyengine-uk stays at 2.165.1, the lock main and #1428 share, and the
feed pin moves to main's 08962f1.

The H2 parity test and 86 related engine-free files pass (5,682 tests,
53 skipped).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@juaristi22

Copy link
Copy Markdown
Collaborator Author

@vahid-ahmadi thanks for the review. All five points are addressed. #1422 has landed, and main is merged in at bc7f59aac.

  1. Composites next to 16-24: the age band is now one of the stage-1 categorical predictors, one-hot encoded with gender, region and main source, so composite records are a category of their own (dae46ae94). The predictor rule, the spec's categorical list and the changelog say so. The next build re-measures the three items with it.
  2. The floor receipt's ten: the receipt withholds through UK_DISCLOSURE_FLOOR and UK_WITHHELD ("<3"). Its descriptions, the hourly-pay gate's note and docstring, and its tests follow (f1092f14c). Two test cases shrink their small side to two households, so they still test the withholding, and the engine test's floored households read "3-9" (8f0ec8fd7). UK: take up #1095's actionable items on policyengine-uk 2.165.1 #1422's floor guards pass on the merged tree.
  3. One tolerance, two checks: the reachable pools keep their own maximum_reachable_pool_deviation at 0.055, beside maximum_caseload_deviation at 0.12 (d68eb5cbb). The caseload check returns to 0.055 when UK: every open data item: uk-data parity, engine locks, the local surface, consumption and high-value homes #1095's UC award item ("UC awards below the top band are too low") lands, per María's ruling of 29 October.
  4. Table 8a and the non-resident surcharge: HMRC's commentary does not say whether Table 8a's receipts include it. HRAD makes up more than half of the residential receipts it reports, so they read as whole SDLT receipts. The module docstring now says which way the surcharge goes: read as HRAD surcharge, it raises other buyers' implied HRAD and lowers households', so households' nil-band purchases are, if anything, understated (f3d9c0af0).
  5. paid_weeks: renamed paid_hours (bc5adc97c).

The H2 fixture is regenerated on the merged tree (bc7f59aac). The full UK engine tier passed locally before main's merge, and CI runs it on the merge. #1095's Phase B joins this PR next (María's call of 31 October). Each item is measured against the national control built from #1428 at 329a99e71 (cut f6e33b93), on policyengine-uk 2.165.1, the lock main and #1428 share.

juaristi22 and others added 2 commits November 2, 2026 14:02
María's national control for Phase B (cut f6e33b93) was built from #1428
at 329a99e, so #1426 takes #1428 at that commit: each arm is the
control's tree plus one of #1426's items. #1428's later SPI banding
change (2dbf7cf) comes with main.

spi_income.py keeps both sides' additions: the employment pay floor
beside the gift anchor and the annuity and trust clear. The UC take-up
gate's note combines #1428's pool joining (María's ruling of 2026-10-31)
with #1426's 0.12 from the pay floor (2026-10-29); its parameters stay at
0.12 for the caseload and 0.055 for the reachable pools, and the
stage-health test's comment says the same. The gate digests are
re-pinned (four move), and the coverage manifest and the charter-H2
fixture are regenerated; the fixture checks byte-identical.

The H2 parity test, the SPI engine tests and 94 related engine-free
files pass (5,802 tests, 54 skipped).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
María's ruling of 2026-11-02: with #1428's pool joining, every family
type ends closer to DWP's count after the redraw than before it on the
T1426 spine (#1426 at 97fd4d0), the SPI employment pay floor included,
so maximum_caseload_deviation needs no wider allowance. The reachable
pools keep their own maximum_reachable_pool_deviation, also 0.055
(Vahid's review). The gate's note says so, the test checks the 0.055
boundary and still tells the two tolerances apart, the changelog
fragment is renamed and rewritten, and the gate battery and spine-part
digests are re-pinned.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@juaristi22

Copy link
Copy Markdown
Collaborator Author

@vahid-ahmadi a follow-up on point 3: #1428 is now merged in at the control's commit, and on the T1426 spine every family type ends closer to DWP's count after the take-up redraw than before it. So the caseload check is back at 0.055 (0ddc23673, María's ruling of 2 November), and nothing is left to track. The pools keep their own maximum_reachable_pool_deviation, also at 0.055.

juaristi22 and others added 9 commits November 2, 2026 15:55
…ge (#1095 B5)

The stage's earnings test now reads the pay salary_sacrifice converts
into pension contributions. Running after salary_sacrifice puts it after
the CGT stages' copies, so each draw is keyed on lineage: a member of a
CGT support copy, incidence clone or residential arm draws as the member
at the same rank of the household it copies (lineage_roots), and the
balanced placement takes a source and its copies in one stratum as one
record of their joint weight (lineage_balanced_positions), so they claim
together. Original rows keep their draws, and the one-record check reads
a record as the rows of one lineage.

The spec block moves after salary_sacrifice with its notes and the
reviewed rate derivation; source_stages.json keeps its key order (the HMRC
contract reads the retained constituents in order). The roster pins, the
coverage manifest and the H2 fixture follow. New tests cover the roots,
the one position per lineage and the stage itself with every carer cloned
at half weight, which claims as the uncloned frame does; keying the draws
on person ids fails it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s two

B-1's first spine stopped at carers_allowance_take_up: "Carer's Allowance
identities must name each record once". Salary sacrifice draws each CGT
copy's pay on its own id, so a copy's earnings band, and so its stratum,
can differ from its source's, and the lineage is then a record of each
stratum. stratum_starts keys each stratum on its own members but refused
an identity repeated across strata, so the starts are now drawn stratum by
stratum, where an identity names one record. Positions where no lineage
spans two strata are unchanged.

Two tests cover it: a lineage in two strata is placed as each stratum
alone, and the stage draws a clone whose pay left its source's band. Both
fail on the single call.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…#1095 B12)

The dense build's post-geography band node now draws each Scottish owner
clone's band from its council area's chargeable dwellings by band, A to H,
at September 2025 (Scottish Government CTAXBASE 2025), at the same uniform
as its value: its rank among Scotland's owners. The counts are vendored
from Chronicle 08962f1b into uk/scotgov_chargeable_dwellings_by_band.json
(256 rows, 32 council areas) and pinned as the graph source
uk_scotgov_band_mix, with a --scotgov-band-mix option beside
--voa-band-mix. Renters keep their band, as in England and Wales, and so
does Northern Ireland. The national build does not localise property, so
it does not change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The two-child-limit target rows counted a child as born after the
cutoff when the engine's birth year was 2017 or later, so a child born
from January to 5 April 2017 read as affected although policyengine-uk
exempts them. The measure now reads the engine's own
uc_is_child_born_before_child_limit (a date of birth before 6 April
2017), as the child element does. The comparison contract and the 15
TCL target notes say so, and a test pins a third child born in early
2017 as unaffected.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Nothing set policyengine-uk's child_limit_non_parental_caring_exception,
so the engine paid no child element for a third or later child born from
6 April 2017 who lives with kinship or friends carers. frs_relationships
now marks a child-table member whom no adult of their own benefit unit
claims as a natural, adopted or step child (grid codes 3 and 4 from the
child, 7 and 8 from the adult) and whom no local authority placed there.
That includes a baby whose parent is a child of the claimant's unit.

The rule is declared in the stage and the spec schema; the column joins
the graph's stage cells and the reviewed extra export columns, and
leaves the signed engine-default register. Multiple births, adoption and
non-consensual conception stay at the default, as María ruled on
2 November: the FRS has no twin field and its birth dates are blank.
Gate digests re-pinned, coverage manifest and H2 fixture regenerated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…itted-work limit (#1095 B10)

spi_benefit_coherence zeroed contributory JSA on every SPI-channel row and
left contributory ESA with the stage-2 forest's draw, which knows nothing of
the benefit unit's claim history. A contributory award follows the person's
National Insurance record and incapacity, which the copy keeps; what the copy
changes is its work. An SPI row now takes its FRS twin's contributory JSA and
ESA where its own earnings (employment income plus any self-employment profit,
over 52 weeks) stay within ESA's permitted-work limit, and none above it
(María's ruling of 29 October 2026).

The limit is policyengine-uk's own esa_exempt_work_higher_earnings_limit at
the frame's year (GBP 183.50 a week in 2024-25), read from a one-person
simulation. The new restore_spi_channel_reports_from_source_within_earnings_limit
operation is registered in the source manifest and the sources schema. The
stage-health gate checks the rule on every SPI row
(restored_within_earnings_limit_columns), and contributory ESA leaves the QRF
tail-concentration surface, since no forest draw survives in it. Gate digests
re-pinned; coverage manifest and H2 fixture regenerated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
was_wealth draws the household's student-loan balance from the WAS and splits
it among the members before any plan exists, so the balance could sit on a
member with no loan while the household's borrower held none. policyengine-uk
caps a borrower's repayment at a positive balance, so the split decides which
repayments the balance can cap.

Once student_loans has set the plans, it now moves each household's balance to
the members who hold one: by their repayments where any of them repays, and
equally otherwise. A household with no borrower keeps was_wealth's split, and
no household's total moves (checked to the penny). The stage declares the
re-split as a derive operation and student_loan_balance among its rewrites,
and its receipt records the households re-split and the balance on borrowers
before and after.

The stage now rewrites a cell the open-surface Carer's Allowance reader before
it binds, so it opens a reader-isolation boundary in the UK graph. Coverage
manifest and H2 fixture regenerated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Brings in #1428's last commit (2dbf7cf, SPI banding on total income net of
expenses), the emitter interface (#1410), and the NZ, Belgium and US work
since 0b1eecf.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…1095 B4)

Two national rows from HMRC's Child Benefit statistics (August 2025, Table 16, United Kingdom): the Self Assessment taxpayers liable to the High Income Child Benefit Charge in 2024-25 (persons whose CB_HITC is above zero) and the charge's revenue (the sum of CB_HITC), both the publisher's provisional figures. María's ruling of 3 November 2026: measured diagnostics first, using 2024-25's figures for the model's 2025.

- Contract: two hmrc_child_benefit rows; the selectors pin the 2024-25 record sets and the tax year, since the feed holds the series from 2012.
- Held out of the solve by two signed exclusions: the model's charge sits well above HMRC's, a gap in who receives Child Benefit at high incomes (microcosm#1288 item 10) rather than a weight question, and HMRC's count covers Self Assessment returns only. The evaluation measures them on every candidate.
- Measured at the calibration year under a signed reviewed hold: the charge's thresholds are the same in 2024-25 and 2025-26.
- Regenerated on the pinned feed: the national references and membership (1,420 active) and the national compile-parity receipt (two ledger-only rows).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@vahid-ahmadi

Copy link
Copy Markdown
Contributor

@juaristi22 Round 2 at 2b49a00ac. I re-checked my five points from 457af0274, read your two replies, and read the Phase B diff against main (312dc4d6b): B5, B7, AC3, B9, B10 and B12. I checked B10 and B7 against the legislation. I ran the PR's engine-free tests for the stages it touches (spi_benefit_coherence, student_loans, frs_relationships, carers_allowance_take_up, stage_health, property_localisation, uc_target_measurements): 281 passed. I didn't build anything or look at the private figures.

Earlier points

  1. Composites next to 16-24: fixed. spi_age_band is the first of SPI_STAGE1_CATEGORICAL_PREDICTORS (spi_income.py:185-190), so _encode_predictor_frames one-hots it. Tape and query codes are both float, so their dummy columns match (spi_income.py:2663, 3028-3030). The query side never produces -1, so the composite column is always 0 for frame people, as it should be.
  2. The floor receipt's ten: fixed. It now uses UK_DISCLOSURE_FLOOR/UK_WITHHELD throughout (spi_income.py:984-1042).
  3. One tolerance, two checks: fixed. maximum_reachable_pool_deviation is separate (stage_health.py:1542-1546, 1586) and both values are 0.055 (gates.json:655-656). See below on your follow-up.
  4. Non-resident surcharge: fixed. The module docstring says which way it goes (property_purchases.py:39-44).
  5. paid_weeks: fixed, now paid_hours (spi_income.py:939).

On your point-3 follow-up. I agree the caseload check can go back to 0.055 and nothing is left to track. One caveat on where the evidence comes from. The gate note cites the T1426 spine at 97fd4d0e4 (gates.json:658). That merges #1428 at 329a99e71, before #1428's SPI banding commit 2dbf7cf56, so the "every family type ends closer after the redraw" reading was taken on the old SPI stage. The body says B-1 to B-3 and t1426m all pass every spine gate, so the 0.055 gate has also passed on the current base. Please name t1426m's commit in the gate note next to 97fd4d0e4, since that is the spine the gate now runs on. If the receipt shows how far inside 0.055 the closest family type sits on t1426m, one figure there would also help. The body already says the T1426 and B-chain A/B measurements were taken against the pre-2dbf7cf56 control, and that ctl-312d/t1426m measure the PR on the current base, so I have nothing to add there.

New findings

  1. Should: B10 gives contributory JSA ESA's permitted-work limit, which is about twice JSA's own earnings limit (spi_benefit_coherence.py:517-530). Contribution-based JSA needs the claimant's earnings not to exceed the prescribed amount (Jobseekers Act 1995 s.2(1)(c); JSA Regs 1996 reg 56: the age-related amount plus the earnings disregard, less 1p). For a claimant aged 25 or over in 2024-25 that is £90.50 + £5 − £0.01 = £95.49 a week. Below that, s.4(1)(b) reduces the award by earnings above the disregard. Failure case: an SPI copy aged 30 earning £7,800 a year (£150 a week), whose twin reports £4,706 of contributory JSA (£90.50 × 52), keeps all £4,706 at this head. Under the law it would get nothing, because £150 is above £95.49. At £50 a week it keeps £4,706 where the award would be about £2,366 a year (£45.50 a week). The docstring says the ESA limit is only a ceiling for JSA (spi_benefit_coherence.py:66-68). But the body puts part of the halving of obr.jobseekers_allowance's pre-calibration shortfall down to B10, so the gap matters here. Fix: test jsa_contrib_reported against reg 56's amount at the copy's age (the engine's JSA age-related amount + £5), and either reduce the award by earnings above £5 or zero it above the prescribed amount. Or, if María keeps the ESA limit for both, have the receipt report the copies' weighted JSA between £95.49 and £183.50 a week, so the JSA row's movement can be read with that in mind.
  2. Nit: B10's "the test reads no hours" doesn't quite hold. spi_benefit_coherence.py:66 says the SPI tape has no hours. But the copy carries its twin's usual hours, which the pay floor reads (SPI_EMPLOYMENT_PAY_FLOOR_HOURS_COLUMN). ESA reg 45(4) also needs fewer than 16 hours a week. Either test the hours too, or reword the docstring to say why the twin's hours aren't used.
  3. Nit: B7's flag is an upper bound on the legal exception (frs_relationships.py:204-218). UC Regs 2013 Sch 12 para 4(2) (before its revocation from 6 April 2026) needs a friend or family carer to meet one of conditions (a) to (h): a child arrangements order, special guardianship, guardian's allowance, a kinship care order, guardianship, a permanence order, or taking the child on where they would otherwise likely be looked after. The FRS grid shows none of these. So every child with no parent or step-parent in their benefit unit is flagged, including informal arrangements that wouldn't qualify. It is inert in this build, so one line in the rule's comment calling it an upper bound would do.

Checked and fine

  • B5: lineage_roots refuses an unmapped or duplicated lineage. Originals keep their person_id draws. lineage_balanced_positions treats a source and its copies in one stratum as one record, and the within-one-record assert is widened to match.
  • B9: allocate_balance_to_borrowers keeps each household's total (with a hard check), leaves households with no borrower or no balance alone, and splits by repayments, or equally where nobody repays.
  • AC3: the measure now reads uc_is_child_born_before_child_limit, so a January to 5 April 2017 birth no longer counts as affected.
  • B10 limit: 16 × £11.44 = £183.04, rounded up to £183.50 (ESA Regs 2008 reg 45(4) and (9A)), which matches the test constant.
  • B12: the loader refuses a council area missing a band or repeating one, Northern Ireland and renters keep their bands, and the Scottish receipt is separate.

CI at 2b49a00ac: all ten checks pass (engine-free and engine-uk on 3.13 and 3.14, engine-us, integration-uk, lint, wheels).

Verdict: my earlier points are all fixed and I agree with the 0.055 return. B10's JSA limit is the one item to settle before the JSA row's movement is quoted.

🤖 Generated with Claude Code

juaristi22 and others added 6 commits November 3, 2026 14:10
…ts household share (#1095 B3: PR 2b J2 + K)

The LCFS consumption stage now reads the National Travel Survey's cars (UKDS SN 5340, End User Licence; the vehicle and household tabs, survey years 2022-2024, at the interview weight W3), as the 16 October plan designed J2 and K.

- J2: each car is zero-emission at VEH1103's licensed-car share times its household income fifth's relative risk among the cars NTS households keep, normalised over the frame's cars so the national share is unchanged (rule per_car_zero_emission_share_by_income_quintile). The frame ranks household_gross_income over the engine's before-housing-costs equivalence scale, as the NTS ranks gross income McClements-equivalised. A car is zero-emission in the DVLA-linked 0 g/km band; a car without a linked band is left out. The uniforms are unchanged, so only each household's threshold moves.
- K: level_road_fuel keeps ONS 07.2.2's household level but levels each fuel to its share of household-paid spend (mix rule household_paid_litres): HMRC's litres times DESNZ's cars share times the share of car miles a household pays for (a kept car's private and commuting miles), at the 2024 pump prices. A car whose miles the release does not split is left out; a blank is missing, never a zero. The road_fuel_level gate rechecks each fuel against the mix it recomputes from the vendored rows.
- Spec: both operations declare the same nts_cars block, which the code refuses to see differ; the NTS household tab joins lcfs_consumption under the bus stage's pin, and the vehicle tab is a new pinned private input (--nts-vehicle-tab).
- obr.fuel_duties_cars stays held, its reason restated with the household-paid shares (María's ruling of 29 October 2026).
- Surfaces: the schema, the source-stage projection, the graph's split sources, the spine tool, the fixture kernels, the H2 fixture (a synthetic vehicle table, W3 on its households), the coverage manifest, and the kept-tens register (the pooling floor of 10 is statistical).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The spi_support_channel stage draws its 10,000 FRS households uniformly
and gave every copy in a region x pension-age stratum the same prior
weight, which reproduced the FRS respondent mix rather than the design's
population (young families, single adults and one-earner couples over-
represented, households of three or more adults under-represented).

Copy j of stratum s now takes M_s * share_s * f_j / sum_k f_k, with f the
copy's source-household design weight (the incoming raw FRS GROSS4) over
the stratum copies' mean, damped to the square root. Each stratum's
channel mass stays exactly its declared share and the pension-age share
stays 0.2; the hosts and their incomes do not move. The shape is declared
on allocate_zero_weight_prior_mass (prior_shape: design_damped,
prior_shape_exponent: 0.5; design_bounded with prior_shape_bounds is also
understood), validated by the schema and refused by the stage parser if it
drifts from the reviewed shape. A design shape requires DESIGN input
weights, so shaping by fitted weights (microcosm#630) is refused, and the
#420 guard that every copy's source household has a positive design weight
is restored.

The square root was chosen by re-solving the release calibration on the
same hosts: the [0.5, 2] clip put more rows at the weight-ratio cap than
equal weights and widened the GBP 125,000-150,000 band's standard error
past the damped shape's; the damped shape kept fewer cap rows, the highest
Kish ESS and loss within 2.2 percent of equal weights. The stage receipt
records the shape and its adjustment's 10th, 50th and 90th percentiles
with banded counts (no minimum or maximum), and the mass-change record
names the shape, so the release input-coverage manifest needs its pin-pass
regeneration.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit d6c0f25)
…ed reason

The prior-shape receipt gave its percentiles and floored clipped shares
whatever the channel's size, so a small scale-ladder rung of one to nine
copies would report statistics resting on fewer than ten source
households. It now writes "<10" in their place, as the take-up receipts do.

The coverage-manifest test pinned the old allocation sentence, so it
would have failed even after the pin pass regenerates the manifest; it
now expects the sentence naming the design-damped shape.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 1d22dd5)
…#1095 residual 12)

d6c0f25 and 1d22dd5 left #1338 before it merged (they broke that build's property_purchases guard on its older base) and stayed on its pre-rebase branch; María's ruling of 3 November 2026 brings them back here, to be measured against B-5a.

- The prior-shape receipt withholds its spread and shares for a channel of one or two copies ("<3"), on the shared floor of 3 (María's ruling of 29 October 2026), where it had used ten; its banded counts follow receipt_counts.
- The pin pass: the release input coverage manifest (the support stage's mass-change reason now names the design-damped shape) and the H2 fixture regenerate.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#1338 review, item 4)

tools/diagnose_uk_spi_prior_concentration.py reports, on any UK spine or calibrated dataset, the cell the SPI prior shape is meant to help: households with a benefit unit that has a child aged 0 to 4 and an adult whose adjusted net income is above £100,000. It gives the cell's weighted households, its Kish effective size over households and over FRS source households (a source and its copies as one unit), the share of its weight on its ten largest source households, its share on SPI-channel copies and its support in source households, rounded; a cell on one or two source households gives none of them. Development diagnostic, aggregate JSON only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s overflow the engine's

On a spine, household and benefit-unit ids run past the engine's 32-bit
entity ids, so the engine's household_id wrapped and the cell's households
were not found in the household table. The receipt now takes memberships,
source households and the SPI flag from the dataset's own tables and only
values from the engine (age, adjusted net income, household weight), in the
tables' row order, which it checks on the stored ages. On a calibrated
dataset the release ids are 1 to N and in table order, so there the engine's
ids and the tables' agree.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…1095 B3: B-5b, VED)

No stage wrote policyengine-uk's car inputs, so every household's vehicle
excise duty was zero. The lcfs_consumption stage gains a declared
assign_car_attributes_from_nts step (nts_car_attributes.py) on the same NTS
kept cars as J2 and K:

- Which cars: a household's vehicles less its members' company cars (an
  employer pays their duty) go one to each other adult, oldest first. A car
  beyond the adults is left out and counted (policyengine-uk holds one car a
  person).
- Which fuel: a household the fuel flag leaves without road fuel keeps
  zero-emission cars only; one that buys fuel holds a fuel-burning car first,
  and each further car is zero-emission at its own per-car share.
- Which car: a kept NTS car of its class in the household's income fifth,
  drawn by keyed uniforms at its interview weight scaled to its fifth and age
  band's kept cars, since fewer of the newest and oldest cars carry the
  registration record the CO2 bands come from. A fifth and age band with kept
  cars but no linked one refuses.
- The inputs: car_fuel_type (a diesel registered from 2021 meets RDE2),
  car_co2_emissions (the band's midpoint on the scale its duty regime reads),
  car_engine_size (none for a zero-emission car), car_first_registration_date
  (1 October 2025 less the car's age, drawn within its NTS band, as an ISO
  date) and car_is_ved_exempt (DLA's higher-rate or PIP's enhanced mobility
  component exempts the member's own car, or the oldest car-holder's in their
  benefit unit). car_list_price keeps the engine's default.
- Surfaces: the spec operation and notes (sources.yaml and
  source_stages.json), the schema branch and allowed kinds, the graph's person
  cells, the stage's person outputs (UK_LCFS_CONSUMPTION_PERSON_OUTPUT_COLUMNS,
  as frs_take_up's), the release export's allow-list in terminal_gates and
  gates.json (so the battery's policy, manifest and fingerprint and the
  release-cut part digests move in the microcosm-data contract), the coverage
  manifest, and the H2 fixture (synthetic vehicle columns; the audited string
  columns gain the fuel type and the date, and the doc's count is computed).
- An engine test holds the step to policyengine-uk: the inputs and their
  defaults, the regimes' start dates, each band midpoint inside its engine
  band, and each regime's duty charged on the step's ISO dates.

VED is measured, never bound: policyengine-uk 2.165.1 does not add it into
household tax, and no target carries it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants