Skip to content

Build the budget totals from the programme registry - #2809

Draft
MaxGhenis wants to merge 2 commits into
mainfrom
claude/budget-totals-from-registry-4rggpz
Draft

Build the budget totals from the programme registry#2809
MaxGhenis wants to merge 2 commits into
mainfrom
claude/budget-totals-from-registry-4rggpz

Conversation

@MaxGhenis

@MaxGhenis MaxGhenis commented Nov 1, 2026

Copy link
Copy Markdown
Collaborator

Addresses #2702
Reproducer: n/a: this changes no result
axiom: n/a: this changes how the package lists programmes and changes no rule or parameter value

What the code does now

policyengine_uk/programs.yaml defines the budget totals as well as model coverage.

  • Each programs entry whose variable a total adds carries budget: tax or budget: benefit, and an optional group for terms that move together (for example the five childcare entitlements, or ESA's two parts).
  • Terms that no programme entry carries sit in a new budget_terms section. Each has a kind: programme_measure (the amount counted for a listed programme, such as esa_income for ESA, with program naming it), programme (a programme the coverage list does not carry yet), reform_measure (not a programme: a hypothetical tax or benefit, or a change from baseline such as vat_change and the employer NI incidence terms) or public_service (public services in kind). These entries stay out of programs, so the coverage page and modelled_policies never list them.
  • budget_dropped_when: gov.contrib.abolish_council_tax sits on council tax, the High Value Council Tax Surcharge and Council Tax Reduction. budget_uprating sits on the State Pension ([all]) and basic income ([]); every other benefit term takes both gov.contrib.benefit_uprating rates.
  • budget_order gives each total's summation order. The loader refuses a registry whose flags and order disagree.

policyengine_uk/programs.py reads the YAML with no import from the package's top level. budget_programs() returns one BudgetProgram per term (variable, label, tax or benefit, group, kind, programme id, dropped-when parameter, uprating rates), taxes first, each in summation order. household_tax and household_benefits build their lists from it, and HOUSEHOLD_TAX_VARIABLES and HOUSEHOLD_BENEFIT_VARIABLES stay in their modules with the same contents in the same order. build_metadata.DATA_BUILD_SURFACE now includes programs.py.

No law changes, so this cites none.

Invariants

  • HOUSEHOLD_TAX_VARIABLES and HOUSEHOLD_BENEFIT_VARIABLES equal main's lists, element by element.
  • The variables each total's formula reads are exactly the flagged terms, less those whose budget_dropped_when parameter is true; household_tax reads them in budget_order.
  • Every budget_programs() variable exists in the system and appears once, and its sign agrees with the list it came from.
  • For any registry, the loader returns the terms in budget_order and refuses one whose order drops or repeats a flagged term.
  • household_tax, household_benefits and household_net_income equal main's values exactly: the totals add the same float32 terms in the same order.

Tests

New tests:

  • tests/test_budget_programs.py (16 tests): drift between the registry and what each total's formula reads (traced, at current law and with council tax abolished); the differential test against main's two lists, frozen as literals in order, and its council tax and uprating rules; budget_programs() from a fresh interpreter, importing policyengine_uk.programs, the household_tax module or the package first; and Hypothesis properties on the real registry and on generated registries.
  • tests/test_budget_totals_pinned.py (17 tests): household_tax, household_benefits and household_net_income for 12 hand-built households in 2023, 2025, 2026 and 2028 under four reforms (current law; abolish_council_tax; both uprating rates; both together with a flat basic income), 192 cells, with exact equality. The values come from main at 2da212a, and the file passes on main's code too.
Run Result
test_budget_programs.py 16 passed
test_budget_totals_pinned.py on this branch, and on main's code 17 passed each
The new files with the related suites: test_program_registry.py, test_fiscal_partition.py, test_household_benefits_properties.py, test_gov_spending_properties.py, test_uc_statutory_pay_properties.py, tests/aggregate_impact, test_build_metadata.py 549 passed, 10 skipped (9 need the certified dataset, 1 needs policyengine_bundles)
tests/code_health 3088 passed
ruff 0.17.2 format --check, check, check --select F811,F821; git diff --check clean
Wheel from uv build holds policyengine_uk/programs.py and policyengine_uk/programs.yaml

Mutation checks on the two new test files (each change made in a scratch copy):

Change Result
Swap income_tax and national_insurance in budget_order.tax 1 failed: the differential test (the pinned sums happen not to move)
Drop the State Pension's budget_uprating: [all] 9 failed: the rules test and every uprating cell of the pinned test
Drop budget_dropped_when from the High Value Council Tax Surcharge 2 failed: the rules test and the traced drift test with council tax abolished
Make household_tax ignore budget_dropped_when 9 failed: the traced drift test and every council tax abolition cell
Remove budget: benefit from Scottish Child Payment, order unchanged the package refuses to load: budget_order.benefit and the entries flagged budget: benefit differ: only ordered ['scottish_child_payment']
Restore main's two formula modules 33 passed, as an unchanged result requires

Generated by Claude Code

claude added 2 commits November 1, 2026 19:11
@claude
Build the budget totals from the programme registry
902cc1d 
programs.yaml now flags each term of household_tax and household_benefits
with `budget`, an optional `group`, and the council tax abolition and
benefit uprating rules. Terms that no programme entry carries sit in a new
`budget_terms` section with a `kind`, and `budget_order` fixes the
summation order. policyengine_uk.programs reads the file, and
budget_programs() returns each term with its label, sign and group.
HOUSEHOLD_TAX_VARIABLES and HOUSEHOLD_BENEFIT_VARIABLES keep their modules,
contents and order.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LwQq6Kyd2hZt9Nh1cPJ8mH
@claude
Import policyengine_uk.programs from the package
e3308b1 
policyengine_uk.programs.budget_programs() is reachable after
`import policyengine_uk` without relying on the variable modules having
imported the submodule.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LwQq6Kyd2hZt9Nh1cPJ8mH
@github-actions

github-actions Bot commented Nov 1, 2026

Copy link
Copy Markdown
Contributor

Aggregate impact on certified data: pass

Base 2da212abcf against head 498e9f74b0, both on enhanced_frs_2024_25.h5 (policyengine-uk-data 1.58.1, sha256 056e80fd8e91), fiscal years 2025-26 to 2031-32. Each year is a full microsimulation run on each side.

Checks: 0 broken by this change, 1 failing on both sides, 0 fixed. Moves flagged for review: 0 (thresholds in report_spec.yaml; a flag does not fail the job).

Workflow run and full report

Notes

  • In the years a figure from fewer than 10 records is suppressed (spending.disabled_students_allowance), every total containing it, and the decile incomes, are rebuilt from their other components, so no published total includes it. Those totals are marked †. A change is withheld where the two sides leave out different figures and can't be lined up.

Moves flagged for review

None.

Spending by benefit and service

Measure 2025-26 2026-27 2027-28 2028-29 2029-30 2030-31 2031-32
Disabled Students' Allowance suppressed suppressed suppressed suppressed suppressed suppressed suppressed

Caseloads

Measure 2025-26 2026-27 2027-28 2028-29 2029-30 2030-31 2031-32
Disabled Students' Allowance suppressed suppressed suppressed suppressed suppressed suppressed suppressed

Standard reform costings (change in gov_balance)

Measure 2025-26 2026-27 2027-28 2028-29 2029-30 2030-31 2031-32
Raise Class 1 main employee NICs rate to 10% 0 0 0 0 0 suppressed suppressed
Raise additional rate by 3pp 0 0 0 0 0 suppressed suppressed

Standard reforms: change in each reform's target

Measure 2025-26 2026-27 2027-28 2028-29 2029-30 2030-31 2031-32
Raise Class 1 main employee NICs rate to 10%: change in national_insurance 0 0 0 0 0 suppressed suppressed
Raise additional rate by 3pp: change in income_tax 0 0 0 0 0 suppressed suppressed

Household net income by decile

No household's net income changes in any year.

No change in any year: receipts by tax head; government balance, each component counted once; model totals; household income totals; population; poverty rates; people in poverty; inequality; mean household net income in each side's own deciles.

Checks failing on both base and head (1)
  • VAT excluding private school fees is never negative (sign:vat): 2025-26: 76 units below zero, a weighted £-53,888,684 in total; 2026-27: 76 units below zero, a weighted £-55,777,260 in total; 2027-28: 76 units below zero, a weighted £-57,727,917 in total
Counted-once check

Each of 92 components was raised by £1 per unit in 2026-27, and every head of its totals was read household by household: its own head must rise by exactly £1, no other tax head may move, no other benefit may rise, and the model's own total must move by what its heads moved. A head that breaks a rule is measured again twice: with every other head held at its baseline value, and with every other head held at its value after the first £1 while the component rises again. It fails where either still shows the response. 0 failed; 0 were skipped.

  • widowed_parents_allowance: pension_credit responds only through other heads in fewer than 10 households (other heads that moved there: carers_allowance; gov_spending; not a failure)
  • widowed_parents_allowance: pension_credit responds only through other heads in fewer than 10 households (other heads that moved there: carers_allowance; household_benefits; not a failure)
Provenance
Side Commit Runtime Environment hash Partition hash Time
base main 2da212abcf uk 2.170.0 (files e654eb5f6f55), core 3.33.12, py 3.13.18 aarch64 77aac721ac5e 7cb61324a46b 2136.93s
head #2809 merged into main 498e9f74b0 uk 2.170.0 (files 4f4c4687ddfb), core 3.33.12, py 3.13.18 aarch64 77aac721ac5e 7cb61324a46b 2086.52s

Dataset: hf://policyengine/policyengine-uk-data-private/enhanced_frs_2024_25.h5@2526edbbb687a3b54dbb11272e5389e4b62d2ac2, sha256 056e80fd8e9151d78f432a25cde863340baee81304c6216a37fb575ce8563784 (verified on both sides), policyengine-uk-data 1.58.1.

Receipts by tax head: head levels
Measure (head) 2025-26 2026-27 2027-28 2028-29 2029-30 2030-31 2031-32
Final charge on cash interest in non-cash ISAs £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Lifetime ISA withdrawal charge £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Income tax £294.46bn £315.23bn £335.83bn £351.28bn £366.96bn £385.36bn £400.00bn
National Insurance (employee and self-employed) £51.70bn £54.14bn £56.13bn £57.73bn £59.76bn £61.64bn £63.51bn
National Insurance (employer) £148.71bn £155.37bn £160.88bn £165.09bn £172.06bn £177.36bn £182.57bn
Employer NI change borne by capital £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Employer NI change borne by consumers £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Capital gains tax £12.60bn £13.15bn £13.60bn £14.11bn £14.67bn £15.24bn £15.80bn
Student loan repayments £7.82bn £8.13bn £8.39bn £8.58bn £8.79bn £9.04bn £9.29bn
VAT excluding private school fees £249.37bn £256.87bn £266.33bn £272.70bn £278.97bn £285.24bn £291.59bn
Change in VAT from its baseline £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
VAT on private school fees £1.95bn £2.01bn £2.09bn £2.14bn £2.20bn £2.26bn £2.31bn
Change in private school VAT from its baseline £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Fuel duties £24.71bn £24.13bn £26.00bn £26.83bn £26.51bn £25.78bn £24.63bn
Alcohol duties £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Tobacco duties £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Vehicle excise duties £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Betting and gaming duties £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
TV licence fee £4.26bn £4.40bn £4.41bn £4.42bn £4.43bn £4.44bn £4.45bn
Council tax £49.21bn £52.16bn £54.89bn £57.95bn £61.16bn £64.47bn £67.91bn
High value council tax surcharge £0.00bn £0.00bn £0.00bn £0.48bn £0.49bn £0.50bn £0.51bn
Northern Ireland domestic rates £0.55bn £0.58bn £0.60bn £0.63bn £0.66bn £0.69bn £0.72bn
Business rates £34.68bn £36.20bn £38.00bn £39.46bn £42.27bn £43.25bn £44.29bn
Stamp duty land tax on the household's own purchases £6.88bn £7.32bn £7.66bn £8.06bn £8.47bn £8.91bn £9.35bn
Stamp duty land tax paid by companies £4.72bn £4.57bn £5.29bn £6.03bn £6.37bn £6.92bn £7.45bn
Land transaction tax (Wales) £0.07bn £0.08bn £0.08bn £0.09bn £0.10bn £0.10bn £0.11bn
Land and buildings transaction tax (Scotland) £0.31bn £0.32bn £0.33bn £0.35bn £0.36bn £0.38bn £0.40bn
Unspecified tax change borne by capital £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Unspecified tax change borne by consumers £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Wealth tax £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Wealth tax excluding the main residence £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Land value tax £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Carbon tax £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Spending by benefit and service: head levels
Measure (head) 2025-26 2026-27 2027-28 2028-29 2029-30 2030-31 2031-32
Lifetime ISA bonus £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
State Pension £121.68bn £125.92bn £126.73bn £129.50bn £132.99bn £136.66bn £140.43bn
Pension Credit £6.66bn £7.03bn £7.29bn £7.48bn £7.60bn £7.84bn £8.03bn
Winter Fuel Payment £1.86bn £1.85bn £1.79bn £1.77bn £1.78bn £1.78bn £1.78bn
Pension Age Winter Heating Payment (Scotland) £0.19bn £0.20bn £0.20bn £0.21bn £0.21bn £0.22bn £0.22bn
Universal Credit £75.48bn £80.52bn £82.95bn £84.96bn £86.11bn £87.61bn £88.21bn
Housing Benefit £8.56bn £8.89bn £9.23bn £9.46bn £9.69bn £9.99bn £10.27bn
Council Tax Reduction £2.07bn £2.20bn £2.31bn £2.45bn £2.58bn £2.72bn £2.86bn
Employment and Support Allowance (income-related) £0.79bn £0.82bn £0.84bn £0.86bn £0.88bn £0.90bn £0.92bn
Employment and Support Allowance (contribution-based) £5.20bn £5.37bn £5.56bn £5.69bn £5.82bn £5.95bn £6.09bn
Jobseeker's Allowance (income-based) £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Jobseeker's Allowance (contribution-based) £0.29bn £0.30bn £0.31bn £0.31bn £0.32bn £0.33bn £0.34bn
Income Support £0.11bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Working Tax Credit £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Child Tax Credit £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Incapacity Benefit £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Personal Independence Payment £26.95bn £28.03bn £29.05bn £30.09bn £30.75bn £31.51bn £32.21bn
Disability Living Allowance £7.49bn £7.79bn £8.07bn £8.36bn £8.55bn £8.76bn £8.95bn
Attendance Allowance £8.68bn £9.03bn £9.36bn £9.69bn £9.90bn £10.15bn £10.38bn
Carer's Allowance £4.48bn £4.66bn £4.83bn £5.00bn £5.11bn £5.27bn £5.42bn
Carer Support Payment (Scotland) £0.31bn £0.32bn £0.33bn £0.34bn £0.35bn £0.36bn £0.37bn
Carer's Allowance Supplement (Scotland) £0.00bn £0.04bn £0.04bn £0.05bn £0.05bn £0.05bn £0.05bn
Severe Disablement Allowance £0.03bn £0.03bn £0.03bn £0.03bn £0.03bn £0.03bn £0.03bn
Industrial Injuries Disablement Benefit £0.50bn £0.52bn £0.54bn £0.55bn £0.56bn £0.58bn £0.59bn
Armed Forces Compensation Scheme £0.61bn £0.63bn £0.65bn £0.66bn £0.68bn £0.69bn £0.71bn
War Widow's or Widower's Pension £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Child Benefit £16.19bn £16.82bn £17.39bn £18.00bn £18.39bn £18.84bn £19.25bn
Scottish Child Payment £0.49bn £0.52bn £0.53bn £0.55bn £0.56bn £0.57bn £0.56bn
Two Child Limit Payment (Scotland, counterfactual) £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Healthy Start vouchers £0.05bn £0.05bn £0.05bn £0.05bn £0.05bn £0.05bn £0.05bn
Maternity Allowance £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Statutory Maternity Pay £3.30bn £3.41bn £3.54bn £3.62bn £3.70bn £3.78bn £3.87bn
Statutory Paternity Pay £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Statutory Sick Pay £1.12bn £1.16bn £1.20bn £1.23bn £1.26bn £1.29bn £1.32bn
Sure Start Maternity Grant £0.01bn £0.01bn £0.01bn £0.01bn £0.01bn £0.01bn £0.01bn
Bereavement Support Payment £0.67bn £0.69bn £0.71bn £0.73bn £0.75bn £0.76bn £0.78bn
Widowed Parent's Allowance £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Tax-Free Childcare £0.56bn £0.57bn £0.57bn £0.58bn £0.60bn £0.61bn £0.59bn
Extended childcare entitlement £2.99bn £3.50bn £3.59bn £3.71bn £3.78bn £3.86bn £3.93bn
Universal childcare entitlement £2.06bn £2.22bn £2.29bn £2.37bn £2.43bn £2.48bn £2.53bn
Targeted childcare entitlement £0.50bn £0.51bn £0.52bn £0.54bn £0.55bn £0.57bn £0.56bn
Childcare Offer for Wales £0.26bn £0.28bn £0.28bn £0.29bn £0.30bn £0.31bn £0.32bn
Care to Learn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Childcare Grant £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Parents' Learning Allowance £0.20bn £0.20bn £0.20bn £0.20bn £0.20bn £0.20bn £0.20bn
Adult Dependants' Grant £0.06bn £0.06bn £0.04bn £0.04bn £0.04bn £0.04bn £0.04bn
Travel Grant £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Disabled Students' Allowance suppressed suppressed suppressed suppressed suppressed suppressed suppressed
16 to 19 Bursary Fund £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
NHS spending £203.89bn £204.30bn £204.56bn £204.91bn £205.34bn £205.85bn £206.35bn
State education spending £0.02bn £0.02bn £0.02bn £0.02bn £0.02bn £0.02bn £0.02bn
Transport subsidy spending £25.05bn £25.10bn £25.14bn £25.18bn £25.23bn £25.30bn £25.36bn
Cost-of-living support payment £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Energy Bills Rebate £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Household Energy Payment £0.00bn £2.11bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Energy Price Guarantee subsidy £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Basic income £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Disability basic income £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Other public spending budget change £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Caseloads: head levels
Measure (head) 2025-26 2026-27 2027-28 2028-29 2029-30 2030-31 2031-32
Lifetime ISA bonus 0k 0k 0k 0k 0k 0k 0k
State Pension 11,264k 11,145k 10,819k 10,700k 10,723k 10,750k 10,776k
Pension Credit 1,486k 1,480k 1,472k 1,452k 1,450k 1,455k 1,464k
Winter Fuel Payment 7,844k 7,783k 7,473k 7,375k 7,391k 7,409k 7,427k
Pension Age Winter Heating Payment (Scotland) 803k 802k 800k 797k 799k 801k 802k
Universal Credit 6,342k 6,347k 6,336k 6,297k 6,292k 6,271k 6,223k
Housing Benefit 1,249k 1,245k 1,244k 1,227k 1,227k 1,229k 1,232k
Council Tax Reduction 3,176k 3,195k 3,200k 3,214k 3,224k 3,235k 3,242k
Employment and Support Allowance (income-related) 71k 71k 71k 71k 71k 71k 71k
Employment and Support Allowance (contribution-based) 719k 720k 721k 722k 724k 726k 728k
Jobseeker's Allowance (income-based) 0k 0k 0k 0k 0k 0k 0k
Jobseeker's Allowance (contribution-based) 67k 67k 67k 67k 67k 68k 68k
Income Support 25k 0k 0k 0k 0k 0k 0k
Working Tax Credit 0k 0k 0k 0k 0k 0k 0k
Child Tax Credit 0k 0k 0k 0k 0k 0k 0k
Incapacity Benefit 0k 0k 0k 0k 0k 0k 0k
Personal Independence Payment 3,567k 3,575k 3,579k 3,585k 3,593k 3,602k 3,611k
Disability Living Allowance 1,291k 1,294k 1,296k 1,298k 1,300k 1,304k 1,307k
Attendance Allowance 1,695k 1,698k 1,700k 1,703k 1,707k 1,711k 1,715k
Carer's Allowance 1,034k 1,037k 1,038k 1,040k 1,042k 1,060k 1,062k
Carer Support Payment (Scotland) 71k 71k 71k 72k 72k 72k 72k
Carer's Allowance Supplement (Scotland) 0k 71k 71k 72k 72k 72k 72k
Severe Disablement Allowance 4k 4k 4k 4k 4k 4k 4k
Industrial Injuries Disablement Benefit 137k 137k 137k 138k 138k 138k 139k
Armed Forces Compensation Scheme 93k 93k 94k 94k 94k 94k 94k
War Widow's or Widower's Pension 0k 0k 0k 0k 0k 0k 0k
Child Benefit 8,123k 8,120k 8,120k 8,130k 8,145k 8,161k 8,178k
Scottish Child Payment 351k 352k 352k 353k 350k 348k 337k
Two Child Limit Payment (Scotland, counterfactual) 0k 0k 0k 0k 0k 0k 0k
Healthy Start vouchers 200k 200k 200k 201k 201k 201k 202k
Maternity Allowance 0k 0k 0k 0k 0k 0k 0k
Statutory Maternity Pay 333k 333k 334k 334k 335k 336k 337k
Statutory Paternity Pay 0k 0k 0k 0k 0k 0k 0k
Statutory Sick Pay 181k 182k 182k 182k 182k 183k 183k
Sure Start Maternity Grant 26k 26k 26k 26k 26k 26k 26k
Bereavement Support Payment 98k 98k 98k 98k 98k 99k 99k
Widowed Parent's Allowance 0k 0k 0k 0k 0k 0k 0k
Tax-Free Childcare 1,043k 1,027k 1,014k 1,009k 1,011k 1,009k 993k
Extended childcare entitlement 795k 800k 796k 797k 798k 798k 792k
Universal childcare entitlement 1,071k 1,072k 1,072k 1,073k 1,076k 1,079k 1,074k
Targeted childcare entitlement 165k 156k 154k 154k 155k 155k 150k
Childcare Offer for Wales 39k 39k 39k 39k 39k 39k 39k
Care to Learn 0k 0k 0k 0k 0k 0k 0k
Childcare Grant 0k 0k 0k 0k 0k 0k 0k
Parents' Learning Allowance 98k 98k 98k 99k 99k 99k 99k
Adult Dependants' Grant 18k 18k 12k 12k 12k 12k 12k
Travel Grant 0k 0k 0k 0k 0k 0k 0k
Disabled Students' Allowance suppressed suppressed suppressed suppressed suppressed suppressed suppressed
16 to 19 Bursary Fund 0k 0k 0k 0k 0k 0k 0k
Cost-of-living support payment 0k 0k 0k 0k 0k 0k 0k
Energy Bills Rebate 0k 0k 0k 0k 0k 0k 0k
Household Energy Payment 0k 21,065k 0k 0k 0k 0k 0k
Energy Price Guarantee subsidy 0k 0k 0k 0k 0k 0k 0k
Basic income 0k 0k 0k 0k 0k 0k 0k
Disability basic income 0k 0k 0k 0k 0k 0k 0k
Government balance, each component counted once: head levels
Measure (head) 2025-26 2026-27 2027-28 2028-29 2029-30 2030-31 2031-32
Government receipts, each component once £892.00bn £934.66bn £980.50bn £1,015.93bn £1,054.24bn £1,091.56bn £1,124.90bn
Government spending £529.38bn † £545.66bn † £550.77bn † £559.52bn † £567.20bn † £575.96bn † £583.61bn †
Government balance, each receipt once £362.62bn † £389.00bn † £429.73bn † £456.41bn † £487.04bn † £515.61bn † £541.29bn †
Benefits in household_benefits but not in gov_spending £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn £0.00bn
Model totals: head levels
Measure (head) 2025-26 2026-27 2027-28 2028-29 2029-30 2030-31 2031-32
gov_tax £892.00bn £934.66bn £980.50bn £1,015.93bn £1,054.24bn £1,091.56bn £1,124.90bn
household_tax £491.97bn £520.41bn £551.21bn £576.01bn £601.00bn £626.70bn £648.42bn
pre_budget_change_household_tax £484.15bn £512.28bn £542.82bn £566.95bn £591.72bn £617.16bn £638.62bn
gov_spending £529.38bn † £545.66bn † £550.77bn † £559.52bn † £567.20bn † £575.96bn † £583.61bn †
household_benefits £529.38bn † £545.66bn † £550.77bn † £559.52bn † £567.20bn † £575.96bn † £583.61bn †
gov_balance £362.62bn † £389.00bn † £429.73bn † £456.41bn † £487.04bn † £515.61bn † £541.29bn †
Household income totals: head levels
Measure (head) 2025-26 2026-27 2027-28 2028-29 2029-30 2030-31 2031-32
Household market income £1,637.96bn £1,702.77bn £1,757.35bn £1,806.41bn £1,855.97bn £1,912.43bn £1,970.44bn
Household net income £1,636.69bn † £1,687.82bn † £1,715.46bn † £1,747.50bn † £1,778.70bn † £1,817.02bn † £1,859.69bn †
Household net income (HBAI definition) £1,449.57bn £1,499.83bn £1,530.87bn £1,565.18bn £1,597.89bn £1,635.60bn £1,676.62bn
Population: head levels
Measure (head) 2025-26 2026-27 2027-28 2028-29 2029-30 2030-31 2031-32
People 69,245k 69,383k 69,473k 69,592k 69,738k 69,912k 70,080k
Benefit units 35,110k 35,181k 35,226k 35,286k 35,360k 35,449k 35,534k
Households 31,046k 31,108k 31,149k 31,201k 31,267k 31,345k 31,420k
Poverty rates: head levels
Measure (head) 2025-26 2026-27 2027-28 2028-29 2029-30 2030-31 2031-32
Absolute poverty, before housing costs: everyone 16.3% 15.5% 16.4% 16.3% 16.5% 16.5% 16.6%
Absolute poverty, before housing costs: children 24.7% 22.1% 23.3% 22.9% 23.1% 23.2% 23.5%
Absolute poverty, before housing costs: working-age adults 14.1% 13.8% 14.7% 14.7% 14.8% 14.8% 15.0%
Absolute poverty, before housing costs: pensioners 13.7% 13.4% 14.0% 13.8% 13.9% 13.9% 13.9%
Absolute poverty, after housing costs: everyone 21.6% 21.0% 22.0% 22.6% 22.9% 23.1% 23.4%
Absolute poverty, after housing costs: children 33.3% 31.3% 32.3% 32.5% 32.9% 33.4% 34.2%
Absolute poverty, after housing costs: working-age adults 19.9% 19.8% 20.8% 21.6% 21.9% 22.0% 22.3%
Absolute poverty, after housing costs: pensioners 12.7% 12.4% 13.2% 13.2% 13.3% 13.4% 13.6%
Relative poverty, before housing costs: everyone 17.0% 16.0% 16.4% 16.3% 16.4% 16.3% 16.6%
Relative poverty, before housing costs: children 25.9% 22.9% 23.3% 22.9% 23.0% 22.9% 23.5%
Relative poverty, before housing costs: working-age adults 14.5% 14.1% 14.6% 14.7% 14.7% 14.7% 15.0%
Relative poverty, before housing costs: pensioners 14.5% 14.0% 13.9% 13.8% 13.8% 13.9% 13.9%
Relative poverty, after housing costs: everyone 21.4% 21.0% 21.3% 21.3% 21.3% 21.5% 21.7%
Relative poverty, after housing costs: children 33.1% 31.1% 31.2% 31.0% 31.1% 31.2% 31.8%
Relative poverty, after housing costs: working-age adults 19.9% 19.8% 20.2% 20.4% 20.5% 20.6% 20.8%
Relative poverty, after housing costs: pensioners 12.5% 12.4% 12.2% 11.7% 11.7% 11.8% 12.0%
People in poverty: head levels
Measure (head) 2025-26 2026-27 2027-28 2028-29 2029-30 2030-31 2031-32
Absolute poverty, before housing costs: everyone 11,278k 10,779k 11,402k 11,365k 11,475k 11,513k 11,651k
Absolute poverty, before housing costs: children 3,688k 3,314k 3,496k 3,445k 3,480k 3,502k 3,549k
Absolute poverty, before housing costs: working-age adults 5,965k 5,893k 6,308k 6,362k 6,420k 6,430k 6,521k
Absolute poverty, before housing costs: pensioners 1,624k 1,572k 1,598k 1,559k 1,575k 1,580k 1,581k
Absolute poverty, after housing costs: everyone 14,944k 14,600k 15,310k 15,721k 15,936k 16,141k 16,426k
Absolute poverty, after housing costs: children 4,975k 4,682k 4,837k 4,888k 4,945k 5,040k 5,179k
Absolute poverty, after housing costs: working-age adults 8,463k 8,461k 8,971k 9,344k 9,491k 9,584k 9,706k
Absolute poverty, after housing costs: pensioners 1,506k 1,457k 1,502k 1,490k 1,500k 1,517k 1,542k
Relative poverty, before housing costs: everyone 11,750k 11,084k 11,388k 11,364k 11,416k 11,415k 11,651k
Relative poverty, before housing costs: children 3,872k 3,423k 3,496k 3,445k 3,466k 3,454k 3,549k
Relative poverty, before housing costs: working-age adults 6,159k 6,013k 6,301k 6,362k 6,389k 6,389k 6,521k
Relative poverty, before housing costs: pensioners 1,720k 1,648k 1,590k 1,557k 1,562k 1,572k 1,581k
Relative poverty, after housing costs: everyone 14,849k 14,552k 14,775k 14,809k 14,874k 15,014k 15,234k
Relative poverty, after housing costs: children 4,942k 4,660k 4,673k 4,663k 4,679k 4,707k 4,806k
Relative poverty, after housing costs: working-age adults 8,426k 8,440k 8,711k 8,823k 8,873k 8,967k 9,062k
Relative poverty, after housing costs: pensioners 1,482k 1,452k 1,391k 1,323k 1,322k 1,339k 1,365k
Inequality: head levels
Measure (head) 2025-26 2026-27 2027-28 2028-29 2029-30 2030-31 2031-32
Gini coefficient, equivalised HBAI income before housing costs 0.3080 0.3052 0.3058 0.3058 0.3063 0.3069 0.3085
Mean household net income in each side's own deciles: head levels
Measure (head) 2025-26 2026-27 2027-28 2028-29 2029-30 2030-31 2031-32
Decile 1: mean household_net_income per household £17,681 † £17,997 † £17,915 † £18,152 † £18,307 † £18,531 † £18,717 †
Decile 2: mean household_net_income per household £27,939 † £29,023 † £28,711 † £29,013 † £29,276 † £29,691 † £30,151 †
Decile 3: mean household_net_income per household £34,320 † £35,485 † £36,073 † £36,611 † £37,197 † £37,737 † £38,187 †
Decile 4: mean household_net_income per household £38,933 † £40,687 † £41,724 † £42,727 † £43,334 † £44,132 † £44,434 †
Decile 5: mean household_net_income per household £43,239 † £45,567 † £46,012 † £46,960 † £47,714 † £48,584 † £50,279 †
Decile 6: mean household_net_income per household £47,927 † £49,210 † £50,893 † £51,939 † £52,942 † £53,942 † £54,680 †
Decile 7: mean household_net_income per household £55,662 † £56,571 † £57,442 † £57,670 † £58,730 † £59,651 † £61,082 †
Decile 8: mean household_net_income per household £68,789 † £70,803 † £71,858 † £73,466 † £73,880 † £75,317 † £77,178 †
Decile 9: mean household_net_income per household £78,890 † £81,013 † £80,861 † £82,646 † £85,464 † £86,946 † £89,509 †
Decile 10: mean household_net_income per household £117,005 † £120,339 † £124,742 † £126,632 † £125,971 † £128,600 † £130,938 †
Standard reform costings (change in gov_balance): head levels
Measure (head) 2025-26 2026-27 2027-28 2028-29 2029-30 2030-31 2031-32
Raise basic rate by 1pp £6.41bn £6.66bn £6.63bn £6.85bn £7.03bn £7.25bn £7.49bn
Raise higher rate by 1pp £3.61bn £3.87bn £4.00bn £4.22bn £4.44bn £4.71bn £4.83bn
Raise personal allowance by ~800GBP/year £-3.80bn £-3.92bn £-3.98bn £-4.04bn £-4.10bn £-4.23bn £-1.95bn
Raise child benefit by 25GBP/week per additional child £-1.99bn £-1.75bn £-1.55bn £-1.38bn £-1.30bn £-1.18bn £-1.07bn
Reduce Universal Credit taper rate to 20% £-20.77bn £-21.32bn £-21.75bn £-21.99bn £-22.34bn £-22.75bn £-22.94bn
Raise Class 1 main employee NICs rate to 10% £10.89bn £11.30bn £11.71bn £12.01bn £12.31bn suppressed suppressed
Raise VAT standard rate by 2pp £24.89bn £25.71bn £26.63bn £27.24bn £27.87bn £28.50bn £29.14bn
Raise additional rate by 3pp £2.90bn £3.15bn £3.22bn £3.41bn £3.61bn suppressed suppressed
Raise the Universal Credit standard allowance by 10% £-3.06bn £-3.18bn £-3.26bn £-3.35bn £-3.44bn £-3.49bn £-3.50bn
Raise the State Pension by 10% £-7.78bn £-8.01bn £-8.24bn £-8.47bn £-8.82bn £-8.55bn £-8.89bn
Standard reforms: change in each reform's target: head levels
Measure (head) 2025-26 2026-27 2027-28 2028-29 2029-30 2030-31 2031-32
Raise basic rate by 1pp: change in income_tax £6.55bn £6.86bn £6.79bn £7.00bn £7.20bn £7.44bn £7.66bn
Raise higher rate by 1pp: change in income_tax £3.61bn £3.92bn £4.01bn £4.23bn £4.45bn £4.72bn £4.84bn
Raise personal allowance by ~800GBP/year: change in income_tax £-3.93bn £-4.07bn £-4.13bn £-4.18bn £-4.22bn £-4.40bn £-1.75bn
Raise child benefit by 25GBP/week per additional child: change in child_benefit £2.33bn £2.14bn £1.94bn £1.76bn £1.65bn £1.52bn £1.40bn
Reduce Universal Credit taper rate to 20%: change in universal_credit £20.74bn £21.31bn £21.78bn £22.04bn £22.37bn £22.78bn £22.99bn
Raise Class 1 main employee NICs rate to 10%: change in national_insurance £11.15bn £11.63bn £12.00bn £12.29bn £12.61bn suppressed suppressed
Raise VAT standard rate by 2pp: change in vat £24.69bn £25.51bn £26.42bn £27.03bn £27.65bn £28.28bn £28.91bn
Raise additional rate by 3pp: change in income_tax £2.90bn £3.15bn £3.22bn £3.42bn £3.62bn suppressed suppressed
Raise the Universal Credit standard allowance by 10%: change in universal_credit £3.06bn £3.19bn £3.28bn £3.37bn £3.43bn £3.49bn £3.51bn
Raise the State Pension by 10%: change in state_pension £10.68bn £11.17bn £11.28bn £11.60bn £12.04bn £12.47bn £12.97bn

@MaxGhenis

MaxGhenis commented Nov 1, 2026

Copy link
Copy Markdown
Collaborator Author

Ready for hub review

Head e3308b11e6d78c9b2097bbcceef123d3652d45f1 on branch claude/budget-totals-from-registry-4rggpz, based on main at 2da212abcf2086337fc088a4a1236f4ae7b6af63. The task pinned pushes to this claude/... branch. Issue #2702 stays open for step 3.

Items

  1. Read first. I read programs.yaml, both total modules and their lists, the loader (tax_benefit_system.py modelled_policies), test_program_registry.py, the three code-health tests, utils/aggregate_impact/partition.py (it imports both lists by module and name, and that still works), fiscal_partition.yaml, the Program registry section of .github/CONTRIBUTING.md, and Classify the Budget helper variables in the fiscal partition #2763. CLAUDE.md has no Program registry section; the section it points to lives in docs/engineering/skills/model-structure.md.
  2. Registry fields (policyengine_uk/programs.yaml).
    • Header documents the fields: lines 6-34.
    • 39 programs entries gain budget (12 tax, 27 benefit), each inserted directly after the entry's variable: line. 13 gain group. council_tax gains budget_dropped_when (line 657) and state_pension gains budget_uprating: [all] (line 312).
    • The new budget_terms section (line 799) holds the 47 terms no programme entry carries, each with a kind: 15 programme_measure with program, 16 programme, 13 reform_measure, 3 public_service.
    • budget_order (line 1119) sets summation order.
    • No verified-year field was added or changed.
  3. Totals from the registry.
    • policyengine_uk/programs.py reads the YAML with only yaml and the standard library. build_budget_programs (line 129) validates the registry, and the package refuses to load if flags and budget_order disagree.
    • household_tax.py:5 and household_benefits.py:9 set HOUSEHOLD_TAX_VARIABLES and HOUSEHOLD_BENEFIT_VARIABLES in their old modules: 29 and 57 names, identical in content and order to main.
    • The formulas (household_tax.py:19, household_benefits.py:25) keep main's arithmetic. Council tax abolition comes from budget_dropped_when, and uprating from budget_uprating through uprated_benefit_variables (programs.py:206).
    • __init__.py:18 imports policyengine_uk.programs explicitly. build_metadata.py:53 adds programs.py to the data-build surface.
    • uv build puts policyengine_uk/programs.py and policyengine_uk/programs.yaml in the wheel.
    • The pre-Budget lists stay unchanged; see the last section.
  4. Stable function.
    • policyengine_uk.programs.budget_programs() (programs.py:118) returns a tuple of frozen BudgetProgram records: variable, label, budget (tax or benefit), group, kind, program, dropped_when and uprating, plus is_tax and is_programme.
    • Taxes come first, then benefits, each in summation order.
    • budget_variables(budget) returns a plain list. test_budget_programs_in_a_fresh_interpreter runs it in a subprocess three ways: importing programs first, the household_tax module first, or the package first.
  5. Tests.
    • tests/test_budget_programs.py covers drift (lines 166 and 192, traced from the formula, with council tax abolished and without), the differential against the frozen literal lists (lines 209 and 214), the council tax and uprating rules from main (line 219), the fresh interpreter (line 250), and Hypothesis properties (lines 297, 337 and 352).
    • tests/test_budget_totals_pinned.py checks 12 households × 2023/2025/2026/2028 × 4 reforms = 192 cells. I computed the values on main before the change and compare them with exact equality.
  6. Docs. .github/CONTRIBUTING.md:55 and docs/engineering/skills/model-structure.md:66. CLAUDE.md is unchanged: it holds no registry section and says it should not duplicate these rules. No other page under docs/ lists the registry fields.
  7. Overlaps.

Tests run

Run Result
test_budget_programs.py 16 passed
test_budget_totals_pinned.py, on this branch and on main's code 17 passed each
The new files plus test_program_registry.py, test_fiscal_partition.py, test_household_benefits_properties.py, test_gov_spending_properties.py, test_uc_statutory_pay_properties.py, tests/aggregate_impact, test_build_metadata.py 549 passed, 10 skipped (9 certified-dataset tests without a Hugging Face token; 1 needs policyengine_bundles)
tests/code_health (includes the three named list tests, unchanged) 3088 passed
ruff 0.17.2 format --check, check, check --select F811,F821; git diff --check clean
Pinned grid, branch against main all 192 cells bit-identical
CI on e3308b11 20 check runs, all success

I ran the 549-test batch before commit e3308b11 added the package import, then reran test_budget_programs.py after it: 16 passed. CI covers the final head.

Mutation and revert checks (scratch worktree)

Change Result
Swap income_tax and national_insurance in budget_order.tax 1 failed: the differential test. The pinned float sums did not move for these households
Drop the State Pension's budget_uprating: [all] 9 failed: the rules test and all 8 uprating cells
Drop budget_dropped_when from the High Value Council Tax Surcharge 2 failed: the rules test and the traced drift test with council tax abolished. The pinned grid has no surcharge payer, so it does not catch this
household_tax ignores budget_dropped_when 9 failed: the traced drift test and all 8 abolition cells
Remove budget: benefit from Scottish Child Payment the package refuses to import with RegistryError: ... only ordered ['scottish_child_payment']
Restore main's two formula modules 33 passed: these are equivalence tests, so they pass on both sides. The first run of this check failed the fresh-interpreter package case, which showed the submodule was reachable only through the variable modules' imports; __init__.py:18 now imports it, and the rerun passed

Aggregate impact report

Aggregate impact on certified data: pass

Base 2da212abcf against head 498e9f74b0 [the PR merge commit], both on enhanced_frs_2024_25.h5 (policyengine-uk-data 1.58.1), fiscal years 2025-26 to 2031-32.
Checks: 0 broken by this change, 1 failing on both sides, 0 fixed. Moves flagged for review: 0
Standard reform costings: 0 in every unsuppressed year.
No household's net income changes in any year.
No change in any year: receipts by tax head; government balance, each component counted once; model totals; household income totals; population; poverty rates; people in poverty; inequality; mean household net income in each side's own deciles.

The check failing on both sides is sign:vat (VAT excluding private school fees goes negative for 76 units), which this PR does not touch.

Skipped

  • Step 3 is for other repositories, as the brief says.
    • policyengine.py: replace UK_PROGRAMS and the imports of the two lists (Update validation page for new UKMOD country report #724) with policyengine_uk.programs.budget_programs(). Use is_tax/budget for the sign and group for grouping, and keep dropped_when/uprating so rows add up under those reforms.
    • policyengine-sim-api: pass budget, group and kind through beside the is_tax flag that Uprating benefit levels for 2025/26 #972 adds.
    • policyengine-app-v2: drop BENEFIT_PROGRAMS and sign rows from the API field.
  • No survey-data test ran locally. The 9 certified-dataset tests in test_aggregate_impact_certified.py skip without a token; CI's Aggregate impact job ran on the certified data.
  • No programs entry was added for the 16 kind: programme terms (for example Statutory Sick Pay and Carer Support Payment), so the coverage page does not change. Each needs verified years that parameters and tests confirm before it moves into programs.

Judgment calls

  1. Non-programmes. vat_change, private_school_vat_change, both employer NI incidence terms, both unspecified incidence changes, the wealth, land value and carbon taxes, basic income, disability basic income and the Two Child Limit Payment are kind: reform_measure. The three in-kind service terms are public_service. They sit in budget_terms, never in programs, and BudgetProgram.is_programme is False for them. test_budget_programs_have_labels_and_kinds checks that none appears in the coverage list. I put the Two Child Limit Payment here because its variable documents that the regulations were not laid and the baseline pays zero.
  2. Components of listed programmes (esa_income, jsa_contrib, working_tax_credit, council_tax_benefit, expected_sdlt and so on) are programme_measure with a program link, not new coverage entries. New entries would show ESA, JSA, tax credits and SDLT twice on the coverage page.
  3. Real programmes the coverage list lacks are kind: programme in budget_terms, with no verified-year fields. The schema requires verified_start_year on programs entries, and I did not verify those years.
  4. budget_order. Membership comes from the flags, but float32 sums depend on order, and the entries cannot be reordered without breaking the open PRs. So a separate list fixes the order, and the loader requires it to match the flags exactly. The cost: adding a term means flagging it and appending it there.
  5. Behaviour in the registry. Council tax abolition is budget_dropped_when on three terms, and uprating is budget_uprating with a default of both rates.
  6. Groups are deliberately narrow: property_transaction_taxes, council_tax, vat, employer_ni_incidence, esa, jsa, tax_credits, statutory_pay, winter_heating_payments, childcare (Tax-Free Childcare, the three English entitlements, the Wales offer), student_support, carer_benefits and public_services_in_kind. Taxes and benefits never share a group.
  7. DATA_BUILD_SURFACE now includes programs.py, because it now determines the totals. This changes the build fingerprint value once.

Pre-Budget lists

  • PRE_BUDGET_CHANGE_HOUSEHOLD_TAX_VARIABLES (17 names) is a subset of the registry's tax terms in the same relative order. It lacks 12: the ISA charge, the High Value Council Tax Surcharge, the four newer duties, private school VAT, the four incidence terms and student loans. Its formula drops only council_tax on abolition.
  • PRE_BUDGET_CHANGE_HOUSEHOLD_BENEFIT_VARIABLES (36 names) is a subset of the benefit terms but not in the same relative order: esa_contrib and jsa_contrib sit later. It lacks 21 terms, including Statutory Paternity Pay, both winter payments, the childcare and student support terms, the in-kind services, Healthy Start, the Two Child Limit Payment and Scottish Child Payment.
  • Both are frozen pre-Budget definitions that the registry neither feeds nor describes. Both are unchanged.

Generated by Claude Code

@MaxGhenis

MaxGhenis commented Nov 2, 2026

Copy link
Copy Markdown
Collaborator Author

Picking this up for the UK hub. Reply here if you're mid-change and I'll stop.

Generated by Claude Code

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@MaxGhenis @claude