Repository navigation
Conversation
…ory model MAIND == 1 marks the spouse who receives a Marriage Allowance transfer, so marriage_allowance stays on those records under policyengine-uk#1963, where it is the amount received and gives a Step 6 tax reduction. Transferors are not flagged, but their SPI personal allowance is the allowance less the transferable amount; set marriage_allowance_relinquished for them. PAS is not an input, so neither side is counted twice. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…riage Allowance The locked policyengine-uk predates PolicyEngine/policyengine-uk#1963. A lock bump changes the dataset build, so it belongs in its own release. The test runs as soon as the installed model defines marriage_allowance_relinquished. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…or transferors The 2020-21 fallback in _get_allowances would have given a 2022-23 tape the wrong transferable amount and found no transferors, without an error. The module already needs policyengine-uk. The invariant grid now fails if a record other than MAIND == 0 relinquishes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Round 2 review: the grid only checked that relinquishers qualify, so a narrower transferor rule passed every test. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Keeps main's unknown-region probe (model_simulates_unknown_region) and its SEINC_NUM fake-tape column, and replaces main's _get_marriage_allowance with this branch's _get_allowances. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Calibration fit against the base build: pass
A drop of more than 2.0% fails. Notes
National targets in a worse error band on the head (9)
Flags are for review: builds of the same code move 2 to 12 national targets to a worse band (#598). Bands are of |estimate / target - 1|; no target's estimate or error is published. Income distribution and single ages against the base build: passHBAI FYE 2025 (DWP), period 2024: equivalised income in £ a week, rates and Gini in per cent. A value outside HBAI's 95% interval is marked (outside).
FlagsNone. These measures report and flag but do not fail yet: their same-code spread is unmeasured (#598). Aggregate impact of the data change: passBase dataset Checks: 0 broken by this change, 4 failing on both sides, 0 fixed. Moves flagged for review: 186 (thresholds in Notes
Moves flagged for review
146 more flags are in the report artifact. Receipts by tax head
Spending by benefit and service
Caseloads
Government balance, each component counted once
Model totals
Components the model counts twice
Household income totals
Population
Poverty rates
People in poverty
Inequality
Mean household net income in each side's own deciles
Standard reform costings (change in gov_balance)
Standard reforms: change in each reform's target
Household net income by decileThe datasets hold different households, so households are not matched; compare the mean income in each side's own deciles. Checks failing on both base and head (4)
Counted-once checkCounted-once check not run: not requested. Provenance
Base dataset: Head dataset: Receipts by tax head: head levels
Spending by benefit and service: head levels
Caseloads: head levels
Government balance, each component counted once: head levels
Model totals: head levels
Components the model counts twice: head levels
Household income totals: head levels
Population: head levels
Poverty rates: head levels
People in poverty: head levels
Inequality: head levels
Mean household net income in each side's own deciles: head levels
Standard reform costings (change in gov_balance): head levels
Standard reforms: change in each reform's target: head levels
|
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Depends on PolicyEngine/policyengine-uk#1963 (fixes policyengine-uk#1947). Do not merge until #1963 is merged.
This PR does not bump the
policyengine-uklock (2.93.0). A lock bump changes the dataset build, so it belongs in its own data release. Models before #1963 ignore the newmarriage_allowance_relinquishedcolumn and treatmarriage_allowanceas before, so merging this changes no model output until the lock moves.What #1963 changes
marriage_allowancebecomes the transferable amount a person receives from their spouse's election. It gives a Step 6 tax reduction instead of coming off taxable income. The transferor side ismakes_marriage_allowance_election/marriage_allowance_relinquished, which cuts the transferor's personal allowance.datasets/spi.pysetsmarriage_allowancefrom the SPI variableMAIND, so the question is which spouseMAINDmarks.Which spouse
MAINDmarks: the recipientHMRC's wording alone does not say. From the SPI 2022-23 Public Use Tape documentation (UK Data Service SN 9422):
The 2020-21 documentation (SN 9121, p. 19) has the same label, marked "NEW FOR 2021".
The tape settles it (aggregates only):
MAIND == 1records (weighted)MAIND == 1withPAS= allowance + transferable amountMAIND == 1withPAS= allowance − transferable amountMAIND == 0withPAS= allowance − transferable amountSo
MAIND == 1is one spouse per couple, the one whose allowance HMRC raises: the recipient. The decisive fact is that noMAIND == 1record has the cut allowance. The count match is weaker evidence, because HMRC calibrates the tape to its published claimants (p. 7: "To calibrate to published Marriage Allowance claimants, estimated values are imputed to cases"), soMAINDand the transferors'PASare partly imputed. That publication describes the relief from the recipient's side: "Gives a tax reduction to a person whose spouse or civil partner has elected for a reduced Personal Allowance." In 2020-21, 20.8% ofMAIND == 1records have the plain £12,500 allowance; I have not established why. Transferors are in the tape unflagged, withPAScut by the transferable amount. That is the cluster policyengine-uk'sdocs/book/validation/spi-validation.ipynbdescribes ("the SPI removes the transferred amount from the individual's Personal Allowance, but does not indicate that the subsequent lower amount is driven by the Marriage Allowance").What changes here
marriage_allowancestays onMAIND == 1: the input is consistent with #1963. The stale comments now point tomarriage_allowance_transferable_amountand ITA 2007 s. 55B(4)-(5). One of them said HMRC rounds down; the code and the statute round up.MAIND == 0,PASequal to the personal allowance less the transferable amount, andTIunder the £100,000 taper threshold getmarriage_allowance_relinquished. Each SPI record is one person, so the model cannot derive an election from a spouse; settingmakes_marriage_allowance_electionwould do nothing, becausemarriage_allowance_relinquishedalso requiresis_marriage_allowance_spouse.create_spidoes not inputPAS, and a test now guards that. The model computes the allowance itself, so the recipient's SPI allowance uplift and the transferor's cut each enter once, through these two inputs.PAS= £11,310 andTI≥ £100,000 are left alone. All 26 have a higher-rate marginal band (MAR), which rules out an election (s. 55C(1)(c), (ca)). None hasTI − PENSRLF − GIFTAID − GIFTINVunder £100,000, and for 19 that figure is within £2 of £102,520, where the taper gives exactly £11,310._get_allowancesno longer falls back to 2020-21 values when policyengine-uk cannot be imported. On a 2022-23 tape that fallback would have found no transferors and raised no error.Nothing published moves. No production code reads the SPI h5: the income imputation reads the tab directly and never uses
MAIND, and the income projection now uses HMRC's aggregate tables.load_spi_datasethas no caller outside tests.SPI validation (2022-23 tape, real
Microsimulationruns)Share of records whose PolicyEngine
income_taxis within £10 of the SPI'sTOTTAX_DEVO_TXP. 834,538 non-composite records (MAIND != -1); 4 runs. Unknown region codes were mapped toSOUTH_EASTso the runs complete (see the note below).MAIND == 1)PAS= £11,310,TI< £100,000)Why recipients match less often under #1963 (run 1 vs run 2, same dataset):
SCOT_TXP)STIR(intermediate)PAS"includes 10% marriage allowance transfer", and its variable relationships (p. 19) list "Personal Allowances = PAS + BPADUE" with no separate Marriage Allowance reduction. This is a difference between the SPI's calculation and the statute, not a dataset error, so nothing here tries to reproduce it.Invariants and tests
For every record: each of
marriage_allowanceandmarriage_allowance_relinquishedis 0 or the transferable amount; no record has both; onlyMAIND == 1receives; exactly the unflagged records (MAIND == 0) with the cutPASand income under the taper threshold relinquish;personal_allowanceis never a dataset input.test_create_spi_marriage_allowance_invariants: exhaustive over 60 combinations ofMAIND×PAS×TI. Hypothesis is not a dependency of this repo. Mutation checks: changing the transferor condition fromMAIND == 0toMAIND != 1, or narrowing the income condition toTI < 50_000, fails it.test_create_spi_marriage_allowance_sides: named cases (recipient, transferor, no transfer, tapered look-alike, composite record).test_spi_marriage_allowance_matches_statute_in_policyengine_uk: runs SPI-shaped records through policyengine-uk. Recipient saves £252 in England and as a Scottish intermediate-rate payer; the recipient's own allowance stays £12,570; the transferor's is £11,310 and their tax is 20% × £690. Passes on #1963's head. On policyengine-uk main it would fail at £264.60, which is what pins the new semantics; so it skips, with that reason, until the installed model definesmarriage_allowance_relinquished. In CI it starts running when the lock reaches a release with #1963.PASandTI. The fiscal-year test now asserts £1,260 exactly instead of "not £1,250".Locally (
test_spi_build.py,test_spi_allowance_deductions.py): 20 passed against #1963's head; 19 passed and 1 skipped against policyengine-uk main.Review
Two independent rounds (Opus 5.5, Subfleet lanes, read-only tools, so the reviewers read the code and sources but did not run the tests).
MAIND == 0gap in the invariant grid. Addressed in ba07c78 and in this description.Because the statute test skips in CI under the current lock, green CI does not show the new inputs work with the merged model. Before merging, both test files must be run against merged policyengine-uk main: 20 passed, none skipped.
Before merge
Note for a follow-up
create_spimapsGORCODE13 ("Address abroad") and 14 ("Address unknown or not available"), 16,547 records, to"UNKNOWN", and policyengine-uk cannot simulate that region (ParameterNotFoundErroronprivate_rental_prices.UNKNOWN).load_spi_datasetworks around it;create_spioutput on its own does not simulate.axiom: n/a: data (SPI dataset inputs; the rules change is policyengine-uk#1963).
🤖 Generated with Claude Code