Skip to content

Keep every baseline benefit in SPM resources under reforms that pay a benefit - #10662

Open
MaxGhenis wants to merge 11 commits into
mainfrom
spm-benefits-parameter
Open

MaxGhenis wants to merge 11 commits into
mainfrom
spm-benefits-parameter

Conversation

@MaxGhenis

@MaxGhenis MaxGhenis commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

WIP: four of the six impact pairs are still running (the End Child Poverty Act in 2027 and 2024, EDAA and BOOST in 2027), so the impact section holds the two that finished. The CI figures come from run 42388504545 at 9b1b43d; CI at 2080a2e is running.

Fixes #10226.

Problem

Four structural reforms that pay a benefit replaced spm_unit_benefits with their own copies of its program list, because the list lived in Python where a reform could not extend it. A simulation that turns on one of these reforms uses its copy of spm_unit_benefits for every household, so a program the copy lacks drops out of SPM resources in every state. The copies had fallen behind:

Reform SPM programs the copy lacked Counted twice Household copy lacked
Texas rebate (tx_rebate) 19 state SSI supplements, including ca_state_supplement and wy_ssp none no household copy
End Child Poverty Act 42 SSI supplements, wa_sfa, wa_rca, wa_child_care_subsidies, wv_child_care_subsidies acp, ebb commodity_supplemental_food_program, household_health_benefits
EDAA End Child Poverty Act the same 46 acp, ebb already read the parameter
BOOST Act the same 46 acp, ebb commodity_supplemental_food_program, household_health_benefits

The issue asks about the Tlaib copies' other differences. Both trace to baseline changes the copies missed:

Change

  • The list. spm_unit_benefits now reads gov.household.spm_unit_benefits, a list parameter holding main's 77 programs in main's order.
    • The formula still appends assigned_aca_ptc under deduct_ptc, then spm_unit_capped_housing_subsidy unless gov.hud.abolition, in main's order.
    • The housing subsidy stays out of the parameter: tests/unit/test_spm_measurement_boundary.py fails any parameter that names an SPM valuation cap.
  • The helper. add_to_list_parameter (in policyengine_us/tools/parameters.py) adds names to every dated value of a list parameter.
    • It skips names a value already holds, so applying a reform again changes nothing. CountryTaxBenefitSystem(reform=R) applies R twice, and Simulation and Microsimulation apply structural reforms once more.
  • Each reform adds only its own benefit, to the SPM list and to the household list, and no longer redefines either aggregate:
    • tx_rebate, which keeps its 2027-only household_state_benefits entry and so reaches household_benefits that way;
    • ecpa_child_benefit, for both End Child Poverty Act versions;
    • boost_middle_class_tax_credit_paid_through_ssa, a new variable: the BOOST credit when SSA pays it, else zero. The old copies counted the credit as a benefit under the same condition.
  • The integrity test. test_aggregate_list_integrity.py drops its checks of the copies. It now fails any reform that redefines a benefit aggregate and points to add_to_list_parameter, and it checks that each member of both benefit lists names a variable the model defines and appears only once.

Invariants

policyengine_us/tests/test_reform_benefit_lists.py tests these with Hypothesis, over each of the four reforms and current law:

  1. add_to_list_parameter keeps every dated value and the members' order, appends only names a value lacks, leaves gaps (None values) alone, and changes nothing when run again. It writes new lists and leaves the names it gets and the member lists it reads unchanged. A clone of a parameter shares its member lists, so an edit in place would reach every clone that shares them. It changes values only through Parameter.update, so the parent node, which caches what it returns for each day, returns the new lists, and core marks the parameter modified, a flag a simulation checks before it serves a variable from its macro cache.
    • Hypothesis draws values that start on any day, gaps, and members that repeat, and the test checks each value's first day and the day before it as well as random days, through the parameter and through its parent node. It builds its expectation from copies it takes before the call.
    • A local mutation check ran the test against eleven broken helpers: stop date off by one day, no stop date, names not deduplicated, gaps not skipped, oldest value first, names added even when present, the given names sorted in place, a member list extended in place, the old member list cleared, values set without Parameter.update, and member lists deduplicated in place. The test fails on all eleven, with Hypothesis derandomized as in CI and with fresh draws. Earlier versions let five through: the first drew no gaps and passed the gap mutant; the second built its expectation after the call and passed the sorting and extending mutants; the third read only the parameter, never its node, and drew no repeated members, so it passed the last two.
  2. Under each reform, each benefit list (spm_unit_benefits, household_benefits, household_state_benefits) in each year from 2015 to 2035 equals the baseline list followed by that reform's own additions, and nothing else.
  3. Applying a reform once more changes no list.
  4. For any one-person household, spm_unit_benefits and household_benefits equal the sum of the baseline list's programs and the reform's additions, as the simulation computes each. The SPM sum also adds the capped housing subsidy. A program dropped or counted twice breaks the sum.
    • Hypothesis draws the state, a year from 2023 to 2027, and up to eight programs with random amounts, including the reforms' own benefits and ACP and EBB.
  5. A dollar of ACP or EBB adds a dollar to spm_unit_net_income under current law and under each reform.

Baseline neutrality needs its own evidence, because the properties take the baseline parameter as their oracle:

  • the parameter holds main's 77 programs in main's order (an AST comparison against main's Python list), and the formula appends the same two names in the same order;
  • the baseline YAML tests that read these aggregates pass unchanged (below);
  • current law gives the same totals before and after this change in the microsimulation runs below.

Tests

  • tests/policy/reform/{tx_rebate,end_child_poverty_act,edaa_end_child_poverty_act,boost}_counts_every_baseline_benefit.yaml: 24 cases, one reform per file, so each file runs in its own subprocess. All 24 pass on this branch, in CI (Baseline (reform), which runs each reform file in its own subprocess). Against main's code (e53b426, run locally from the main tree), 21 fail on the values they check and none errors. The 3 that pass there are the Texas case that checks the rebate adds to the baseline benefits, and EDAA's two household cases, because EDAA's household copy already read the parameter.
  • The existing YAML that reads these aggregates under a reform passes in CI at this head, which tests the merge with main: the Tlaib folder in Contrib (congress), contrib/states/tx/rebate/tx_rebate.yaml in its Contrib states shard, and the Trump dividend, Alaska energy relief and Head Start composition files in Baseline (reform). Every Baseline and Contrib job passes.
  • test_household_spm_resource_parity.py reads the SPM list from the parameter plus what the formula appends. In CI (Rest group remaining-1) its 57 cases pass, as do test_aggregate_list_integrity.py (25) and test_spm_measurement_boundary.py, and, in remaining-2, test_rest_python_groups.py (64).
  • test_reform_benefit_lists.py runs in its own Rest group, reform-benefit-lists, because it builds a full reformed system for each reform. Locally it peaked at 5.7 GiB RSS; remaining-heavy already peaks at 10.5 GiB on main (release run 42082143005), and a shared process would stack the two on a 16 GB runner. In CI (Rest shard 5) the group ran 21 passed and 1 skipped (current law applies no reform, so the reapply case has nothing to check) in 12 minutes 19 seconds, peaking at 5.96 GiB RSS. remaining-heavy peaked at 10.52 GiB, against 10.48 GiB in main's release run 42082143005. The planner has no timing for the new group yet, so shard 5 ran 37 minutes against the other shards' 20 to 29; main's next release run records the group's time.

Impact

Every figure here comes from a full microsimulation of one policy and year on hf://datasets/policyengine/populace-us/populace_us_2024.h5@populace-us-2024-spm-20260909: 4 runs, one per policy, year and side. The before runs use main at e53b426; the after runs use this branch at ee75695, whose model code matches the head (later commits change only tests and the Makefile). The script, reform_lists.py, records each run's commit and dataset, and render_reform_lists.py builds these tables and checks from its output. Poverty rates give the weighted share of people in SPM poverty, counting everyone under 18 as a child and everyone 65 or older as a senior.

2027

Policy SPM benefits before ($bn) After Change SPM poverty before After Change (pp) People in SPM poverty, change Child SPM poverty, change (pp) Household benefits, change ($bn)
Current law 2,171.61 2,171.61 +0.00 14.42% 14.42% +0.000 +0 +0.000 +0.00
Texas rebate 2,184.70 2,187.77 +3.07 14.36% 14.29% -0.065 -225,435 -0.111 +0.00

Where each change comes from

Each change in an aggregate equals the total of the programs the old copy lacked, minus the total of those it counted that the baseline does not. The copies' gaps account for every dollar, to within a few hundred dollars of float32 rounding:

2027 ($m):

Policy Aggregate Change Lacked Counted extra Expected Residual
Current law spm_unit_benefits +0.0 0.0 (0) 0.0 (none) +0.0 +0 dollars
Current law household_benefits +0.0 0.0 (0) 0.0 (none) +0.0 +0 dollars
Texas rebate spm_unit_benefits +3,071.3 3,071.3 (19) 0.0 (none) +3,071.3 +325 dollars
Texas rebate household_benefits +0.0 0.0 (0) 0.0 (none) +0.0 +0 dollars

Largest state changes in SPM benefits, 2027 ($m):

  • Texas rebate: 13 states change; CA +2,502.9, NY +223.4, WI +119.4, PA +71.8, NJ +52.7, OK +36.0, NH +21.0, RI +15.6
    state changes sum to +3,071.3; national +3,071.3
Own benefit and poverty detail, 2027
Policy Own benefit ($bn), before → after SPM poverty Child Senior Deep SPM poverty People in SPM poverty, change People in deep SPM poverty, change
Current law none 14.4211% → 14.4211% 18.6483% → 18.6483% 11.1210% → 11.1210% 3.3337% → 3.3337% +0 +0
Texas rebate tx_rebate 16.16 → 16.16 14.3593% → 14.2945% 18.4939% → 18.3829% 11.2047% → 11.1210% 3.3219% → 3.3104% -225,435 -39,993

Under the Texas rebate, senior SPM poverty after the fix equals current law's rate. The rebate adds up to $1,500 to each poor senior's resources, but in a run of the Texas households alone (tx_only_probe.py, which reproduces the national run's Texas SPM benefits, $142.796bn), the closest of the 48 poor Texas senior records falls $1,502 short of its threshold, so none crosses it.

The renderer ran 38 checks on these runs, and all pass. Among them: under current law every recorded number, state totals included, matches before and after; each reform's own benefit, market incomes, population, household_state_benefits and every program's total match on both sides; each change in net income equals the change in the benefit aggregate it adds; and the branch's list parameter holds the 77 baseline programs plus only the policy's own addition.

Conflicts

#9462 (draft), #10086 (draft) and #10569 edit spm_unit_benefits.py and all four reform copies. After this lands, each moves its additions into parameters/gov/household/spm_unit_benefits.yaml and drops its reform-copy edits.

axiom: n/a: SPM and household benefit aggregate lists; no rule changes

🤖 Generated with Claude Code

MaxGhenis and others added 5 commits November 4, 2026 19:52
Four structural reforms that pay a benefit (the Texas rebate, the End
Child Poverty Act, its Economic Dignity for All Agenda version and the
BOOST Act) replaced spm_unit_benefits with copies of its program list.
The copies fell behind. Texas dropped 19 state SSI supplements. The
three Tlaib reforms dropped 42 supplements, Washington's cash aid and
the Washington and West Virginia child care subsidies, and counted ACP
and EBB twice, since spm_unit_net_income adds both itself.

spm_unit_benefits now reads gov.household.spm_unit_benefits, and each
reform adds its own benefit there, and to household_benefits, with
add_to_list_parameter. The ECPA and BOOST household totals regain CSFP
and household health benefits. BOOST gets a variable for its credit
paid through SSA, so the lists can name it.

Fixes #10226.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
parameters(year) builds and caches the whole parameter tree for each
instant, and the list test draws 21 years, so one run peaked at 7.5 GB.
Reading the one list parameter avoids that.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The file builds a full reformed system for each of four reforms, the
kind of module that group isolates from the 14.4 GB remaining process.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… a Rest group

- The add_to_list_parameter property now draws None values (gaps) and values
  that start on any day, and checks each value's first day and the day before
  it as well as random days. A local mutation check ran it against six broken
  helpers: it fails on all six, while the old strategy, which drew no None
  values, let the helper that stops skipping them through.
- test_reform_benefit_lists.py leaves remaining-heavy for its own Rest group,
  reform-benefit-lists, so its 5.7 GiB peak RSS (local) never adds to the
  10.5 GiB remaining-heavy holds (release run 42082143005). remaining-heavy
  keeps main's files and comment.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis
MaxGhenis marked this pull request as ready for review November 5, 2026 21:24
MaxGhenis and others added 6 commits November 7, 2026 10:52
… no list in place

The property built its expected lists after calling add_to_list_parameter.
The parameter holds the drawn member lists themselves, so a helper that
sorted the names or extended a member list in place moved the expectation
with it and passed. The test now copies the names and every member list
before the call, builds the expectation from those copies, and asserts the
helper left both unchanged. A clone shares its member lists, so an in-place
edit would reach every clone that shares them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… repeated members

The helper property read only the parameter itself. A helper that set each
value's list directly, skipping Parameter.update, passed it, though the parent
node would keep returning its cached old lists to formulas and the parameter
would stay unmarked, so a simulation could serve a variable from its macro
cache. The property now reads through the parent node before and after the
call, and checks the modified flag after any change.

It also drew member lists without repeats, which hid a helper that dedupes the
member lists in place, and it now asserts that core keeps the drawn lists, so
its checks for edits in place cannot pass by default.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Main's #10663 added a parity test that reads spm_unit_benefits' members
with the year-less spm_members() of its Python list; the next commit
passes it the year this branch's dated parameter needs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Main's #10663 read spm_unit_benefits' members from its Python list, which
had no years. This branch moves the list into a dated parameter, so
spm_members takes a year; check each year the other parity tests check.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CountryTaxBenefitSystem applies a reform it is given, clears the modified
flag of every parameter below the root, then applies the reform again.
add_to_list_parameter changed values, and so set the flag, only when a
name was missing, and the Texas rebate's household_state_benefits edit
skipped its update when the rebate was already listed. So the second
application left the lists unmarked: each of the four reforms passed as
an object ended with every list it changes, and gov.household, unmarked,
while the Texas rebate switched on through its in_effect parameter, which
applies the structural reform only after the flags are cleared, kept them.
Core skips its macro cache for a variable when a parameter the variable
declares carries the flag.

add_to_list_parameter now marks the parameter, and each node above it,
on every call, and the Texas rebate marks household_state_benefits after
its check. No figure changes: core reads the flag only when a simulation
turns its macro cache on, which it leaves off by default, and none of the
variables that read these lists declares its parameters.

The helper property now checks the parameter's and its parent's flags
before and after the first call, then clears both and checks that the
second call, which adds nothing, sets them again. A new test checks that
each reformed system marks exactly the lists its reform changes, and that
current law marks none.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CountryTaxBenefitSystem resets the modified flag of each parameter and
node that parameters.get_descendants() yields, and that generator skips
the root itself. Three comments and the helper's docstring said it
clears every flag. This changes comments only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis

Copy link
Copy Markdown
Contributor Author

#10725 adds state_property_tax_refunds to spm_unit_benefits, so this PR's gov.household.spm_unit_benefits list needs it too. Without it, the six refunds drop out of SPM resources.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Keep one spm_unit_benefits list so reforms that add a benefit stop dropping baseline programs

1 participant