Repository navigation
Formula-owned SSI export pins SSI in PolicyEngine simulations #18
Description
Summary
The published US populace artifact exports person.ssi, even though ssi is formula-owned in PolicyEngine-US. When PolicyEngine loads the HDF5, USSingleYearDataset flattens every table column into inputs and Microsimulation sets any column that matches a PolicyEngine variable. That turns ssi from a computed output into a pinned input, so SSI reforms can fail to change final SSI.
Why this matters
This surfaced while rerunning SSI asset-limit analysis against policyengine/populace-us populace_us_2024.h5. The reform appeared to have zero cost because the published ssi column overrode the formula path.
A direct check confirmed the issue:
- with the published artifact as-is, the SSI reform cost was
0.0 - after clearing the loaded
ssiinput arrays in the simulation, the same reform cost was about$8.7B
So this is not just an unused extra column; it changes reform behavior.
Expected behavior
Populace exports should not persist non-structural PolicyEngine variables that are owned by formulas. Those variables should be omitted so the rules engine calculates them from source inputs under baseline or reform policy.
A static formula_owned_excluded list is not enough: the export path should dynamically check the installed PolicyEngine variable registry and drop/fail any present formula-owned variable, while preserving structural ID and membership columns.
Observed affected US columns
In the published populace_us_2024.h5, the following non-structural exported columns are formula-owned under the current PE-US registry:
employment_income_last_yearhas_itinhas_tinhousehold_sizein_nycis_adultis_childis_seniorspm_unit_capped_work_childcare_expensesssitaxable_unemployment_compensationtraditional_ira_contributionsweeks_worked
The critical regression identified so far is ssi.
eCPS comparison
I checked local eCPS-style files as references. ssi was not present in either checked eCPS input file, so this specific bug is not caused by eCPS parity.
However, the published populace US artifact also contains many extra known PE variables relative to those eCPS files, which suggests the current export path does not enforce a closed eCPS input surface.
Related UK finding
A local UK populace candidate artifact has the same class of problem: non-structural formula-owned PE-UK variables were present in the saved H5. Examples include current_education, state_pension_reported, student_loan_repayments, and rail_subsidy_spending.
Fix direction
Add export-time guards in populace so that:
- formula-owned engine variables are dynamically identified from the live PolicyEngine registry and omitted/rejected before save;
- structural columns such as entity IDs and memberships are exempted;
- contracts can optionally enforce a closed allowed column surface for eCPS-style exports;
- regression tests cover
ssispecifically.
Activity
MaxGhenis commentedon Jun 14, 2026
Follow-up aggregate audit before broadening the fix:
Dropping every formula-owned column from the published US artifact is not safe as a silent behavior change. Comparing current stored/pinned aggregates to formula-recomputed aggregates after removing all 13 formula-owned exported columns from
populace_us_2024.h5gives large deltas for several columns:employment_income_last_yearin_nycspm_unit_capped_work_childcare_expensesssitaxable_unemployment_compensationweeks_workedSo the export fix should prevent unreviewed formula-owned columns from shipping, but only explicitly reviewed/classified columns should be dropped and converted to formula calculation.
ssiremains the regression that motivated this issue.UK side shows the same caution. Some formula-marked PE-UK columns are real stored source/compatibility layers: dropping
employment_income,capital_gains, oremployee_pension_contributionsmakes their aggregates zero, and droppingstudent_loan_repaymentsprevents the UK simulation from building during dataset uprating. Other formula paths also shift aggregates materially (rail_subsidy_spending+11%,is_married+57%, enhanced disability flag -88%).MaxGhenis commentedon Jun 14, 2026
Design correction after reviewing the recent no-silent-drop work:
The export boundary should assert, not mutate.
formula_owned_excludedshould document variables that upstream build stages have removed, but if any active formula-owned variable reacheswrite_dataset(), the writer should fail and write nothing. This matches the recentexported_nonzero_gatepattern: gates name bad columns; build stages make explicit removal/population decisions upstream.Also, the formula-owned check must be period-specific. Example:
weeks_workedhas a PE-US formula starting in 2025, but for a 2024 artifactget_formula("2024")isNone, so it is a valid 2024 input and must not be rejected.Current US 2024 active formula-owned exported columns after the period-specific check:
employment_income_last_year: OK to remove upstream per Max; old proposal/provenance layer, not worth maintaining a separate imputation.has_tin,has_itin: aggregates unchanged if one is dropped because PE-US has compatibility fallback/default logic. Should probably remove from export unless we need non-default TIN modeling; current artifact is allTrue.household_size: stale stored structural summary. PE formula from entity membership gives 332.8M weighted people vs stored 383.7M; 33% of unweighted household rows mismatch. Remove upstream and rely on structure, after checking downstream expectations.is_adult,is_child,is_senior: age-derived.is_childmatches exactly;is_adult/is_seniorhave stale low-weight mismatches. Remove upstream and rely on age formulas.in_nyc: not safe to remove as-is. Artifact hasin_nycandstate_fips, but nocounty/county_str; PE formula recomputes to zero. Fix by exporting county/county_str or making this a documented compatibility/source input in PE-US.spm_unit_capped_work_childcare_expenses: not safe to remove as-is. Formula recomputation changes aggregate by about -47.5% in the one-column audit, and local eCPS also carries this formula-marked column. Needs a PE-US/data decision: source input compatibility vs upstream component repair.ssi: should not be exported as final benefit. It masks SSI reforms; clearing it made the SSI asset-limit reform nonzero. Needs upstream input/takeup calibration review because formula baseline is lower.taxable_unemployment_compensation: likely should not export final person-level formula output; formula allocates tax-unit taxable UI fromunemployment_compensation. One-column recompute is -32.9%, so check income-tax/AGI impacts before removal.traditional_ira_contributions: not safe to remove blindly. Formula from desired contributions and limit scale is +13.1% vs stored; local eCPS also carries this formula-marked column. Needs PE-US/data decision around final-vs-desired IRA contribution surface.MaxGhenis commentedon Jun 14, 2026
Decision update:
spm_unit_capped_work_childcare_expensesis formula-owned and should not be exported as a populace input. The fix branch now includes an SPM-unit regression test that writing this column fails the export before any HDF5 is written; upstream build stages should remove it before calling the writer, after separately deciding whether any source input/dependency repair is needed to match intended childcare-expense aggregates.MaxGhenis commentedon Jun 14, 2026
Fixed by #37. The writer now refuses formula-owned PolicyEngine-US variables before writing an HDF5 dataset, including the SSI regression case, so final benefit/output columns cannot silently pin reforms as inputs.