Repository navigation
Conversation
Add census_money_income for people, families and households. One list parameter per source in Census Current Population Reports P60-200, Appendix A holds the model variables that source covers, so the 18 sources and their citations sit in gov.household.census_money_income_sources. A benefit the model computes for a group gives each member an equal share. Check the CBO lists against CBO's published definitions (The Distribution of Household Income in 2021, Appendix D, and its sources) and correct them: Medicare counts net of premiums, the premium tax credit reduces federal taxes, corporate income tax counts in market income as well as in taxes, retirement income for past services and other nongovernmental income join market income, and WIC, the Medicare Savings Programs, state SSI supplements and state cash assistance count as means-tested transfers. Tests: Census and CBO YAML cases from the published definitions, and test_income_concept_parity.py, which requires every source any income list counts to be in each concept or in its documented exclusions, and runs every listed and excluded source through paired households to show each concept moves dollar for dollar with what it lists and not at all with what it leaves out. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ducational assistance out of CBO income Appendix A of P60-200 lists military survivor pensions among survivor benefits and military retirement among pension or retirement income, so the Census lists now take military_retirement_income apart: military_retirement_pay under pension or retirement income and military_retirement_pay_survivors under survivor benefits. The parity tests count the aggregate through its parts and fail if it gains a part no list holds. educational_assistance holds Pell Grants and other government aid beside private scholarships (IPUMS INCEDUC). CBO's market income takes only "other nongovernmental sources of income", and the model can't split the variable, so it stays out of the CBO measures, as on main, recorded in CBO_EXCLUDED with the reason. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MaxGhenis
force-pushed
the
census-money-income
branch
from
November 8, 2026 21:19
bdaf579 to
bebdb12
Compare
MaxGhenis
marked this pull request as ready for review
November 8, 2026 21:28
test_military_retirement_readers.py fails any non-tax list that names military_retirement_pay without military_retirement_pay_survivors, so a reader can't miss survivor annuities. Census money income counts both, but Appendix A puts them under two sources, so they sit in two list files and the file-level check can't see that. SPLIT_YAML_FILES now names the pension list's partner, and the guard passes the file only if the partner lists the survivor annuities. A new test fails if an entry stops being needed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Current Population Survey has left withdrawals by people under 58 out of total income since its 2019 redesign, which collects 2018 income. Census money income now counts distributions at any age through 2017 and from age 58 from 2018, through census_retirement_distributions and its minimum age parameter. This also answers the two reviews of 2b59cbd. Each Census list holds one block from 0000-01-01, and the CBO lists drop their year splits, because the system backdates household_state_benefits to 2015. The parity tests now simulate each year from 2022 to 2026, follow `adds` recursively for double counting, reject repeated list entries and pin each alias's parts. New cases split group benefits by the group that pays them and reconcile aged and disabled households in every state. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ome and the CBO measures #10725 added state_property_tax_refunds to household benefits. Appendix A leaves out tax refunds, and CBO's lists name no state tax relief, so both concepts leave the refunds out, as they do New Jersey's ANCHOR. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Max asked whether PolicyEngine-US should make its income concepts explicit, starting with Census money income. #10693 adds veterans' and railroad benefits to household benefits and cites Census's money income definition; this PR makes that definition a variable and checks the CBO income measures against CBO's published definitions.
census_money_income(person),census_family_money_income(family) andcensus_household_money_income(household) add the 18 sources in Current Population Reports P60-200 (1998), Appendix A, which the survey's current subject definitions still list. Each source is one list undergov.household.census_money_income_sources, citing the Appendix A pages that define it. Retirement account distributions count from age 58, as the survey has counted them since its 2019 redesign.cbo_household_*in line with CBO's The Distribution of Household Income in 2021 (publication 60341), with a citation on every list.Census money income
employment_income,self_employment_income,sstb_self_employment_income,partnership_s_corp_income,farm_operations_income,farm_rent_incometotal_unemployment_compensation,strike_benefits(Appendix A counts union strike benefits here)workers_compensationsocial_securityssiand 45 state supplementstanf,general_assistance,co_oap,ma_eaedc,wa_sfa,wa_rcaveterans_benefitssurvivor_benefits,military_retirement_pay_survivorsdisability_benefits,pfml_benefitspension_income,military_retirement_pay,census_retirement_distributions,railroad_benefitsinterest_incomedividend_incomerental_income,estate_incomeeducational_assistancealimony_incomechild_support_receivedfinancial_assistancegi_cash_assistance,ak_permanent_fund_dividend,ak_energy_relief,basic_income,trump_dividend,ca_la_infant_supplement,ca_la_expectant_parent_paymentLeft out. Appendix A leaves out capital gains (
capital_gains,non_sch_d_capital_gains,other_net_gain_person), tax refunds (salt_refund_income, and property tax relief such asnj_property_tax_relief) and lump-sum insurance payments (page A-3), and noncash benefits (page A-2): SNAP, WIC, school meals, CSFP, housing assistance, energy subsidies, child care subsidies, Head Start, Medicaid, CHIP, the Medicare Savings Programs, Medicare, the premium tax credit (paid to the insurer) and the 12 state health subsidies, plus discounts and rebates (ACP, EBB, CARE, FERA, vehicle and electrification rebates). The list treats renters' property tax relief like homeowners'. Discharged debt (debt_relief) is not money received. Income from bribes and other illegal activity (illicit_income) is not a regular payment under other income, and no other source covers it.miscellaneous_incomestays out because no Appendix A source fits it: the default data fills it with Census other income on survey records and with Form 4797 business property gains and losses on tax records (microcosm#393), and the model cannot separate the two. The test file gives the reason for every variable in household market income or household benefits that the Census concept leaves out.Gaps. The model lacks three Appendix A sources: Black lung payments, military family allotments and foreign government pensions. It has foster child payments only for Los Angeles County and no variable for interest earned inside retirement accounts, which the survey has counted as interest since 2019. Many listed variables hold $0 in the default dataset (listed under Impact), so sources 8 and 17 total $0 there, and source 18 falls short of Census's total because of
miscellaneous_incomeabove.Conventions
basic_income(tax unit) and Los Angeles's infant supplement (household). The Current Population Survey records the payment for the person who reports it, so the person-level amounts differ while the family and household totals match.SSIKID). The family and household totals add every member.military_retirement_incomeapart: retirement pay under source 10, survivor annuities under source 8.railroad_benefitsholds the Railroad Retirement Board's retirement, disability and survivor annuities in one variable, so it stays under source 10, and the disability and survivor lists say so. Neither choice changes the total.POTHVAL) equal to the sources without them, and all 9,770 records with distributions at 58 or older include them. The test uses age in the income year; Census uses age at the survey.CBO income measures
Checked against 60341 Appendix D (definitions, PDF page 41), Appendix A note 3 (page 27) and the tax definitions (page 42):
medicare_gross_cost; CBO measures Medicare as average cost net of offsetting receipts (note 3; also CBO 61157). The list now countsmedicare_cost, which subtracts Part A and Part B premiums; Part D premiums stay in.income_taxleaves out the PTC (gov.irs.credits.refundableomits it), so the newcbo_federal_tax_subtractionslist subtractsassigned_aca_ptcfrom CBO federal taxes.cbo_corporate_income_tax(an optional household input, default 0) counted only in taxes; it now also counts in market income, so it nets out of income after taxes as CBO intends.railroad_benefits(the Current Population Survey records railroad retirement as retirement income: IPUMSINCSS) andsurvivor_benefits(survivor pensions and annuities).child_support_receivedandfinancial_assistance.educational_assistancestays out, as on main: it holds Pell Grants and other government aid beside private scholarships (IPUMSINCEDUC), the model doesn't split them, and CBO's transfer lists name no education grants.wic(child nutrition programs: WIC is section 17 of the Child Nutrition Act, 42 U.S.C. 1786) andmsp_cost(Medicaid pays Medicare premiums and cost sharing, 42 U.S.C. 1396a(a)(10)(E)) and the 45 state SSI supplements and four state cash programs (co_oap,ma_eaedc,wa_sfa,wa_rca) under CBO's SSI and state and local general assistance. CBO also counts Part D low-income subsidies and ACA cost-sharing subsidies, which the model does not value; the list says so.Veterans' benefits stay out of the CBO measures. CBO's income categories have not named veterans' benefits since its report on 2014 (published March 2018); working paper 2017-09 counted them as means-tested transfers. Main also leaves them out. The test file records this as a documented exclusion, along with state paid family leave (outside CBO's four social insurance programs), guaranteed income pilots, the Alaska dividend and energy relief, and contributed reforms' payments.
Judgment calls
Each of these is a single reading where the sources leave room; each is disclosed in the list it touches and in the test file's exclusion reasons.
educational_assistance: outside CBO's measures, since its government part (Pell Grants first) is not "nongovernmental" income and the model can't separate its private scholarships.pfml_benefitsin Census disability benefits, outside CBO.partnership_s_corp_incomein Census earnings (self-employment).Invariants and tests
tests/test_income_concept_parity.pystates five invariants and checks each for every year from 2022 to 2026. The last three hold each source as an input, because a program's own formula responds to income:market_income_sources; CBO subtractions come from it; neither concept counts two variables that reach a common variable throughadds, at any depth (CBO countsmedicaid_costthroughmedicaidand the military parts throughmilitary_retirement_income; Census countsmilitary_retirement_incomethrough its parts), and each such variable must add exactly the parts the tests name.census_household_money_incomereconciles withhousehold_market_income + household_benefitsless the sources Census excludes plus the ones it adds, in each of the 51 states, for a working-age household with children, an aged person and a disabled person. That reconciliation reads the same lists as the variables, so it tests how the variables add them, not where a program belongs.YAML:
census_money_income.yaml(9 cases from Appendix A and the survey: a person with income from every source the model holds, the excluded receipts, an SPM-unit TANF split, group benefits split by the paying group, roommates in two families, a state cash program, and the age test at 58, at 57 and before the redesign) and, incbo_household_income.yaml, three updated cases and one new one.test_aggregate_list_integrity.pynow also checks that no year block of the new lists silently drops a variable.tests/code_health/test_military_retirement_readers.pyfails any benefit list that namesmilitary_retirement_paywithoutmilitary_retirement_pay_survivors. The Census lists count the two under different sources, so its newSPLIT_YAML_FILESlets the pension list pass only while the survivor list holds the annuities.Mutation checks. Removing the group-size division in
census_money_incomefails the dollar-for-dollar, totals and reconciliation tests. Movingsnapinto public assistance, addingrailroad_benefitstomarket_income_sourcesand droppingwicfrom CBO means-tested transfers gives 15 failures, each naming the source. Countingmilitary_retirement_incomebeside its parts failstest_no_source_counts_twice; dropping the survivor annuities failstest_census_classifies_every_sourceandtest_exclusions_are_current; givingmilitary_retirement_incomea third part failstest_aliases_sum_exactly_their_parts. On the guard, taking the annuities out of the survivor list failstest_benefit_source_lists_do_not_read_raw_military_retirement_pay; listing them in the pension list itself, or dropping the raw input from it, failstest_split_yaml_files_still_need_their_partner.On the logic the review added (run at
79f39e6152), each of ten mutations fails the test named:test_census_counts_retirement_distributions_from_age_58for 2018 and 2026 and a YAML age case.retirement_distributionsbeside the limited variable, orsocial_security_retirementbesidesocial_security:test_no_source_counts_twicein every year.test_no_source_counts_twiceandtest_census_classifies_every_sourcein every year.tanftwice:test_no_list_names_a_source_twice.medicaidaddchip:test_aliases_sum_exactly_their_parts.test_totals_sum_members_and_match_an_independent_allocationin every year and a YAML case.test_census_classifies_every_sourcefor 2022.Why Python. The coverage and double-counting invariants iterate over every list and year, which YAML cannot do; the dollar-for-dollar and exclusion checks run every source (139 in 2024) through paired households; the totals check compares the model with an independent numpy allocation, not with its own formula. No existing test checks concept membership;
test_aggregate_list_integrity.pychecks year blocks only, and the PR extends it to the new lists.Cost. New file: 958 lines. The tests build only household simulations on the shared system, with no reformed systems or datasets.
92caead63e,test_aggregate_list_integrity.pyalone: 31 tests, 0.57 s of test time, 24.24 s elapsed, 1.15 GB peak RSS. This branch at79f39e6152, that file plustest_income_concept_parity.py: 113 tests, 88.30 s of test time, 144.68 s elapsed, 3.90 GB peak RSS.remaining-2group (run 42606943944 at79f39e6152): 63 tests in 111.1 s of test time, the largest 11.4 s, third by test time among the group's 73 files. That group took 29:19.97 and peaked at 7.52 GB, below the 8.19 GB that shard 4'sremaining-1group reached in the same run. The aggregate-list file, inremaining-1, took 0.46 s.Impact
Four full microsimulation runs through the host's heavy-run queue, on
hf://datasets/policyengine/populace-us/populace_us_2024.h5@populace-us-2024-spm-20260909: this branch for 2024, 2025 and 2026 (298 to 338 s each) and main for 2026 (246 s). The branch ran at79f39e6152. Main ran at92caead63e.Census money income next to Census's published aggregates
Census table PINC-08 reports each source's recipients and mean for people 15 and older; its aggregate is their product and carries their rounding. Income year 2024 comes from the 2025 release and 2025 from the 2026 release. Census has not published 2026 income, so the 2026 column stands alone. Figures are billions of dollars for people 15 and older: PINC-08 reads age as of March of the following year, and the model reads it in the income year.
(a) The source includes a benefit the model computes for a group.
census_money_incomesplits it equally among the group's members. For the comparison with PINC-08, which records a payment for the person who reports it, these 15-and-older figures split it among the members 15 and older, each of whom counts as a recipient.PolicyEngine's total exceeds Census's by 5% in 2024 and by 4% in 2025. The sources differ in both directions, as the ratios show. The table reports these gaps and doesn't attribute them: the PR adds no calibration and changes no dataset.
census_retirement_distributionscounts from 58. In the default dataretirement_distributionstotals $890.4bn in 2024 and $1,063.7bn in 2025, and people 58 and older receive $526.4bn and $628.9bn of it. On the public use files, withdrawals by people under 58 total $57.4bn and $52.9bn; PINC-08's account row, for people 58 and older, shows $300.7bn and $321.6bn.miscellaneous_incomeon survey records, and tax records put Form 4797 business property gains and losses in the same variable (microcosm#393), so the concept leaves that variable out: it totalsParts of a source and recipient counts
employment_incomeself_employment_income,sstb_self_employment_income,partnership_s_corp_incomefarm_operations_income,farm_rent_incomemilitary_retirement_payrailroad_benefitscensus_retirement_distributionsPINC-08's rows within retirement income include payments reported as survivor or disability benefits. (a) marks the sources described under the first table.
Zero in this dataset. 40 variables the Census lists count total exactly $0 in all three years:
al_ssp,basic_income,ca_capi,ca_la_expectant_parent_payment,ca_la_infant_supplement,co_state_supplement,dc_ossp,de_ssp,financial_assistance,fl_oss,ga_ssp,general_assistance,gi_cash_assistance,hi_oss,ia_ssa,id_aabd,in_ssp,ks_sspp,ky_ssp,la_oss,md_paa,military_retirement_pay,military_retirement_pay_survivors,mo_ssp,mt_ssp,nc_sa,ne_aabd,nm_ssi_state_supplement,oh_rss,or_ssp,pfml_benefits,railroad_benefits,sc_ssi_state_supplement,strike_benefits,survivor_benefits,trump_dividend,tx_ssi_state_supplement,va_ag,wa_rca,wa_sfa. The lists still count them, and the dollar-for-dollar tests cover each on household situations; they move no national total here.CBO income measures, 2026
household_market_incomecbo_household_market_incomecbo_household_social_insurance_benefitscbo_household_means_tested_transferscbo_household_federal_taxescbo_household_income_before_transfers_and_taxescbo_household_income_after_transfers_and_taxesmedicare_costmedicare_gross_costassigned_aca_ptccbo_corporate_income_taxrailroad_benefitssurvivor_benefitschild_support_receivedfinancial_assistancewicmsp_costco_oap,ma_eaedc,wa_sfa,wa_rca)The last column of the first table rebuilds each measure before the change (the branch's value less the components it moved) and compares it with main's run. The differences are at most $3,987 on totals in the trillions, so each measure moved by the listed components and nothing else.
WIC. The $11.7bn shows WIC at full take-up among those eligible:
wicfollowstakes_up_wic_if_eligible, which defaults to true, and the default data doesn't set it. The data'swould_claim_wiccolumn reaches no variable in this package.Coordination
test_aggregate_list_integrity.py; whichever lands second rebases.axiom: n/a: statistical income concepts (Census money income and the CBO household income measures), not legal provisions
🤖 Generated with Claude Code