Skip to content

Stop Colorado CCAP reading parameters before 2015 - #10646

Merged
MaxGhenis merged 6 commits into
mainfrom
co-early-year-guards
Nov 3, 2026
Merged

MaxGhenis merged 6 commits into
mainfrom
co-early-year-guards

Conversation

@MaxGhenis

@MaxGhenis MaxGhenis commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

What and why

Colorado CCAP crashed for January to September 2015. CountryTaxBenefitSystem backdates every parameter to 2015-01-01 (FIRST_MODELED_YEAR, policyengine_us/tools/parameters.py) and no further. CCAP reads its parameters at the October 1 before each month, so January to September 2015 read them at 2014-10-01, where the CCAP parameters (first dated 2022-10-01) have no value. Any 2015 Colorado household reaches CCAP through co_income_tax → co_cdcc → cdcc → childcare_expenses, whatever gov.simulation.branch_to_determine_itemization says. With the flag on, the package default, it also gets there through tax_unit_itemizes → tax_liability_if_itemizing. Either way it raised

ParameterNotFoundError: The parameter 'gov.states.co.ccap.entry.fpg_rate.DENVER_COUNTY_CO' was not found in the 2014-10-01 tax and benefit system.

CI didn't catch it because no test reached Colorado CCAP for January to September 2015.

A shared co_ccap_parameter_instant(period) (policyengine_us/variables/gov/states/co/ccap/co_ccap_parameter_instant.py) fixes it. It keeps the October rule, but a month whose October falls before FIRST_MODELED_YEAR reads at FIRST_MODELED_YEAR-01-01 instead. All eight CCAP formulas that computed the October instant now call it, and the helper backdates nothing further. At 2015-01-01 the CCAP parameters hold the values the model already applies from October 2015, and gov.hhs.smi still holds HHS's FY 2015 values, as at 2014-10-01.

The sales tax refund's age test let in filers who turned 18 during the year. Under C.R.S. 39-22-2003(1)(a)(II), a person qualifies who "is at least eighteen years of age as of December 31 of the taxable year preceding such taxable year", and on its 2015 Form 104 Colorado states the test as "born before 1997". co_sales_tax_refund_person_eligible compared the year's age with 18. It now tests birth_year < year - age_threshold, which equals age >= 19 on the model's age, since birth_year subtracts age from the period's year. The paths in (1)(a)(I), Colorado tax or withheld wages, stay age-independent. The 2022 age 18 cases now expect false, and new age 19 and 2015 cases hold the boundary.

The model doesn't test full-year Colorado domicile, which (1)(a)(I) and (II) both require, so part-year residents qualify as if they lived in Colorado all year.

Test plan

  • For policy changes: YAML tests whose expected values come from an external source. The 2015 Colorado SMI cases use HHS's FY 2015 and FY 2016 figures (85,182 and 85,915) as gov.hhs.smi records them. The other cases guard against regressions. The CO tax case sets the branching flag on and a new co_cdcc.yaml case sets it off, so neither depends on the package default.
  • Source conflict: not applicable (the statute and the form agree).
  • ruff format / ruff check on the changed files
  • Changelog fragment in changelog.d/

impact: we estimate that two Colorado tax units in the default dataset lose a $19 sales tax refund in 2025, each a single filer aged 18. Each one's Colorado income tax rises $19 and its household net income falls $19. Together they weigh 0.005, so the weighted totals move $0.10. Federal income tax and benefits don't change. PE-US pays no Colorado sales tax refund for 2026, so we measured 2025 (pair).
axiom: us-co:statutes/39/39-22-2003 matches (rulespec-us fe62a527af tests age as of the preceding December 31 against 18) | CCAP read date: n/a (no provision's reading changes)

🤖 Generated with Claude Code

CCAP reads its parameters at the October 1 before each month, so January to
September 2015 read them at 2014-10-01. Parameters are backdated only to
2015-01-01, so the CCAP parameters had no value there and a 2015 Colorado
household raised ParameterNotFoundError whenever itemization branching
reached the CDCC and child care subsidies. A shared helper now reads those
months at the backdating floor instead, adding no earlier parameter value.

The regression case sets the branching flag itself, so it fails without the
fix even though CI's update_itemization.sh turns branching off.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LwQq6FjgHJQCKydAZomnYR
@MaxGhenis

Copy link
Copy Markdown
Contributor Author

Ready for hub review

Head: 80c583a89a15fc8d41ad9e324f97a73143cf9bfa (one commit on origin/main 4e71481). Draft; CI hasn't run, because the Pull request workflow skips drafts (run 41548711695: skipped).

Item 1: CCAP crash under itemization branching — FIXED

Reproduced on main. A 2015 CO single filer with no inputs, run through the YAML runner, raised ParameterNotFoundError: gov.states.co.ccap.entry.fpg_rate.DENVER_COUNTY_CO ... 2014-10-01 from co_ccap_fpg_eligible.py:36. The path was co_income_tax_before_non_refundable_credits → co_taxable_income → taxable_income → tax_unit_itemizes:68 → tax_liability_if_itemizing → income_tax → cdcc → childcare_expenses → co_ccap_eligible. A Python Simulation with year-keyed inputs raised the same error. 2016–2024 didn't crash.

Cause. backdate_parameters copies each parameter's first value back to FIRST_MODELED_YEAR-01-01 = 2015-01-01 and no further (policyengine_us/tools/parameters.py, system.py:143). The CCAP formulas read parameters at the October 1 before each month, so January to September 2015 read at 2014-10-01, where every CCAP parameter (first dated 2022-10-01) is absent. October to December 2015 already read the backdated values.

Fix.

  • New helper policyengine_us/variables/gov/states/co/ccap/co_ccap_parameter_instant.py:4. It keeps the October rule, but a month whose prior October is before FIRST_MODELED_YEAR reads at FIRST_MODELED_YEAR-01-01 instead (line 18). This applies the tools/parameters.py rule that a formula reading an earlier period "must not reach before this year".
  • Called at co_ccap_child_eligible.py:20, co_ccap_smi.py:19, co_ccap_base_parent_fee.py:19, co_ccap_add_on_parent_fee.py:19, co_ccap_parent_fee.py:19, entry/co_ccap_fpg_eligible.py:22, entry/co_ccap_smi_eligible.py:20 and re_determination/co_ccap_re_determination_income_eligible.py:20. These are all eight places that computed the October instant. Fixing only the FPG lookup would have moved the crash to the next one.
  • No parameter value is added before any existing date. A script compared every gov.states.co.ccap and gov.hhs.smi leaf at 2014-10-01 and 2015-01-01: none differs where 2014-10-01 has a value. HHS's FY 2015 Colorado SMI ($85,182, dated 2014-10-01) therefore still applies to January to September 2015.

Regression tests, each run in its own process:

  • tax/income/co_income_tax_before_non_refundable_credits.yaml:65: 2015, gov.simulation.branch_to_determine_itemization: true set in the case, single CO filer → 0. It doesn't depend on update_itemization.sh.
  • ccap/co_ccap_smi.yaml:76: CO size 4 gives 85,182/12 in 2015-01 and 2015-09, and 85,915/12 in 2015-10.
  • ccap/entry/co_ccap_fpg_eligible.yaml:43: 2015-01, Adams County. Hand computation: 144/12 × 2.35 = 28.20, so income 28.1 is eligible and 28.3 isn't.
  • ccap/co_ccap_child_eligible.yaml:45: 2015-01, age 12 dependent → eligible.

Mutants, each in its own process, with the code restored and diffed after each:

  • (a) Clamp removed (if False:). Killed by the CO tax file (1 failed / 8), co_ccap_fpg_eligible.yaml (1/5) and co_ccap_child_eligible.yaml (1/5). co_ccap_smi.yaml survives this one, as expected, because HHS SMI has a 2014-10-01 value.
  • (b) Clamp to FIRST_MODELED_YEAR-10-01. Killed by co_ccap_smi.yaml (1 failed / 9).

Tests run at head, one file per process, package default flag (true):

  • All 14 files under tests/.../states/co/ccap/: 8, 4, 5, 8, 6, 9, 2, 6, 4, 4, 5, 2, 3 and 2 passed, 0 failed.
  • co_income_tax_before_non_refundable_credits.yaml: 8 passed.
  • co_sales_tax_refund.yaml: 34 passed.
  • co_sales_tax_refund_person_eligible.yaml: 8 passed.
  • ruff format and ruff check are clean on the changed files.

Item 2: sales tax refund age test — BLOCKED, code unchanged

  • Statute not read. The item says to read C.R.S. 39-22-2003 before changing anything, and I couldn't. law.justia.com and web.archive.org are refused by this session's egress policy. Two web searches returned no statutory text. So I can't say whether the statute or the form sets the rule, and I made no change.
  • What the code says. co_sales_tax_refund_person_eligible.py:24 has a comment saying the statute measures age as of December 31 of the year before the tax year, and that the model approximates this with the current year's age. :26 tests age >= 18.
  • Conversion to the model's age. If that comment is right, it agrees with the 2015 book ("18 or older for the entire year", Form 104 "born before 1997"): both mean born on or before Dec 31 of (year − 19). birth_year is year − age (household/demographic/age/birth_year.py), so the rule becomes birth_year < year − 18, which is age >= 19 on the model's age.
  • Ready-to-apply change, once the statute is confirmed: compare person("birth_year", period) < period.start.year - p.age_threshold, or equivalently age > p.age_threshold.
  • An existing test would move. That change flips co_sales_tax_refund_person_eligible.yaml:133, "Head exactly age 18 -> eligible" (2022), from true to false. That test currently contradicts the code's own statutory comment. It is not a partner test.
  • Full-year residence isn't modelled, by the statute's or the form's test. I report it here and didn't build it.
  • Overlap. Open PR Guard historical state rebate eligibility and document attribution #10309 edits co_sales_tax_refund_person_eligible.py, but only its documentation, and the CO sales tax refund tests. This PR touches neither file, so it shouldn't conflict.

Judgment calls

  • Clamp, not ineligibility. For January to September 2015 I clamped the read instant rather than returning "ineligible". Ineligibility would have made January to September disagree with October to December 2015, which already use the backdated rules. The clamp invents no value.
  • Dropped test case. I removed a planned 2015 wages case. It showed that the federal standard deduction for 2015 is $12,000: gov/irs/deductions/standard/amount.yaml starts at 2018 and is backdated. A hand computation at $6,300 failed (model 1,574.20 against 1,838.11), and that bug is outside this item.
  • Branch. I used the branch named in the brief, co-early-year-guards.

Proposed follow-ups

  • Federal standard deduction is backdated from 2018 into 2015–2017. standard/amount.yaml (and aged_or_blind/amount.yaml) start at 2018-01-01, so 2015–2017 returns use TCJA amounts, e.g. a single filer gets $12,000 in 2015 rather than $6,300, alongside a $4,000 exemption.
  • Colorado sales tax refund: age on the statute's date. Read C.R.S. 39-22-2003(1)(a)(II) and, if it measures age at Dec 31 of the prior year, test birth_year < year − threshold. Update the age-18 test at co_sales_tax_refund_person_eligible.yaml:133 to match.

Generated by Claude Code

MaxGhenis and others added 2 commits November 2, 2026 19:37
C.R.S. 39-22-2003(1)(a)(II) requires age 18 "as of December 31 of the taxable year preceding such taxable year", and the 2015 Form 104 says "born before 1997". The formula compared the year's age with 18, so a filer who turned 18 during the year qualified. The eligibility test now reads birth_year, and the case at age 18 now expects false.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Age 19 now holds the boundary these cases meant to test, and a new case pins a filer who turns 18 during the year.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis
MaxGhenis marked this pull request as ready for review November 3, 2026 00:38
@codecov

codecov Bot commented Nov 3, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.00%. Comparing base (b0a8c09) to head (4594c7b).
⚠️ Report is 167 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##              main    #10646    +/-   ##
==========================================
  Coverage   100.00%   100.00%            
==========================================
  Files            1        10     +9     
  Lines           15       174   +159     
  Branches         0         3     +3     
==========================================
+ Hits            15       174   +159     
Flag Coverage Δ
unittests 100.00% <100.00%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@MaxGhenis

MaxGhenis commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor Author

Hub review in progress

The review is finished and posted: #10646 (comment) (VERDICT: REQUEST_CHANGES at 17179a9aeb6f50aa52d54c108ba70c0a1f13228d). This interim comment is superseded.


Generated by Claude Code

@MaxGhenis

Copy link
Copy Markdown
Contributor Author

Ready for hub review
VERDICT: REQUEST_CHANGES
17179a9
METHODOLOGY_CHOICE: no

The head hasn't moved since 17179a9aeb. All 34 jobs of the Pull request run at that head (41554314353) concluded success. The PR is no longer a draft. The code is sound. Both fixes reproduce, both are correct for every year they reach, and every test that reads the changed variables passes. I'm asking for changes because the PR body, changelog and new test comment misstate the defect: the crash didn't need itemization branching. Those are text fixes. One small regression case should come with them.

Findings

P1. The body's account of why CI missed the crash is false: main crashes with branching off too

The body says the crash needs branch_to_determine_itemization: true and that "CI didn't catch it because every workflow runs update_itemization.sh first, and that script turns branching off." That's false. With the flag off, co_income_tax for a 2015 Colorado single filer raises the same error on main. It gets there through the Colorado CDCC, not through itemization. A reader (or the impact run) would wrongly conclude that runs with the flag off, CI included, were never affected.

Command, on main (4e71481e19), with a reform that sets the flag to False for 2015 onward and a household with no inputs (single adult aged 30, CO):

flag False
co_income_tax_before_non_refundable_credits [0.]
co_sales_tax_refund [37.]
co_income_tax ParameterNotFoundError
variables/gov/states/co/tax/income/credits/cdcc/co_cdcc.py", line 36
variables/gov/irs/credits/cdcc/capped_cdcc.py", line 17
variables/gov/irs/credits/cdcc/cdcc.py", line 14
variables/gov/irs/credits/cdcc/cdcc_potential.py", line 13
variables/gov/irs/credits/cdcc/cdcc_relevant_expenses.py", line 24
variables/gov/irs/credits/cdcc/cdcc_qualifying_childcare_expenses.py", line 16
variables/household/expense/childcare/tax_unit_childcare_expenses.py", line 15
variables/household/expense/childcare/childcare_expenses.py", line 15
variables/gov/states/co/ccap/co_ccap_eligible.py", line 18
variables/gov/states/co/ccap/entry/co_ccap_entry_eligible.py", line 16
variables/gov/states/co/ccap/entry/co_ccap_entry_income_eligible.py", line 13
variables/gov/states/co/ccap/entry/co_ccap_fpg_eligible.py", line 36

With a child added, co_income_tax, cdcc, childcare_expenses and co_ccap_eligible@2015-03 all raise on main whichever way the flag is set. At head, all of them compute either way: co_income_tax -37, cdcc 0, co_ccap_eligible@2015-03 true. The real reason CI missed the crash: no test under tests/policy/baseline/gov/states/co/ on main uses period: 2015 (grep -rln "period: 2015" returns nothing).

Replacements:

  • PR body. Replace "With gov.simulation.branch_to_determine_itemization: true, the package default, a 2015 Colorado household reaches CCAP through tax_unit_itemizes → … → childcare_expenses, and raised" with "Any 2015 Colorado household reaches CCAP through co_income_tax → co_cdcc → cdcc → childcare_expenses, whatever gov.simulation.branch_to_determine_itemization says. With the flag on, the package default, it also gets there through tax_unit_itemizes → tax_liability_if_itemizing. Either way it raised". Replace "CI didn't catch it because every workflow runs update_itemization.sh first, and that script turns branching off." with "CI didn't catch it because no Colorado test ran in 2015."
  • changelog.d/co-early-year-guards.fixed.md. Replace "which raised ParameterNotFoundError for January to September 2015 under itemization branching" with "which made every 2015 Colorado income tax calculation raise ParameterNotFoundError".
  • co_income_tax_before_non_refundable_credits.yaml:59-64. The comment should say that this case covers the itemization path and that the CDCC path crashes without branching. Add the flag-off case below as policyengine_us/tests/policy/baseline/gov/states/co/tax/income/credits/cdcc/co_cdcc.yaml, or in that folder's existing file if there is one. I ran it from a scratch file: at head it gives 1 passed, and at main 1 failed with ParameterNotFoundError: … 'gov.states.co.ccap.entry.fpg_rate.DENVER_COUNTY_CO' … 2014-10-01.
- name: 2015 single filer with no income, without itemization branching
  period: 2015
  absolute_error_margin: 0
  input:
    gov.simulation.branch_to_determine_itemization: false
    state_code: CO
    filing_status: SINGLE
  output:
    co_cdcc: 0

P2. The CPS age reading decides whether the new age rule helps or hurts the population, and nothing in the PR checks it

The household rule is right. The model defines birth_year = period.start.year - age (household/demographic/age/birth_year.py), so age means age at December 31 of the period year. On that meaning, birth_year < year - 18 is exactly "18 by December 31 of the prior year". Hand values:

  • 2022, age 18: born 2004, which is 17 on 2021-12-31, so ineligible. 2004 < 2004 is false ✓.
  • 2022, age 19: born 2003 ✓.
  • 2015, age 18: born 1997, ineligible under "born before 1997" ✓.
  • 2015, age 19: born 1996 ✓.

The population file's age isn't age at year end, though, and the direction of the change depends on which date it measures. Assume birthdays are spread evenly and that age is measured a fraction f of the way through a year (ASEC interviews run February to April, so f ≈ 0.2):

  • Age measured in the tax year itself. A person recorded as 18 is 18 on 2021-12-31 with share 1 − f ≈ 80%. A person recorded as 17 never is. The old rule (age >= 18) is wrong for about 20% of recorded 18-year-olds. The new rule (age >= 19) is wrong for about 80% of them. The new rule is worse, about 4x on that cell.
  • Age measured in the year after the tax year (an ASEC interview in March Y+1 for tax year Y). Nobody recorded as 18 qualifies, and about 80% of those recorded as 19 do. The old rule is wrong for every recorded 18-year-old. The new rule is wrong for about 20% of recorded 19-year-olds. The new rule is better.

I can't read the data pipeline from this session, so I can't say which case holds, or whether ages are rolled forward for later simulation years. The affected cell is Colorado heads and spouses recorded at 18 (or 19) with irs_employment_income == 0 and co_income_tax_before_non_refundable_credits == 0. Each one moves co_sales_tax_refund by one tier amount for their modified AGI. The lowest 2025 tier is $19 (per tests/core/test_co_sales_tax_refund_invariants.py). The hub should count that cell, by age, on the Colorado file before landing. If ages are measured in the tax year, the honest options are the old >= test with a comment, or a fractional adjustment, rather than this change.

I found no house pattern for a prior-year-end age test: grep for "age", period.last_year and age - 1 in variables/ returns nothing. The nearest pattern, a birth_year compared with a cutoff (ri_social_security_modification.py:25, mi_*_eligible.py, ut_retirement_credit_max.py:22), is the one this PR follows. Reading age at period.last_year would be worse: with no value for the prior year, a YEAR variable without a formula returns its default of 40.

P3. The helper's docstring and the body overstate what January to September 2015 read

co_ccap_parameter_instant.py:11-13 says those months read "the same backdated values the model applies from that October". The body says "January to September 2015 use the same backdated values the model already applies from October 2015". Both are false for gov.hhs.smi. 52 of its 63 leaves differ between 2015-01-01 and 2015-10-01, and the PR's own case asserts 85,182 for 2015-01 and 85,915 for 2015-10. The CCAP leaves (87) do match. Replace the docstring's last sentence with:

    first modeled year instead. That date has the same values as the October
    before it would have had: the CCAP parameters' backdated values and HHS's
    FY 2015 SMI. No value is added before it.

P3. The body doesn't report that full-year residence isn't modelled

The brief asked the PR to report this. The round-one hand-off mentions it, but the body doesn't, and co_sales_tax_refund_person_eligible.py has no comment on it. The formula never tests domicile for the entire year, under (1)(a)(I) or (II). Add to the body: "Full-year Colorado domicile, which C.R.S. 39-22-2003(1)(a)(I) and (II) both require, isn't modelled. Part-year residents qualify as if full-year. Not built here."

P3. age_threshold.yaml's description no longer says when age is measured

The description still reads "filers this age or older". Since the age is now measured at the end of the prior year, it should read "Colorado limits its sales tax refund to filers this age or older as of December 31 of the year before the tax year." #10309 rewrites the same description. Make the change in whichever PR lands second, to avoid a conflict.

P3 (follow-up, identical on main). Only co_ccap_smi.yaml would catch losing the October rule

My mutant if period.start.month >= 10: → if True: makes every month read the current year's October 1. Only co_ccap_smi.yaml catches it (7 of 9 fail). All 13 other CCAP files pass, including the FPG, parent fee and child eligibility files. The FPG and fee parameters are flat over each test's dates, so their tests can't see the read date. That's a coverage gap on main, not this PR's defect.

Checked and sound

  • Defect reproduced. On main, co_income_tax_before_non_refundable_credits, co_sales_tax_refund and co_income_tax for 2015 raise ParameterNotFoundError … DENVER_COUNTY_CO … 2014-10-01. At head they compute 0, 37 and -37.
  • CCAP, per call site. On main, eight files build the October instant (grep -- '-10-01"' under states/co/). At head all eight call co_ccap_parameter_instant, and none computes the instant itself: co_ccap_child_eligible:20, co_ccap_smi:19, co_ccap_base_parent_fee:19, co_ccap_add_on_parent_fee:19, co_ccap_parent_fee:19, entry/co_ccap_fpg_eligible:22, entry/co_ccap_smi_eligible:20, re_determination/co_ccap_re_determination_income_eligible:20. The only other CCAP parameter read, co_ccap_countable_income's adds, reads at the month and is backdated.
    • The helper changes only months 1 to 9 of years up to 2015. October onward and every month from 2016 keep main's instant.
    • January to September 2015 read 2015-01-01. All 87 gov.states.co.ccap leaves are missing at 2014-10-01 and equal at 2015-01-01 and 2015-10-01. All 63 gov.hhs.smi leaves are equal at 2014-10-01 and 2015-01-01, so FY 2015 SMI still applies (Colorado 85,182).
    • At head, with the flag on and with it off: co_ccap_subsidy@2015-01 is 0, @2015-10 is 0, and co_ccap_eligible@2015-03 is true.
  • Other states. At head, child_care_subsidies for 2015 computes without error in all 50 states and DC, for a household with a child and $20,000 of wages. No other state's October-instant formula crashes this way.
  • Threshold dates. age_threshold.yaml starts at 2021-01-01 and reads 18 at 2015-01-01 and 2015-12-31 (in_effect reads true there too). Both 2015 cases pass.
  • Tests at head, one file per process, flag at the package default.
    • All 14 CCAP files: 8, 4, 5, 8, 6, 9, 2, 6, 4, 4, 5, 2, 3 and 2 passed.
    • co_income_tax_before_non_refundable_credits 8, co_sales_tax_refund_person_eligible 11, co_sales_tax_refund_eligible 6, co_sales_tax_refund 34, co/tax/income/integration 11, co_family_affordability_credit 6 and co_tabor_cash_back 14 passed.
    • Contrib co_progressive_income_tax 5 and abolish_federal_income_tax 17 passed. tests/core/test_co_sales_tax_refund_invariants.py passed 2.
  • The new tests against main's code.
    • These fail at main: co_ccap_fpg_eligible (1), co_ccap_child_eligible (1), co_income_tax_before_non_refundable_credits (1, even after running update_itemization.sh, because the case sets the flag itself), co_sales_tax_refund_eligible (1) and co_sales_tax_refund_person_eligible (2: "Head turning 18 during the year" and "2015 head born in 1997").
    • The co_ccap_smi 2015 case passes at main, as the hand-off says. It guards where the clamp lands, not the crash.
  • Mutants, each in its own process on a scratch worktree, restored after each:
    • < → <= in the age test fails "Turns 18 during the year, not age eligible", "Head turning 18 during the year -> ineligible" and "2015 head born in 1997 -> ineligible". integration, co_sales_tax_refund, co_family_affordability_credit and co_tabor_cash_back all survive it.
    • Mine, threshold − 1 (age ≥ 20), fails "Age eligible", "Two age eligible", "Head 18 at the end of the preceding year -> eligible" and "2015 head born in 1996 -> eligible".
    • Clamp removed (if False:) fails co_ccap_fpg_eligible, co_ccap_child_eligible and co_income_tax_before_non_refundable_credits, each 1 case, with the crash. co_ccap_smi survives.
    • Clamp to 2015-10-01 fails only "Household size 4, State CO, January and October 2015".
    • Mine, October rule removed: see the coverage P3 above.
  • Fractional age. Age 18.5 in 2022 gives birth_year 2003 (truncated), so it's eligible. The rule acts as age > 18, which suits integer CPS ages.
  • Readers.
    • co_sales_tax_refund_person_eligible is read by co_sales_tax_refund_eligible, which is the defined_for of co_sales_tax_refund, and by co_sales_tax_refund's eligible_count. That feeds co_refundable_credits (through the refundable list in every year) and then co_income_tax.
    • p.age_threshold has one reader, this formula.
    • Impact variables to read: co_sales_tax_refund, co_refundable_credits, co_income_tax, state_income_tax and household_net_income.
    • In the 2026 baseline the refund pays nothing in 2026 (in_effect false from 2026-01-01). It pays in 2025 and 2027 onward, and also in 2015–2020, because in_effect backdates from 2021 (Guard historical state rebate eligibility and document attribution #10309 makes that 2015 true, 2016–2020 false). The age change can move all of those years. The CCAP change moves no output that computed on main. It only turns January to September 2015 errors into values.
  • Guard historical state rebate eligibility and document attribution #10309. git merge-tree shows no textual conflict in either order, or against current origin/main. On a scratch merge of the two heads, co_sales_tax_refund.yaml (43), co_sales_tax_refund_person_eligible.yaml (11), co_sales_tax_refund_eligible.yaml (6) and Guard historical state rebate eligibility and document attribution #10309's test_state_rebate_year_guards.py (24) all pass. Guard historical state rebate eligibility and document attribution #10309's new 2015 refund cases all use age 40, so the new age rule doesn't move them. Neither PR needs to land first. Whichever lands second should carry the age_threshold.yaml description change above.
  • Scope. No file under .github/ or tests/policy/baseline/partners/. Open PRs that touch these files: Guard historical state rebate eligibility and document attribution #10309 only (searched co_sales_tax_refund_person_eligible, co_ccap_parameter_instant, co_ccap_smi, co_ccap_fpg_eligible, co_ccap_child_eligible, co_income_tax_before_non_refundable_credits).
  • Body lines. impact: the hub will measure and axiom: (hub files pe-parity before landing) are both present. The diff moves baseline outputs (co_sales_tax_refund for heads and spouses recorded at 18 with no wages and no Colorado tax), so it needs the population check under P2.

What I couldn't check

  • The statute and forms. This session's network policy refused colorado.public.law, leg.colorado.gov and tax.colorado.gov. I couldn't read C.R.S. 39-22-2003, the 2015 DR 0104 book or HHS's SMI tables. The quoted (1)(a)(I)/(II) text, the 2015 "born before 1997" line and the 85,182/85,915 sources are unverified here. govinfo.gov was refused too. A web search's summary of CDOR's State Sales Tax Refund ITT (June 2024) describes the age test as "at least 18 years old before the tax year began", which agrees with the PR, but that's a secondhand summary, not the text.
  • The population data. I can't read the data pipeline, so I couldn't settle the CPS age-reference question (P2), size the affected cell, or run the impact.
  • Hub-machine logs named in the hand-off. I reran everything they cover.

Generated by Claude Code

MaxGhenis and others added 3 commits November 2, 2026 23:16
Main raised ParameterNotFoundError for 2015 Colorado income tax with branching off too, through
co_cdcc, cdcc and child care expenses. Add the review's flag-off case, and correct the
changelog, the test comment and the helper's docstring (SMI holds FY 2015 values in
January to September 2015).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The refund formula tests whether a filer was 18 by December 31 of the year before the tax year (C.R.S. 39-22-2003(1)(a)(II)), so we added that date to the threshold's description. The rest of the description comes from #10309, which landed first.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis

Copy link
Copy Markdown
Contributor Author

Hub impact pair, progress

The base side of the 2025 impact pair has finished. The head side runs next.

  • Base: 4170b4235b57e2d8a4ddd73ed01ce5f234906373, the merge base of the PR head with main.
  • Head to measure: 4594c7b5cd3491735e756dde1d6f7d46aa1bb062. The PR head has moved since the brief, which named 5f8b04879ed30c11d0598ee755d7855f17022813.
  • Dataset: hf://datasets/policyengine/populace-us/populace_us_2024.h5@populace-us-2024-spm-20260909. The local file's SHA-256 is 6496cc4393d4d3c6574f76eca231de5898c803b9067645591fd5c4d3e65aee84, which matches DEFAULT_DATASET_SHA256.
  • Year 2025, 8 household chunks, run through calculate_chunked from Add household-chunked microsimulation calculations #10561.
  • Wall time: 10.9 min (653 s) in the helper; 11 min 33 s for the whole process, by GNU time.
  • Maximum resident set size: 3.55 GiB (3,726,464 kB).

Amounts are in dollars. co_sales_tax_refund_eligible sums weights of eligible tax units.

Variable Base weighted sum
co_sales_tax_refund 127,951,087.96
co_income_tax 10,217,703,975.92
co_sales_tax_refund_eligible 3,003,699.80
income_tax 2,421,866,758,136.52
state_income_tax 548,929,924,569.89
household_net_income 16,344,885,148,794.13
household_benefits 1,954,659,343,329.57

Generated by Claude Code

@MaxGhenis

Copy link
Copy Markdown
Contributor Author

Ready for hub review
IMPACT PAIR: differs
head 4594c7b
base 4170b42

The PR head has moved since the brief, which named 5f8b04879ed30c11d0598ee755d7855f17022813. This pair measures 4594c7b5cd3491735e756dde1d6f7d46aa1bb062, the head when the run started. The base is its merge base with main.

Dataset. hf://datasets/policyengine/populace-us/populace_us_2024.h5@populace-us-2024-spm-20260909. The local file's SHA-256 is 6496cc4393d4d3c6574f76eca231de5898c803b9067645591fd5c4d3e65aee84, which matches DEFAULT_DATASET_SHA256. Both trees set the same DEFAULT_DATASET and DEFAULT_DATASET_SHA256.

Year and chunks. 2025, 8 household chunks. Both sides ran the same calculate_chunked file on the same chunks, in one environment (the lockfiles don't differ), with PYTHONPATH choosing the tree. Each side's summary.json names the tree it ran.

Variable Base sum Head sum Change Rows that differ Weighted rows up Weighted rows down Largest abs. difference
co_sales_tax_refund (tax unit) 127,951,087.96 127,951,087.86 -0.102 2 0 0.005369 19.00
co_income_tax (tax unit) 10,217,703,975.92 10,217,703,976.03 0.102 2 0.005369 0 19.00
co_sales_tax_refund_eligible (tax unit) 3,003,699.80 3,003,699.79 -0.005369 2 0 0.005369 1.00
income_tax (tax unit) 2,421,866,758,136.52 2,421,866,758,136.52 0 0 0 0 0
state_income_tax (tax unit) 548,929,924,569.89 548,929,924,569.99 0.102 2 0.005369 0 19.00
household_net_income (household) 16,344,885,148,794.13 16,344,885,148,794.03 -0.1016 2 0 0.005369 19.00
household_benefits (household) 1,954,659,343,329.57 1,954,659,343,329.57 0 0 0 0 0

Amounts are in dollars. co_sales_tax_refund_eligible sums weights of eligible tax units. household_net_income's change of −0.1016 comes from float64 rounding in compare.py: its sums are about 1.6e13, where adjacent float64 values are 0.002 apart, so the cents of both sums are noise too. The two moved households' exact weighted change is −19 × 0.005369 = −0.1020, the same size as the refund's.

What moves. Five of the seven variables move, in two Colorado tax units and their two households: co_sales_tax_refund, co_sales_tax_refund_eligible, co_income_tax, state_income_tax and household_net_income. income_tax and household_benefits don't move. Each of the two tax units loses a $19 refund. The two carry weights of 0.001006 and 0.004363, so each weighted total moves by about $0.10, and the weighted count of eligible tax units falls by 0.0054.

Why they move. I read the code, then ran each tree on just these two households. Both runs reproduce the base and head values that compare.json lists for these two tax units and their two households. At base, co_sales_tax_refund_person_eligible tests age >= 18. At head it tests birth_year < year - 18, and birth_year is year - age truncated to a whole year, so a filer needs a model age above 18 to qualify on age. Every age in the dataset is a whole number, so that means 19. Each moved tax unit is a single Colorado filer aged 18 in 2025 (birth year 2007), with no employment income and no Colorado tax before nonrefundable credits, so age was the only test they passed:

Tax unit Household Weight Filer's age Employment income CO tax before nonrefundable credits CO modified AGI Refund, base → head
134079 99483 0.001006 18 0 0 0.00 19 → 0
1133795 1099300 0.004363 18 0 0 12,433.50 19 → 0

co_refundable_credits includes the refund. co_income_tax subtracts co_refundable_credits, and state_income_tax subtracts state_refundable_credits, which adds co_refundable_credits, so both go from −19 to 0 in each moved tax unit. household_net_income adds the refund through household_refundable_tax_credits, so each household's net income falls by 19. Other filers in these households keep the refund: a 19-year-old spouse in tax unit 134080 still qualifies on age. The federal SALT deduction reads state_withheld_income_tax, not co_income_tax, and income_tax doesn't move. The PR's CCAP change, co_ccap_parameter_instant, returns the same read date as before for every month from October 2015 on, so it can't move a 2025 output.

Resources.

Side Wall time in the helper Process wall time Maximum resident set size
Base 10.9 min (653 s) 11 min 33 s 3.55 GiB (3,726,464 KiB)
Head 10.7 min (640 s) 11 min 16 s 3.55 GiB (3,725,960 KiB)

Each side ran in about 11 minutes and peaked at 3.55 GiB. The hub's earlier eight-chunk, nine-output run for 2026 took about 70 minutes and 9.75 GiB per side.

Dropped variables. None. All seven exist on both sides on the same entity (five on the tax unit, two on the household), the helper accepted all seven, and Microsimulation accepted dataset_end_year. Each side peaked far below the brief's 14 GB limit, so the pair kept 8 chunks.

Before the full sides, I ran precheck.py (below) on each tree, with the first 400 Colorado households in 2 chunks. The helper accepted all seven variables on both trees, and both runs exited 0.

Commands.

curl -sI https://huggingface.co/api/datasets/policyengine/populace-us | head -1   # the proxy's CONNECT line
curl -sSI https://huggingface.co/api/datasets/policyengine/populace-us | grep -E '^HTTP'   # HTTP/2 200
git clone -q https://github.com/PolicyEngine/policyengine-us.git pe && cd pe
git fetch -q origin pull/10646/head:pr-10646 pull/10561/head:pr-10561
HEAD_SHA=$(git rev-parse pr-10646); BASE_SHA=$(git merge-base origin/main $HEAD_SHA); echo $HEAD_SHA $BASE_SHA
git worktree add -q --detach ../head $HEAD_SHA
git worktree add -q --detach ../base $BASE_SHA
git show pr-10561:policyengine_us/tools/calculate_chunked.py > ../calculate_chunked.py
git diff --stat $BASE_SHA $HEAD_SHA -- pyproject.toml uv.lock   # empty: one environment serves both sides
cd ../head && pip install -q uv 2>/dev/null; uv sync > ../uv_sync.log 2>&1; echo "exit=$?" >> ../uv_sync.log
apt-get install -y -q time   # the VM had no GNU time

# pair_side.py and compare.py are the brief's scripts, unchanged; they sit next to calculate_chunked.py.
# Each side ran detached (setsid nohup), base first and then head, one at a time;
# a ps call once a minute logged its memory.
cd ../pe; mkdir -p ../out
H=$PWD/../calculate_chunked.py; PY=$PWD/../head/.venv/bin/python
export OMP_NUM_THREADS=4 OPENBLAS_NUM_THREADS=4 MKL_NUM_THREADS=4
PYTHONPATH=$PWD/../base /usr/bin/time -v $PY ../pair_side.py $H 8 2025 ../out/base > ../out/base.log 2> ../out/base.err
PYTHONPATH=$PWD/../head /usr/bin/time -v $PY ../pair_side.py $H 8 2025 ../out/head > ../out/head.log 2> ../out/head.err
$PY ../compare.py ../out/base ../out/head ../out/compare.json

# SHA-256 of the file the default dataset resolved to
F=$(PYTHONPATH=$PWD/../head $PY -P -c "from policyengine_us.system import DEFAULT_DATASET, _resolve_dataset_path; print(_resolve_dataset_path(DEFAULT_DATASET))" 2>/dev/null | tail -1); sha256sum "$F" | tee ../out/dataset.sha256
Extra checks: the smoke test and the two moved households
# Smoke test before the sides: the first 400 Colorado households in 2 chunks, each tree
PYTHONPATH=$PWD/../base /usr/bin/time -v $PY ../precheck.py $PWD/../calculate_chunked.py 2025 400 > ../out/precheck_base.log 2> ../out/precheck_base.err; echo "exit=$?" >> ../out/precheck_base.log
PYTHONPATH=$PWD/../head /usr/bin/time -v $PY ../precheck.py $PWD/../calculate_chunked.py 2025 400 > ../out/precheck_head.log 2> ../out/precheck_head.err; echo "exit=$?" >> ../out/precheck_head.log
# After the pair: the two households holding the moved tax units, each tree
for side in base head; do
  PYTHONPATH=$PWD/../$side $PY ../inspect_moved.py $PWD/../calculate_chunked.py 2025 1133795 134079 > ../out/inspect_$side.log 2> ../out/inspect_$side.err
done

precheck.py:

"""Smoke test before the full pair: python precheck.py <helper.py> <year> <households>.

Checks that each variable exists, names its entity, and passes the helper's
guard on a small Colorado subset of the default dataset. Writes nothing.
"""
import importlib.util
import json
import sys
import time

import numpy as np

helper, year, n_households = sys.argv[1], int(sys.argv[2]), int(sys.argv[3])
VARIABLES = (
    "co_sales_tax_refund",
    "co_income_tax",
    "co_sales_tax_refund_eligible",
    "income_tax",
    "state_income_tax",
    "household_net_income",
    "household_benefits",
)
spec = importlib.util.spec_from_file_location("hub_calculate_chunked", helper)
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)

import policyengine_us  # noqa: E402
from policyengine_us import Microsimulation  # noqa: E402
from policyengine_us.system import system  # noqa: E402

print("package", policyengine_us.__file__, flush=True)
print(
    "entities",
    json.dumps({name: system.variables[name].entity.key for name in VARIABLES}),
    flush=True,
)
start = time.monotonic()
dataset = module._load_dataset(Microsimulation.default_dataset)
source = module._EntityTables(dataset)
household = source.years[0].household
print("households", len(household), "columns", sorted(household.columns)[:40], flush=True)
keep = np.zeros(source.household_count, dtype=bool)
if "state_fips" in household:
    colorado = np.flatnonzero(household.state_fips.to_numpy() == 8)
    print("colorado households", len(colorado), flush=True)
    keep[colorado[:n_households]] = True
else:
    keep[:n_households] = True
masks = {entity: keep[owners] for entity, owners in source.positions.items()}
small = source.chunk(masks)
del dataset, source
result = module.calculate_chunked(
    lambda chunk: Microsimulation(dataset=chunk, dataset_end_year=year),
    VARIABLES,
    year,
    2,
    dataset=small,
)
print("weighted_sums", json.dumps(result.weighted_sums, sort_keys=True), flush=True)
for entity, table in result.entities.items():
    print(entity, len(table), list(table.columns), flush=True)
print("seconds", round(time.monotonic() - start, 1), flush=True)

inspect_moved.py:

"""Show the inputs behind the moved tax units: python inspect_moved.py <helper.py> <year> <tax_unit_id>...

Builds one Microsimulation on just the households holding those tax units, with
the helper's own chunking, and prints the refund test's inputs. Writes nothing.
"""
import importlib.util
import sys

import numpy as np
import pandas as pd

helper, year = sys.argv[1], int(sys.argv[2])
tax_unit_ids = [int(x) for x in sys.argv[3:]]
spec = importlib.util.spec_from_file_location("hub_calculate_chunked", helper)
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)

import policyengine_us  # noqa: E402
from policyengine_us import Microsimulation  # noqa: E402

print("package", policyengine_us.__file__)
dataset = module._load_dataset(Microsimulation.default_dataset)
source = module._EntityTables(dataset)
person = source.years[0].person
households = person.loc[
    person.person_tax_unit_id.isin(tax_unit_ids), "person_household_id"
].unique()
keep = np.isin(source.years[0].household.household_id.to_numpy(), households)
masks = {entity: keep[owners] for entity, owners in source.positions.items()}
small = source.chunk(masks)
sim = Microsimulation(dataset=small, dataset_end_year=year)
pd.set_option("display.width", 250)
pd.set_option("display.max_columns", 30)
people = pd.DataFrame(
    {
        name: np.asarray(sim.calculate(name, year, use_weights=False))
        for name in (
            "person_id",
            "person_household_id",
            "person_tax_unit_id",
            "age",
            "birth_year",
            "is_tax_unit_head",
            "is_tax_unit_spouse",
            "is_tax_unit_dependent",
            "irs_employment_income",
            "co_sales_tax_refund_person_eligible",
        )
    }
)
print(people.to_string(index=False))
units = pd.DataFrame(
    {
        name: np.asarray(sim.calculate(name, year, use_weights=False))
        for name in (
            "tax_unit_id",
            "tax_unit_weight",
            "filing_status",
            "co_income_tax_before_non_refundable_credits",
            "co_modified_agi",
            "co_sales_tax_refund_eligible",
            "co_sales_tax_refund",
            "co_refundable_credits",
            "co_income_tax",
            "state_income_tax",
            "income_tax",
        )
    }
)
print(units.to_string(index=False))
homes = pd.DataFrame(
    {
        name: np.asarray(sim.calculate(name, year, use_weights=False))
        for name in (
            "household_id",
            "state_code_str",
            "household_weight",
            "household_net_income",
            "household_benefits",
        )
    }
)
print(homes.to_string(index=False))
compare.json
{
 "household_net_income": {
  "entity": "household",
  "base_sum": 16344885148794.127,
  "head_sum": 16344885148794.025,
  "change": -0.1015625,
  "records_changed": 2,
  "weighted_changed": 0.005369204212911427,
  "weighted_up": 0.0,
  "weighted_down": 0.005369204212911427,
  "largest_abs": 19.0,
  "rows": [
   {
    "id": 1099300,
    "base": 117213.5625,
    "head": 117194.5625
   },
   {
    "id": 99483,
    "base": 25322.146484375,
    "head": 25303.146484375
   }
  ]
 },
 "household_benefits": {
  "entity": "household",
  "base_sum": 1954659343329.5674,
  "head_sum": 1954659343329.5674,
  "change": 0.0,
  "records_changed": 0,
  "weighted_changed": 0.0,
  "weighted_up": 0.0,
  "weighted_down": 0.0,
  "largest_abs": 0.0,
  "rows": []
 },
 "co_sales_tax_refund": {
  "entity": "tax_unit",
  "base_sum": 127951087.95737438,
  "head_sum": 127951087.8553595,
  "change": -0.10201488435268402,
  "records_changed": 2,
  "weighted_changed": 0.005369204212911427,
  "weighted_up": 0.0,
  "weighted_down": 0.005369204212911427,
  "largest_abs": 19.0,
  "rows": [
   {
    "id": 1133795,
    "base": 19.0,
    "head": 0.0
   },
   {
    "id": 134079,
    "base": 19.0,
    "head": 0.0
   }
  ]
 },
 "co_income_tax": {
  "entity": "tax_unit",
  "base_sum": 10217703975.924839,
  "head_sum": 10217703976.026854,
  "change": 0.10201454162597656,
  "records_changed": 2,
  "weighted_changed": 0.005369204212911427,
  "weighted_up": 0.005369204212911427,
  "weighted_down": 0.0,
  "largest_abs": 19.0,
  "rows": [
   {
    "id": 1133795,
    "base": -19.0,
    "head": 0.0
   },
   {
    "id": 134079,
    "base": -19.0,
    "head": 0.0
   }
  ]
 },
 "co_sales_tax_refund_eligible": {
  "entity": "tax_unit",
  "base_sum": 3003699.7992115617,
  "head_sum": 3003699.7938423576,
  "change": -0.005369204096496105,
  "records_changed": 2,
  "weighted_changed": 0.005369204212911427,
  "weighted_up": 0.0,
  "weighted_down": 0.005369204212911427,
  "largest_abs": 1.0,
  "rows": [
   {
    "id": 1133795,
    "base": 1.0,
    "head": 0.0
   },
   {
    "id": 134079,
    "base": 1.0,
    "head": 0.0
   }
  ]
 },
 "income_tax": {
  "entity": "tax_unit",
  "base_sum": 2421866758136.5215,
  "head_sum": 2421866758136.5215,
  "change": 0.0,
  "records_changed": 0,
  "weighted_changed": 0.0,
  "weighted_up": 0.0,
  "weighted_down": 0.0,
  "largest_abs": 0.0,
  "rows": []
 },
 "state_income_tax": {
  "entity": "tax_unit",
  "base_sum": 548929924569.88617,
  "head_sum": 548929924569.98816,
  "change": 0.10198974609375,
  "records_changed": 2,
  "weighted_changed": 0.005369204212911427,
  "weighted_up": 0.005369204212911427,
  "weighted_down": 0.0,
  "largest_abs": 19.0,
  "rows": [
   {
    "id": 1133795,
    "base": -19.0,
    "head": 0.0
   },
   {
    "id": 134079,
    "base": -19.0,
    "head": 0.0
   }
  ]
 }
}

Generated by Claude Code

@MaxGhenis
MaxGhenis merged commit df1d08b into main Nov 3, 2026
37 checks passed
@MaxGhenis

Copy link
Copy Markdown
Contributor Author

US + core hub merge audit, us#10646 at 4594c7b5cd3491735e756dde1d6f7d46aa1bb062 (squash): Stop Colorado CCAP from reading its parameters before 2015, and test the sales tax refund's age at the end of the year before the tax year.

  • Review: independent reviews (Opus 5.5). A cloud review at 17179a9aeb asked for changes (review). At 5f8b04879e the hub added the review's flag-off co_cdcc.yaml case and took its changelog, test comment and docstring text. No formula or parameter value changed. A GitHub branch update merged main 4170b4235b into 5f8b04879e, giving a219ca6cfe, and every file's changes match. 4594c7b5cd changes only the age_threshold description. A delta review of that range found no wrong result and approved with four notes. The hub took the one that rewords a sentence in the description.
  • Source conflict: none. The statute and Colorado's 2015 Form 104 agree.
  • Gates the hub checked live at merge:
    • gh pr checks exits 0;
    • the latest Pull request run at the head concluded success;
    • MERGEABLE;
    • not a draft;
    • no reviewer's latest review requests changes;
    • the semantic-overlap guard;
    • head pinned.
  • Impact (pair at 4594c7b5cd against main 4170b4235b): in 2025, two Colorado tax units in the default dataset, each a single filer aged 18, lose a $19 sales tax refund, so each one's Colorado income tax rises $19 and its household net income falls $19. Together they weigh 0.005, so the weighted totals move $0.10. Federal income tax and benefits don't change. PE-US pays no Colorado sales tax refund for 2026.
  • Axiom: us-co:statutes/39/39-22-2003 matches (rulespec-us fe62a527af tests age as of the preceding December 31 against 18) | CCAP read date: n/a (no provision's reading changes)

@MaxGhenis
MaxGhenis deleted the co-early-year-guards branch November 3, 2026 09:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants