Repository navigation
Stop Colorado CCAP reading parameters before 2015 - #10646
Conversation
CCAP reads its parameters at the October 1 before each month, so January to September 2015 read them at 2014-10-01. Parameters are backdated only to 2015-01-01, so the CCAP parameters had no value there and a 2015 Colorado household raised ParameterNotFoundError whenever itemization branching reached the CDCC and child care subsidies. A shared helper now reads those months at the backdating floor instead, adding no earlier parameter value. The regression case sets the branching flag itself, so it fails without the fix even though CI's update_itemization.sh turns branching off. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LwQq6FjgHJQCKydAZomnYR
|
Ready for hub review Head: Item 1: CCAP crash under itemization branching — FIXEDReproduced on main. A 2015 CO single filer with no inputs, run through the YAML runner, raised Cause. Fix.
Regression tests, each run in its own process:
Mutants, each in its own process, with the code restored and diffed after each:
Tests run at head, one file per process, package default flag (
Item 2: sales tax refund age test — BLOCKED, code unchanged
Judgment calls
Proposed follow-ups
Generated by Claude Code |
C.R.S. 39-22-2003(1)(a)(II) requires age 18 "as of December 31 of the taxable year preceding such taxable year", and the 2015 Form 104 says "born before 1997". The formula compared the year's age with 18, so a filer who turned 18 during the year qualified. The eligibility test now reads birth_year, and the case at age 18 now expects false. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Age 19 now holds the boundary these cases meant to test, and a new case pins a filer who turns 18 during the year. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #10646 +/- ##
==========================================
Coverage 100.00% 100.00%
==========================================
Files 1 10 +9
Lines 15 174 +159
Branches 0 3 +3
==========================================
+ Hits 15 174 +159
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
Hub review in progress The review is finished and posted: #10646 (comment) (VERDICT: REQUEST_CHANGES at Generated by Claude Code |
|
Ready for hub review The head hasn't moved since FindingsP1. The body's account of why CI missed the crash is false: main crashes with branching off tooThe body says the crash needs Command, on main ( With a child added, Replacements:
- name: 2015 single filer with no income, without itemization branching
period: 2015
absolute_error_margin: 0
input:
gov.simulation.branch_to_determine_itemization: false
state_code: CO
filing_status: SINGLE
output:
co_cdcc: 0P2. The CPS age reading decides whether the new age rule helps or hurts the population, and nothing in the PR checks itThe household rule is right. The model defines
The population file's
I can't read the data pipeline from this session, so I can't say which case holds, or whether ages are rolled forward for later simulation years. The affected cell is Colorado heads and spouses recorded at 18 (or 19) with I found no house pattern for a prior-year-end age test: P3. The helper's docstring and the body overstate what January to September 2015 read
P3. The body doesn't report that full-year residence isn't modelledThe brief asked the PR to report this. The round-one hand-off mentions it, but the body doesn't, and P3.
|
Main raised ParameterNotFoundError for 2015 Colorado income tax with branching off too, through co_cdcc, cdcc and child care expenses. Add the review's flag-off case, and correct the changelog, the test comment and the helper's docstring (SMI holds FY 2015 values in January to September 2015). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The refund formula tests whether a filer was 18 by December 31 of the year before the tax year (C.R.S. 39-22-2003(1)(a)(II)), so we added that date to the threshold's description. The rest of the description comes from #10309, which landed first. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Hub impact pair, progress The base side of the 2025 impact pair has finished. The head side runs next.
Amounts are in dollars.
Generated by Claude Code |
|
Ready for hub review The PR head has moved since the brief, which named Dataset. Year and chunks. 2025, 8 household chunks. Both sides ran the same
Amounts are in dollars. What moves. Five of the seven variables move, in two Colorado tax units and their two households: Why they move. I read the code, then ran each tree on just these two households. Both runs reproduce the base and head values that
Resources.
Each side ran in about 11 minutes and peaked at 3.55 GiB. The hub's earlier eight-chunk, nine-output run for 2026 took about 70 minutes and 9.75 GiB per side. Dropped variables. None. All seven exist on both sides on the same entity (five on the tax unit, two on the household), the helper accepted all seven, and Before the full sides, I ran Commands. curl -sI https://huggingface.co/api/datasets/policyengine/populace-us | head -1 # the proxy's CONNECT line
curl -sSI https://huggingface.co/api/datasets/policyengine/populace-us | grep -E '^HTTP' # HTTP/2 200
git clone -q https://github.com/PolicyEngine/policyengine-us.git pe && cd pe
git fetch -q origin pull/10646/head:pr-10646 pull/10561/head:pr-10561
HEAD_SHA=$(git rev-parse pr-10646); BASE_SHA=$(git merge-base origin/main $HEAD_SHA); echo $HEAD_SHA $BASE_SHA
git worktree add -q --detach ../head $HEAD_SHA
git worktree add -q --detach ../base $BASE_SHA
git show pr-10561:policyengine_us/tools/calculate_chunked.py > ../calculate_chunked.py
git diff --stat $BASE_SHA $HEAD_SHA -- pyproject.toml uv.lock # empty: one environment serves both sides
cd ../head && pip install -q uv 2>/dev/null; uv sync > ../uv_sync.log 2>&1; echo "exit=$?" >> ../uv_sync.log
apt-get install -y -q time # the VM had no GNU time
# pair_side.py and compare.py are the brief's scripts, unchanged; they sit next to calculate_chunked.py.
# Each side ran detached (setsid nohup), base first and then head, one at a time;
# a ps call once a minute logged its memory.
cd ../pe; mkdir -p ../out
H=$PWD/../calculate_chunked.py; PY=$PWD/../head/.venv/bin/python
export OMP_NUM_THREADS=4 OPENBLAS_NUM_THREADS=4 MKL_NUM_THREADS=4
PYTHONPATH=$PWD/../base /usr/bin/time -v $PY ../pair_side.py $H 8 2025 ../out/base > ../out/base.log 2> ../out/base.err
PYTHONPATH=$PWD/../head /usr/bin/time -v $PY ../pair_side.py $H 8 2025 ../out/head > ../out/head.log 2> ../out/head.err
$PY ../compare.py ../out/base ../out/head ../out/compare.json
# SHA-256 of the file the default dataset resolved to
F=$(PYTHONPATH=$PWD/../head $PY -P -c "from policyengine_us.system import DEFAULT_DATASET, _resolve_dataset_path; print(_resolve_dataset_path(DEFAULT_DATASET))" 2>/dev/null | tail -1); sha256sum "$F" | tee ../out/dataset.sha256Extra checks: the smoke test and the two moved households# Smoke test before the sides: the first 400 Colorado households in 2 chunks, each tree
PYTHONPATH=$PWD/../base /usr/bin/time -v $PY ../precheck.py $PWD/../calculate_chunked.py 2025 400 > ../out/precheck_base.log 2> ../out/precheck_base.err; echo "exit=$?" >> ../out/precheck_base.log
PYTHONPATH=$PWD/../head /usr/bin/time -v $PY ../precheck.py $PWD/../calculate_chunked.py 2025 400 > ../out/precheck_head.log 2> ../out/precheck_head.err; echo "exit=$?" >> ../out/precheck_head.log
# After the pair: the two households holding the moved tax units, each tree
for side in base head; do
PYTHONPATH=$PWD/../$side $PY ../inspect_moved.py $PWD/../calculate_chunked.py 2025 1133795 134079 > ../out/inspect_$side.log 2> ../out/inspect_$side.err
done
"""Smoke test before the full pair: python precheck.py <helper.py> <year> <households>.
Checks that each variable exists, names its entity, and passes the helper's
guard on a small Colorado subset of the default dataset. Writes nothing.
"""
import importlib.util
import json
import sys
import time
import numpy as np
helper, year, n_households = sys.argv[1], int(sys.argv[2]), int(sys.argv[3])
VARIABLES = (
"co_sales_tax_refund",
"co_income_tax",
"co_sales_tax_refund_eligible",
"income_tax",
"state_income_tax",
"household_net_income",
"household_benefits",
)
spec = importlib.util.spec_from_file_location("hub_calculate_chunked", helper)
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
import policyengine_us # noqa: E402
from policyengine_us import Microsimulation # noqa: E402
from policyengine_us.system import system # noqa: E402
print("package", policyengine_us.__file__, flush=True)
print(
"entities",
json.dumps({name: system.variables[name].entity.key for name in VARIABLES}),
flush=True,
)
start = time.monotonic()
dataset = module._load_dataset(Microsimulation.default_dataset)
source = module._EntityTables(dataset)
household = source.years[0].household
print("households", len(household), "columns", sorted(household.columns)[:40], flush=True)
keep = np.zeros(source.household_count, dtype=bool)
if "state_fips" in household:
colorado = np.flatnonzero(household.state_fips.to_numpy() == 8)
print("colorado households", len(colorado), flush=True)
keep[colorado[:n_households]] = True
else:
keep[:n_households] = True
masks = {entity: keep[owners] for entity, owners in source.positions.items()}
small = source.chunk(masks)
del dataset, source
result = module.calculate_chunked(
lambda chunk: Microsimulation(dataset=chunk, dataset_end_year=year),
VARIABLES,
year,
2,
dataset=small,
)
print("weighted_sums", json.dumps(result.weighted_sums, sort_keys=True), flush=True)
for entity, table in result.entities.items():
print(entity, len(table), list(table.columns), flush=True)
print("seconds", round(time.monotonic() - start, 1), flush=True)
"""Show the inputs behind the moved tax units: python inspect_moved.py <helper.py> <year> <tax_unit_id>...
Builds one Microsimulation on just the households holding those tax units, with
the helper's own chunking, and prints the refund test's inputs. Writes nothing.
"""
import importlib.util
import sys
import numpy as np
import pandas as pd
helper, year = sys.argv[1], int(sys.argv[2])
tax_unit_ids = [int(x) for x in sys.argv[3:]]
spec = importlib.util.spec_from_file_location("hub_calculate_chunked", helper)
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
import policyengine_us # noqa: E402
from policyengine_us import Microsimulation # noqa: E402
print("package", policyengine_us.__file__)
dataset = module._load_dataset(Microsimulation.default_dataset)
source = module._EntityTables(dataset)
person = source.years[0].person
households = person.loc[
person.person_tax_unit_id.isin(tax_unit_ids), "person_household_id"
].unique()
keep = np.isin(source.years[0].household.household_id.to_numpy(), households)
masks = {entity: keep[owners] for entity, owners in source.positions.items()}
small = source.chunk(masks)
sim = Microsimulation(dataset=small, dataset_end_year=year)
pd.set_option("display.width", 250)
pd.set_option("display.max_columns", 30)
people = pd.DataFrame(
{
name: np.asarray(sim.calculate(name, year, use_weights=False))
for name in (
"person_id",
"person_household_id",
"person_tax_unit_id",
"age",
"birth_year",
"is_tax_unit_head",
"is_tax_unit_spouse",
"is_tax_unit_dependent",
"irs_employment_income",
"co_sales_tax_refund_person_eligible",
)
}
)
print(people.to_string(index=False))
units = pd.DataFrame(
{
name: np.asarray(sim.calculate(name, year, use_weights=False))
for name in (
"tax_unit_id",
"tax_unit_weight",
"filing_status",
"co_income_tax_before_non_refundable_credits",
"co_modified_agi",
"co_sales_tax_refund_eligible",
"co_sales_tax_refund",
"co_refundable_credits",
"co_income_tax",
"state_income_tax",
"income_tax",
)
}
)
print(units.to_string(index=False))
homes = pd.DataFrame(
{
name: np.asarray(sim.calculate(name, year, use_weights=False))
for name in (
"household_id",
"state_code_str",
"household_weight",
"household_net_income",
"household_benefits",
)
}
)
print(homes.to_string(index=False))compare.json{
"household_net_income": {
"entity": "household",
"base_sum": 16344885148794.127,
"head_sum": 16344885148794.025,
"change": -0.1015625,
"records_changed": 2,
"weighted_changed": 0.005369204212911427,
"weighted_up": 0.0,
"weighted_down": 0.005369204212911427,
"largest_abs": 19.0,
"rows": [
{
"id": 1099300,
"base": 117213.5625,
"head": 117194.5625
},
{
"id": 99483,
"base": 25322.146484375,
"head": 25303.146484375
}
]
},
"household_benefits": {
"entity": "household",
"base_sum": 1954659343329.5674,
"head_sum": 1954659343329.5674,
"change": 0.0,
"records_changed": 0,
"weighted_changed": 0.0,
"weighted_up": 0.0,
"weighted_down": 0.0,
"largest_abs": 0.0,
"rows": []
},
"co_sales_tax_refund": {
"entity": "tax_unit",
"base_sum": 127951087.95737438,
"head_sum": 127951087.8553595,
"change": -0.10201488435268402,
"records_changed": 2,
"weighted_changed": 0.005369204212911427,
"weighted_up": 0.0,
"weighted_down": 0.005369204212911427,
"largest_abs": 19.0,
"rows": [
{
"id": 1133795,
"base": 19.0,
"head": 0.0
},
{
"id": 134079,
"base": 19.0,
"head": 0.0
}
]
},
"co_income_tax": {
"entity": "tax_unit",
"base_sum": 10217703975.924839,
"head_sum": 10217703976.026854,
"change": 0.10201454162597656,
"records_changed": 2,
"weighted_changed": 0.005369204212911427,
"weighted_up": 0.005369204212911427,
"weighted_down": 0.0,
"largest_abs": 19.0,
"rows": [
{
"id": 1133795,
"base": -19.0,
"head": 0.0
},
{
"id": 134079,
"base": -19.0,
"head": 0.0
}
]
},
"co_sales_tax_refund_eligible": {
"entity": "tax_unit",
"base_sum": 3003699.7992115617,
"head_sum": 3003699.7938423576,
"change": -0.005369204096496105,
"records_changed": 2,
"weighted_changed": 0.005369204212911427,
"weighted_up": 0.0,
"weighted_down": 0.005369204212911427,
"largest_abs": 1.0,
"rows": [
{
"id": 1133795,
"base": 1.0,
"head": 0.0
},
{
"id": 134079,
"base": 1.0,
"head": 0.0
}
]
},
"income_tax": {
"entity": "tax_unit",
"base_sum": 2421866758136.5215,
"head_sum": 2421866758136.5215,
"change": 0.0,
"records_changed": 0,
"weighted_changed": 0.0,
"weighted_up": 0.0,
"weighted_down": 0.0,
"largest_abs": 0.0,
"rows": []
},
"state_income_tax": {
"entity": "tax_unit",
"base_sum": 548929924569.88617,
"head_sum": 548929924569.98816,
"change": 0.10198974609375,
"records_changed": 2,
"weighted_changed": 0.005369204212911427,
"weighted_up": 0.005369204212911427,
"weighted_down": 0.0,
"largest_abs": 19.0,
"rows": [
{
"id": 1133795,
"base": -19.0,
"head": 0.0
},
{
"id": 134079,
"base": -19.0,
"head": 0.0
}
]
}
}Generated by Claude Code |
|
US + core hub merge audit, us#10646 at
|
What and why
Colorado CCAP crashed for January to September 2015.
CountryTaxBenefitSystembackdates every parameter to 2015-01-01 (FIRST_MODELED_YEAR,policyengine_us/tools/parameters.py) and no further. CCAP reads its parameters at the October 1 before each month, so January to September 2015 read them at 2014-10-01, where the CCAP parameters (first dated 2022-10-01) have no value. Any 2015 Colorado household reaches CCAP throughco_income_tax→co_cdcc→cdcc→childcare_expenses, whatevergov.simulation.branch_to_determine_itemizationsays. With the flag on, the package default, it also gets there throughtax_unit_itemizes→tax_liability_if_itemizing. Either way it raisedCI didn't catch it because no test reached Colorado CCAP for January to September 2015.
A shared
co_ccap_parameter_instant(period)(policyengine_us/variables/gov/states/co/ccap/co_ccap_parameter_instant.py) fixes it. It keeps the October rule, but a month whose October falls beforeFIRST_MODELED_YEARreads atFIRST_MODELED_YEAR-01-01instead. All eight CCAP formulas that computed the October instant now call it, and the helper backdates nothing further. At 2015-01-01 the CCAP parameters hold the values the model already applies from October 2015, andgov.hhs.smistill holds HHS's FY 2015 values, as at 2014-10-01.The sales tax refund's age test let in filers who turned 18 during the year. Under C.R.S. 39-22-2003(1)(a)(II), a person qualifies who "is at least eighteen years of age as of December 31 of the taxable year preceding such taxable year", and on its 2015 Form 104 Colorado states the test as "born before 1997".
co_sales_tax_refund_person_eligiblecompared the year'sagewith 18. It now testsbirth_year < year - age_threshold, which equalsage >= 19on the model'sage, sincebirth_yearsubtractsagefrom the period's year. The paths in (1)(a)(I), Colorado tax or withheld wages, stay age-independent. The 2022 age 18 cases now expect false, and new age 19 and 2015 cases hold the boundary.The model doesn't test full-year Colorado domicile, which (1)(a)(I) and (II) both require, so part-year residents qualify as if they lived in Colorado all year.
Test plan
gov.hhs.smirecords them. The other cases guard against regressions. The CO tax case sets the branching flag on and a newco_cdcc.yamlcase sets it off, so neither depends on the package default.ruff format/ruff checkon the changed fileschangelog.d/impact: we estimate that two Colorado tax units in the default dataset lose a $19 sales tax refund in 2025, each a single filer aged 18. Each one's Colorado income tax rises $19 and its household net income falls $19. Together they weigh 0.005, so the weighted totals move $0.10. Federal income tax and benefits don't change. PE-US pays no Colorado sales tax refund for 2026, so we measured 2025 (pair).
axiom: us-co:statutes/39/39-22-2003 matches (rulespec-us
fe62a527aftests age as of the preceding December 31 against 18) | CCAP read date: n/a (no provision's reading changes)🤖 Generated with Claude Code