[MaxGhenis]

Rebased onto main after #1227 merged (2026-10-14, de90c6c7e). #1227's final version of the overlap-refusal test (crossing and identical rows) is carried into the rewritten test file.

Methodology principle for Max. No published source decides between the options below. The choice is the first instance of a general rule for stale SOI decompositions: when uprated levels conflict, rake the cells to both margins instead of shipping both. So it is queued as a principle decision rather than merged on the usual gates. The recommendation, option (c), is implemented here and ready to merge. Options (b) and (d) are each a few lines if preferred.

What is inconsistent after #1227

_uprate_cross_period_eitc_decompositions scales the TY2023 SOI Table 2.5 EITC-by-AGI-and-children rows to the TY2024 filing-season EITC by AGI:

• each bin takes its AGI group's active / source ratio, so every AGI group keeps its TY2023 child mix;
• each split's all-AGI total row takes the all-AGI ratio (×1.0418 amount, ×0.9753 returns), so the totals keep the TY2023 all-AGI child mix.

The AGI groups grew at different rates (amount ×0.85 at $5,000–$10,000 to ×1.73 at $50,000 and over; returns ×0.81 to ×1.38). So per split, the total row and the sum of its 28 bins disagree, and both levels sit on national_state and full:

Split	Amount: total / bins	Returns: total / bins
Childless	1.157	1.062
One child	1.018	1.001
Two children	0.975	0.961
Three or more	0.993	0.951

The options, on the pinned feed

These are release-equivalent compiles of consumer_facts_us_51aa40f.jsonl, from experiments/eitc-child-split-rake-20261012/results/.

	(a) status quo	(b) totals := Σ bins	(c) rake (this PR)	(d) drop totals
What moves	nothing	8 total rows, ×0.864–1.052	224 bins, ×0.857–1.226; totals keep their values	8 total rows removed
Total row = Σ its bins	no (0.951–1.157)	yes	yes	n/a
Childless share of EITC, amount / returns	3.71% / 27.73% (totals), 3.21% / 26.11% (bins)	3.21% / 26.11%	3.71% / 27.73%	3.21% / 26.11%
national_state	6,102 at 3fad14f85468	6,102 at a211cbec63ec	6,102 at 45adaa24b243	6,094 at 41a1fa2b013f
full	33,250 at c3aaea9c1e8c	33,250 at a83fc4bad02f	33,250 at 60a52e2f204d	33,242 at 469b8ed2c5db
Table 2.5 share of loss weight, national_state / full	1.555% / 0.605%	1.555% / 0.605%	1.548% / 0.603%	1.272% / 0.494%
Table 2.5 rows' weights vs (a)	—	×0.930–1.026	×0.926–1.101	×1.0003–1.0029
Largest weight move of any other row	—	8×10⁻⁶	1.5×10⁻⁴	2.9×10⁻³

• Totals. All four conserve the active EITC ($69.04B, 23.84M returns).
• Scope. None changes any target outside Table 2.5.
• (b) vs (d). They carry the same information, the bins' child mix. (b) adds weight on the bins' sums; (d) drops the totals' weight.

Why (c)

1. No TY2024 child mix is published (checked live 2026-10-12).
   • SOI Table 2.5 stops at TY2023: 24in25ic.xls returns 404.
   • SOI preliminary data stop at TY2022; the geographic data with EITC by children stop at TY2023.
   • The TY2024 filing-season file and IRS EITC Central give EITC by AGI or by state only.
2. Backtest on published Table 2.5, TY2013–2023. Each option uprates TY s to TY s+1, as the compiler does, and is scored against the published TY s+1 table. The active margins are either SOI's mid-November file (as production uses) or the true TY s+1 margins. There are seven year pairs without a structural change to the credit, scored in 11 runs (four pairs against both kinds of margins).
   • Keeping the all-AGI mix gave mean split-total errors of 2.9% (amount) and 2.1% (returns) on mid-November margins. The bins gave 5.4% and 5.0%.
   • On true margins it was 2.1% and 1.6%, against 3.0% and 2.7%.
   • The bins under-predicted childless EITC in all 11 runs (by 1.1–19.7%).
   • On split × AGI-group cells, (c) was as good or better on average.
   • The exceptions are TY2015→16, and TY2022→23 amount, when childless EITC really fell.
   • On individual bin rows it is roughly a wash: (c)'s sqrt-weighted capped error is 0.1299 against the bins' 0.1291 for amount, and 0.1204 against 0.1226 for returns, on mid-November margins. The case for (c) is national totals and consistency; no calibration solve here shows release quality improving.
3. Consistency with the rest of the surface. national_state already carries SOI Historic Table 2 TY2022 state EITC-by-children rows. Their national child mix is 0.55–0.63 pp TVD from (c) and 1.62–2.24 pp from the bins' mix in (b) and (d).
4. The mix (c) keeps is confirmed. SOI's full-population TY2023 state data (August 2026; not yet in the pinned feed) give childless 3.68% of the amount and 27.80% of returns. Table 2.5 TY2023 gives 3.71% and 27.73%. The bins project a further fall to 3.21% and 26.11% that no published source supports.

Full write-up: experiments/eitc-child-split-rake-20261012/README.md.

The change

• _rake_eitc_child_splits runs after the per-row uprating. For each decomposition (kind, source period, target period), it builds the child-split × AGI-group matrix of source bins (_eitc_rake_plan) and scales it (_rake_two_margins) to the splits' uprated total rows and the active AGI groups.

• _rake_two_margins finds the IPF solution: the unique positive scaling r_i·s_j·x_ij that meets both margins and keeps every cross-product ratio of the source.

  • It first checks whether that scaling exists (_require_scalable, the Menon 1968 / Brualdi 1968 condition, over subsets of the shorter side, at most 16). The subset sums are exact (fractions.Fraction). It refuses only when the margins are certainly infeasible: an excess beyond rounding, or an exact fill that other splits also reach. Margins within rounding of the boundary go to the solver.
  • It then takes damped Newton steps on the convex dual, in log shares so that no positive cell underflows or overflows. The largest column of each connected support block is pinned, and the solve is Jacobi-scaled, with Levenberg–Marquardt damping when rounding makes it singular.
  • Rows are normalized directly against the seed, in units of each row's largest cell.
  • It finishes with a few exact IPF sweeps. Plain IPF needed more than 10,000 sweeps on ill-conditioned inputs (review round 1); this converges in a few steps.
  • Rows hold to rounding and columns to 1e-12 relative.

• When it rakes. A decomposition is raked when:

  • every split has one all-AGI total row and leaf bins that tile AGI;
  • and the active AGI bins give the total rows and every bin an AGI group.

  Otherwise it keeps its per-row factors: uprated on the all-returns total, every row already has one factor, and with no total row there is nothing to reconcile to. A group with no EITC in either year takes a zero factor.

• Refusals (ValueError):

  • negative EITC;
  • margins that disagree in total;
  • an AGI group with active EITC but no source EITC in any split;
  • a split with no EITC in any active group;
  • margins no positive scaling can meet;
  • more than 16 splits and more than 16 groups;
  • a published row that is zero although a bin inside it is not, whether it spans a whole AGI group or sits inside one.

• Metadata.

  • Every raked row carries uprating_reconciliation: two_margin_rake.
  • Each bin also records uprating_factor (its effective factor), uprating_agi_group_factor (the factor the per-row uprating recorded) and uprating_child_split_adjustment. The adjustment is relative to the recorded factor, so the group factor times the adjustment equals the bin's factor.
  • Total rows keep their Count each EITC child split once in the Table 2.5 uprating source total #1227 value and factor. A subtotal row, if a feed ever has one, becomes the sum of its raked bins.

• Refactor. The per-row AGI metadata moved into _eitc_agi_uprating_metadata, with no change for Count each EITC child split once in the Table 2.5 uprating source total #1227's rows.

Invariants

These hold for every decomposition the rake applies to:

1. Each split's bins sum to its total row.
2. Each active AGI group's bins (all splits) sum to the group's active EITC.
3. Conservation: the total rows sum to the active EITC, and so do the bins.
4. The total rows keep Count each EITC child split once in the Table 2.5 uprating source total #1227's all-AGI factor, active / source total rows.
5. Every bin of one split in one AGI group takes one factor, so the source shape inside the cell is kept.
6. The source's cross-product ratios between splits and groups are preserved. This is what makes it IPF rather than some other reconciliation.
7. Uniform growth across groups makes the rake the identity.
8. _rake_two_margins succeeds exactly when a positive scaling exists.

Tests

• test_us_eitc_decomposition_uprating.py (41 tests).
  • Example tests:
    • Count each EITC child split once in the Table 2.5 uprating source total #1227's total-row doubling;
    • a 2×2 rake against the hand solution of a² + 2,900a − 800,000 = 0;
    • a subtotal row;
    • a group with no EITC in either year;
    • a group with active but no source EITC, which is refused;
    • strict slack below rounding, which is raked (review round 2);
    • a zero subtotal over positive bins, which is refused (review round 2);
    • rounded equality with strict slack, which is raked (review round 3);
    • a zero subtotal inside a group, which is refused (review round 3);
    • an ill-conditioned decomposition;
    • the adjustment against a rounded subtotal;
    • the no-total-row and all-returns cases, which are left unraked;
    • Count each EITC child split once in the Table 2.5 uprating source total #1227's overlap refusal;
    • eight _rake_two_margins refusals;
    • eleven feasible ill-conditioned inputs, including all three rounds' counterexamples (a singular Newton system, a seed whose sum overflows, rounded-equal subset sums, a cell that underflowed in normalization).
  • Hypothesis properties (150 examples each):
    • invariants 1–6 and invariant 7, through compile_us_fiscal_target_registry;
    • invariant 8, against an independent scipy.optimize.linprog oracle (the largest t such that some matrix on the seed's support, every positive cell ≥ t, meets both margins). It also checks margins and cross-product ratios;
    • positive seeds spanning ten orders of magnitude.
  • A differential test of _rake_two_margins against the closed-form 2×2 solution (bisection on the cross-product ratio).
• test_us_fiscal_targets.py.
  • New test_pinned_feed_eitc_child_split_bins_sum_to_their_total_rows: per split Σ bins = total row; per active group Σ bins = the group's EITC; adjustments within 0.85–1.23.
  • Re-pinned national_state 3fad14f85468 → 45adaa24b243.
• Local runs. The pinned-feed tests skip in CI because the feed is absent, so they were run locally:
  • test_us_eitc_decomposition_uprating.py: 41 passed;
  • test_us_fiscal_targets.py and test_us_fiscal_refresh_builder.py with -k "eitc or pinned_feed or decomposition": 25 passed;
  • test_us_spine_blindness.py: 498 passed. Round 1's CI failed this static check on lambda row: row[1:].

Separate findings, not changed here

• An IRS cell misplacement in TY2023 Table 2.5 (comment above, 2026-10-14). The three-or-more $25,000–$30,000 cell holds about 311,000 returns that belong at $30,000–$35,000. It distorts the per-row factors of those two AGI groups under the status quo and under every option here, and the rake carries it into the raked bins just as the status quo does. Its repair or guard is a separate task.

• microcosm#1274 (open) adds CI tests on the real Table 2.5 rows. One of its Hypothesis properties pins the status-quo bin factors for any filing-season input. Whichever lands second must update it to the rake's invariants. Note that for arbitrary inputs, the rake can be infeasible (active EITC where the childless split has no bins), which the compile now refuses.

• Mid-November totals vs complete year. The mid-November filing-season EITC totals mostly run 3–6% below complete-year Table 2.5 (TY2017–2023: returns 0.939–0.971; amount 0.957–0.973, except TY2018 at 1.012). They are the TY2024 EITC target levels and two release critical-fit gates. All four options conserve them, so this is a separate question, proposed as a follow-up.

• CD file vintage. 22incd.csv is SOI TY2022 data that Chronicle stamps TY2023, already tracked as CD record sets stamp 22incd.csv as ty2023 — a tax-year vintage error; add Table 1.4 taxable-interest facts (populace#489) chronicle#117.

• TY2023 state data. SOI's TY2023 state data (23in55cmcsv.csv, with EITC by children) are not yet in the pinned feed; Add IRS SOI Historic Table 2 TY2023 national, state broad and state EITC facts chronicle#295 is adding them. Their child mix sits 0.19–0.24 pp TVD from (c), so they would widen the gap to (b) and (d).

Review

• Round 1 (GPT-6.1 Sol, subfleet run --task review --tier hard, head a2a529d): REQUEST_CHANGES.
  • Two majors:
    • a group with no EITC in either year blocked the rake;
    • the IPF iteration cap rejected feasible, ill-conditioned margins.
  • Three minors:
    • the adjustment against rounded subtotals;
    • option (b)'s provenance;
    • qualifying the README claims.
  • It favoured (c) once those were fixed. All five are addressed above.
• Round 2 (head 9d94deb): REQUEST_CHANGES.
  • All five round-1 findings were fixed or partly fixed.
  • One major: the feasibility tolerance refused strict slack below rounding as an exact fill.
  • Three minors: extreme magnitudes (a singular Newton system, an overflowing seed sum); no exact check above 16 splits; a zero subtotal over positive bins.
  • All addressed above.
• Round 3 (head 776ea71): REQUEST_CHANGES.
  • The 16-row bound was fixed.
  • One major: float subset sums rounded strict slack to an exact fill. Exact Fraction sums now.
  • Two minors: a 1e-308 cell underflowed in normalization (log shares now, and direct row normalization), and a zero subtotal inside a group kept a stale factor (refused now).
• Round 4: pending.

axiom: n/a: target-compilation methodology for calibration targets (no policy encoding)

🤖 Generated with Claude Code

[MaxGhenis]

A source anomaly that bears on this rake.

IRS's own TY2023 Table 2.5 (23in25ic.xls, sha256 c6a58f63…, byte-identical to the live file on 2026-10-14) misplaces one child-split cell pair. The three-or-more $25k–$30k cell holds 635,160 returns and $3.81B, and the $30k–$35k cell is 0.

• The evidence. In every other row, the all-returns column J equals Z+AP+BF+BV. In rows 31–33 the gap is −1,187, −310,212 and +311,398.
  • All-returns minus the other splits implies three-plus counts of about 324,948 (25k–30k) and 311,398 (30k–35k).
  • Each split's bins still add to its total row. TY2022 has no such anomaly.
• The uprating effect. The 25k–30k group's source is inflated, so every split's 25k–30k bin takes 0.832 (returns) instead of about 0.966. The 30k–40k group takes 1.030 instead of about 0.948.
• In the release. In populace-us-2024-spm-20260915, calibration cut three-plus EITC returns at 30k–35k from 174,894 to 16,231.
• Why it matters here. Raking the child × AGI matrix to the total rows and AGI groups would carry the misplaced cell into the raked bins. The repair, or a guard for it, is being split into its own task. It may need Chronicle to extract the all-returns AGI rows (B/J/K, rows 10–37) so microcosm can check child splits against all-returns per row.

Context: microcosm#1274, which adds CI tests on the real TY2023 rows after #1227.

🤖 Generated with Claude Code