Rebase congressional-district capital gains onto Table 1.4 and drop mislabeled CD SALT and PTC columns on the full surface

[MaxGhenis]

Fixes #1038 (capital gains). Stacked on #1036 and #1224.

• The SALT/PTC column exclusions moved to Drop congressional-district SALT and PTC facts read from the wrong IRS column #1224, a bug fix that lands on its own. Until it merges, this branch carries the same commits, byte-identical.
• The base is main so CI runs, so until Choose the capital-gains rebase control by concept and period, not feed order #1036 and Drop congressional-district SALT and PTC facts read from the wrong IRS column #1224 merge the diff also shows their changes. This PR's own change is the capital-gains share rebase.

Waiting on Max's ruling (d539), together with #1036 (d468). It changes published full-surface target values, and choosing shares over exclusion is a methodology call.

Summary

On the default --target-surface full, the congressional-district file (22incd.csv) compiles its N01000/A01000 rows onto capital_gains_gross. Those columns are "net capital gain (less loss)" at Form 1040 line 7: gain, capped loss and distributions-only returns. The model measures something else: the positive part of Schedule D, whose IRS counterpart is Table 1.4 col 37/38 (#1035).

On the pinned feed (51aa40f), the district file's states and its districts each summed to:

• 29,845,710 returns, 2.41x the national Table 1.4 target of 12,392,020 on the same surface;
• $1,522.2B, 1.198x the target's $1,270.9B.

This PR makes those rows shares of the district file's own US row, scaled to the Table 1.4 control, exactly as #1036 does for Historic Table 2:

• the district file's US row retires;
• every share row declares soi_source_concept, soi_control_concept and its denominator, soi_share_total_source_record_id;
• CD children keep scaling to their CD state parent.

Option (a) shares, or (b) exclusion

Quantified on the sha-matched IRS files and the compile dumps. Scripts and JSON: ~/PolicyEngine/_reviews/cd-cg-concept-20260927/lane-b/. Critic re-derivations: critic.md.

The share assumption is #1036's. A district's share of line-7 returns stands in for its share of Schedule D gain returns. AGI composition is the measurable source of error. Reweighting each of the 428 numbered districts' N01000 by AGI class, with Table 1.4's per-class gain share, moves its share by:

• counts: median 2.1%, 95th percentile 4.7%, max CA-18 +9.7%;
• amounts: median 0.5%, 95th percentile 2.4%.

AGI composition is one measurable source of error, not a bound on the total: non-AGI variation in the gain/loss/distributions mix cannot be measured from IRS geography and could add to or offset it.

Exclusion would discard more.

• District capital-gains incidence runs from 8.3% to 31.5% of returns (5th to 95th percentile), against 13.7% to 23.4% across states.
• Within a state, the district share left after AGI has log SD 0.22 (counts) and 0.20-0.22 (amounts). That is 9-20x the share proxy's AGI error.
• Without these rows, district capital gains would be constrained only by the 9 HT2 at-large copies.

Agreement with the HT2 state rows already on full. Both files' state rows stay on full; 45 measures pair two state targets. After dividing out each pair's common coverage-and-aging offset:

• capital-gains return counts differ by a median 0.4% and at most 1.4%, among the tightest pairs (ninth of 45 by median gap);
• capital-gains amounts differ by a median 1.8%, with the widest 95th percentile of the 45 pairs: Montana −28%, West Virginia −18%, Mississippi −17%.

The amount gap comes from item suppression in the district file's A01000; Montana's state total is 32.5% below HT2's. Other pairs have worse single states, for example qualified dividends in Hawaii at −58%.

Split of the control. The district file has no other-area rows, so its US row equals the sum of its states, and its rebased states and districts each sum to the control. HT2's US row includes other areas, so its rebased states sum to 99.2% (returns) and 98.0% (amounts).

• So in aggregate the district-file capital-gains targets sit above the matching HT2 shares by about 0.8% (returns) and 2.1% (amounts). State by state they scatter around that; Montana's amount is 0.73x HT2's.
• This is accepted. The model has households in the 50 states and DC only, and the national row on the same surface already asks them for the whole control.
• The alternative is to scale the district file by HT2's 51-state share. It would mix the two families' totals in one factor.

Interplay

The congressional_district_children_scale_to_state_parent profile rule.

• One factor per family, quantity and period scales the district file's states and districts alike, and the file's districts sum exactly to its states.
• So all 872 district children still reconcile with factor 1. The rule never used the US row.

The ACS-local state_cd surface.

• It already takes district capital gains as within-state shares of HT2, which a national factor does not move.
• Its capital-gains rebase factors scale by the reciprocal of this PR's factor. That leaves every factor-band verdict and its contract counts unchanged (test_pinned_feed_state_cd_surface_matches_its_contract passes).

#1030 (source-vintage correction).

• A share divides the district rows by the district US row of the same restamped package, so the stamp cancels. The value lands at the control's period (2023) and ages from there.
• test_congressional_district_capital_gains_ignore_the_package_stamp compiles and ages the package stamped ty2023 and truthfully at 2022 and asserts identical values. Its facts carry the pinned 22incd.csv digest and raw key.
• Read restamped Chronicle facts at their data year before aging #1030 as written refuses every restamped fact that was rebased, so Read restamped Chronicle facts at their data year before aging #1030 plus this PR refuses every compile.
• Read restamped Chronicle facts at their data year before aging #1030 has been adopted, rebased onto main (head 03ceca6ad), and carries the reviewed narrowing (lane-a/source_vintage_share_narrowing.patch):
  • admit a rebased restamp only when its soi_share_total_source_record_id is a correction of the same package and stamp;
  • move source_period and uprating_from_period to the data year;
  • never move uprating_to_period. Starting the aging at 2022 would apply the Table 1.4 2022→2023 fall twice (×0.761).
• Merge order. Whichever of the two merges second runs CI on a merge ref containing the first, and the stamp test fails until the narrowing is in.

What changes (fiscal_targets.py)

• _rebase_stale_soi_capital_gains_distributions rebases every row of a share family: a registered family whose concept is not the model's, today historic_table_2 and congressional_district (_soi_capital_gains_share_family).
• Each row is divided by its own family's national all-AGI row at its own period, via _soi_capital_gains_share_national_totals, which is keyed by family and refuses conflicting duplicates. A row with no control, or no national row of its own family, is dropped, never shipped as a level. That is the established rule (test_stale_soi_capital_gains_without_source_total_is_dropped).
• Every share row names its denominator in soi_share_total_source_record_id.

Effect on the pinned feed

Release compile on 51aa40f: target period 2024, aging, packaged CD crosswalk, Medicaid substitution.

	#1036 on main	this PR (with #1224)
District-file capital-gains returns: states / districts (each sum)	29,845,710 (2.41x national)	12,392,020
District-file capital-gains amounts: states / districts	$1,522.19B (1.198x)	$1,270.86B
California returns / amount	3,808,930 / $214.08B	1,581,478 / $178.73B
Compiled targets (= full)	33,250 (f09571684885)	31,784 (6e1ac91262bc)
national_state	6,102 (882f4764a248)	6,102 (84504a7afd57)
District-file families past their national target on full	3 of 38 compared	0 of 35

• Values: 974 district-file capital-gains values move, by 0.415202720927 (returns) and 0.834893818418 (amounts).
• Removed: the 2 district-file capital-gains US rows, plus Drop congressional-district SALT and PTC facts read from the wrong IRS column #1224's 1,464.
• Metadata only: 120 HT2 capital-gains rows (102 on national_state) gain soi_share_total_source_record_id. That is the whole national_state change: no national_state value or membership moves.
• Hierarchy: all 872 district children reconcile with factor 1.
• ACS local: state and state_cd keep their counts and band verdicts. state_cd's capital-gains amount rebase factors move from 0.77–1.00 to 0.93–1.20 (median 0.98).

Invariants

• Shares conserve the control. Every district-file capital-gains row is raw × control / district US. The states, and each state's districts, sum to control × (Σ states / district US), never past the control. HT2 rows drawn beside them divide by, and name, their own US row. No district-file capital-gains row ships at the country level. This is a Hypothesis property over random packages, HT2 rows, controls and feed orders.
• Stamp-invariance. A differential test: stamped ty2023 or truthfully at 2022, the aged values are identical, and amounts age from the control's 2023 period.
• Denominators. Conflicting national totals refuse. Equal totals name the smaller id in either order.
• District-file families stay inside their national targets on full. Pinned, two-sided: every compared family keeps its state and district rows and sums to within (0.85, 1] of its national target. Before this change, the capital-gains returns family was at 2.41x and the amount family at 1.198x.

Tests

These runs were local, on the pinned 51aa40f feed, run sequentially with peak RSS 3.6 GB. CI has no feed and skips the pinned-feed tests.

• engine_free/us/test_us_fiscal_targets.py at 21a638dec: 210 passed, none skipped. That includes all six pinned-feed tests:
  • the receipt;
  • the fences (31,784 / 6e1ac91262bc / 84504a7afd57);
  • Choose the capital-gains rebase control by concept and period, not feed order #1036's HT2 control and 63-family state invariant;
  • the new district-file control test;
  • the two-sided district-file family invariant.
• Neighbouring suites at 21a638dec: the ACS-local release tool, the state_cd sparse surface (its contract included), source coverage, release target parity, the Chronicle feed and the fiscal refresh builder: 495 passed, none skipped.
• Follow-up commit 90ad595a6 (the ACS-local count pins and the state_cd doc paragraph) was verified with the same contract test on Drop congressional-district SALT and PTC facts read from the wrong IRS column #1224.
• Static checks: tools/ci_test_plan.py verify passes; ruff check is clean.

New tests:

• district-file rows are shares, while HT2 rows keep their own denominator;
• with no control, or no own-family US row, the row is dropped;
• conflicting totals refuse, and equal totals name the smaller id in either order;
• share-family classification;
• the Hypothesis share-conservation property, with HT2 rows and feed order in the draw;
• the stamp-invariance differential;
• the two pinned-feed tests above.

Review

The 2026-09-27 independent review (review-1040.md) asked for changes. Each finding is addressed below. A GPT-6.1 Sol re-review at 90ad595a6 (review-1040b.md) returned APPROVE WITH NITS. It found no P0–P2 in this PR's own change and confirmed that each prior answer holds, including that the stamp test is a real #1030 tripwire. Its four P3 nits are addressed at d3a625350:

• the return counts agree "among the tightest" pairs (ninth of 45 by median gap);
• the CD-over-HT2 tilt holds in aggregate, not state by state (Montana's amount is 0.73x);
• AGI composition is "one measurable source of error";
• what the generic invariant catches is stated exactly;
• the parity-manifest note now names the capital-gains shares, the retired US row and the column exclusions.

d3a625350 also carries #1224's review fix byte-identical (the column check runs before the district crosswalk merges source districts).

• P1-1 (Read restamped Chronicle facts at their data year before aging #1030 merge safety).
  • Read restamped Chronicle facts at their data year before aging #1030 is 567 commits behind main and its owner session is gone. An adoption chip is filed: rebase it onto main and carry lane A's reviewed narrowing. The narrowing admits a rebased restamp only when its soi_share_total_source_record_id is a correction of the same package and stamp, moves source_period/uprating_from_period to the data year, and never moves uprating_to_period. With it, all 21 of Read restamped Chronicle facts at their data year before aging #1030's tests pass.
  • The merge order is stated here and in docs/us-soi-capital-gains-concepts.md. Whichever PR merges second runs CI on a merge ref containing the first, and test_congressional_district_capital_gains_ignore_the_package_stamp fails until the narrowing is in.
  • Read restamped Chronicle facts at their data year before aging #1030 cannot merge without being rebased onto main, which reruns that CI, so a stale green run cannot carry it in.
• P2-1 (interest paid). Not added to the register.
  • N19300/A19300 (line 8a mortgage interest) is a component of the line-10 deduction the measure id means, not another concept. Its remedy is the same share-or-drop question as capital gains, and the rows also feed state_cd's itemized bridge.
  • The register comment and docs say so; the "56 compared" claim is corrected to 50 twins; a chip is filed.
• P2-2 (ACS-local). Disclosed. The ACS-local counts are in docs/us-acs-local-soi-target-surface.md: totals 607 → 120, full 31,321 → 29,860, state/state_cd unchanged. The column exclusion is now its own PR, linked above.
• P2-3 (tilt). Stated plainly in the doc and here: each CD capital-gains target sits above the matching HT2 share by 0.8% (returns) and 2.1% (amounts) in aggregate. It is accepted with the reason given; the alternative is named.
• P2-4 (silent drop). The established rule is kept: a share row without its own family's national row is dropped, never divided by another family's total. test_stale_soi_capital_gains_without_source_total_is_dropped on main codifies it, and the interest twin does the same. The pinned invariant is now two-sided: every compared CD family must keep its state and district rows and sit in (0.85, 1] of its national target. So a family that vanishes, or loses a whole geography level, fails on the pinned feed; a family losing a few states can still pass that generic check, and for capital gains the dedicated pinned test (51 states and 436 districts per measure, exact control sums) catches it.
• P2-5 (fail-open column). Fixed. A listed congressional-district fact with no column refuses (MissingSoiSourceColumnError).
• P2-6 (qualified-dividends comparison). Reworded on the demeaned comparison. Capital-gains returns agree among the tightest pairs (ninth of 45 by median gap); amounts are the loosest pair at the 95th percentile. The worst states are named.
• P3-1: the worst states are named instead of a p95.
• P3-2: accepted as fail-closed.
• P3-3: a comment notes that CD boundary vintages share one family.
• P3-4: a comment notes that a corrected PTC package compiles without a further gate, by design.
• P3-5: the key stays on every share row. On national_state that is a metadata-only change on the 102 HT2 rows.
• P3-6: the parity-manifest note ("congressional-district rows inform shape while national and state actuals retain control ownership") is now literally true for capital gains, so it is unchanged.
• P3-7: the Hypothesis property now draws HT2 rows alongside and checks that each family divides by, and names, its own US row.

axiom: n/a: microsim target compilation, no policy rule changes.

🤖 Generated with Claude Code

[MaxGhenis]

Heads-up from #1244, which makes the SOI congressional-district guards vintage-free. Two things to adjust when this rebases after it, or that #1244 would need if this lands first:

1. Duplicate helper. Match the SOI congressional-district file by family, at any vintage #1244 adds _soi_record_set_family (identical to the copy here) next to _is_soi_congressional_district_record_set, so the second to land gets ruff F811. Keep one copy.
2. PTC tests. This PR deletes _is_soi_cd_premium_tax_credit_amount_conflict. Match the SOI congressional-district file by family, at any vintage #1244's new tests/engine_free/us/test_us_soi_congressional_district_vintage.py calls it:
   • _ptc_conflict in test_district_record_set_is_matched_by_family;
   • the property test;
   • the pinned-feed fact test.
     Its test_district_ptc_amount_never_compiles also builds district PTC facts with no layout.source_column_id, which the column fence here would refuse with MissingSoiSourceColumnError. Those assertions should move to the column-exclusion mechanism, for example asserting the A85530 column is excluded at any district vintage.

The _is_soi_cd_premium_tax_credit_amount_conflict deletion merges without a conflict against #1244, since its changed lines are two unchanged lines away.

[MaxGhenis]

#1244 has merged (8fd39a21b). The heads-up above now applies on this PR's next rebase onto main.