Bind Historic Table 2 and district-file business, partnership and rental 'net income (less loss)' rows to signed model concepts

[MaxGhenis]

Fixes #1228. Chronicle side: PolicyEngine/chronicle#346.

This changes what 312 national_state targets bind, so it waits on Max's ruling (d1432) before merge.

Summary

Historic Table 2 (HT2) and the IRS congressional-district file publish business (N/A00900), partnership/S-corp (N/A26270) and rental/royalty (N/A25870) income as signed "net income (less loss)" columns: the amount nets losses against income, and the count is every return with an entry. The target compiler bound them to one-sided model concepts:

• business and partnership took the positive part and counted units with a positive total;
• rental kept the sign but counted positive totals.

Two more gaps sat on the same rows:

• business read self_employment_income, which in this frame excludes SSTB Schedule C income;
• rental added Form 4835 farm rent, which Schedule E line 26 leaves out.

On the certified Route A release, these rows drive the calibrated Schedule C net to $821B, against IRS ~$460–472B, and the rental net to 2.6× its target.

This PR binds each column to the signed tax-unit total of its line and counts nonzero totals. A re-solve of Route A from its own checkpoint shows the effect:

• Schedule C net: $507B;
• partnership nonzero returns: 9.52M (HT2 9.55M);
• rental net: $98B (target $96B);
• every other target family unchanged, and ESS −0.6%.

What each IRS column counts

The 2022/2023 HT2 and 2022 district-file doc guides define:

Measure ids	Column	Definition	Line
schedule_c_income_*	N/A00900	business or professional net income (less loss)	Schedule 1 line 3
partnership_scorp_income_*	N/A26270	partnership/S-corp net income (less loss)	Schedule E line 32
rental_royalty_income_*	N/A25870	total rental and royalty net income (less loss)	Schedule E line 26

Identities against Table 1.4's split "net income" / "net loss" columns (feed-pinned workbooks):

• Schedule C. HT2 N00900 equals T1.4 NI + NL returns within 0.7% in each of TY2020–TY2023 (TY2022: 30,354,680 vs 30,356,401). Profit returns are only 70–73% of it. A00900 equals NI − NL within 1.9–3.4%.
• Partnership/S-corp. In TY2020, the last year with combined columns, N26270 (9,011,930) equals combined NI + NL returns (9,001,513) within 0.12%; 65.5% are net-income returns. A26270 equals the summed nets within 0.8% in every year (0.2–0.35% in TY2021–TY2023).
• Rental. Publication 1304 defines Table 1.4's "Total rental and royalty net income or loss" as Schedule E "line 26 + lines 39 and 40" (REMIC and farm rental); the doc guides tie A25870 to line 26 alone. A25870 is 0.919 of the Table 1.4 total net in TY2022 and TY2023, and 0.983 / 0.992 once farm rental is removed.
• The count convention is printed. Publication 1304's summary table gives each "(less loss)" line's returns as its net-income plus net-loss returns: TY2020 business 27,727K = 19,977K + 7,750K; partnership/S-corp 9,002K = 5,902K + 3,100K.

What changes

• fiscal_targets.py
  • SOI_NET_INCOME_LESS_LOSS_VARIABLES holds three signed concepts:
    • business_net_income_less_loss → total_self_employment_income (SE + SSTB SE);
    • partnership_and_s_corp_net_income_less_loss → tax_unit_partnership_s_corp_income;
    • rent_and_royalty_net_income_less_loss → rental_income.
  • Caveat: these are the frame's closest columns, not exact ones. The PUF donor builds partnership_s_corp_income from the nonpassive fields only ((E26190 − E26180) + (E25980 − E25960), not the line-32 total E26270), so the frame starts at $315B net against a $1,158B target and calibration triples those carriers, as it already did under the release binding (US frame partnership/S-corp income is nonpassive-only: the donor builds it from E25980-E25960 + E26190-E26180, not line 32 (E26270) #1239).
  • The six measure ids bind them, and each spec carries soi_source_concept (its return line).
  • Only HT2 and the district file are reviewed to publish these ids. The same id from any other family raises UnregisteredSoiNetIncomeLessLossFamilyError, because Table 1.4 splits these lines into one-sided columns. Capital-gains rebase control is picked by feed order and crosses IRS concepts (HT2 N01000 vs Table 1.4 col 37) #1035 is the capital-gains case of one id naming two IRS measures.
  • The one-sided legs (business_net_profits / _losses, partnership_and_s_corp_income / _losses, rent_and_royalty_net_income / _losses) stay reserved for Table 1.4's columns (chronicle#346). No pinned fact uses them. Two changed so a future leg binds correctly:
    • the business legs now read total_self_employment_income;
    • rent_and_royalty_net_income is now a positive part like every other income leg (it keeps farm rent, as Table 1.4's total does).
• tools/build_us_fiscal_refresh_release.py: _signed_component keeps the sign for the registered concepts, and _soi_component_row counts a nonzero total for them.
• variable_labels.py: labels for the three concepts.
• Docs: docs/us-soi-net-income-less-loss-concepts.md.
• Experiment: experiments/1228-net-income-less-loss/.

Effect on the pinned feed

consumer_facts_us_51aa40f, compiled at 2024 with aging, the packaged CD crosswalk and Medicaid substitution:

• 3,294 specs rebind. 312 are the HT2 rows on national_state (6 ids × 51 states + US). The other 2,982 sit on full only: 2,928 district-file rows (each id's US, 51 state and 436 district rows) and 54 HT2 rows of the nine single-district states, which full classes as district rows.
• No target value changes, and no spec enters or leaves either surface.
• national_state moves e1f9906bf0e0 → 38c1db8880da (6,102 specs); full moves f5bd921d6b21 → 9f6c65fe8734 (33,250).
• Aging is keyed on source_measure_id, so it is unchanged.
• Loss weights are unchanged: on national_state each row is its own concept group and no value changes. Recomputed and checked equal.

Effect on the Route A calibration (re-solve from the release's own checkpoint)

experiments/1228-net-income-less-loss/:

• 5,694 targets × 352,932 households, with the release's solver settings.
• Five arms on Modal; only the 312 rows differ.
• Checks:
  • the CSR reproduces every recorded initial and published estimate exactly;
  • the release-concept rows rebuild from person inputs to 5.4e-8;
  • the release arm replays to loss 0.028420 (recorded 0.028426), with weight correlation 0.9999.

2024 national total	IRS	Release binding	Concept-exact (this PR)	Rebase onto T1.4 net income (#1036-style)	Drop 260 rows
Schedule C net (incl. SSTB)	$460B (HT2 $472B)	$821B	$507B	$613B	$1,321B
Schedule C profits	$609B	$837B	$532B	$628B	$1,333B
Schedule C profit units	21.97M	30.64M	28.08M	21.89M	19.03M
Schedule C units with an entry	30.35M	31.29M	29.37M	22.49M	19.55M
SSTB Schedule C (untargeted before)	–	$348B	$96B	$103B	$320B
Partnership/S-corp net	$1,158B	$1,021B	$1,120B	$1,119B	$811B
Partnership/S-corp profit units	≤ 7.34M	9.48M	7.26M	7.27M	5.13M
Rental net, line 26	$95.9B	$248.6B	$98.4B	$118.2B	$95.1B
Rental units with an entry, line 26	9.40M	14.09M	9.32M	10.67M	14.25M
The 5,382 other targets within 10%		93.96%	93.96%	94.13%	94.43%
Kish ESS		77,794	77,306	81,036	84,834

IRS = Table 1.4 TY2022 aged ×1.12035 (the rows' own factor); counts are not aged.

Concept-exact state fit: 82–100% of each measure's state rows land within 10%; the release binding's state rows, read on these concepts, land 0–57%. No other target family's mean capped error rises by more than 0.0025.

The remaining gap is the frame, not the binding (#1238). The puf_tax_detail stage lists self_employment_income_before_lsr under nonnegative_outputs, although the donor's E00900 is signed. So the frame carries 0.39M weighted Schedule C loss units before calibration, against IRS 8.39M.

• Any binding of the Schedule C count fills it with profit units.
• Concept-exact gets the net and count right, with profits at 0.87× and profit units at 1.28×.
• The rebase gets profits right and the net 1.3× high.

Once the frame carries losses and Chronicle carries the legs, concept-exact rows plus national Table 1.4 profit/loss targets can get both. Both are proposed as follow-ups.

Invariants

• Leg conservation (Hypothesis, 300 examples), for every signed tax-unit vector and nonnegative weights:

  • the less-loss count = net-income-leg count + net-loss-leg count;
  • the less-loss amount = net-income-leg amount − net-loss-leg magnitude.

  For business these are the published identities N00900 = col 31 + col 33 and A00900 = col 32 − col 34, and for partnership/S-corp in TY2020's combined columns. For rental (Table 1.4's legs are lines 26 + 39 + 40) and TY2021+ partnership/S-corp (legs per entity type), the property checks the model's own decomposition of one column.

• End to end: test_net_income_less_loss_rows_materialize_signed_totals_and_nonzero_counts materializes all six concepts through the SOI slice on the fake engine. It covers a unit whose members net to a Schedule C loss, an SSTB-only unit and a farm-rent-only unit, hand-computed; it fails if business reads self_employment_income.

• Bounds and sign symmetry (same property): the less-loss count ≥ the net-income count, and the amount ≤ the net-income amount. Negating every value keeps the count and negates the amount.

• Family-only binding (Hypothesis, 200 examples): for any id, tax year and record-set shape, a fact binds its line's signed concept exactly when its family is HT2 or the district file, and refuses otherwise.

• Pinned feed (test_pinned_feed_net_income_less_loss_rows_bind_signed_concepts):

  • every business, partnership and rental row binds its line's concept;
  • only those rows bind a signed concept, and all come from HT2 or the district file;
  • no row binds a one-sided leg;
  • national_state carries exactly 312;
  • each measure's 51 state rows sum to at most its US row. This is a data pin; the US row includes territories and other areas.

• Determinism and parity: the release-concept rebuild (5.4e-8) and loss-weight reproduction (exact) in the experiment.

Tests

• New unit tests: sign rules and indicator rules for the three concepts; the registered signed set and its PE-US columns; compile of HT2 US/state and district rows for all six ids (values unchanged, negative district amounts stay signed); refusal from five unreviewed families; _soi_source_family shapes (the district family is congressional_district[_YYYY] only).
• New end-to-end test: the hand-computed materialization above.
• New properties: the two Hypothesis properties above.
• New pinned-feed tests: the invariant above; the re-pin of test_pinned_feed_national_state_surface_restores_the_fences.
• Mutation check: each mutation fails at least one new test (_reviews/.../out/mutations.txt):
  • positive-only count: 2 tests fail;
  • removed family guard: 6;
  • business without SSTB: 1;
  • positive-part amount: 4.
• Local:
  • the new and touched tests pass, including 3 pinned-feed tests on 51aa40f;
  • ruff check and ruff format --check are clean on touched files.

Review

The Codex review lanes were held by Subfleet's disk floor, so the independent review ran as an in-session Opus 5.5 adversarial reviewer (_reviews/soi-net-income-less-loss-20261012/review-1-opus.md).

• Round 1 (head 9efe451): REQUEST CHANGES, on documentation and test coverage only. It confirmed:

  • the IRS concepts, the PE-US columns and the single materialization path;
  • that the fail-closed guard cannot refuse a pinned fact;
  • the pinned rebinding with no value change;
  • the experiment harness.

  It asked for five changes: row attribution, the partnership lineage caveat, a qualified leg identity, the changelog, and an end-to-end test. All five are addressed in ec3df20, along with its nits. The review's own row attribution (312 TY2023 HT2 rows) was itself off; the dump shows 54 single-district-state HT2 rows.

• Round 2 (head ec3df20): APPROVE. All required changes and nits are resolved, and the review found no new defects.

Interactions

• Choose the capital-gains rebase control by concept and period, not feed order #1036 adds _soi_record_set_family with the same semantics as _soi_source_family here; whichever merges second should dedupe.
• Both PRs re-pin national_state in the same test; the second re-pins again.
• Rebase congressional-district capital gains onto Table 1.4 and drop mislabeled CD SALT and PTC columns on the full surface #1040 (district capital gains) touches different rows.

Not changed here (follow-ups)

• Signed Schedule C on the PUF channel, US frame clips PUF Schedule C losses to zero (puf_tax_detail nonnegative_outputs) #1238 (nonnegative_outputs). Without it, no binding can match both Schedule C profits and net.
• Line-32 partnership/S-corp income in the frame, US frame partnership/S-corp income is nonpassive-only: the donor builds it from E25980-E25960 + E26190-E26180, not line 32 (E26270) #1239 (the donor is nonpassive-only). This bears on Populate passive_partnership_s_corp_income (NIIT passive pass-through leg, pe-us#9306): SCF non-active-business shape, SOI/8960 level #722.
• National Table 1.4 profit/loss targets once IRS SOI Table 1.4: add business, rental, royalty, farm-rental, partnership and S-corp net income/loss legs; give HT2/district 'net income (less loss)' columns their own concept ids chronicle#346 lands.
• Rental frame level. Before calibration the frame's line-26 positive part is $261B, against Table 1.4's rent and royalty net-income legs of ~$164B aged.

axiom: n/a: calibration-target concept mapping (microsim data), no policy rule change

🤖 Generated with Claude Code