Pool each state's congressional-district rows of one concept into one national loss budget

[MaxGhenis]

Summary

On the national full surface, the #195 per-concept loss budget never pooled a congressional-district (CD) row. This PR makes it pool them again and adds a guard so a new per-row key cannot silently disable it a second time.

• The bug. The district concept key (_ledger_concept_budget_key in us_runtime/target_loss_weights.py) reads every metadata key outside US_FISCAL_TARGET_CONCEPT_METADATA_EXCLUSIONS.
  • On 2026-09-05, efa6052 ("Normalize calibration hierarchy ownership") made the ledger compiler stamp three per-row labels on every compiled row:
    • ledger_fact_label
    • ledger_layout_groupby_value_label
    • diagnostic_target_label
  • The first two name the district ("WV congressional district 1"). The third repeats the fact label whenever the build period equals the fact period.
  • Budget congressional district target weights by concept #195 had set the exclusions in June, before these labels existed, so none of them was excluded. Every compiled CD row has been its own concept group since.
  • On the pinned feed that is 24,340 singleton groups, and the budget did nothing.
• It was not intended.
  • Budget congressional district target weights by concept #195's body says to "split CD rows within each parent state/concept so adding more districts does not mechanically increase that concept's objective share".
  • The module docstring states the same aim.
  • Weight the ACS local release's calibration targets with the national release's loss formula, from one shared implementation #1104 found the gap, fixed it for the ACS local mapping only, and measured the national gap in experiments/us-acs-local-target-loss-weights-20261004/ ("A finding for the national release (not changed here)"). Lane C of the Congressional-district capital-gains, SALT and PTC targets on the full surface cross IRS concepts #1038 code map found it again on 2026-09-27.

Changes

• LEDGER_PER_ROW_LABEL_METADATA names the three labels, and the national exclusions now include them.

  • The ACS local key inherits them, so its own extra set shrinks to state_cd_cd_file_value.
  • Its effective exclusions are therefore the old set plus diagnostic_target_label.

• validate_district_concept_groups now guards the national mapping, renamed us_fiscal_ledger_metadata.v2. The ACS local guard calls it too. It refuses:

  • district rows of one concept in one state (district_concept_identity) that land in more than one group; the error names the differing keys;
  • a district group that spans states;
  • a district row without state_fips.

  A future per-row key now fails the weights loudly.

• Spec bundle: us/spec/calibration.yaml is regenerated with tools/generate_us_bundle_from_constants.py. Its exclusion list gains the three labels. The spec-engine pins move with it:

  • authored and configuration field counts (+3);
  • the calibration pointer claim (312 → 315 pointers);
  • the pointer-inventory digest;
  • the legacy-behaviour mode counts;
  • the US spec_sha256 (fbf25dba… → fe6d751d…);
  • docs/evidence/spec-engine/us-f0-coverage.json.

  Every new value was observed from the code, not transcribed.

• The formula id US_FISCAL_TARGET_LOSS_WEIGHTING is unchanged. On the full surface the loss-vector digest in target_loss_basis moves, so the exact-k incumbent gate refuses to compare old-weighted runs with new ones.

Effect on loss weights (pinned feed 965a29ac…, --target-surface full, 33,249 targets)

These figures come from experiments/us-national-cd-concept-budget-20261012/ (script, results.json, README).

District rows (24,340)	Groups	Singletons	Largest
Before	24,340	24,340	1
After	3,125	665	52 (CA)

After the fix, each group is exactly one concept identity. The 665 singletons are the seven single-district states (AK, DE, DC, ND, SD, VT, WY), 95 concepts each.

Share of the loss	Targets	Equal weights	Before	After
District	24,340	73.2%	45.6%	11.0%
State	8,409	25.3%	42.1%	68.8%
Country	500	1.5%	12.3%	20.2%

• Non-district targets: every weight is multiplied by one factor per basis, ×1.6373 for amounts and ×1.6332 for counts.

• District targets: the median weight is 0.157× what it was. The minimum, 0.033×, is in California's 52-district groups. Single-district states change by the basis factor only.

• By family: irs_soi goes from 90.5% to 84.5%. Every other family rises by about 1.63×.

• Unchanged:

  • Route A and any national_state run. They have no district rows, and the recorded weights and loss-basis hash 206ada09… still reproduce bit for bit.
  • The ACS local weights. acs_local_unchanged.py checks this against the pre-change module, bit for bit, on two kinds of surface:
    • the 09-23 release registry, whose digest is still ba36f887…;
    • the state, totals, full and state_cd surfaces of the pinned feed.
  • Target values and every published release.

  Only future --target-surface full calibrations change. Certified releases use national_state (d122).

Invariants (each tested)

1. Differential. For every registry Hypothesis draws, the shared module returns, bit for bit, what the frozen pre-move helpers (test_support/.../us_target_loss_weights_reference.py) return for the same registry with the per-row labels removed. On label-free registries that is the pre-move result itself. Route A's 5,694 recorded weights still reproduce.

2. Key invariance. A district row's national key does not change when any metadata naming its place, record or label is dropped or rewritten. These are all exclusions except state_fips and ledger_geography_level, which the key reads on purpose.

3. Partition = identity. District rows share a national key if and only if they are one concept in one state (district_concept_identity). This is tested on random registries, on 31 real compiled rows from the pinned feed (new fixture, written by the experiment script), and on the whole pinned compile.

4. Guard. The national weights refuse:

   • a per-row key that splits a concept, and the error names it;
   • a stateless district row;
   • a group that spans states.

   Non-district rows are never checked.

5. The labels are real. The compiler's own _ledger_metadata and _diagnostic_target_label_metadata stamp exactly LEDGER_PER_ROW_LABEL_METADATA on a district fact. The diagnostic label appears only when the periods match. The pre-change key split two such districts, and the new key pools them.

6. Feed-gated count (skips on CI). The pinned compile gives:

   • 24,340 district rows and 3,125 groups, max 52;
   • singletons of 95 per single-district state;
   • district loss share 0.456 → 0.110;
   • each basis's non-district rows rescaled by one common factor.

Merge rationale

This restores the behaviour that #195 documented and #1104 already applied to ACS local. It does not move target values or any release surface. Under the "obvious bug fix" rule it needs no ruling, so it merges on the usual gates: green CI, mergeable, and an independent review in agreement.

axiom: n/a: calibration loss-weighting infrastructure, no policy rule.

Review

Pending: independent GPT-6.1 Sol review (Subfleet review/hard, job 20261012-111905-cd-budget-review-r1) at 89c9c28; the later commits add only experiment evidence. Local runs: the 57 loss-weight tests pass, including the feed-gated one, and tools/spec_engine_coverage.py --check passes (42,187/42,187 fields, 41/41 inventory checks). A broader targeted run over the US release-tool and spec-engine test files is paused by a host hold.

🤖 Generated with Claude Code