Skip to content

G8-r2c F: replace NZ's 474 native income rows with 24 coarse stand-in rows - #1476

Closed
MaxGhenis wants to merge 8 commits into
nzhub/g8r2c-e-wage-mapfrom
nzhub/g8r2c-f-coarse-refs
Closed

MaxGhenis wants to merge 8 commits into
nzhub/g8r2c-e-wage-mapfrom
nzhub/g8r2c-f-coarse-refs

Conversation

@MaxGhenis

@MaxGhenis MaxGhenis commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

G8-r2c hunk F (#1183): NZ's 474 native income rows become 24 coarse stand-in rows. Draft, with E (#1475) as its base; G (#1478) stacks on it. Paths as in #1475; tool file:line is at G's head (64ef20af).

Landing (R2C-21a, #1183 comment 6604753490). E, F and G land together. The hub fast-forwards G's branch to the fix rounds' top f43e8fbb (#1486), retargets G (#1478) to main and lands it with one merge commit, which brings E's and F's commits. This PR never lands alone, and no check gates F by itself.

Change

  • Coarse table ird_taxable_income_standin_ty2024 (tools/generate_nz_target_references.py:1355, _coarse_table): person, main's ird_taxable_income_proxy_cents and _universe_status, right-closed, nil_band: absent, floor_zero, first_lower_cut_cents: 4000000, diagnostics.support_household_entity: household, cuts at $50k ... $1m and an open tail: 12 bands, each with count_reference and amount_reference.
  • Membership (coarse_band_members, :1260): each native row joins the band whose (lower, upper] holds its own upper cut; the generator refuses a band that doesn't end on a native cut or a set that doesn't tile rows 50-244. Result: b01 50-59, b02 60-69, b03 70-79, b04 80-89, b05 90-109, b06 110-134, b07 135-159, b08 160-209, b09 210-229, b10 230-233, b11 234-243, b12 244, as design §3's table.
  • 24 rows (coarse_income_rows, :1299) in A's summed form (R2C-17): 12 count rows (value_operation: sum, no operands, measure people) and 12 identity amount rows (measure ird_taxable_income_proxy_nzd, R2C-10). Each pins its band's native labels in dimension_values and its expected_member_count, family income, prerequisites reference_profile:aging_index and measurement:ird_taxable_income_ty2027_v1, status requires_prerequisites. The 5% tolerance is the income family's in calibration_per_family_fit (within: 0.05, d1376); rows carry no tolerance field, as every NZ row.
    • The calibration document's allowed_value_operations adds sum (:871).
    • _reference_selection never exempts a row with a measurement binding (:1021).
  • R2C-4. The 474 native rows move to the non-executable N/income_diagnostic_references.json (key diagnostic_references, executable: false, :896), and their exemptions go. The national table loses its companions (declared and validated only, indicators unchanged), and the aging_index transform scales the 24 rows and the coarse table (:1591).
  • N/reference_inventory.json counts the diagnostic file; N/country_package.json registers it; docs/nz-calibration-targets.md describes the coarse rows.

Invariants and tests

Invariant Test Red under its mutant
Each coarse band lists exactly its native rows' labels (feed audit, R2C-16) test_nz_target_references.py::test_each_coarse_band_lists_exactly_its_native_rows_labels drop a member from b05
Each aged coarse row compiles to the sum of its native rows' aged values (differential, through A's summed path) test_coarse_amounts_equal_the_aged_sum_of_their_native_rows —
The central first cut is 4,468,436 cents; $125k is 13,963,862.5 test_nz_target_references.py::test_the_central_first_coarse_cut_is_4468436_cents, test_nz_reference_intervals.py::test_central_ageing_keeps_fractional_cent_cuts_exactly —
No native IRD row sits in an executable document test_no_native_ird_taxable_row_sits_in_an_executable_document a native row back in target_references.json
The rows stay parked and unexempt without a producer test_the_24_coarse_rows_stay_parked_unexempt_without_a_producer exempt one row
12 bands with companions and per-band support test_nz_reference_intervals.py::test_coarse_table_has_12_bands_each_with_companions_and_support drop support_household_entity
The scaled coarse table tiles the positive range for any factor test_property_the_scaled_coarse_table_tiles_the_positive_range (Hypothesis, 80 examples, through scale_interval_table and classify_records) —

The NZ composition tests in test_nz_transport_declarations.py add a compile-only measurement producer when the graph has none (as they already add synthetic money producers), and the file's resolved-view cache now keys on the transport graph, because A's proof reads it (a52efb77).

R2C-21: the one red test at F

test_transport_composed_graph.py::test_packaged_spec_refuses_with_every_activation_gap_named stays red at F on E's text, as R2C-21 rules: the packaged spec's activation now also lists the 24 parked rows (placeholder reference target_references.ird_taxable_income_standin_…; awaits measurement:ird_taxable_income_ty2027_v1 (0 nodes declaring …)), so the exact count fails. G's producer lifts all 24 and the test passes there on E's text.

R2C-19 and R2C-22 shared-test edits

  • test_spec_engine_country_bundles.py:263 (income_diagnostic_references.json, R2C-19 item 1, 8167c424)
  • test_reference_profiles.py:1163, :1165, :1171, :1173, :1175, :1187 (R2C-22, c9f5b0c2)
  • test_transport_composed_graph.py:2424-2431 and :2434-2435 (R2C-22; local name coarse)

R2C-22's mutants: a resolver that also scales the national table turns both tests red (the second at tables[unscaled] == by_id[unscaled]); giving the low profile the central factor id turns the second red.

Hashes

The hub hashed 42 engine-free transport kernels (no simulate.*) at each head, each in its own tree: int 3e5f5edb, E 8ec98c1c, F 623afa09 and G a3fe1952, after which G's head 64ef20af changes only two test files. E, F and G hash identically to int, as does the int before D's merge (ba08d0f9), so this hunk moves no kernel hash.

Kernel int 909fe3cb int 3e5f5edb (= ba08d0f9, E, F, G)
calibrate.ordered_adam@1 0d4577effa147ebe 0d4577effa147ebe
calibration.prepare@1 180823901b9a0d99 180823901b9a0d99
concepts.encode@1 7815fae8ed573af6 7815fae8ed573af6
concepts.encode_groups@1 062f88411ad605ab 062f88411ad605ab
diagnostics.calibration@1 30654dfc7038f560 30654dfc7038f560
export.prepare@1 af31fa6e1690b90c af31fa6e1690b90c
export.readback@1 96be393c760ca768 96be393c760ca768
gates.battery@1 ac050bab09f48dc2 ac050bab09f48dc2
geography.assign_atomic@1 d60f7ec388903678 d60f7ec388903678
geography.derive@1 d60f7ec388903678 d60f7ec388903678
geography.gate@1 d60f7ec388903678 d60f7ec388903678
geography.person_location@1 3efcbe3741d04497 3efcbe3741d04497
geography.support_from_facts@1 ad0a8e73d20405be ad0a8e73d20405be
geography.support_import@1 d60f7ec388903678 d60f7ec388903678
measure.combine@1 d609832882ffc113 d609832882ffc113
money.annual_transfers@1 c8a067cd750ed2fe c8a067cd750ed2fe
money.minor_units@1 c8a067cd750ed2fe c8a067cd750ed2fe
money.normalize@1 f10fbeeaf87467b6 f10fbeeaf87467b6
reporting.carry_income_decile@1 80335f0183c27410 80335f0183c27410
reporting.income_decile@1 80335f0183c27410 80335f0183c27410
reporting.receipt_group@1 80335f0183c27410 80335f0183c27410
takeup.assign@1 b2a3a50cc376108b b2a3a50cc376108b
takeup.bands@1 1ceca3ba7c7c9b92 1ceca3ba7c7c9b92
takeup.cell_diagnostics@1 1ceca3ba7c7c9b92 1ceca3ba7c7c9b92
takeup.compare@1 1605d7e8519f4b0f 1605d7e8519f4b0f
takeup.gap@1 54f79560d896fab4 54f79560d896fab4
targets.compile@1 dc91d1479f8f84b1 dc91d1479f8f84b1
targets.problem@1 fe401c736551e351 fe401c736551e351
transport.boundary@1 fc9d10670f6a092f fc9d10670f6a092f
transport.create@1 865c21926cee0ac0 865c21926cee0ac0
transport.create@2 865c21926cee0ac0 865c21926cee0ac0
transport.currency@1 9c3457ddb908aa96 9c3457ddb908aa96
transport.donor_ranges@1 62def362139a52e4 62def362139a52e4
transport.donor_receipt@1 62def362139a52e4 62def362139a52e4
transport.input_closure@1 23134112061495b7 23134112061495b7
transport.package@1 e86be552287a6eb3 e86be552287a6eb3
transport.quantile_map@1 b25bb6538b19dd64 b25bb6538b19dd64
transport.quantile_map@2 b845aac2c57f667b 76c81605aa416dc9 (moved)
transport.scenario_override@1 062f88411ad605ab 062f88411ad605ab
transport.support_bounds@1 2303c60a7ec6135c 2303c60a7ec6135c
transport.unit_attributes@1 f50ec74a1bfe35bb f50ec74a1bfe35bb
transport.weights_audit@1 1f3d475013456127 1f3d475013456127

transport.quantile_map@2 moves only through int's merge of B's tail_estimators.py docstring fix, which it digests.

Tests

Closed list at int ba08d0f9, E 3e850e07 and F f5ed82ba (G at #1478), no --maxfail, -n 2. Counts read passed / skipped / failed. Merges of D's head (91bf4c0f) moved int to 3e5f5edb, E to 8ec98c1c and F to 623afa09, and against each earlier head they change only T/compose.py, test_transport_measurement_prefit.py and test_transport_measurement_proof.py. The hub reran those two files at each new head (-n 0, with the feed):

File int 3e5f5edb E 8ec98c1c F 623afa09
test_transport_measurement_prefit.py 47 47 47
test_transport_measurement_proof.py 33 33 33

The E-G review ran F's list at 623afa09 and found the same one red. Landing waits for green CI on #1478 against main.

The closed list at the earlier heads:

File int E F
frame/test_transport_donor.py 94 / 2s 94 / 2s 94 / 2s
frame/test_transport_quantile.py 79 79 79
test_country_spec.py 143 143 143
test_nz_as_area_crosswalk.py 67 / 3s 67 / 3s 67 / 3s
test_nz_as_bridge.py 34 34 34
test_nz_axiom_input_closure.py 19 / 2s 19 / 2s 19 / 2s
test_nz_benefit_gross_up.py 16 16 16
test_nz_earnings_bands.py — 4 4
test_nz_family_reconciliation.py 16 16 16
test_nz_joint_geography.py 22 / 11s 22 / 11s 22 / 11s
test_nz_money_normalization.py 56 56 56
test_nz_msd_benefit_rates.py 6 6 6
test_nz_msd_pre_activation_reconciliation.py 5 5 5
test_nz_reference_aging.py 22 22 22
test_nz_reference_intervals.py 14 14 16
test_nz_spec_package.py 1203 1203 1203
test_nz_target_references.py 54 56 60
test_nz_transport_declarations.py 31 42 42
test_nz_transport_toy_graph.py 23 25 25
test_reference_interval_support.py 12 12 12
test_reference_intervals.py 107 107 107
test_reference_profiles.py 129 129 129
test_reference_summed_bands.py 23 23 23
test_spec_engine_country_bundles.py 22 22 22
test_spec_only_country_packages.py 37 37 37
test_transport_annual_transfers.py 75 75 75
test_transport_binding_identity.py 32 32 32
test_transport_bridge_differential.py 37 37 37
test_transport_codecs_geography.py 22 22 22
test_transport_column_kernels.py 31 31 31
test_transport_composed_graph.py 469 469 468 / 1 failed
test_transport_declared_units.py 18 18 18
test_transport_dependent_children.py 24 24 24
test_transport_entitlement_graph.py 67 67 67
test_transport_evidence_kernels.py 151 151 151
test_transport_gate_bindings.py 131 131 131
test_transport_gate_kernels.py 31 31 31
test_transport_graph.py 15 15 15
test_transport_grossup_selectors.py 26 26 26
test_transport_measure_kernels.py 37 37 37
test_transport_measurement_prefit.py 47 47 47
test_transport_measurement_proof.py 27 27 27
test_transport_money_kernels.py 40 40 40
test_transport_population_kernels.py 32 32 32
test_transport_quantile_map_v2.py 38 38 38
test_transport_receipt_v2.py 117 117 117
test_transport_receipt_v2_kernel.py 14 14 14
test_transport_reference_selection.py 60 60 60
test_transport_registry.py 59 59 59
test_transport_reporting_kernels.py 31 31 31
test_transport_run_entry_factors.py 75 75 75
test_transport_run_entry_forms.py 229 229 229
test_transport_source_household_lineage.py 8 8 8
test_transport_takeup_kernels.py 38 38 38
test_transport_target_kernels.py 68 68 68
test_transport_terminal_kernels.py 18 18 18
test_transport_weight_quality.py 30 30 30
uk/test_transport_gate_bindings.py 7 7 7
Total 4338 / 18s 4357 / 18s 4362 / 18s / 1 failed

Skips (18 at each head), each needing a source this container lacks: test_nz_joint_geography.py 11 (NZ_AS_AREA_SOURCES and NZ_JOINT_GEOGRAPHY_SOURCES unset); test_nz_as_area_crosswalk.py 3 (pinned ArcGIS bytes); test_nz_axiom_input_closure.py 2 (Axiom dense engine and a rulespec-nz checkout); frame/test_transport_donor.py 2 (pinned real donor not cached). Every NZ_CHRONICLE_ARTIFACT feed audit ran, against the artifact I built from Chronicle 8f4cfd57 (both digests match the pin).

Also at each head: ci_test_plan.py verify prints verification=ok; the three source-free NZ builders' --check and generate_nz_target_references.py --check (with the artifact) match; ruff is clean on the changed files.


Generated by Claude Code

The generator adds the coarse table ird_taxable_income_standin_ty2024
(person, right-closed, no nil band, floor_zero, first lower cut $40,000,
cuts at $50k to $1m and an open tail: 12 bands) with count and amount
companions on every band and per-band source-household support
(support_household_entity: household), which measurement_prefit needs.
It writes 24 rows in A's summed form: 12 count rows with
value_operation sum and no operands, 12 identity amount rows measuring
ird_taxable_income_proxy_nzd (R2C-10, R2C-17), each pinning its band's
native labels and expected_member_count. The rows keep the prerequisites
reference_profile:aging_index and measurement:ird_taxable_income_ty2027_v1,
stay requires_prerequisites and are not exempt. The calibration
document allows sum.

R2C-4 moves the 474 native rows to the non-executable
income_diagnostic_references.json (key diagnostic_references) and drops
their exemptions. The national table loses its companions and becomes
declared and validated only. The aging_index transform now scales the 24
rows and the coarse table.

Invariants: each coarse band is the sum of whole native rows 50-244
(generator check, a feed-gated label audit and a differential that
compiles each aged coarse row against its native rows' aged sum); the
central first cut is exactly 4,468,436 cents; no native row sits in an
executable document; the rows stay parked and unexempt without a
producer; the scaled coarse table tiles the positive range for any
positive factor (Hypothesis, through scale_interval_table and
classify_records). The NZ composition tests add a compile-only
measurement producer, as they already add synthetic money producers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LwQq6Fwj4yUEPsD3wfNstU
The hub's R2C-19 item 1 lifts the freeze on
test_nz_generation_zero_views_come_from_the_country_spec_seam for F: NZ's
legacy resources now include income_diagnostic_references.json, the
non-executable home of the 474 native IRD rows (R2C-4).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LwQq6Fwj4yUEPsD3wfNstU
test_nz_transport_declarations memoizes resolved reference views by the
content of the documents the resolver reads. A's measurement proof now
reads the transport graph too, so a view resolved for a graph without a
measurement producer was reused for a graph with one, and the composition
tests saw "no proof". The cache key now includes the graph.

Checked: the full file passes in one process after the change; before
it, the five composition tests failed only when an earlier test in the
same process had cached a producer-less view.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LwQq6Fwj4yUEPsD3wfNstU
R2C-4 moves the 474 native rows out of the executable documents, so
the aging_index profiles scale the coarse stand-in table, not the
national one. The hub's R2C-22 lifts the freeze for:

- test_reference_profiles
  test_nz_default_factor_receipt_is_compact_and_covers_the_exact_roster:
  the receipt names ird_taxable_income_standin_ty2024 (12 bands) and 24
  companions, whose source carries A's expected_member_count.
- test_transport_composed_graph
  test_declared_scenarios_scale_national_cuts_with_their_own_factor: the
  per-scenario cut check moves to the coarse table (local name coarse),
  and the national and age tables must stay unscaled.

Mutants: a resolver that also scales the national table turns both
tests red (the second at tables[unscaled] == by_id[unscaled]); giving
the low profile the central factor id turns the second test red.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LwQq6Fwj4yUEPsD3wfNstU
@MaxGhenis

Copy link
Copy Markdown
Contributor Author

Closing as included: #1478 landed this PR's commits on main in merge commit f8f543a4, at G's reviewed head f43e8fb. The NZ hub reviewed the stack as one change (review 213 v4, #1183 comment 6615417145).

🤖 Generated with Claude Code

@MaxGhenis MaxGhenis closed this Nov 4, 2026
@MaxGhenis
MaxGhenis deleted the nzhub/g8r2c-f-coarse-refs branch November 4, 2026 00:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants