Skip to content

Add NZ joint territorial authority, region and AS-area support - #1393

Merged
MaxGhenis merged 1 commit into
mainfrom
nzhub/g8m2-joint-geography
Oct 25, 2026
Merged

MaxGhenis merged 1 commit into
mainfrom
nzhub/g8m2-joint-geography

Conversation

@MaxGhenis

@MaxGhenis MaxGhenis commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

Independent of the #1375/#1382 stack.

What this does

Adds NZ's joint territorial authority × region × Accommodation Supplement area support (G8-m2). It is spec-only, with no transport activation. NZ's atomic geography unit becomes the joint TA × REGC × AS cell, so TA, region and AS area all derive from one atomic area by lookup.

  • Generator: tools/build_nz_joint_geography.py, which has --check.
    • It pins all 29 Stats NZ MBHG-2026 page URLs and verifies each page's SHA-256, byte size and feature count before reading geography. The URLs are recorded provenance for the fetcher; the loader doesn't read them. These pages carry REGC2025 and TALB2026; they are CC BY 4.0.
    • It then assigns each official SA1 to exactly one TA × REGC cell.
    • It reuses G8-m's SA1 → AS placement.
  • Support (nz/atomic_geography_support.json, 140 cells, in the existing support_from_surface schema):
    • Within each TA, observed 2023 Census SA1 shares split that TA's June 2025 ERP anchor through the existing largest-remainder kernel, which is unchanged.
    • All 67 TA anchors are conserved exactly.
    • Regional ERP is reproduced only approximately. It is reported as a diagnostic, not a target: the largest regional gap is -94 people in region 09.
    • Marginal shares are never multiplied.
  • Edge cases:
    • Region 99 keeps its own SA1 cells under TA 067 (Census 612 → ERP 620).
    • The 33 Census residents under TA 999 stay in audit-only outside-ta|… cells. They get no ERP anchor and no sampling mass, and are never reassigned to a real TA. This follows the crosswalk's existing Census-to-ERP discrepancy policy.
  • Audit: docs/nz-joint-geography-report.json holds source counts, every straddling TA's cells, every outside-TA SA1, the pins and Census conservation:
    • all 32,817 Census SA1s map once, out of 33,170 MBHG SA1s;
    • the 353 SA1s without Census records get no invented population;
    • anchored Census 4,993,281 plus audit-only 33 equals the full SA1 Census total of 4,993,314.
  • Packaging:
    • country_package.json registers the resource;
    • geography_spine.json describes the joint unit;
    • spec/sources.yaml re-pins that policy resource;
    • the golden changes only fingerprint, resources and resource_hashes.
  • Shared code: no shared or transport module changes, so no compile-module hash moves.

Invariants (tested)

  1. Anchor conservation: for every TA, support cells sum exactly to its ERP anchor. This holds for arbitrary integer populations (property) and for the actual pinned Chronicle targets.
  2. Partition: every Census SA1 maps to exactly one joint cell. Cells are unique, and anchor coverage is exact.
  3. No marginal product: each cell's share is the observed joint SA1 count over the TA's total. An independent oracle rejects the marginal-product mutant.
  4. Regions come from SA1s, never from TAs: all seven straddling TAs are checked against independent source counts, and the oracle rejects the region-from-TA mutant.
  5. Region 99 and TA 999: each is handled as above. The oracle rejects the drop-region-99 and TA-999-as-real mutants.
  6. Pinned bytes before interpretation: a page with a wrong hash refuses before any geography is read, and the output doesn't depend on page order (permutation property).
  7. Determinism: --check regenerates the committed support and audit byte for byte.

Verification

  • CI at 389e33a, on the merge ref with main at fe1e2ad: all ten checks pass. Its engine-free (3.13) log shows every file below run: 21,112 passed, 210 skipped, 3 xfailed.
  • Lane (GPT-6.1 Sol, at this tree): 8 files, 1,507 passed. Five oracle-rejection cases are red as intended: marginal_product, region_from_ta, drop_region_99, ta999_as_real_ta, unpinned_page.
  • Hub, at normal priority:
    • test_transport_codecs_geography.py: 22 passed;
    • test_nz_joint_geography.py: 33 passed;
    • test_fetch_nz_as_area_sources.py: 10 passed;
    • test_nz_as_area_crosswalk.py: 70 passed;
    • ruff check and format, ci_test_plan.py verify, graph_acceptance_burndown.py --verify, git diff --check and the generator's --check are all clean.
    • The other files (atomic_geography, spec_only, spec_engine bundles, country_spec, nz_spec_package) weren't rerun locally because of the fleet disk hold; CI ran them above.
  • Independent review (GPT-6.1 Sol), APPROVE_WITH_NITS:
    • It reran test_nz_joint_geography.py (33 passed) and the generator's --check, and regenerated with the page pins reversed: identical bytes.
    • It checked anchor conservation with its own rational largest-remainder implementation: all 140 cells agree and every TA equals its anchor (both totals 5,324,430).
    • It instrumented the loader: all 29 byte and hash checks run before JSON decoding, and all 29 feature-count checks before any geography field is read.
    • It ran four source mutants of the generator, each killed by an existing test with a passing control and a verified restore: drop region 99, skip page verification, TA 999 as real, and each SA1 taking its TA's modal region.
    • Its one nit, that this body overstated URL verification, is fixed above. No code change was needed.

Merge

🤖 Generated with Claude Code

Generate a spec-only atomic geography support for NZ whose area is the
joint TA x REGC x AS cell, built from the pinned SA1-to-TA-and-region
concordance and the existing G8-m AS placement. Within each TA, observed
Census SA1 shares split the TA's ERP anchor through the existing
largest-remainder kernel, so all 67 TA anchors are conserved exactly and
regional totals are reproduced only approximately (a diagnostic, not a
target). Marginal shares are never multiplied.

Region 99 keeps its own SA1 cells under TA 067. The 33 Census residents
under TA 999 stay in audit-only cells with no ERP anchor and are never
reassigned to a real TA.

Pin all 29 REGC page URLs, hashes, sizes and feature counts, and verify
the bytes before reading any geography. The generator has --check. NZ
transport is not activated; the resource is registered and described
only.

Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis
MaxGhenis marked this pull request as ready for review October 25, 2026 02:41
@MaxGhenis
MaxGhenis merged commit 4e31e42 into main Oct 25, 2026
10 checks passed
@MaxGhenis

Copy link
Copy Markdown
Contributor Author

Merged by the NZ hub (session local_e407a527) at 389e33a with --merge. Gates: gh pr checks exit 0; MERGEABLE; not draft; no CHANGES_REQUESTED review. Reviewed head: 389e33a. Independent review: GPT-6.1 Sol APPROVE_WITH_NITS at 389e33a (NEEDS MAX: no); its one nit (PR body overstated URL verification) is fixed in the body, no code change. Spec-only NZ geography; no shared or transport module changes, so the UK Budget compile-module hold doesn't apply. Review evidence: ~/reviews/nz-hub/jobs/new-g8m2/review-full.md.

@MaxGhenis
MaxGhenis deleted the nzhub/g8m2-joint-geography branch October 25, 2026 02:41
MaxGhenis added a commit that referenced this pull request Oct 25, 2026
#1393 added atomic_geography_support.json and changed country_package.json,
geography_spine.json and spec/sources.yaml, so the NZ golden conflicted.
The golden was regenerated from the merged tree with the test's own
summary function. Keys only this branch changed keep its values, keys only
main changed take main's, and the fingerprint, resource list and
country_package.json hash are the merged tree's.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MaxGhenis added a commit that referenced this pull request Oct 25, 2026
#1375 merged main (#1393), which added atomic_geography_support.json and
changed country_package.json, so the NZ golden conflicted. The golden was
regenerated from the merged tree with the test's own summary function. Keys
only one side changed keep that side's values; the fingerprint, the
resource list and the country_package.json hash, which both sides changed,
are the merged tree's.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MaxGhenis added a commit that referenced this pull request Oct 26, 2026
The region-99 support note in the generator, calibration_bindings.json and
the calibration-targets doc said G8-m2 (#1393) was not in this stack's base.
The merge of #1394's head brought in #1382's fix4 head, which carries #1375's
merge of main, so #1393 (4e31e42) is now an ancestor. Drop the dependency
clause; the support statement itself is unchanged. calibration_bindings.json
is regenerated with the generator, and the NZ golden follows its hash.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant