Skip to content

Stats NZ territorial-authority resident population, 30 June 2025 (NZ, G8-d2) - #355

Merged
MaxGhenis merged 3 commits into
mainfrom
nzhub/g8d2-ta-population
Oct 14, 2026
Merged

MaxGhenis merged 3 commits into
mainfrom
nzhub/g8d2-ta-population

Conversation

@MaxGhenis

@MaxGhenis MaxGhenis commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

What

A new package, stats_nz/subnational_population_ta_2025: 67 Stats NZ estimated resident population facts, one per territorial authority, at 30 June 2025 (provisional). It is G8-d2 of the NZ Microcosm plan (microcosm#1183), the territorial-authority population input.

  • Source: Table 2 of the same subnational workbook the existing subnational_population_estimates_2025 package already pins (sha256 001e8a89…50a6, unchanged). The package reads one cell per fact from column E, headed 2025 P.
  • Geography: geography_level: local_authority, geography_id: nz-ta-NNN, geography_vintage: ta_2025. That is the convention mbie/tenancy_bond_rents_tla_2026 already uses, so these facts join the TLA rent facts.
  • Codes: the workbook has names only. The codes come from a newly pinned file, the Stats NZ ArcGIS Territorial_Authority_2025 code, name and ASCII-name attributes (68 features, sha256 db033184…c225, 14,315 B).
    • All 67 workbook rows match a classification entry on the publisher's own ASCII name, with macrons removed from the workbook text and case folded. Row order never assigns a code.
    • The classification's code 999 (Area Outside Territorial Authority) has no workbook row.
  • R2: the hub published the classification to raw/nz/stats_nz/territorial_authority_2025_codes/2025/db033184…/ and verified the remote object by SHA-256.

Not emitted (preserved in the parsed worksheet):

  • the 21 Auckland local-board rows. They need a geography level Chronicle doesn't have; that is follow-up G8-d2b.
  • the 2023 and 2024 P columns, and the change and percent columns;
  • the New Zealand(3) row. The existing package already emits 5,324,700 from Table 1!D25.

No sums, shares, reconciliation or ageing. The 67 TA cells sum to 5,324,430 against the published 5,324,700; the workbook's footnote 3 (inlets, islands, ships, oil rigs outside any territorial authority) and its rounding note qualify the comparison; the workbook doesn't say how the 270 gap splits between them, and neither figure is adjusted.

Shared files

  • chronicle/source_package.py: one alias.
  • chronicle/bundle.py: one line in the curated NZ roster (NZ_BUNDLE_SOURCES), the same kind of edit G8-d1 (Census 2023 weekly rent paid by households, by regional council (NZ, G8-d1) #341) made. The lane's role couldn't write it, so the hub applied it.
  • tests/test_source_package_alias_drift.py: the NZ roster counts 66 → 67.
  • tests/test_chronicle_bundle.py: the default-bundle pins move by this package's measured single-source delta. The full bundle test runs in CI only.
    • facts +67 (1,030,137 → 1,030,204);
    • geographies +1 (local_authority:nz-ta-067, Chatham Islands, which MBIE doesn't cover);
    • packages +1, tables +1;
    • warnings +2: the classification spells Ōtorohanga and Ōpōtiki with macrons, MBIE without. Both are kept as published.
  • Docs: the checklist row, the population source notes, the existing package's README (it no longer says TA tables are not ingested), and a changelog fragment.

Tests

tests/test_chronicle_nz_subnational_population_ta.py, 94 cases.

  • Its expected map reads the workbook cells and the classification JSON directly, independent of the package YAML.
  • It requires exactly 67 facts, each equal to its workbook cell and coded from the classification.
  • It has exact-cell tests for ten sample rows, and asserts that code 999 and every local-board row are absent.

Mutants, each failing the tests (lane run):

  • the 2024 P column instead of 2025 P;
  • a local board (Rodney) emitted as a TA;
  • Matamata-Piako and Hamilton codes swapped;
  • the New Zealand(3) row emitted;
  • a stale guard header (Whangarei without the macron);
  • Auckland replaced by its local-board sum (1,816,110).

The existing population package still passes with 85 facts. validate-package and the single-source build-suite pass.

Invariants

  • One workbook cell per fact: no sums, shares or computed values.
  • Every geography id is a code in the pinned TA 2025 classification, used once. Matching is by publisher name, never by row order.
  • Values are the published rounded figures, never un-rounded or reconciled.
  • Reference date 30 June 2025; boundaries at 1 January 2025.

Review

  • Hub verification at 3e2cd70 (one file per run, worktree code):
    • new package tests 94 passed;
    • test_chronicle_nz_population.py 30 passed (85 facts);
    • alias drift 28 passed;
    • validate-package valid, 0 errors, 0 warnings.
  • r1, 20261013-150745 (GPT-6.1 Sol) at 3e2cd70: APPROVE_WITH_NITS, Needs Max: no.
    • It read Table 2 itself: all 67 values equal column E, and the local boards, row 96 and other columns emit nothing.
    • 73 guards, each failing when its cell changes.
    • Its own matcher gives the same 67 codes, with 999 unused.
    • The classification manifest mirrors the regional one, and its sha256 and size match.
    • The bundle pin deltas are consistent with the package and with MBIE's 66 TA ids (MBIE lacks only 067; the +2 warnings are 018 and 027, macrons only).
    • The chronicle/bundle.py change is the one roster line.
    • All six lane mutants and five of its own fail; an order-only control passes.
  • Its nits, fixed in 81fdaa6:
    • N1: the classification JSON is now binary in .gitattributes, beside its regional sibling.
    • N2: the README records the verified R2 publication and says row order never assigns a code.
    • N3: ruff format (0.14.10, the uv.lock version) on the new test file; 94 pass.
    • N4: the footnote/rounding sentence above, reworded.
  • r2 confirmation, 20261013-162418 (GPT-6.1 Sol) at 81fdaa6: APPROVE.
    • 94 pass; ruff clean; the test-file diff is formatting only (AST-equal).
    • The classification JSON is binary and its blob still hashes to db033184….
    • Its "Needs Max" (the README's R2 claim, which it could not check remotely) was judged no. On 2026-10-14 at 02:27Z the hub streamed ledger-raw/raw/nz/stats_nz/territorial_authority_2025_codes/2025/db033184…/territorial-authority-2025-codes.json back from R2: sha256 db033184…c225, 14,315 bytes, equal to the manifest. It is an internal package statement and publishes no number.
  • Its process note: the PR touches paths outside the ledger-source-ingestor allow-list (chronicle/bundle.py, chronicle/source_package.py, db/data/**, docs/**, changelog.d/**). These are the per-package files every NZ source package touches, as in Census 2023 weekly rent paid by households, by regional council (NZ, G8-d1) #341 and Add Stats NZ 2023 Census household and family counts by regional council #335.

🤖 Generated with Claude Code

MaxGhenis and others added 2 commits October 13, 2026 14:47
Emit 67 provisional all-age ERP observations from individual Table 2 E cells,
using pinned TA 2025 publisher codes and names. Preserve the whole worksheet,
exact labels and guards; exclude local boards, earlier years, changes and
the national row. Reuse the unchanged workbook and manifests.

Single-source build_bundle measurement (year=2023, only new package; output deleted):
facts=67; geography_count=67; source_count=1 (stats_nz); packages=1;
tables=1 (stats_nz:Subnational population estimates: At 30 June 2025 (provisional), Table 2);
warnings=0; errors=0; period_count=1; entity_count=1; duplicate counts=0.
by_period: calendar_year:2025=67; by_entity: person=67; by_source: stats_nz=67.
by_geography delta: 67 keys local_authority:nz-ta-NNN (the 67 Table 2 TA codes,
001 to 076 with gaps), each +1 fact.

Default bundle delta relative to main pins:
facts=+67 (1030137 -> 1030204); geographies=+1 (20548 -> 20549),
new key local_authority:nz-ta-067; 66 other keys already covered by MBIE;
sources=+0 (56); packages=+1 (379 -> 380); tables=+1 (321 -> 322);
warnings=+2 (118 -> 120); period_count=+0 (511); entity_count=+0 (12).
The two merged name warnings were measured through the existing warning
function using declared package geography metadata (no combined bundle build):
local_authority:nz-ta-018 (Otorohanga / Ōtorohanga District) and
local_authority:nz-ta-027 (Opotiki / Ōpōtiki District).
Counts: stats_nz +67 (1948 -> 2015), CY2025 +67, person +67 (496036 -> 496103);
new Table 2 source_table +67; every by_geography key above +1.
No new period/entity/source keys, aggregate/semantic duplicate keys, errors,
or skipped sources. Full bundle and merged-consumer-contract test remain CI-only.

Validation: validate-package and single-source build-suite pass (67 facts,
2226 preserved cells, complete lineage); one-source build_bundle passes.
Tests: new package 94 passed (also 94 passed after restoring mutants),
existing population package 30 passed with unchanged 85 facts; six required
mutants all rejected and restored.
The NZ bundle roster line (NZ_BUNDLE_SOURCES in chronicle/bundle.py) was
outside the ingestor role's paths; the NZ hub applied it, as for G8-d1.
Classification publication and remote verification are owned by the NZ hub.

Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The NZ hub published the classification with publish-raw and streamed the
object back from raw/nz/stats_nz/territorial_authority_2025_codes/2025/db033184…/;
its SHA-256 matched the manifest, which is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis
MaxGhenis marked this pull request as ready for review October 13, 2026 19:08
- Mark the TA 2025 classification JSON `binary` beside its regional-council
  sibling, so a renormalisation can never rewrite its pinned CRLF bytes.
- README: record the hub's verified R2 publication; row order never assigns
  a code (not "not ... alone").
- `ruff format` (0.14.10, uv.lock) on the new test file; 94 pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis
MaxGhenis merged commit 5c4912a into main Oct 14, 2026
3 checks passed
@MaxGhenis

Copy link
Copy Markdown
Contributor Author

Merged by the NZ hub (session local_e407a527) at 81fdaa6 with --squash. Gates: gh pr checks exit 0; MERGEABLE; not draft; no CHANGES_REQUESTED review. Reviewed head: 81fdaa6. Reviews (GPT-6.1 Sol): r1 APPROVE_WITH_NITS at 3e2cd70 (fidelity: all 67 = Table 2 col E; 11 mutants red), nits fixed in 81fdaa6; r2 confirmation APPROVE at 81fdaa6. Hub verification: new 94, nz_population 30, alias 28; validate-package valid. R2 classification object re-verified by sha256 02:27Z 10/14. Needs Max judged no (internal package; no published number).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant