Repository navigation
NZ: re-declare the ageing factors from the September 2026 QES - #1507
Merged
Merged
Conversation
MaxGhenis
force-pushed
the
nzhub/qes-sep2026-redeclare
branch
3 times, most recently
from
November 4, 2026 19:22
a8ae6fc to
2703a89
Compare
MaxGhenis
force-pushed
the
nzhub/qes-sep2026-redeclare
branch
from
November 4, 2026 19:23
2703a89 to
b44229c
Compare
Stats NZ's September 2026 quarter QES moves nine of NZ's ten earnings ageing factors; the observed TY2026-over-TY2024 ratio stays 1.086069. The old declarations held through 2026-11-03. Since then the transport CLI refuses every NZ run with --resume auto or forbid; only a --resume require replay of stored work skips its currency check. Re-pin NZ's Chronicle feed to the NZ-suite artifact of Chronicle 89178aaf (12,370 rows; facts sha256 8580143f..., manifest ea0cc7ef...), the merge of chronicle#406. That commit carries chronicle#405 (the September 2026 QES vintage), #406 (tests of each QES record set's vintage and year guards) and chronicle#377. #377 gives each dimension value one label and moves consumer facts to chronicle.consumer_fact.v5: v4's fields plus an optional package_dimension_value_labels (a source package's own wording where it differs), which no NZ row carries. So the pin's schema list and schema digest move too. tools/generate_nz_reference_aging.py takes the September 2026 operands, adds the third QES vintage to its hash rule, and declares the ten factors, valid from 2026-11-04 to 2027-02-02. The hub's own checker, outside this repository, computed the ten factors from those operands at 60 digits, and the generator's resolver reproduced all ten when it ran. Each NZ generator then ran against the artifact: reference_aging.json, reference_inventory.json and msd_pre_activation_reconciliation.json (the last two change only the feed identity). tools/build_nz_transport_graph.py rewrote earnings_bands.json, which scales each wage cut by its profile's P1 factor and copies that factor and its receipt hash. The golden nz_country_spec.json pins every resource's hash, so it moved last. test_nz_reference_aging.py follows the generator. Four more NZ test files and docs/nz-calibration-targets.md pinned values that depend on the old factors or the v4 schema; they now pin the new ones. The three clock pins in test_transport_composed_graph.py move from 2026-10-24 to 2026-11-05, inside the new window. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MaxGhenis
force-pushed
the
nzhub/qes-sep2026-redeclare
branch
from
November 4, 2026 20:44
b44229c to
4fcfc14
Compare
MaxGhenis
marked this pull request as ready for review
November 4, 2026 23:11
Contributor
Author
|
Merged by the NZ hub (session local_e407a527) at 4fcfc14 with --merge. Gates: |
This was referenced Nov 4, 2026
anth-volk
added a commit
that referenced
this pull request
Nov 5, 2026
main re-pinned the US feed to Chronicle 41b2fb2b (#1506) and the NZ feed onto chronicle.consumer_fact.v5 (#1507). The one conflict is in docs/us-chronicle-feed-repin.md, where #1506's new re-pin section and this branch's section on re-pinning onto a newer row contract sit in the same place: both are kept, the general section first. #1506's sentence saying the US loader carries a row's stamp unchecked now adds that the release and target-parity tools check the stamps against the pin. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
anth-volk
added a commit
that referenced
this pull request
Nov 5, 2026
Review round 3 of #1510 found that tools/build_us_target_parity_manifest.py holds the feed to the committed US pin by digest, compiles it and writes committed files, but checked no row contracts, so a US re-pin that left the hand-typed list stale passed its own rebuild steps. - us_runtime.chronicle_feed: require_us_chronicle_feed_row_contracts, the exact-or-declared check: rows of the committed feed must carry exactly the pin's list; rows of any other feed, declared row contracts only. - The release tool's _check_us_ledger_feed_row_contracts now calls it, and the target-parity generator calls it after its digest check, before compiling or writing. - Wording: validate_pinned_row_contracts and the UK row check say the release certifier holds its feed to the operator's digests, so it applies the exact rule only when given the committed feed. The changelog says the same, names the US generator, and lists the readers left unchecked because no reviewed pin holds their feed (the US preflight and ACS tools, the scoring and diagnostic tools, the UK high-value home stock, BRMA rent-quantile and Scotland BRMA-overlap builders, and the childcare take-up fit tools). - The UK and US re-pin guides name the certifier's rule and the generator's check; notes that said the NZ pin carries the v4 digest now say NZ is on v5 (#1507). Refs #1509 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Re-declares New Zealand's earnings ageing factors from Stats NZ's September 2026 quarter Quarterly Employment Survey (QES) and re-pins NZ's Chronicle feed to
89178aaf, Chroniclemainat the merge of PolicyEngine/chronicle#406 (tests of each QES record set's vintage and year guards), which follows PolicyEngine/chronicle#405 (the September package).Why
NZ's declarations expired after 3 November 2026 (
valid_until). The transport CLI's run-entry check refuses an NZ run with--resume autoorforbidwhen today's UTC date falls outside a selected factor's window (reference_factors.py:747-766,transport/cli.py:710-736); only a--resume requirereplay of stored work skips it (transport/cli.py:727-728). G8-r2 hub rulings 3 §2 set the method: once Stats NZ publishes a target quarter, the observed value replaces its projection, and the projection runs from the latest contiguous observed quarter. For TY2027 that gives N = (Jun26 + Sep26 + Sep26·q + Sep26·q²) / 4, with q = (1 + g)^(1/4) and g from PREFU 2026 (2.7%, ±0.5 points for the low and high profiles). A re-declaration is a new feed pin plus reviewed expected factors and receipts.What changes
nz/chronicle_feed.json), all six moving fields: Chronicle89178aaf, 12,370 facts,facts_sha2568580143f…,manifest_sha256ea0cc7ef…, schemachronicle.consumer_fact.v5(acd33a09…). Seven operand quarters (2024-Q3 to 2026-Q2) take new fact keys and September's source hash, and 2026-Q3 joins at 1,714.86.package_dimension_value_labels, a source package's own wording of a value where it differs (consumer_fact.v5.schema.json:244at89178aaf). None of the feed's 12,370 rows carries it, and no Microcosm code at this head names it. Microcosm's loader carries each row'sschema_versionwithout gating on it (ledger_artifact.py:464-470), andis_accepted_consumer_fact_schema_versionhas no caller outside tests. The gate sits in the feed pin:validate_chronicle_feed_pincompares the schema ids the rows carry, the manifest's list and the schema digest with the pin exactly (ledger_artifact.py:257-291), so the pin moves to v5 with them. Between the old pin (8f4cfd57) and this one:schema_version; the other 16,stats_nz_household_net_worth2023 facts, also lose a trailing space inlayout.groupby_value_label, a field no NZ resource, generator or test reads;tools/generate_nz_reference_aging.py): the one-line hash rule becomes_qes_hash(period)(TY2024 quarters from June 2025, 2024-Q2 from June 2026, later quarters from September 2026)._OPERANDSgains 2026-Q3 (line 12054) and the seven moved keys, whose line numbers stay, and_EXPECTEDtakes the new factors. Validity runs from 2026-11-04 to 2027-02-02, the day before Stats NZ releases the December quarter (3 February 2027, NZ time). A decision reference names this re-declaration.reference_aging.json,reference_inventory.jsonandmsd_pre_activation_reconciliation.json;earnings_bands.json, bytools/build_nz_transport_graph.py, which copies each P1 factor and its receipt hash and scales the 235 wage bands (only those values move);nz_country_spec.json, which pins every resource's hash.test_nz_reference_aging.pytakes the new factors and window. Its re-anchor test now adds a synthetic December quarter, since September is real now, and its rounding test picks a P1 factor whose truncation differs. Four NZ test files anddocs/nz-calibration-targets.mdmove pins derived from the factors: the central coarse cuts, the IRD weights, the P1 factors and design 80's worked band, the donor factors and their numerator formula, and the feed's schema version. Three clock pins intest_transport_composed_graph.pymove from 10/24 to 11/05, inside the new window (Belgium hub's line review: OK (R2C-19). Belgium reconfirmed it atb44229cd4at 19:52Z on 11/04 and at4fcfc14efat 21:02Z, since that head keeps the same trees. The file's patch-id stayscc60fdd73b4c7cfc, and with the pins back at 10/24 all three tests fail on the currency check).changelog.d/1507-nz-qes-september-2026-redeclaration.changed.md.Factors (millionths)
ird_ty2024_to_ty2027low, central, highp1_ty2026_to_ty2027low, central, highdonor_cy2024_to_ty2027low, central, highobserved_ty2026_over_ty2024Invariants
chronicle_feed.jsonpins seven fields.reference_aging.jsonrepeats them 21 times (at its top level and in each declaration and receipt), andreference_inventory.jsonandmsd_pre_activation_reconciliation.jsononce each; every copy equals the pin. The golden pins the sha256 of 54 files, these four among them; the other 50 carry no copy, andearnings_bands.json, one of them, carries each P1 factor's receipt hash.validate_chronicle_feed_pinchecks six of the seven fields against the loaded rows and manifest (ledger_artifact.py:257-291);source_commitstays the caller's declaration (ledger_artifact.py:94-98).test_each_reviewed_operand_resolves_once_with_exact_lexical_evidence).reference_factors.py:611-614), so a one-millionth change refuses.require_current_factorsaccepts 2026-11-04 and 2027-02-02 and refuses 2026-11-03 and 2027-02-03 (test_factor_currency_uses_inclusive_declared_boundaries).float(cut × factor / 100)computed fromFraction, and each profile'sfactor_receipt_sha256equals its receipt inreference_aging.json(test_every_profiles_edges_equal_cut_times_factor_exactly,validate_earnings_bands).Checks
Independent arithmetic: the hub's
expected_factors.py(Decimal at 60 digits, outside the repo) produced_EXPECTED. The generator resolves every declaration from the artifact and raises on any difference from its expected factor; it exited clean, so the repo's resolver agrees with the hub's arithmetic on all ten.Gated files, one at a time with the feed staged (
NZ_CHRONICLE_ARTIFACT), peak tree RSS and footprint in GiB:"Head" means tree
427910fc6, the change commit's tree:a623c14efnow,4608f5457before the message fix. It differs from this PR's head tree74c61ef44only by the changelog fragment.test_ledger_artifact.py216c40578test_nz_family_reconciliation.py216c40578test_nz_joint_geography.py216c40578test_nz_msd_pre_activation_reconciliation.py216c40578test_nz_reference_aging.py216c40578test_nz_reference_intervals.pytest_nz_money_normalization.pytest_nz_earnings_bands.pytest_nz_target_references.pytest_nz_transport_declarations.pytest_transport_composed_graph.py, the three clock-pin teststest_country_spec.py -k nzThe first five ran at the hub's earlier commit
216c40578, which the squash replaced. Its tree differs from the head's inearnings_bands.json, the golden, four other NZ test files, the doc and the changelog fragment; none of the five files changed after it. Two of them (test_ledger_artifact,test_nz_joint_geography) load the whole NZ spec, which includesearnings_bands.json; CI reruns their feed-free tests at the head, and cloud review 220 reran both files whole with the feed (55 passed; 22 passed, 11 skipped).At
216c40578,test_nz_reference_intervals.pyfailedtest_central_ageing_keeps_fractional_cent_cuts_exactly, and the hub's memory guard stoppedtest_nz_target_references.pyat 2.063 GiB after 26 of its 61 tests, 6 of them failed. The test-pin moves fixed both files, and their reruns at the head passed.The 11 skips in
test_nz_joint_geography.pycome from itsstaged_jointfixture (:557-567), which needs the census source directories (NZ_AS_AREA_SOURCES,NZ_JOINT_GEOGRAPHY_SOURCES), not the feed.Generators: each ran against the local artifact and exited 0.
test_nz_transport_declarations.pyruns the target-reference and MSD generators'--checkwith the feed. Cloud review 220 ran all four--checks against its own rebuild of the feed, and each exited 0.CI: run 41909406184 on
4fcfc14ef, merged with main3e7a7045(Encode person_role bindings and their closure defaults on the person table (S2b PR 2) #1480 and Prove extension calibration nodes' measurement producers in compose #1488 landed today and touch no file here): pending. The feed-gated tests skip in CI.Reviews, both at
b44229cd4, whose trees4fcfc14efkeeps:89178aaf(same digests), recomputed all ten factors at 60 digits, ran all twelve gated files with the feed (the counts above) and turned three mutants red. Nit 1, a P1 rounding test that can't tell one rounding from intermediate rounding, goes to NZ's follow-up PR. Nit 2, the epoch table's missing v5, belongs to open microcosm#1510.20261104-145428-nz1507-claims): APPROVE_WITH_NITS, needs Max: no. From git objects alone it recomputed the ten factors, all 1,407 finite band edges and the golden's 54 resource hashes, and checked every cited line range. Its five fixes all concerned wording, and this body and the change commit's message now carry them: the feed-identity and exact-factor invariants, the shared-fact counts, the operand line numbers, and the replay exception to the currency check.publisher_dimension_value_labels; Chronicle's schema at89178aafcalls itpackage_dimension_value_labels. Chronicle's tests assert the old name's absence from the v5 schema, the harness documentation and the tested output rows. The body and the commit message now use the schema's name, and a scan found the field on none of the feed's 12,370 rows.20261104-160400-nz1507-delta): APPROVE_WITH_NITS, needs Max: no. It found all five fixes in place and no numerical or code defect. Its three nits concerned this body's wording (two in the hub correction, one passive), and this body now carries them.🤖 Generated with Claude Code