Add exact income-band intervals and boundary diagnostics

changelog.d/nz-reference-ageing-factors.added.md

@@ -0,0 +1,15 @@
+Add declaration-driven, exact feed-derived reference ageing factors with certified
+root bounds, once-only rounding, selected evidence identities and authenticated
+compiler replay. Declare the reviewed NZ ageing profiles and their explicit money
+and count companions while keeping the income references parked for measurement
+closure.
+
+Retain authenticated decimal tokens only with explicit lexical-float loading for
+selected factors; ordinary loads keep plain floats and their serializer and memory
+behavior. Bind factor application to the selected money/count roster and preserve
+count operations. Certify exact-root ties in the independent oracle and bound
+supported schedules so exact receipts remain serializable. Float selector pins
+match token-retaining floats while keeping booleans, integers and floats distinct.
+Scan extra feed-identity fields for RuleSpec commits. Bind factor evidence to
+reviewed feed identities while allowing prose-only pin edits; refuse replay when
+the source commit cannot be authenticated.

changelog.d/nz-reference-intervals.added.md

@@ -0,0 +1,24 @@
+Add exact income-band intervals: a shared, declaration-driven interval module
+with no default semantics (closure, nil band, negative-income policy, first lower
+cut, entity, columns and diagnostics window are all declared per table),
+integer-cent source cuts, exact rational scaling that never rounds an aged cut,
+universe-aware classification that refuses unresolved records or invalid amounts
+before any indicator exists, and fixed-weight boundary diagnostics weighted
+through `resolve_weights(entity)`.
+
+`scale_money_and_intervals` now scales named interval tables with the same
+selected factor as their money companions and binds each scaled table to its
+source and factor. Interval outputs are their own calibration binding kind.
+Preparation passes the effective interval document into column preparation and
+records its digest, payload and input fingerprints; the problem kernel binds a
+boundary diagnostics report without changing any target. Composition supplies
+each variant its own effective interval companion. Declare the NZ national
+(237 bands) and age (183 bands) taxable-income tables under Microcosm's declared
+convention; the 474 IRD references stay parked.
+
+Require the first cut explicitly. When a nil band is present, zero is nil under
+either closure, so with a zero first cut the first positive band starts above
+zero. Report below-first-cut mass and floored negative records, including
+weighted counts and signed pre-floor amounts by destination. These diagnostics
+describe fixed weights before fitting; reporting later weight changes on
+unfitted records remains G8-r2c/G8-x work.

docs/nz-calibration-targets.md

@@ -99,6 +99,46 @@ All statistical spreadsheets on the un-gated `ird.govt.nz` CDN (HTTP 200 confirm
 
 Notes: **I1, I3, I4 are the load-bearing IRD targets.** IRD income tables are **individual-level** — joint/couple income unpublished. **FamilyBoost: no public statistics dataset confirmed** — only a modelled `Value_Type` in Treasury DistributionExplorer [UNCONFIRMED — re-check].
 
+### I1 taxable-income bands: a declared convention (G8-r2b, 2026-10-24)
+
+IRD publishes no band closure or cents semantics for the I1 tables. The
+workbook introduces an "income between $0.01 and $100" band, and four label
+pairs share an endpoint ($20,000, $50,000, $80,000 and $300,000: national rows
+29/30, 59/60, 89/90 and 229/230; age rows 27/28, 57/58 and 87/88). Microcosm
+therefore applies its own declared convention, not IRD's semantics (G8-r open
+rulings 2, section 1):
+
+- t = max(0, x) in integer cents, with no dollar rounding;
+- t == 0 is the nil band, including qualifying loss records;
+- each positive band is `previous_upper < t <= upper`, and the open tail is
+  `t > its lower cut` (national top band over $1,000,000, age table over
+  $180,000);
+- the ordered right-closed cuts give each shared endpoint to the lower band.
+
+`build/nz/reference_intervals.json` holds the two tables: the national table
+(237 bands, rows 8–244, each naming its parked count and amount references)
+and the age table (183 bands, rows 6–188, no references and no fit rows).
+`tools/generate_nz_target_references.py` generates both from the row formulas
+and checks every cut against the pinned feed's own band labels. Ageing scales
+the national cuts exactly by the selected factor and keeps fractional cents:
+under the central factor 1.117109, $100 becomes 11,171.09 cents and $1,000
+becomes 111,710.9 cents. Before matrix construction the calibration problem
+reports, for every cut, the weighted mass within $1, and for every band the
+fixed-weight change under the left-closed alternative. These diagnostics never
+change a target, and publisher reclassification is unavailable, because the
+published histogram does not identify mass exactly on a cut.
+
+The shared diagnostics also report records below a table's first lower cut,
+including their weighted count and amount. Floored negative records retain
+their separate count and signed pre-floor amount, grouped by destination.
+These are fixed-weight, pre-fit measurements; reporting the later weight
+changes on these unfitted records remains G8-r2c/G8-x work.
+
+The income and universe columns (`ird_taxable_income_proxy_cents`,
+`ird_taxable_income_proxy_universe_status`) name the d1952 stand-in measure
+rather than IRD's definition. No producer exists yet (G8-r2c), no active row
+requests them, and all 474 IRD rows stay parked.
+
 ## 3. MSD
 
 **Critical structural finding:** the **quarterly Benefit Fact Sheets are COUNT-ONLY** (every sheet opened; only hardship reports dollars). **Dollar-level data comes from the Annual Report** expenditure schedule (explicit CC BY 4.0).

docs/nz-take-up-engine-plan.md

@@ -129,9 +129,12 @@ tenure targets close to each region's published Census household total before
 the existing ERP bridge. Calibration variant V4 concentrates only the size
 residual onto one- and two-person households. The implementation preserves raw
 discrepancies and independent national diagnostics, and supplies validated size
-columns before activation. G8-r2 reuses the same resolver for ageing profiles
-and interval preparation; central ageing remains explicitly deferred and V1/V2
-ageing transformations remain unavailable until that work lands.
+columns before activation. G8-r2 reuses the same resolver for ageing: each
+low, central or high profile scales the IRD money references and, exactly, the
+national taxable-income band cuts by its own feed-derived factor, and the
+interval tables declare Microcosm's band convention (see
+[the target inventory](nz-calibration-targets.md)). The 474 IRD references
+stay parked until the B4 stand-in measure's producer lands (G8-r2c).
 - **Pre-calibration:** IRD wage and salary bands, MBIE bond rents by TA, Stats
   NZ household financial net worth, TA population, and the same MSD recipient
   facts as the calibration set (the take-up draws calibrate to them).

packages/microcosm-build/src/microcosm/build/calibration_columns.py

@@ -1,7 +1,8 @@
 """Prepare simple calibration columns from an explicit, engine-free contract.
 
 Geography and canonical concepts must already be present. This helper creates
-constants, validated roster counts and conjunctions of declared predicates;
+constants, validated roster counts, conjunctions of declared predicates and
+interval indicators classified by :mod:`microcosm.build.reference_intervals`;
 receipt assignment and statistical-unit construction belong to their own producers.
 """
 
@@ -15,15 +16,26 @@
 import numpy as np
 import pandas as pd
 
+from microcosm.build.reference_intervals import (
+    interval_indicators,
+    validate_interval_document,
+)
+
 
 def prepare_calibration_columns(
-    tables: Mapping[str, pd.DataFrame], bindings: Mapping[str, Any]
+    tables: Mapping[str, pd.DataFrame],
+    bindings: Mapping[str, Any],
+    *,
+    intervals: Mapping[str, Any] | None = None,
 ) -> dict[str, pd.DataFrame]:
     """Return independent tables with the contract's prepared columns.
 
     All required inputs are validated before any column is produced. Existing
     prepared columns are recomputed, so a changed geography or age cannot retain
     stale indicators. Missing/unknown inputs never become all-false categories.
+    An interval output names its entity, an effective table of ``intervals``
+    and one of its bands; every record of each named table is validated, and
+    any refusal raises, before an output exists.
     """
     if bindings.get("schema_version") != 1:
         raise ValueError("Calibration bindings require schema_version 1.")
@@ -48,6 +60,7 @@ def prepare_calibration_columns(
                 raise ValueError(f"Calibration input {entity}.{name} needs a domain.")
             _validate_input(tables[entity][name], domain, f"{entity}.{name}")
 
+    interval_columns = _interval_columns(tables, columns, required, intervals)
     out = {entity: table.copy(deep=True) for entity, table in tables.items()}
     derived = _prepare_roster_counts(out, bindings.get("roster_counts", ()), required)
     prepared: dict[str, dict[str, object]] = {}
@@ -69,6 +82,9 @@ def prepare_calibration_columns(
                 f"Calibration output {entity}.{name} is repeated or structural."
             )
         seen.add(key)
+        if "interval" in entry:
+            prepared.setdefault(entity, {})[name] = interval_columns[key]
+            continue
         if "constant" in entry:
             value = entry["constant"]
             if "predicates" in entry or not _finite_number(value):
@@ -115,6 +131,81 @@ def prepare_calibration_columns(
     return out
 
 
+def _interval_columns(
+    tables: Mapping[str, pd.DataFrame],
+    columns: object,
+    required: Mapping[str, Any],
+    intervals: Mapping[str, Any] | None,
+) -> dict[tuple[str, str], np.ndarray]:
+    """Classify every named table once; refuse before any output is produced."""
+    entries = [
+        entry for entry in columns if isinstance(entry, Mapping) and "interval" in entry
+    ]
+    if not entries:
+        return {}
+    if intervals is None:
+        raise ValueError("Interval outputs need the effective interval document.")
+    document = validate_interval_document(intervals)
+    requested: dict[str, list[tuple[str, str, str]]] = {}
+    for entry in entries:
+        entity, name, binding = (
+            entry.get("entity"),
+            entry.get("name"),
+            entry["interval"],
+        )
+        if set(entry) != {"entity", "name", "interval"}:
+            raise ValueError(
+                f"Interval output {entity}.{name} declares only its entity, name "
+                "and interval."
+            )
+        if not isinstance(binding, Mapping) or set(binding) != {"table", "band"}:
+            raise ValueError("An interval output names exactly its table and band.")
+        table = document.get(binding["table"])
+        if table is None or binding["band"] not in table.band_ids:
+            raise ValueError(
+                f"Interval output {entity}.{name} names an undeclared table or band."
+            )
+        if entity not in required or entity != table.entity:
+            raise ValueError(
+                f"Interval output {entity}.{name} must name its table's entity."
+            )
+        if name != table.indicators[table.band_ids.index(binding["band"])]:
+            raise ValueError(
+                f"Interval output {entity}.{name} must be its band's declared indicator."
+            )
+        if (
+            table.declaration["source_period"] != table.declaration["target_period"]
+            and not table.scaled
+        ):
+            raise ValueError(
+                f"Interval table {table.id!r} has source-period cuts; scale it to its "
+                "target period before preparing indicators."
+            )
+        requested.setdefault(table.id, []).append((entity, name, binding["band"]))
+    result: dict[tuple[str, str], np.ndarray] = {}
+    computed = {}
+    for table_id in requested:
+        table = document[table_id]
+        data = tables.get(table.entity)
+        if data is None or any(
+            column not in data
+            for column in (table.income_column, table.universe_column)
+        ):
+            raise ValueError(
+                f"Interval table {table_id!r} needs {table.entity}."
+                f"{table.income_column} and {table.entity}.{table.universe_column}."
+            )
+        computed[table_id] = interval_indicators(
+            table,
+            data[table.income_column].tolist(),
+            data[table.universe_column].tolist(),
+        )
+    for table_id, outputs in requested.items():
+        for entity, name, _ in outputs:
+            result[entity, name] = computed[table_id][name]
+    return result
+
+
 def _finite_number(value: object) -> bool:
     return (
         isinstance(value, Real) and not isinstance(value, bool) and math.isfinite(value)
@@ -196,6 +287,11 @@ def _validate_input(values: pd.Series, domain: Mapping[str, Any], name: str) ->
         for value in values.tolist():
             _identity(value)
         valid = True
+    elif domain.get("kind") == "signed_integer":
+        valid = all(
+            isinstance(value, Integral) and not isinstance(value, (bool, np.bool_))
+            for value in values.tolist()
+        )
     elif domain.get("kind") == "nonnegative_integer":
         valid = all(
             _finite_number(value) and value >= 0 and value == int(value)

packages/microcosm-build/src/microcosm/build/ledger_artifact.py

@@ -64,7 +64,9 @@
     "CHRONICLE_FEED_RESOURCE",
     "ChronicleFeedPinMismatchError",
     "LedgerConsumerArtifact",
+    "LexicalFloat",
     "add_ledger_artifact_args",
+    "chronicle_feed_identity",
     "load_ledger_consumer_artifact",
     "resolve_ledger_artifact",
     "validate_chronicle_feed_pin",
@@ -78,6 +80,51 @@
 ALLOWED_LEDGER_ASSERTIONS = frozenset(("observation", "source_projection"))
 DEFAULT_LEDGER_ASSERTION = "observation"
 CHRONICLE_FEED_RESOURCE = "chronicle_feed.json"
+_FEED_IDENTITY_FIELDS = (
+    "source_commit",
+    "facts_sha256",
+    "manifest_sha256",
+    "artifact_schema_version",
+    "consumer_fact_schema_versions",
+    "consumer_fact_schema_sha256",
+    "fact_row_count",
+)
+
+
+def chronicle_feed_identity(feed_pin: Mapping[str, Any]) -> dict[str, Any]:
+    """Select reviewed source commit and loader-verified fields, excluding prose.
+
+    The source commit is a caller declaration unless the authenticated manifest
+    also carries it; the artifact loader verifies the remaining six fields.
+    """
+    missing = sorted(set(_FEED_IDENTITY_FIELDS) - feed_pin.keys())
+    if missing:
+        raise ValueError(
+            f"Chronicle factor feed identity is missing fields: {missing}."
+        )
+    if not isinstance(feed_pin["source_commit"], str) or not feed_pin["source_commit"]:
+        raise ValueError("Chronicle factor feed identity requires a source commit.")
+    return {field: feed_pin[field] for field in _FEED_IDENTITY_FIELDS}
+
+
+class LexicalFloat(float):
+    """An opt-in float retaining its authenticated JSON numeric token.
+
+    Request ``lexical_floats=True`` when loading operands for exact arithmetic.
+    The default loader returns plain floats, avoiding the instance and token
+    overhead and preserving compatibility with serializers requiring exact
+    float types. Standard JSON serialization retains the same numeric bytes.
+    """
+
+    __slots__ = ("lexical_token",)
+
+    def __new__(cls, token: str):
+        instance = super().__new__(cls, token)
+        instance.lexical_token = token
+        return instance
+
+    def __reduce__(self):
+        return type(self), (self.lexical_token,)
 
 
 class ChronicleFeedPinMismatchError(ValueError):
@@ -250,6 +297,7 @@ def load_ledger_consumer_artifact(
     expected_facts_sha256: str | None = None,
     expected_manifest_sha256: str | None = None,
     feed_pin: Mapping[str, Any] | None = None,
+    lexical_floats: bool = False,
 ) -> LedgerConsumerArtifact:
     """Load a Ledger consumer artifact directory or bare consumer-facts file.
 
@@ -259,6 +307,8 @@ def load_ledger_consumer_artifact(
     reviewed against; any mismatch raises before facts are used. ``feed_pin``
     additionally checks all six reviewed artifact fields, without replacing
     the manifest integrity checks or either optional hash check.
+    ``lexical_floats=True`` retains numeric tokens for factor verification;
+    otherwise JSON decimals load as plain ``float`` instances.
     """
     artifact_path = Path(path)
     if not artifact_path.exists():
@@ -314,7 +364,7 @@ def load_ledger_consumer_artifact(
     if feed_pin is not None:
         artifact = LedgerConsumerArtifact(
             path=artifact_path,
-            facts=_load_fact_rows(facts_path),
+            facts=_load_fact_rows(facts_path, lexical_floats=lexical_floats),
             facts_sha256=facts_sha256,
             manifest=manifest,
             manifest_sha256=manifest_sha256,
@@ -340,7 +390,7 @@ def load_ledger_consumer_artifact(
     if artifact is None:
         artifact = LedgerConsumerArtifact(
             path=artifact_path,
-            facts=_load_fact_rows(facts_path),
+            facts=_load_fact_rows(facts_path, lexical_floats=lexical_floats),
             facts_sha256=facts_sha256,
             manifest=manifest,
             manifest_sha256=manifest_sha256,
@@ -389,15 +439,19 @@ def resolve_ledger_artifact(args: Namespace) -> LedgerConsumerArtifact | None:
     )
 
 
-def _load_fact_rows(path: Path) -> tuple[dict[str, Any], ...]:
+def _load_fact_rows(
+    path: Path, *, lexical_floats: bool = False
+) -> tuple[dict[str, Any], ...]:
     rows: list[dict[str, Any]] = []
     with path.open() as file:
         for line_number, line in enumerate(file, start=1):
             stripped = line.strip()
             if not stripped:
                 continue
             try:
-                row = json.loads(stripped)
+                row = json.loads(
+                    stripped, parse_float=LexicalFloat if lexical_floats else float
+                )
             except json.JSONDecodeError as exc:
                 raise ValueError(
                     f"Invalid Ledger facts JSONL row {line_number}: {exc.msg}"

packages/microcosm-build/src/microcosm/build/ledger_targets.py

@@ -20,6 +20,7 @@
 from types import MappingProxyType
 from typing import Any
 
+from microcosm.build.ledger_artifact import LexicalFloat
 from microcosm.calibrate import (
     CalibrationHierarchy,
     CalibrationHierarchySeed,
@@ -5058,7 +5059,9 @@ def _dimension_values_match(fact: object, expected: object) -> bool:
 
 
 def _strict_scalar_equal(actual: object, expected: object) -> bool:
-    return type(actual) is type(expected) and actual == expected
+    actual_type = float if type(actual) is LexicalFloat else type(actual)
+    expected_type = float if type(expected) is LexicalFloat else type(expected)
+    return actual_type is expected_type and actual == expected
 
 
 def _measure_concepts(fact: object) -> tuple[str, ...]: