Design: persisted take-up propensities vs simulation-time choice (the health coverage utility layer) #360

Activity

1. MaxGhenis commented on Jul 8, 2026

   

   MaxGhenis

   on Jul 8, 2026

   Contributor

   More actions

   Strong framing, and I think the proposed split is the right one. Persist the latent parameters in the frame; let the engine evaluate a deterministic choice over them. Below is the case for that direction grounded in machinery already in the repo, a concrete consistency invariant, a baseline-preserving migration, and the open questions I would not want the design doc to gloss.

   Why persisted θ (not engine-side choice) is the correct default

   Three doctrines already in this codebase point the same way:

   1. The persistence-scope lesson (#340). An input absent from the persisted H5 silently falls back to its engine default, invisible to every weights-side gate and catchable only by a column-manifest check. Put the behavioral parameter engine-side and you inherit exactly that failure on the behavioral margin: turning the utility layer on/off, or bumping the engine, shifts results with no column-level diff and no gate to trip. Persisting θ makes the behavioral parameter a first-class input — diffable, gate-able, and coverable by the same required-column manifest that PR #350 just used to make the SNAP eligibility inputs a hard release precondition. The static take-up flags already live under this rule; #353's is_snap_abawd_discretionary_exempt parameter states it outright (economy: false, "like takeup, assigned during microdata construction rather than at simulation time"). θ is the natural extension of a convention that is already load-bearing, not a new principle.

   2. The take-up contract is the machinery to extend — and it already carries the tripwire for the drift this issue fears. take_up_contract() derives each flag's engine facts (engine_class: model_simulated vs data_seeded) from pinned-engine metadata, and test_us_take_up_contract.py fails when a checked-in fact drifts from the pinned engine — the #301 formula-owned-guard doctrine applied to take-up. So when PE-US #76's aca_take_up_seed lands, that test fires rather than silently flipping ACA from data_seeded to model_simulated; the engine_state_note on the contract confirms the pinned engine carries no such seed today, so the flag is data_seeded right now. That gives the boundary a home and an enforcement point: add a treatment class for the latent-parameter case (populace persists θ; the engine reads θ in a randomness-free choice formula) distinct from both seed (static flag) and model_simulated (engine owns the draw), and let the contract test guard the transition. The contract's provenance_rule extends cleanly too — the elasticity/slope that turns θ into a reform response must itself be traceable to an administrative or published source (HIPSM's calibrated elasticities), or it is the rate_unsourced / ledger#77 fabricated-parameter class again, just one derivative up.

   3. The release-as-reproducible-artifact principle. Persisted θ makes reform response reproducible in the sense the issue itself requires — the same households switch under the same reform, run over run — and reform-visible, because the response is a deterministic function of a persisted column that a promotion/validation suite can pin as an artifact property. An engine-time draw couples the result to engine RNG state and re-randomizes; that response can't be certified as a property of the artifact. θ-as-data keeps the behavioral layer inside the reproducibility boundary the release is promoted against.

   Make the consistency constraint a tested invariant, not an aspiration

   The sharpest line in the proposal is "the utility layer must be a refinement of the calibrated baseline, never a second opinion on it." I'd harden it into a fixed-point test: θ is backed out at baseline so that choice(θ, baseline prices) reproduces the count-calibrated flag exactly, and CI asserts that composing the engine choice formula over the persisted θ on the baseline dataset returns the persisted baseline flag. Without that test the utility layer can quietly move the baseline enrollment that #331/#332/#334 calibrated — the #334 off-domain-propensity worry, inverted. The invariant is what lets the layer be "a refinement," and only a test makes it true release over release.

   Migration path (baseline-preserving, per-program)

   • v0 (today): static flags / draw columns. Keep them; they stay the baseline anchor.
   • v1: add θ as a frame column beside the flag, seeded with the same stable-identity machinery current stages use (blake2b source-identity keying, Assign SNAP take-up from reported receipt and the FNS participation rate #294), backed out so baseline choice reproduces the flag. Add θ to the release-required person/unit column manifest (the Carry SNAP eligibility and exemption inputs through Populace US outputs #350 pattern) so a release missing it fails rather than defaulting to no-response.
   • v2: introduce the engine choice formula behind an in_effect toggle defaulted off, so baseline artifacts are unchanged until the layer is validated; enable per-program once the fixed-point test and an elasticity check pass.
   • Sequencing: ACA first is the right instinct (richest menu/prices; Investigate ACA take-up and plan-choice inputs driving high PTC estimates #70/Implement ACA marketplace take-up input materialization #74/Migrate ACA health-input gates after PE-US take-up contract changes #76 already active), but it should follow reconciliation with Migrate ACA health-input gates after PE-US take-up contract changes #76's engine-side aca_take_up_seed — otherwise v2's engine choice formula and Migrate ACA health-input gates after PE-US take-up contract changes #76 collide on who writes the flag, which is precisely the drift this issue wants to pre-empt.

   Open questions I would not want the doc to overclaim

   1. Identifiability. A single baseline moment pins one point of θ's distribution; the distributional form — and therefore the reform slope — is an external assumption the calibration does not identify. Many θ distributions reproduce the same baseline count. The design should say plainly that reform-response credibility rests on the elasticity literature, with calibration pinning only the intercept.
   2. θ removes the randomness coupling, not the model-version coupling. The choice function still lives in the engine, so results still depend on its formula version (guarded, at least, by the contract drift test). That is fine and normal for versioned rules, but the doc should claim "decoupled from engine RNG and re-randomization," not "decoupled from the engine" — otherwise the promotion-suite guarantee is oversold.
   3. Cross-program correlation. Medicaid/ACA/SNAP take-up tastes are correlated, and HIPSM models the joint choice; independent per-program θ misses substitution across the coverage menu. A per-program v1 is a defensible start, but name the independence assumption as a known limitation rather than letting it be implicit.
   4. Entity of assignment. θ has to be seeded at the deciding entity (spm_unit for SNAP, tax_unit/person for ACA); getting that wrong reintroduces the cross-entity broadcast hazards the geography slicing already had to navigate.

   On the interim rule (ask #2)

   Agree, and it is already the de facto state — the pinned engine carries no aca_take_up_seed and no formula on the ACA flag, so "flags stay data-seeded; no engine formula bakes in take-up until the design lands" is holding the current line, not adding a constraint. The one addition I'd make explicit: name test_us_take_up_contract.py as the enforcement point, so an accidental engine-owned migration is caught in CI rather than in a release diff.

   Net: I'd green-light the persisted-θ / deterministic-engine-choice architecture as the target, gate it behind the fixed-point invariant and an in_effect toggle, extend the take-up contract's vocabulary and drift test to cover the latent-parameter class, and settle the ACA-vs-#76 ownership before any engine formula writes a take-up flag.