Skip to content

Epic #242 S3+S4: HydraNet roster reconfig + roster-conformance + D×K sample-count contract - #367

Open
Polichinel wants to merge 6 commits into
developmentfrom
chore/s3-roster-reconfig
Open

Epic #242 S3+S4: HydraNet roster reconfig + roster-conformance + D×K sample-count contract#367
Polichinel wants to merge 6 commits into
developmentfrom
chore/s3-roster-reconfig

Conversation

@Polichinel

Copy link
Copy Markdown
Collaborator

Epic #242 story S3 (#246), with the S4 sample-count contract reconciliation folded in (it's the only path to a green suite — the roster's D×K sampler trips the old contract).

S3 — roster + tidy-up

  • Reconfigure the 8 HydraNets to the LOCKED roster (pre-registration 05, 2026-08-08):
    gated_NB (nb, soft_gate: violet 42 / bright_starship 43 / bold_comet 44),
    th_gated_NB (nb, threshold_gate 0.5: blazing_meteor 45 / heavy_freighter 46),
    mixture_NB (mixture_nb, soft_gate: pink_pirate 42 / blue_stranger 43 / purple_alien 44).
    All share the v2 foundation (mse · softplus · body_supervision=all · weighted_bce pw2 ·
    rollout_feedback=sample · bn_recalibrate · D×K=4×4 · lr_/by_ targets), all datafactory /
    africa_me_legacy. rusty_bucket points at all 8 (concat, 8×16=128) and declares
    classification_targets for the C-132 gate pooling.
  • Retire golden_hour + stellar_horizon — the two 3-member C-47 parity ensembles, superseded
    by the 8-member rusty_bucket (the viewser↔datafactory comparison they existed for is moot; S2
    migrated all three viewser models to datafactory).
  • Replace test_datafactory_parity.py (trio-mirror, tobit) with test_roster_conformance.py:
    pins each model to its roster family + the shared v2 foundation + grid/targets + datafactory,
    and the rusty_bucket ensemble incl. the C-132 gate channel + no stale targets key.
    EXPERIMENT_IN_PROGRESS is now unused (violet is pinned, not skipped).
  • Downstream consequence fixes: stale-loss docstring guard repointed; delivery-map shadow
    count 117→115; env-sharing pin views_ensemble 13→11.

S4 — sample-count contract (ADR-015 §6)

An ADR-067 family head emits D×K = n_posterior_samples × n_head_samples = 16 draws per cell,
not D=4. The contract equated "samples per model" with n_posterior_samples alone, so every
family model's on-disk width (16) and rusty_bucket's expected_samples_per_model (16) tripped it
(9 red tests). Fix: a shared conftest.get_produced_sample_count (D×K; K defaults to 1 for
non-family models — pre-#242 behaviour byte-identical), routed through the ensemble contract, the
PF output-width checks, and the drift report. ADR-015 §6 records the reconciliation.

Register

C-132 updated with the durable fix (see cross-repo PR below); C-71 noted the roster landed and
flags Cluster E (C-47/C-71/C-72/C-74/C-87) for a /review-rr pass.

Verification

Full suite: 7574 passed, 0 failed; ruff clean. No prediction artifacts or the transient
gate-hack committed.

⚠️ Merge-together dependency

The classification_targets declaration only actually pools the gate once the framework fix lands:
views-platform/views-pipeline-core#422 (C-132). Merge these two together.

🤖 Generated with Claude Code

Polichinel and others added 5 commits August 10, 2026 00:37
…the occurrence/gate channel; every HydraNet ensemble AP understated

Discovered in Epic #242 S6 (views-hydranet ensemble dossier EXP-03). The pool loops
ctx.targets = config.targets|regression_targets; HydraNet ensembles declare only lr_*
(no by_* classification), so concat never pools the gate. AP crippled (sb-h1 0.316 vs
members 0.38-0.47); recovered to near-best (0.456) by pooling by_* — no retrain, no code
change. Fix direction: pool the occurrence channel by default for gated ensembles + a test.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ter + wire rusty_bucket

S3 (#246). The fleet moves from the pre-epic tobit configs to the v2 gated_NB roster
(pre-registration 05, LOCKED 2026-08-08):

  gated_NB     (nb,         soft_gate)          violet 42 / bright_starship 43 / bold_comet 44
  th_gated_NB  (nb,         threshold_gate 0.5) blazing_meteor 45 / heavy_freighter 46
  mixture_NB   (mixture_nb, soft_gate)          pink_pirate 42 / blue_stranger 43 / purple_alien 44

All eight share the v2 foundation (mse · softplus · body_supervision=all ·
weighted_bce pos_weight 2 · rollout_feedback=sample · bn_recalibrate · D×K=4×4 ·
lr_*/by_* targets) and all read views-datafactory / africa_me_legacy (S2 migrated the
three viewser models; heavy_freighter scoped global→africa for the local run, its
global template banked in the hydranet dossier).

rusty_bucket now points at all eight (concat, 8×16=128 pooled) and declares
`classification_targets: [by_*]` so the pool carries the occurrence gate (C-132) —
the durable form of the transient `targets` gate-hack, which the #380-fixed manager
now refuses.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…r parity with roster-conformance

S3 (#246). The viewser↔datafactory trio-mirror parity programme (C-47) is resolved:
S2 migrated all three viewser models to datafactory, so "two loss-identical trios
compared via golden_hour vs stellar_horizon" no longer describes reality — the roster
reorganised every model by family across both trios and moved them all off tobit.

- Retire the two 3-member parity ensembles golden_hour + stellar_horizon (superseded
  by the single 8-member rusty_bucket; the comparison they existed for is moot).
- Replace test_datafactory_parity.py (trio-mirror, tobit) with test_roster_conformance.py:
  pins each model to its roster (family / composition / gate / seed), the shared v2
  foundation, identical grid + targets, datafactory source, and the rusty_bucket
  8-member ensemble incl. the C-132 classification_targets gate-pooling + no stale
  `targets` key. EXPERIMENT_IN_PROGRESS is now unused (the roster is pinned, not skipped).
- Downstream consequence fixes: repoint the stale-loss docstring guard to the new file;
  delivery-map shadow count 117→115; env-sharing pin views_ensemble 13→11.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… sampler (ADR-015 §6)

S4. An ADR-067 HydraNet family head draws K = n_head_samples from the distribution per
D = n_posterior_samples MC-dropout pass, so the EMITTED posterior width per cell is
D×K, not D. The roster runs D=4 × K=4 = 16 (rusty_bucket pools 8×16 = 128) — but the
sample-count contract still equated "samples per model" with n_posterior_samples (4)
alone, so every family model's on-disk y_pred width (16) and rusty_bucket's
expected_samples_per_model (16) tripped the contract (9 red tests).

- conftest: add get_produced_sample_count = n_posterior_samples × n_head_samples
  (K defaults to 1 for every non-family model, so pre-#242 behaviour is byte-identical).
- Route the ensemble expected_samples_per_model contract, the PF output-width checks
  (test_pfe_production_readiness), and the non-blocking drift report through the
  produced count.
- ADR-015 §6 records the reconciliation: the two knobs stay distinct (D = MC-dropout
  depth, K = family draws per pass); the produced width is their product; the 128
  standard is a target on the produced count.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ion note

- C-132: record the durable fix (incomplete #380 migration; both _build_context sites
  now route through combined_targets, pipeline-core branch chore/c132-pool-gate) and
  the durable config form (classification_targets, pinned by test_roster_conformance).
  Kept Open pending merge of both PRs.
- C-71: the roster LANDED (S3) — the trio-mirror parity premise this Cluster-E entry
  guards is retired (parity test replaced, golden_hour/stellar_horizon retired, violet
  pinned not skipped). Flag Cluster E (C-47/C-71/C-72/C-74/C-87) for a /review-rr pass.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…t declare the gate

A falsification pass (2026-08-10) showed the C-132 fix is config-gated, not automatic:
a prediction_frame / hydranet_ucdp concat ensemble that omits classification_targets
still silently drops the by_* occurrence gate, and only rusty_bucket was pinned.
test_every_hydranet_pf_ensemble_declares_the_occurrence_gate generalises the check
across all ensembles — green today (rusty_bucket declares it), fails loud the moment a
gate-less HydraNet ensemble is added, so C-132 cannot silently recur (e.g. on the
global ensemble).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@Polichinel

Copy link
Copy Markdown
Collaborator Author

⚠️ Disclosure: this PR was authored by an AI agent. Please read the self-critical review notes (cross-repo dependency, un-ratified upstream ADR-067, governance edits) before evaluating it: #368.

Polichinel added a commit that referenced this pull request Aug 10, 2026
feat(#242): roster conformance for 7 models, retire the parity suite (S3, split from #367)
Polichinel added a commit that referenced this pull request Aug 10, 2026
docs(ADR-015): §6 — the count that matters is the count a model produces (S4, split from #367)
@Polichinel

Copy link
Copy Markdown
Collaborator Author

Superseded in part. Two pieces of this are now on development, split per the review in #368's thread:

Three deliberate departures, each with a reason:

  1. test_roster_is_fully_pinned was dropped. It asserts EXPERIMENT_IN_PROGRESS is absent, which converts an escape hatch into a prohibition. test_datafactory_parity.py was the only reader of that marker anywhere in the repo, so deleting the file would have removed the fence silently. feat(#242): roster conformance for 7 models, retire the parity suite (S3, split from #367) #369 re-homes it as a set-pin, so adding or removing an exemption is a reviewed edit.
  2. violet_visitor was not reconfigured. It is fenced, mid-experiment, and there is uncommitted maintainer work in its config_queryset.py. It is exempt from the value pins and the source-migration checks, and remains subject to every structural check.
  3. The rusty_bucket rewiring is deferred — a consequence of (2), not a preference. The ensemble declares expected_samples_per_model: 16; the seven pinned members emit D×K = 16; violet emits 8. Rewiring now fails the ADR-015 contract test, correctly.

Two findings from reviewing this that are not in #368:

  • Deleting golden_hour/stellar_horizon leaves investigations/compare_parity.py:26ENSEMBLE_PAIR = ("golden_hour", "stellar_horizon") — pointing at removed directories. investigations/ is not under CI, so the five green checks say nothing about it.
  • ensembles/golden_hour/reports/preanalysis_plan_calibration.md (208 lines) is the sole evidential basis for register entries C-44, C-45, C-46, C-47. This PR deletes it.

Still to come from this branch: the two ensembles are being retired to README-only reusable slots rather than deleted outright (maintainer's call — they become slots for future HydraNet ensembles), with the preanalysis plan archived first. The C-132 work waits on views-pipeline-core#422 merging and a release being pinned here.

Closing this in favour of the split is reasonable once the remaining pieces land; leaving it open as the reference is also fine. Your call.

@Polichinel

Copy link
Copy Markdown
Collaborator Author

Finding from the views-pipeline-core side, on the deferred rusty_bucket scope. Not a request — your call.

The classification_targets addition to ensembles/rusty_bucket/configs/config_meta.py will be rejected by CoreConfigSniffer at config load, and this is independent of whether pipeline-core#422 merges.

Run against the real sniffer, three variants:

today (regression only)              : PASSES
this PR's config as written          : RAISES
  -> "classification_targets is non-empty but none of
      {classification_point_metrics, classification_sample_metrics}
      are present. Add at least one classification metric key."
this PR's config + a classification metric : PASSES

_check_targets_and_metrics runs inside sniff_all, and both ensemble managers call it before any side effect (prediction_frame_ensemble.py:328, ensemble.py:158). So this is a load-time refusal, not a runtime surprise.

Suggested fix, entirely your call: add a classification metric key alongside the targets — e.g. "classification_sample_metrics": ["AP"] — since AP is the metric the gate-pooling work is measured on anyway. Whether AP is the right choice is your judgement; the sniffer only requires that some classification metric is declared when classification targets are.

Why I am flagging it now rather than after #422 merges: the discussion on this PR and on pipeline-core#427 both characterise the pipeline-core change as the blocker and this config as inert until it lands. That is half right. The pipeline-core change is needed for the gate to actually be pooled — but the config as written does not get that far, because it does not load. Merging #422 first would not reveal it; the config would still be refused.

One correction to something I said on pipeline-core#427, for the record: I claimed the failure mode here would be a silent wrong answer — the gate quietly dropped. It is not silent. The sniffer refuses loudly, which is the better outcome and means this is a smaller problem than I described.

Separately: the gate-pooling effect itself is real and measured — views-hydranet's EXP-03 log records h1 AP moving 0.316 → 0.456 (sb), 0.177 → 0.355 (ns), 0.135 → 0.225 (os) once the gate is pooled. I had earlier suggested #422 could not deliver the improvement it claimed; that was wrong, and the measurement is what changed my mind.


Verified 2026-08-10 by running CoreConfigSniffer._check_targets_and_metrics against all three config shapes. Cross-ref: pipeline-core #422, #427, core_config_sniffer.py:261; your #368; views-models#371 (the CI gap that would have caught this).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant