gate: per-cell budget for the mis-budgeted GMM row; exclude the confirmed adaptive-alloc regression from the probe

[oshaughnessy-junior]

Two things, both driven by measurements that finished overnight. The second corrects a claim I made repeatedly in #47, #51 and #55.

1. The mis-budgeted strict cell -- measured and fixed

GMM mix_d6_n3_s303 sits on the n_eff = 100 starvation floor at the matrix budget, so as a merge-blocking row it is close to a coin flip on every branch. 8 fresh seeds per budget:

budget	n_eff min / med / max	clears 100	median |bias|
x1 (matrix default)	59 / 105 / 159	5/8	0.014
x2	105 / 169 / 214	8/8	0.010
x4	209 / 293 / 473	8/8	0.012

Bias is flat across budgets, so this is threshold margin, not a defect in the cell.

Correction to the two options I offered earlier: neither was expressible. The matrix budget is nmax_per_dim * d for every cell, and --strict-samplers is per-sampler, not per-cell -- so "re-budget this cell" and "drop this row from strict" both needed a mechanism that did not exist. (Dropping GMM from strict would have dropped all 24 GMM rows.)

This adds CELL_BUDGET_MULT, in the same shape as the existing per-case WARM_CASES budgets, and sets this cell to x4. Why x4 and not x2: x2 clears 8/8, but its minimum is 105 -- 5% above the floor, which is not a margin worth trusting for a row that has read 66 and 119 on unchanged code. x4 gives min 209 (2.1x margin) and costs ~4% of the gate's total evaluations, being one cell of ~96.

2. --portfolio-adaptive-alloc is a CONFIRMED regression on that target

Re-running the probe confirmation on the fixed probe reverses what I reported in three previous PRs. I described adaptive_alloc ON / d4_n1_s303 as pre-existing realization noise, "verified against the base." That verification ran on the buggy probe, which compared the flag against itself and structurally could not report "worse". It was never evidence.

seed 988654: base=PASS(213)    flag=FAIL(124)
seed 989654: base=PASS(252)    flag=FAIL(125)
seed 990654: base=PASS(244)    flag=FAIL(292)
seed 991654: base=STARVED(67)  flag=FAIL(177)
seed 992654: base=PASS(136)    flag=FAIL(140)
-> CONFIRMED (4 worse / 1 not-worse)

The flag arm fails at every seed, and frequently with higher n_eff than the default arm (292 vs 244, 177 vs 67, 140 vs 136). So it is failing on shape metrics -- more effective samples, worse recovered posterior. That is the "confidently wrong" signature documented elsewhere in this work (n_eff measures weight concentration, not coverage), now appearing in one of our own opt-in features.

Production is unaffected: the flag is opt-in, defaults off, and the pipeline never sets it. But it is not safe to promote, and the higher n_eff must not be read as evidence it helps -- that is exactly the reading that made the estimator-clip experiment look like a success while it was biasing lnZ by -11.5 nats.

Recorded as FOLLOWUPS.md item 4 with the scoping work needed: sweep the flag across the full matrix at several seeds, recording JS / pull / width alongside n_eff, to establish whether the allocation signal systematically over-concentrates on whichever member reports the best per-chunk n_ess. Deliberately not attempted here -- it is an investigation, not a gate-tuning change.

3. Those two probe rows are now EXCLUDED from the probe

Follow-up to section 2, after review discussion. The adaptive_alloc ON and adaptive+clip ON rows are commented out of FLAG_CONFIGS in probe_portfolio_optin_flags.py, so the probe keeps working as a regression detector for the flags that pass instead of becoming a permanently red row people learn to scroll past.

This is containment, not a fix. The exclusion is written as commented-out config lines (greppable), recorded in FOLLOWUPS.md item 4, and asserted by test_adaptive_alloc_is_excluded_from_the_probe_configs so it cannot be quietly lost. Reinstating those two lines is step one of any fix, and they will fail until the flag is actually repaired. Production is unaffected meanwhile -- opt-in, defaults off, pipeline never sets it.

Probe on d4_n1_s303 with the reduced list: weight_clip ON, varaha floor .25, varaha band .25-.75, band + gmm cap3, all base=PASS flag=PASS, 0 flagged rows.

FLAG_CONFIGS was hoisted to module level in the same change, because it had to be: the confirmation-machinery tests keyed on portfolio_adaptive_alloc and, once it was removed, passed by having nothing left to flag. They now inject a synthetic config list rather than coupling to whichever flags happen to ship.

4. Review fixes to the section-1 mechanism

Both P2 findings on CELL_BUDGET_MULT were right, and the second was more than cosmetic.

The override was bypassed by the pytest entry point. The matrix has two of them -- shape_recovery.main() (what run_shape_recovery.sh drives) and the parametrization in test_shape_recovery.py -- and the first cut applied the table only in main(). The one cell this table exists to un-starve was therefore still starved under RIFT_SHAPE_PRESET=standard pytest, which the suite documents as an equivalent way to run the matrix.

It also rescaled an explicit --nmax-per-dim. The CLI documents nmax = this * ndim, and the x1/x2/x4 study that produced the x4 figure in section 1 passes --nmax-per-dim -- so it would silently have been measuring x4/x8/x16 under the labels x1/x2/x4.

Both paths now go through one cell_budget() helper; an explicit --nmax-per-dim disables the table and prints that it did; --no-cell-budget-mult disables it at preset defaults. Verified by capturing what main() actually dispatches to run_one, rather than by recomputing the formula:

path	budget for GMM d6_n3_s303
cell_budget() (pytest)	4800000
main(), preset default	4800000	agrees; x4 live
main() --nmax-per-dim 200000	1200000	= npd*ndim, override off
main() --no-cell-budget-mult	1200000	= npd*ndim, override off
main(), non-override cell d4_n1	800000	untouched

Also

FOLLOWUPS.md items 1-3 marked resolved with outcomes (probe confirm-on-fail #54, _rvs cache audit #55, ordering constraint #57).

Tests: 46 in expensive_before_merging/integrators (28 of them test_probe_confirm.py) + 36 in test/integrators (test_portfolio_restrict_and_warm 22, test_replica_pooling 9, test_rvs_weight_derivation 5), all passing.

On re-running the full gate: no RIFT library code is touched by this PR -- the diff is the gate harness, its tests and FOLLOWUPS.md -- so a base-vs-candidate run would be comparing identical integrator code and is a no-op by construction. What a run would still show is the runtime cost of the x4 cell, which is the thing worth knowing before merge; happy to run it for that if you want the number, but it is not a correctness gate here.

[oshaughnessy-junior]

Gate result

COMPARE_EXIT=0, 0 blocking regressions. The re-budgeted cell now reads n_eff = 355 and is comfortably clear of the 100 floor, versus a median of 105 and 5/8 clearing at the matrix budget.

Worth stating explicitly, because it looks odd: both arms report the identical 355. That is correct and by design -- run_shape_recovery.sh always executes ${HERE}/shape_recovery.py, i.e. the candidate's harness, and only puts the arm's checkout on PYTHONPATH. So the harness (including CELL_BUDGET_MULT) is common to both arms while the library under test differs, which is what keeps the comparison apples-to-apples. The practical consequence is that the x4 applies twice, so the real cost is ~8% of total gate evaluations rather than ~4%. Still one cell of ~96.

The probe now fails -- correctly, and this needs a decision

PROBE_EXIT=1, 1 confirmed opt-in regression:

adaptive_alloc ON  d4_n1_s303   base=PASS  flag=FAIL   <-- FLAGGED
   seed 988654: base=PASS(213)    flag=FAIL(124)
   seed 989654: base=PASS(252)    flag=FAIL(125)
   seed 990654: base=PASS(244)    flag=FAIL(292)
   seed 991654: base=STARVED(67)  flag=FAIL(177)
   seed 992654: base=PASS(136)    flag=FAIL(140)
   -> CONFIRMED (4 worse / 1 not-worse)

This is the probe working as intended for the first time. It is not caused by this branch -- the regression is in --portfolio-adaptive-alloc, which is opt-in, defaults off, and is never set by the pipeline.

But it will now turn every subsequent PR red, because the flag is genuinely broken on that target and the probe no longer masks it. That is a policy call rather than something to fix here:

1. Fix the flag (needs the scoping investigation in FOLLOWUPS.md item 4 first -- it may be specific to d=4/ncomp=1 or systematic to the allocation signal);
2. Drop adaptive_alloc from the probe's configuration list until it is fixed, with a comment pointing at item 4;
3. Accept a red probe on unrelated PRs in the interim.

My preference is (2): it keeps the probe useful as a regression detector for the other four flag configurations, which currently all pass, and it makes the exclusion explicit and greppable rather than silently tolerated. (1) is the real fix but should not gate unrelated work. I have not done either -- say which and I will implement it.

Everything else in the probe passes: weight_clip, adaptive+clip, varaha floor, varaha band, band + gmm cap3.

[oshaughnessy-junior]

The multiplier is applied only while main() builds jobs. The documented pytest entry point (test_shape_recovery.py:36) constructs its budget directly as _PRESET["nmax_per_dim"] * ndim, so RIFT_SHAPE_PRESET=standard pytest ... still runs this GMM cell at x1. Please centralize budget calculation in a helper and use it from both paths; otherwise the supposedly resolved starvation row remains inconsistently budgeted.

[oshaughnessy-junior]

This multiplier also applies after the user explicitly passes --nmax-per-dim, contradicting the CLI contract that nmax = this * ndim. For example, targeting this cell with --nmax-per-dim 200000 silently runs at 4.8M rather than 1.2M evaluations, which makes controlled x1/x2 measurements misleading. Please apply the override only to preset defaults, or expose and document a way to disable it.

[oshaughnessy-junior]

Review summary: two P2 findings posted inline. (1) The per-cell override is bypassed by the documented pytest gate entry point. (2) It silently changes the meaning of an explicit --nmax-per-dim value. I would request changes, but GitHub does not allow the PR owner to submit a change-request review on their own PR. The two failing container canaries appear unrelated (pygsl_lite/SWIG dependency builds).

[oshaughnessy-junior]

Both review findings addressed, plus the follow-up decision on the failing probe rows (option 2: exclude).

P2 — "the per-cell override is bypassed by the documented pytest gate entry point." Correct, and it defeated the point of the change: the matrix has two entry points (shape_recovery.main(), driven by run_shape_recovery.sh, and the pytest parametrization in test_shape_recovery.py), and the override lived only in the first. So the one cell the table exists to un-starve was still starved under RIFT_SHAPE_PRESET=standard pytest. Both paths now go through a single cell_budget() helper.

P2 — "it silently changes the meaning of an explicit --nmax-per-dim." Also correct, and worse than cosmetic: the CLI documents nmax = this * ndim, and the x1/x2/x4 budget study that produced the x4 figure passes --nmax-per-dim — so with the first cut that study would have been silently measuring x4/x8/x16 while labelling the columns x1/x2/x4. An explicit --nmax-per-dim now disables the table and prints that it did; --no-cell-budget-mult disables it at preset defaults too.

Verified by capturing what main() actually dispatches to run_one (not by recomputing the formula):

path	budget for GMM d6_n3_s303
cell_budget() (pytest)	4800000
main(), preset default	4800000	agrees; x4 live
main() --nmax-per-dim 200000	1200000	= npd*ndim, override off
main() --no-cell-budget-mult	1200000	= npd*ndim, override off
main(), a non-override cell (d4_n1)	800000	untouched

adaptive_alloc probe rows — excluded, not tolerated. The two --portfolio-adaptive-alloc rows are commented out of FLAG_CONFIGS. This is containment of the confirmed regression in FOLLOWUPS item 4 (5 fresh seeds, flag arm fails at every one, at three of them with higher n_eff than the default arm — it degrades shape, not efficiency), so the probe keeps working as a detector for the flags that pass rather than becoming a standing red row people learn to ignore. The flag is opt-in, defaults OFF, and the pipeline never sets it, so production is unaffected meanwhile.

The exclusion is deliberately written as commented-out config lines so it is greppable, is recorded in FOLLOWUPS.md, and is asserted by test_adaptive_alloc_is_excluded_from_the_probe_configs so it cannot be quietly lost. Reinstating those two lines is step one of any fix.

FLAG_CONFIGS was hoisted to module level in the same change: the confirmation-machinery tests were keying on portfolio_adaptive_alloc and so silently stopped exercising anything when it was removed (they passed by having nothing to flag). They now inject a synthetic config list instead of coupling to whichever flags happen to ship.

Validation. Probe on d4_n1_s303 with the reduced list: weight_clip ON, varaha floor .25, varaha band .25-.75, band + gmm cap3 — all base=PASS flag=PASS, 0 flagged rows. Tests: 46 in expensive_before_merging/integrators (28 of them test_probe_confirm.py) + 36 in test/integrators (test_portfolio_restrict_and_warm 22, test_replica_pooling 9, test_rvs_weight_derivation 5), all passing.

No RIFT library code is touched by this PR — the diff is the gate harness and its tests only — so a base-vs-candidate shape gate run would be comparing identical integrator code and is a no-op by construction.

[oshaughnessy-junior]

This test is vacuous after FLAG_CONFIGS moved to module scope. inspect.getsource(PB.main) contains only configs = FLAG_CONFIGS, so it never contains portfolio_adaptive_alloc; uncommenting either known-bad entry in FLAG_CONFIGS would still pass. Please inspect PB.FLAG_CONFIGS directly, for example by asserting the key is absent from every active flags dict.

[oshaughnessy-junior]

evaluate() returns status strings (PASS, FAIL, STARVED, or ERROR), all of which are truthy. Therefore this pytest gate passes even when the sampler fails or errors, undermining the entry point whose budget consistency this PR just fixed. Please unpack as status, reasons and assert status == "PASS".

[oshaughnessy-junior]

Re-review summary: the original two findings are resolved. Two P2 test defects remain and are posted inline: the adaptive-allocation exclusion test inspects PB.main instead of module-level PB.FLAG_CONFIGS, so it is vacuous; and the pytest shape gate asserts a non-empty status string rather than status == "PASS", so failures remain truthy. All current GitHub checks are green.

[oshaughnessy-junior]

Both P2s confirmed and fixed. I verified each by reintroducing the defect first, because a test that cannot fail is exactly what is under discussion.

"The exclusion test is vacuous after FLAG_CONFIGS moved to module scope." Correct — and demonstrably so. Appending the known-bad entry back onto FLAG_CONFIGS left the old form passing:

OLD form (inspect.getsource(PB.main)) catches a re-enabled flag : False   <- vacuous
NEW form (inspect PB.FLAG_CONFIGS)    catches a re-enabled flag : True
   -> "adaptive_alloc re-enabled in the probe configs: ['adaptive_alloc ON']"
still passes on the real config list                            : True

I added a second assertion while there: >= 2 active opt-in arms must remain. Otherwise emptying FLAG_CONFIGS — or reducing it to the flags-off row — would satisfy "adaptive_alloc is absent" while the probe tested nothing at all. That is the same vacuity one level up, and it is the failure mode this whole exchange keeps producing.

"The pytest gate asserts a non-empty status string." Correct, and worse than it looks: that entry point has never enforced anything. evaluate() returned booleans when the test was written in 98c644d1; 6467ac91, later the same day, changed the contract to status strings and updated compare_shape_results.py and shape_recovery.py but not this caller:

-        return False, ["ERROR " + r["error"]]
-        return False, reasons
-    return len(reasons) == 0, reasons
+    return ("FAIL" if reasons else "PASS"), reasons

Every status is truthy, so from 2026-07-22 onward assert ok passed on ERROR, STARVED and all four FAIL paths. Verified against each:

evaluate() →	old assert ok	new
PASS	passes	passes
FAIL (JS / pull / width / lnZ, all four)	passes	blocks
ERROR	passes	blocks
STARVED	passes	skips, visibly

So the budget-consistency fix in this PR was making two entry points agree when one of them could not fail. Worth noting given that was the first finding.

One deliberate deviation from the requested fix, on STARVED. You asked for assert status == "PASS". I implemented FAIL/ERROR assert, STARVED skip, for two reasons:

1. It contradicts the gate's own definition. 6467ac91 is titled "STARVED verdict (non-blocking absolute, differential-only gating)", its message recording that whole d=8 rows legitimately starve at production budgets, so absolute pass/fail is uninformative there. Strict equality makes this entry point disagree with compare_shape_results.py about what STARVED means.
2. Empirically it reds the default preset. Running the strict version:

FAILED test_shape_recovery[GMM-4-2-101]
   AssertionError: GMM on mix_d4_n2_s101: STARVED -- n_eff=42 < 100
1 failed, 3 passed

quick budgets nmax_per_dim=50000, so d=4 gets 200k evaluations and that cell reads n_eff=42 against the 100 floor. It is not a sampler defect — the same cell passes at the standard budget. Shipping a gate that fails out of the box on its default preset trains people to ignore it.

The skip carries the n_eff and says explicitly that it is not a pass, so it stays visible in the pytest summary rather than being silently absorbed:

SKIPPED [1] GMM on mix_d4_n2_s101: STARVED -- n_eff=42 < 100: shape untestable at this
budget [not a pass: use run_shape_recovery.sh + compare_shape_results.py to gate
starvation against a base run]
3 passed, 1 skipped

I am aware this leaves a hole: a branch that starves everything would skip everything and go green. I think that is acceptable because the merge gate is the differential run, not this one — but if you would rather close it, say so and I will either add a session-level guard that fails when more than some fraction of the matrix starves, or re-budget quick so the cell clears (~2.5x on that cell, which costs quick its name). Happy to take the strict version instead if you disagree with the reading of 6467ac91.

The under-budgeted quick cell is recorded as FOLLOWUPS.md item 5, including how it stayed invisible.

Tests: 46 in expensive_before_merging/integrators, 36 in test/integrators, all passing; the pytest shape gate now runs 3 passed / 1 skipped instead of 4 vacuously passed.