fix(ci): re-baseline storyboard floors to measured main - #6885
Conversation
The #6815 coverage rebaseline set current-source floors above what main actually measures (creative 48/50 clean with 200 steps vs 49/209 floor; creative-builder 48 vs 50, 172 vs 184; sales 553 steps vs 556) — every gap is authored known-failing skips citing packaged-runner blockers, with zero step failures. The workflow has been red on main since the merge and the mirrored local pre-push gate blocks every compliance- source push. Set floors to the values measured on main's own CI run so the ratchet grades regressions from reality again. Closes #6876. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
![aao-secretariat[bot]](https://avatars.githubusercontent.com/u/221690821?s=60&v=4){kind=small}
| # reported separately in the aggregate summary. | ||
| MIN_CLEAN: 126 | ||
| MIN_PASSED: 556 | ||
| MIN_PASSED: 553 |
There was a problem hiding this comment.
MUST FIX: Third floor mirror not updated. tests/run-storyboards-sharding.test.cjs asserts the old literals against both files this PR edits and will go red on merge: MIN_PASSED: 556 (L259), "sales:126:556" (L260), and the baselines table ['sales', 126, 556], ['creative', 49, 209], ['creative-builder', 50, 184] (L293/295/296). The PR body says it sets "both floor mirrors" — there are three. Update the test to 553 / 200 / 172 (creative 48, creative-builder 48) or the suite fails. The end-to-end matrix run validated the storyboard run, not these literal assertions.
There was a problem hiding this comment.
Ladon verdict: Request changes
Request changes — 1 blocking finding.
Blocking findings
- tests/run-storyboards-sharding.test.cjs:259 — Stale third floor mirror breaks tests. The PR re-baselines storyboard floors in two mirrors (workflow + matrix script), but a third mirror in the sharding test still asserts the old literals and will fail on merge.
This is a reproducible test failure on a load-bearing path (high), so per decision-table row 1 the outcome is request-changes. Update the floor literals in tests/run-storyboards-sharding.test.cjs to match the new values before merge.
Note: this PR also touches .github/workflows/training-agent-storyboards.yml, a gated path, with review_decision: REVIEW_REQUIRED — human/CODEOWNERS approval will be required regardless.
Blocking findings
- tests/run-storyboards-sharding.test.cjs:259 — Stale third floor mirror still asserts old literals and breaks tests on merge
run-storyboards-sharding.test.cjs pins the (tenant, min_clean, min_passed) tuples across the workflow and the local matrix runner; update its baseline table to the re-measured floors. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
1 participant
Summary
Fixes the storyboard workflow, red on main since this morning's #6815 coverage rebaseline set current-source floors above what main measures. CI's own failing run (32842430342) measures creative 48 clean / 200 steps (floor was 49/209), creative-builder 48 / 172 (was 50/184), sales 126 / 553 (step floor was 556) — every gap is authored known-failing skips citing packaged-runner blockers, with zero step failures anywhere. This sets both floor mirrors (workflow +
scripts/run-storyboards-matrix.sh) to those measured values so the ratchet grades regressions from reality again.Validated end-to-end: the local pre-push matrix ran all seven tenants green with results matching CI's measurements exactly. Closes #6876's floor half; the "local matrix nondeterminism" also reported there turned out to be disk exhaustion on the dev machine (runs degraded 126→117→86 clean as free space fell to 1.9 GB; deterministic and CI-identical after cleanup) — noted on the issue.
🤖 Generated with Claude Code