test(integration): TestNightly<Suite> convention + no-match guard for the consolidated nightly#490
Conversation
Running every nightly suite in one process (one CronJob, one -test.run regex over the whole set) means one os-level environment: the three suites whose required image can't be the shared SEID_IMAGE default now read their own env var name instead of colliding on it. - TestBenchmark: SEID_IMAGE -> SEID_IMAGE_MOCK (mock_balances flavor) - TestChaosSuite: SEID_IMAGE -> SEID_IMAGE_CHAOS (mock_chain_validation-mock_balances flavor) - TestChainUpgrade: SEID_IMAGE/SEID_UPGRADE_IMAGE -> SEID_UPGRADE_FROM_IMAGE/SEID_UPGRADE_TO_IMAGE (a pinned historical version pair, never "latest") release_test.go, workflow_test.go, giga_migration_test.go, and giga_mixed_release_test.go are unchanged: all four already want the same vanilla nightly image and keep reading the shared SEID_IMAGE. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…htly run Renames the seven suites the nightly schedule runs (TestBenchmark, TestChaosSuite, TestChainUpgrade, TestRelease, TestWorkflowStateSync, TestGigaStoreMigration, TestGigaMixedRelease) to TestNightly<Suite>, so `-test.run '^TestNightly'` selects the whole nightly set by convention — a new suite joins the schedule by taking the TestNightly prefix, with no separate allowlist to maintain in this repo or the platform CronJob. TestGenesisCeremonyProducesBlocks keeps its name: it's an onboarding check, run standalone, never nightly. Builds on the prior commit's per-suite image env vars (the actual one-process, one-environment constraint the combined run has to satisfy; this commit is purely the selection mechanism on top of that). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
PR SummaryLow Risk Overview Naming: Seven scheduled suites are renamed to One process, multiple images: Benchmark, chaos, and upgrade suites read dedicated env vars ( Safety: Refactor: Reviewed by Cursor Bugbot for commit 8d68711. Bugbot is set up for automated code reviews on this repo. Configure here. |
Two bar-raisers from the systems + reuse review of the consolidation: - TestMain no-match guard: go test exits 0 when -test.run matches nothing, which against a live cluster (SEI_NODE_CLUSTER set) means the harness binary and the pattern are out of sync (an image built before a suite rename) — a false-healthy nightly that ran nothing. TestMain now fails the run when zero suites started against a cluster. Counts executed suites (incremented in requireCluster, the universal entry point) rather than enumerating registered tests, so it needs no per-suite wiring and upholds the TestNightly-prefix-is-the-only-join-step invariant. Bounds the zero-match case only; positive "all 7 ran" proof is the completion SLI on the metrics layer. - Extract witnessNamespace + assertWitnessFresh into harness_test.go: the aggregate-RPC-host namespace derivation and the witness-freshness gate were copied verbatim between the state-sync and giga-mixed suites. Both now call the shared helpers; giga-mixed's witness section is reordered to match (derive ns, build the set, assert freshness) for one reading order. Panic-isolation (per-suite recover) was considered and declined: it is per-suite wiring a new suite can silently forget, and the completion SLI detects a dropped suite regardless of cause (panic, OOM, timeout) — the robust layer for that concern is the metric, not scattered recovers. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
dd0c279 to
8d68711
Compare
Summary
Prepares the integration harness for the platform-side consolidation of the 5 per-suite nightly CronJobs into one. Test-only; no controller/CRD/SDK change.
SEID_IMAGEdefault read their own:SEID_IMAGE_MOCK(benchmark),SEID_IMAGE_CHAOS(chaos),SEID_UPGRADE_FROM_IMAGE/SEID_UPGRADE_TO_IMAGE(upgrade's pinned pre/post pair). Release/workflow/giga/giga-mixed keep the sharedSEID_IMAGE.TestNightly<Suite>naming convention — all 7 nightly suites renamed so-test.run '^TestNightly'selects the whole set by convention; a new suite joins by taking the prefix, no allowlist to maintain.TestGenesisCeremonyProducesBlocksstays outside the prefix (onboarding check, run standalone).go testexits 0 when-test.runmatches nothing; against a live cluster that means the harness binary and the pattern are out of sync (an image built before a rename) — a false-healthy nightly that ran nothing.TestMainnow fails the run when zero suites started against a cluster. Counts executed suites atrequireCluster(the universal entry point), so no per-suite wiring.witnessNamespace+assertWitnessFreshextracted toharness_test.go; the namespace-derivation and witness-freshness gate were duplicated verbatim between the state-sync and giga-mixed suites (a must-change-together correctness coupling).Companion / sequencing
The platform PR (sei-protocol/platform#feat/nightly-cron-consolidation) collapses the 5 CronJobs into one
nightly-harness-suiterunning-test.run '^TestNightly'. Merge order matters: this PR must land first so its ECR push-build produces a harness image carrying theTestNightly*names; the platform CronJob then pins that image atomically. The no-match guard is the backstop if that ordering ever slips.Deliberately not included (recorded in the review ledger): per-suite panic recovery — the completion SLI (count of
--- PASS/FAIL: TestNightly== 7, an observability follow-up) is the robust layer for detecting a dropped suite, not scattered recovers.Review
xreview ledger:
bdchatham-designs/designs/seinode-task/xreview/workflow-complete-at-release.mdrounds 20-22. Four-lens cross-repo review (kubernetes-specialist, platform-engineer, sre-engineer as assigned dissenter, idiomatic-reviewer) + a systems-engineer raise-the-bar pass + a verify round on the guard/refactor (systems + idiom, both RATIFY). Gates: gofmt,go vet -tags integration,go test -c -tags integration,-test.run '^TestNightly'selection all clean.🤖 Generated with Claude Code