fix(test): make UBI fast-seed rung test time-independent (anchor to now)#624
Conversation
There was a problem hiding this comment.
Code Review
This pull request improves the resilience of the integration test test_demo_seeding_ubi_fast.py against transient Elasticsearch connectivity issues on CI runners by verifying that seeded events are searchable before asserting the rung and adding a retry loop for the classification step. The review feedback suggests enhancing error handling in these new additions: first, by catching TypeError when parsing the Elasticsearch response to handle unexpected JSON structures gracefully, and second, by wrapping the classify_rung call in a try...except block within the retry loop so that transient network exceptions do not prematurely fail the test.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
test_synthetic_ubi_seed_round_trip_hits_rung_3 has been red on main since ~2026-06-28: it hardcoded seed_anchor_iso='2026-05-29', but classify_rung counts events only within a trailing [now - 30d, now] window. ~30 days after the anchor date every seeded event falls outside the window, the count drops to 0, and the rung collapses to rung_1 — deterministically, for everyone, regardless of ES health. (It surfaced on the 2026-07-11 security PR simply as the first CI run past the time-bomb.) Fix: anchor the synthetic events to (now - 1 day) so they always land inside classify's window — matching the real demo paths (demo_seeding.py, seed_meaningful_demos.py), which already anchor at now. This is the only call site that runs the real now-windowed classify_rung against a hardcoded past anchor. Unblocks 'backend (contract + integration)' as a future required status check under the new main branch protection. Signed-off-by: SoundMindsAI <eric.starr@soundminds.ai>
38e0b5f to
9811bb6
Compare
Review adjudication (Gemini Code Assist)Both inline comments target the first revision of this branch (a searchable-
|
Summary
test_synthetic_ubi_seed_round_trip_hits_rung_3has been red onmainsince ~2026-06-28 — a deterministic time-bomb, not an ES flake (my initial diagnosis on PR #623 was wrong; see below).The test hardcoded
seed_anchor_iso="2026-05-29", butclassify_rungcounts events only within a trailing[now - 30d, now]window (DEFAULT_READINESS_WINDOW_DAYS = 30). Once "now" drifts >30 days past the anchor, every seeded event falls outside the window → count 0 → rung collapses torung_1, for everyone, regardless of ES health. It surfaced on the 2026-07-11 security PR simply because that was the first CI run past the bomb.Fix
Anchor the synthetic events to
now - 1 dayso they always land inside classify's window — matching the real demo paths (demo_seeding.py:1448,seed_meaningful_demos.py:2469), which already anchor atnow. This is the only call site that runs the real now-windowedclassify_rungagainst a hardcoded past anchor (the unit tests anchor-relative internally, so they're unaffected).Diff is two lines of substance: a
datetimeimport and the relative anchor.How the real cause was found
The first attempt on this branch added a searchable-
_countverify + classify retry, on the theory that an ES "name resolution" flake was under-counting the read. CI then reported "verified 640 searchable in ES … got rung_1" — proving the events were present and searchable, so the flake theory was wrong and the count was a window problem, not a presence problem. That scaffolding was reverted; this PR is just the anchor fix.Test plan
ruff check+mypy+py_compilecleanbackend (contract + integration)— now expected to pass (events land inside the classify window)Once green and stable,
backend (contract + integration)becomes a candidate for a required status check under the newmainbranch protection.🤖 Generated with Claude Code