Skip to content

test: make the Recall heartbeat planning setup and locked-store checks deterministic - #1538

Merged
kvlonge merged 1 commit into
mainfrom
fix/recall-test-determinism
Nov 17, 2026
Merged

kvlonge merged 1 commit into
mainfrom
fix/recall-test-determinism

Conversation

@kvlonge

@kvlonge kvlonge commented Nov 17, 2026

Copy link
Copy Markdown
Contributor

Why

Two Recall tests dropped merge-queue entries on unrelated changes. The ClickHouse invocations plan test still had a race: its setup heartbeat set only invocation_id, so its start time defaulted to 1970. The table's TTL drops such a row as it is inserted and removes the empty part about 2 s later, after which the plan reads nothing from recall_heartbeats whenever the build wrote no heartbeat of its own. The locked-store test checked only the notice and a hang guard, not the work the locked build did.

Changes

  • The setup heartbeat now starts at now64(6), so the table keeps its part until the plan is read.
  • The locked-store test counts work: the locked build's notice reports dropped records and no failures, and after the lock is released only the builds that ran unlocked are recorded. No wall-clock budget is involved.

DuckShop: no case applies; this changes test setup and assertions only, not a project shape, data value or lifecycle sequence.

Verification

  • On ClickHouse 26.1 locally, the old setup row loses the heartbeats read after 2 s in 6 of 6 runs and the new one keeps it up to 6 s in 6 of 6. With a build that wrote no heartbeat simulated, the e2e test fails 3 of 3 with the old setup and passes 3 of 3 with the new one.
  • With the store lock removed from the locked-store helper, both new checks fail.
  • Both test files pass against a local ClickHouse 26.1 server; make check-ci passes.

…s deterministic

The ClickHouse invocations plan test gave recall_heartbeats a part with a
setup row that set only invocation_id, so its start time defaulted to 1970.
The table's TTL removes such a row as it is inserted and drops the empty
part shortly after, so whenever the build wrote no heartbeat of its own the
plan read nothing from the table. The setup row now starts now.

The locked-store test also counts the work the locked build did: its
closing notice reports dropped records and no failures, and after the lock
is released only the other builds are recorded.
@chio-labs-delivery
chio-labs-delivery Bot added this pull request to the merge queue Nov 17, 2026
@kvlonge
kvlonge removed this pull request from the merge queue due to a manual request Nov 17, 2026
@kvlonge
kvlonge added this pull request to the merge queue Nov 17, 2026
Merged via the queue into main with commit 6cf94f3 Nov 17, 2026
32 checks passed
@kvlonge
kvlonge deleted the fix/recall-test-determinism branch November 17, 2026 07:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant