Skip to content

test: step the scheduler's database clock instead of waiting on wall time - #1534

Merged
kvlonge merged 4 commits into
mainfrom
fix/scheduler-latency-window
Nov 17, 2026
Merged

kvlonge merged 4 commits into
mainfrom
fix/scheduler-latency-window

Conversation

@kvlonge

@kvlonge kvlonge commented Nov 17, 2026

Copy link
Copy Markdown
Contributor

Why

The scheduler latency test checked an exact tick count inside a real-time window, with due times set before migration, location setup and publication. On a slow Windows runner it decided 2 of 3 ticks: either setup ran past a due time, or one cycle stalled long enough for two times to be overdue together. The test could not say which, because it printed its timings only after the count assertion.

Changes

  • Every database time the scheduler reads already comes from one place: the transaction clock that refresh_clock reads. In test builds only, a task-local clock can override it, so a test sets the database time instead of waiting for it. Production code paths and signatures are unchanged.
  • A new stepped-clock test runs one scheduler cycle per step on SQLite and PostgreSQL. Each step asserts leadership, tick count and the wait to the next due time, and the end of the test asserts which due times ticked. The cases are on-time ticks, a standby that stays fenced until the stopped leader's lease lapses and then ticks only the latest overdue time, and a late cycle that folds two overdue times into one tick.
  • One real-clock test remains, with a single due time and a 3 s bound that only a hang misses. It fails clearly if setup ran past the due time, and it prints its cycle timings before asserting.
  • The 500 ms and 1,500 ms wall-clock wake bounds are gone. The stepped clock checks the same behaviour by count and exact wait.

DuckShop: no case added. This is a scheduler test change with no project shape, data value or build lifecycle sequence.

Verification

  • cargo test -p sqlbuild-orchestration --lib -- scheduler:: on SQLite and PostgreSQL: 19 passed.
  • I changed tick planning to tick the earliest missed time, and the stepped-clock test failed at the standby takeover step.
  • make check-ci passed on the head.
  • One independent review: both findings fixed (assert the ticked times, correct a comment).

…time

Due-time selection, lease takeover and the next-wake read now run against a
task-local database clock set by the test, and assert tick counts and exact
waits per cycle. One real-clock run remains, with a single due time and a
generous bound, to catch a scheduler that hangs; it fails clearly when setup
runs past the due time.
@kvlonge
kvlonge disabled auto-merge November 17, 2026 04:32
@kvlonge
kvlonge added this pull request to the merge queue Nov 17, 2026
Merged via the queue into main with commit 02d3c9d Nov 17, 2026
32 checks passed
@kvlonge
kvlonge deleted the fix/scheduler-latency-window branch November 17, 2026 05:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant