Skip to content

ci: build native wheels once and shard the unit and integration tests - #1475

Merged
kvlonge merged 3 commits into
mainfrom
ci/shard-static-tests
Nov 14, 2026
Merged

kvlonge merged 3 commits into
mainfrom
ci/shard-static-tests

Conversation

@kvlonge

@kvlonge kvlonge commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

Why

"Static checks and unit + integration tests" is the slowest job on every pull request's critical path: about 31–35 minutes on one runner, spent as a 9-minute native build, 2–3 minutes of static checks and 18–22 minutes of tests on 2 xdist workers. One file, the warehouse history publication tests, takes about 15 minutes of that serial time.

Changes

  • A native_build job builds the extension and CLI wheels once. A checks job (make check-ci) and three test shards install those wheels instead of compiling:
    • core: unit and integration tests outside publication/;
    • publication: publication/ except the warehouse history tests;
    • warehouse history: test_warehouse_metadata.py, spread test by test (--dist load).
  • check-test-shards, now in check-ci, fails when a unit or integration test is in no shard or in two. It generalises the DuckDB E2E shard check, which keeps its behaviour. make test is unchanged.
  • Verify requires the new jobs exactly as it required the old one.
  • Release Please version-bump heads: Lockfile consistency waits for the lockfile refresh commit instead of failing on the bump.
  • Compiler-differential code scans find the repository from their own path, as tests import the installed wheel.
  • A workflow comment records that re-running an older Verify suite never repaints the required check; start a new suite instead.

Verification

  • make check-ci passes; shard coverage 12,225 tests (core 12,032, publication 145, history 48); E2E 2,641.
  • A planted test in no shard fails with owners: none; dropping the publication --ignore fails with two owners.
  • actionlint shows no new findings.
  • Before: jobs 132320066524 (30.8 min) and 132353553697 (35.4 min).
  • After (run 43576682096): wheels 9.0 min, then static 4.9, core 4.0, publication 7.4 and history 5.0 min in parallel. Critical path 16.5 min (fast core 13.0); 30.3 runner-minutes, about as before.

DuckShop: not applicable; this changes CI job layout only.

…ob and three test shards

The extension and CLI build once into wheels that the static job and each shard install.
check-test-shards, now part of check-ci, proves the shards run every unit and integration
test exactly once, using the shard coverage check the DuckDB E2E shards already had.
On a Release Please version-bump head, Lockfile consistency waits for the lockfile refresh
commit, whose run checks the lockfiles. The depth step documents that re-running an older
Verify suite never repaints the required check.
@kvlonge
kvlonge marked this pull request as ready for review November 14, 2026 09:47
@kvlonge
kvlonge enabled auto-merge November 14, 2026 09:50
@kvlonge
kvlonge added this pull request to the merge queue Nov 14, 2026
Merged via the queue into main with commit 67c4ecc Nov 14, 2026
87 of 89 checks passed
@kvlonge
kvlonge deleted the ci/shard-static-tests branch November 14, 2026 11:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant