Repository navigation
ci: verify the full matrix once in the merge queue and tier timing jobs - #1460
Merged
Merged
Conversation
… groups Work in progress, first of five planned CI speed-ups. Not yet opened as a pull request. build-distributions.yml takes a platforms input; its source job filters the one platform list into the wheel matrix and rejects unknown names. Verify passes linux-x86_64 on pull requests and merge groups, which Compose needs; manual runs and Release Distributions still build every platform. Validated: the filter script run locally for an empty selection (all five), one platform, two platforms and an unknown name (fails); both workflows parse as YAML; make check-release-gate passes. Not yet run in GitHub Actions.
A passing fast core reports as Verify when the repository variable SQLBUILD_MERGE_QUEUE is true, because the merge queue runs the full matrix on every group before it merges. A merge group whose whole diff is release metadata skips the suites, as release pull requests already do. Pull requests and merge groups run the 3k-model compile jobs and the 1k dense job; a nightly scheduled run on main adds 5k and 10k and every wheel platform, and opens or updates an issue when it fails. Changes to the orchestration and CLI Rust test crates run native checks and the scheduler recovery suite instead of every suite.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
A typical pull request waits on the whole matrix twice: once for the label run, and again when it merges. Release pull requests rerun every suite, even though they change only version lines. The 5k/10k-model timing jobs and four of five wheel platforms add most of the wall time.
Changes
build-distributions.ymltakes aplatformsinput. Verify builds only the linux-x86_64 wheel on pull requests and merge groups. Scheduled and manual runs, Release Distributions and Publish still build all five.SQLBUILD_MERGE_QUEUEvariable istrue, a passing fast core on a pull request reports asVerify, and every merge group runs the full matrix once before merging.mainadds the 5k and 10k jobs and every wheel platform, and opens or updates anightly-verifyissue when it fails.After merge, someone with admin access sets the repository variable
SQLBUILD_MERGE_QUEUE=true. Themainruleset already has the merge queue.Verification
main.check_release_gatepasses, as do its 47 tests.DuckShop: not applicable. This changes CI only, not the compiler or project behaviour.