You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Split the pull request tests into the same six shards as the release tests #4144
The pull request workflow's Test job runs the whole make test suite on one runner. On #4140's last run it took 28 minutes (13:58 to 14:26 UTC). The release run already splits the same suite over six runners (#4134), where each shard finishes in 3 to 5 minutes.
The split also runs only on release commits, so a pull request can merge with a change that breaks it. That is what happened in release run 41063091599: a test that starts pytest itself failed only on shard 6 (#4139), and nothing ran the split before the merge.
Proposal
pr.yml's test job becomes a matrix of six shards (fail-fast: false). Each shard runs make test with the shard plugin loaded through PYTEST_ADDOPTS, as .github/scripts/run_release_tests.sh does for push.yml's release-tests job, and uploads the tests it collected and ran.
A new job downloads every shard's report and runs python -m scripts.release_test_shards check-coverage, which fails unless each collected test ran in exactly one shard. CI complete needs it.
The shards' coverage reports go to Codecov in one upload from a job that needs every shard, so Codecov gets one upload for the test suite, as before the split.
A test checks that pr.yml and push.yml use the same shards.
The cross-database integration tests took 29 minutes on the same run, so the pull request workflow stays about that long until that job is also faster.
Activity