You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Keep the release test split's settings out of pytest subprocesses #4139
Release run 41063091599 (the first with #4134) failed on the release test shard 6 of 6, so no deploy job ran.
run_release_tests.sh turns on the split by setting PYTEST_ADDOPTS="-p tests.release_test_shard_plugin" and the three RELEASE_TEST_SHARD_* variables. A pytest process that a test starts inherits all four, so the shard plugin runs inside it too:
tests/unit/test_cloud_run_push_workflow.py::test_live_staging_tests_get_one_pytest_worker_each starts pytest --collect-only on the two live staging test files. The plugin in that process splits just those two files over six shards, keeps none of them on shard 6, and pytest exits with "no tests collected (9 deselected)".
The plugin writes the shard's report when it collects, so the inner process also overwrites the outer process's report. On a shard where the inner run keeps its tests, the test passes, but check-release-test-shards would then fail because that shard's report lists the wrong tests.
Which shard the test file lands on depends on the files and test counts on master, so this fails every release run while the test's file lands on shards 3 to 6, and corrupts the report on shards 1 and 2.
Fix
The plugin reads its settings once, when pytest configures it, and then removes the RELEASE_TEST_SHARD_* variables and its own -p option from PYTEST_ADDOPTS in the process environment, so pytest processes that tests start run unsplit.
write_report refuses to replace a report that already exists, so a second writer fails loudly instead of overwriting a shard's report.
Activity