You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
ci(unit-tests): coverage combine can silently miss a worker's data file, under-reporting a PR's coverage #1387
The unit-tier coverage number can silently lose a worker's data. PR #1345, an MH-only change at f81c3927a, reports 85.85%. #1341 at f6e21cf67 reports 88.89%, and both branch from the same base, c9676c56. The loss sits in packages #1345 never touches: interface 33.55% vs 92.11%, toolkit 65.43% vs 83.57%. Within those, interface/misc.py reads 18% vs 87% and toolkit/pcapng.py 19% vs 100%.
Every test ran in both jobs. Job 112994306184 (#1345) reports 2883 passed, 114 skipped, and job 112993436318 (#1341) reports 2887 passed, 114 skipped. The difference of 4 is #1345's own tests. Neither log shows a cancelled shard, a crashed xdist worker, a collection error or a CoverageWarning. The one difference is coverage combine:
So one worker's .coverage.* file was missing at combine time, and the run still reported green. The leg runs coverage run --rcfile=.github/coverage.toml -m pytest -q -n auto --dist load (unit-tests.yml:148) with parallel plus patch = ["subprocess"]. The exact mechanism by which a worker's file is lost is inferred, not traced.
This is not a one-off. Across the 15 most recent coverage comments, interface is a constant 92.11% everywhere except #1345. toolkit alternates between 85.42% and 83.57% on unrelated diffs (#1341, #1344, #1287 sit at the lower value), which suggests smaller drops of the same kind.
Fix:
Find out why a worker's data file goes missing: the sigterm/atexit save, xdist worker teardown, or the patch hook.
Make the combine step fail or warn when the number of combined files falls short of the worker count plus the controller, so a short count can never be posted as the PR's coverage.
Activity
test(project): keep the step stubs from measuring a second coverage c…
ci(unit-tests): cap the size of files tests write on every pytest step (
ci: move the repeated workflow shell into util scripts (#1967)
ci: move the repeated workflow shell into util scripts (#1967) (#1995)