Skip to content

ci(unit-tests): coverage combine can silently miss a worker's data file, under-reporting a PR's coverage #1387

Closed

Description

@JarryShaw

The unit-tier coverage number can silently lose a worker's data. PR #1345, an MH-only change at f81c3927a, reports 85.85%. #1341 at f6e21cf67 reports 88.89%, and both branch from the same base, c9676c56. The loss sits in packages #1345 never touches: interface 33.55% vs 92.11%, toolkit 65.43% vs 83.57%. Within those, interface/misc.py reads 18% vs 87% and toolkit/pcapng.py 19% vs 100%.

Every test ran in both jobs. Job 112994306184 (#1345) reports 2883 passed, 114 skipped, and job 112993436318 (#1341) reports 2887 passed, 114 skipped. The difference of 4 is #1345's own tests. Neither log shows a cancelled shard, a crashed xdist worker, a collection error or a CoverageWarning. The one difference is coverage combine:

So one worker's .coverage.* file was missing at combine time, and the run still reported green. The leg runs coverage run --rcfile=.github/coverage.toml -m pytest -q -n auto --dist load (unit-tests.yml:148) with parallel plus patch = ["subprocess"]. The exact mechanism by which a worker's file is lost is inferred, not traced.

This is not a one-off. Across the 15 most recent coverage comments, interface is a constant 92.11% everywhere except #1345. toolkit alternates between 85.42% and 83.57% on unrelated diffs (#1341, #1344, #1287 sit at the lower value), which suggests smaller drops of the same kind.

Fix:

  1. Find out why a worker's data file goes missing: the sigterm/atexit save, xdist worker teardown, or the patch hook.
  2. Make the combine step fail or warn when the number of combined files falls short of the worker count plus the controller, so a short count can never be posted as the PR's coverage.
  3. Re-run fix(mh): keep zero-length PadN, reserved fields and BU/BA flag bits on rebuild #1345's Python 3.14 job to confirm.

Metadata

Metadata

Assignees

No one assigned

    Labels

    ciPull requests that change CI or workflow configuration (ci: subject prefix)

    Projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions