Skip to content

ci(unit-tests): save each xdist worker's coverage at session finish and keep the process log (#1423) - #1424

Merged
JarryShaw merged 1 commit into
mainfrom
ci/1423-coverage-worker-save
Oct 8, 2026
Merged

JarryShaw merged 1 commit into
mainfrom
ci/1423-coverage-worker-save

Conversation

@JarryShaw

Copy link
Copy Markdown
Owner

What is the purpose of your pull request?

  • ci — workflows or build tooling

Description of your pull request and other information

Established: each missing pid is a real xdist worker (4 controller children, cpu_count 4), and it died after reporting its results, which is why every test passed. xdist ignores a worker's EOF after workerfinished. No log has a traceback. A process with no data still writes a file, so nothing ran its atexit save to completion. Not established: what killed it. execnet SIGKILLs after 10 s, but a measured save takes 0.24 s, so a crash during finalisation is the other candidate. Nothing in the #1409–#1412 diff spawns or kills a process.

Ruled out: runner image/Python (hits on both 20260927/3.14.7 and 20261004/3.14.8); test count (first hit ran 3195, the same as a clean run); chdir (data path is absolute); a test touching coverage (none does).

Changes:

  1. tests/conftest.py: pytest_sessionfinish saves a worker's coverage before shutdown.
  2. .github/coverage.toml: sigterm = true.
  3. unit-tests.yml: the guard prints each missing pid's cmd and atexit fate, and coverage-processes.log is uploaded as an artifact. Doc citations are renumbered.

tests/project: 404 passed, 1 skipped. The 5 new tests fail without the fix.

Closes #1423

…nd keep the process log (#1423)

- tests/conftest.py: a pytest_sessionfinish hook saves an xdist worker's
  auto-started coverage before shutdown, so a worker that dies after
  reporting its results no longer loses its whole data file.
- .github/coverage.toml: sigterm = true, so a SIGTERM saves the data too.
- unit-tests.yml: the guard names each missing pid's command line and
  whether its atexit save ran; coverage-processes.log is now uploaded as
  the coverage-processes-log artifact on every coverage leg.
- docs: unit-tests.yml citations renumbered by line content; the
  coverage section mentions the save and the new artifact.

tests/project: 404 passed, 1 skipped; the 5 new tests fail without the fix.
@JarryShaw JarryShaw added ci Pull requests that change CI or workflow configuration (ci: subject prefix) review: running A cross-review is in flight against the current head - no verdict yet labels Oct 8, 2026
@github-actions

github-actions Bot commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

Coverage: 89.43% (unit tier, Python 3.14, 7eef5df56, Unit Tests run success)

Package Statements Missed Branches Partial Cover
pcapkit (top level) 104 4 20 4 93.55%
pcapkit/const 18791 968 2352 869 90.90%
pcapkit/corekit 2038 89 634 25 94.76%
pcapkit/dumpkit 234 2 84 2 98.74%
pcapkit/foundation 2637 98 928 48 94.78%
pcapkit/interface 112 7 40 5 92.11%
pcapkit/protocols 16504 219 4420 183 98.04%
pcapkit/toolkit 539 74 168 3 84.58%
pcapkit/utilities 429 4 122 4 98.55%
pcapkit/vendor 4409 2342 1006 157 43.25%

Per-file detail: the coverage-html artifact of this run.

@JarryShaw

Copy link
Copy Markdown
Owner Author

Verdict: GOOD TO GO at 7eef5df56 (Sonnet cross-review; the author ran on Opus).

  • Real xdist run: coverage 7.16.2, pytest-xdist 3.8.0, in a throwaway venv. Each pid writes one .coverage.*, with no DataError. Arcs executed after the session-finish save (in pytest_unconfigure) still land in the worker's file.
  • Reproduction: SIGKILLing a worker at pytest_unconfigure leaves its file in place and the guard reports missing []. On the base conftest.py the same kill gives missing ['24103'], which is ci(unit-tests): Python 3.14 coverage guard intermittently finds an xdist worker with no data file #1423's failure.
  • No-op elsewhere: plain pytest and pytest -n 2 without coverage leave no stray files and raise no errors.
  • sigterm = true: it only installs a save-and-re-raise handler while coverage runs, and nothing in the repo sends SIGTERM.
  • Workflow: the existing guard strings are unchanged, and the new upload step mirrors its siblings (upload-artifact@v7, !cancelled() && matrix.coverage).
  • Docs: all 11 renumbered citations land on the content their prose describes, and none are left stale.
  • Tests: tests/project plus the tier-guard and support-helper modules give 544 passed, 2 skipped.
  • Note, not blocking: combine dedupes by the content hash embedded in the filename (coverage/data.py, hash_for_data_file). After this hook's early save, a later append keeps the old name, so two workers with byte-identical data at session end would dedupe even if their later arcs differ. That realistically needs two workers that ran no tests. CI's current Combined 30 files, skipped 36 is true duplicates, so there is no under-count today.

@JarryShaw JarryShaw added review: good-to-go Cross-review at the current head says ready; CI state is separate and removed review: running A cross-review is in flight against the current head - no verdict yet labels Oct 8, 2026
@JarryShaw
JarryShaw merged commit 02ee60a into main Oct 8, 2026
41 checks passed
@JarryShaw
JarryShaw deleted the ci/1423-coverage-worker-save branch October 8, 2026 17:24
@JarryShaw JarryShaw removed the review: good-to-go Cross-review at the current head says ready; CI state is separate label Oct 8, 2026
@JarryShaw JarryShaw added this to the PyPCAPKit v1.5 milestone Oct 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci Pull requests that change CI or workflow configuration (ci: subject prefix)

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

ci(unit-tests): Python 3.14 coverage guard intermittently finds an xdist worker with no data file

1 participant