Skip to content

test(foundation): retry a pyshark engine run once on a tshark crash (#1534) - #1536

Merged
JarryShaw merged 1 commit into
mainfrom
test/1534-tshark-crash-retry
Oct 10, 2026
Merged

JarryShaw merged 1 commit into
mainfrom
test/1534-tshark-crash-retry

Conversation

@JarryShaw

Copy link
Copy Markdown
Owner

What is the purpose of your pull request?

  • test — tests only

Description of your pull request and other information

_run in test_engine_agreement_runtime.py retries a run whose error is a TSharkCrashException once (matched by class name), with a RuntimeWarning; a persistent crash stays ERROR. No Gap row or skip rule changes.

New test_engine_agreement_retry_unit.py stubs pcapkit.extract: one crash → OK + warning, two → ERROR after two runs, ValueError → not retried. On origin/main the first two fail.

Ran: new test, tests/foundation/engines, tests/project, tests/test_tier_guard.py (Python 3.14), and the agreement module 6× under Python 3.11 with pyshark: one run hit a real crash on ipv4.pcap, retried, and passed.

Closes #1534

…1534)

tshark sometimes exits 255 on a capture it reads cleanly on a rerun, and
pyshark raises TSharkCrashException. The engine agreement module caches one
run per engine and capture, so that one crash failed all nine pyshark
aspects of the capture.

- _run retries a run whose error is a TSharkCrashException once, matched by
  class name so the module still imports without pyshark, and logs the
  retry with a RuntimeWarning; a crash that persists stays an ERROR.
- test_engine_agreement_retry_unit.py stubs pcapkit.extract: one crash then
  success is OK with the retry logged, two crashes are ERROR after two runs,
  and any other exception is not retried.
@JarryShaw JarryShaw added review: running A cross-review is in flight against the current head - no verdict yet test Pull requests that add or correct tests (test: subject prefix) labels Oct 9, 2026
@JarryShaw

Copy link
Copy Markdown
Owner Author

Verdict: GOOD TO GO at e274fd866. Cross-reviewed on Sonnet; the author ran Opus.

  • The retry only fires on a real tshark crash. The check matches the exact class name. The error string is built as f'{type(exc).__name__}: {exc}', so a ValueError whose message mentions the crash is not retried.
    • pyshark 0.6 raises TSharkCrashException bare, at capture.py:358 and :379, and pcapkit does not wrap it.
  • A real crash was provoked: a truncated arp.pcap made tshark exit with code 14 under the real pyshark 0.6.
    • The harness ran extract twice and emitted one RuntimeWarning, then cached ERROR.
    • A retry that succeeds is still compared against the default engine.
  • No pytest config, conftest or workflow turns RuntimeWarning into an error.
  • Fail-before: 2 of the 3 new tests fail; the ValueError guard passes, as designed. The new file is genuinely unit tier.

Not reproduced: the flake itself, across 806 serial runs on py3.11 with tshark 4.6.9. The evidence that the retry cures it is the author's observed crash-then-pass.

Nits:

  • The module docstring doesn't mention the retry.
  • The other pyshark test modules have no retry, but there one crash fails only one test.

@JarryShaw JarryShaw added review: good-to-go Cross-review at the current head says ready; CI state is separate and removed review: running A cross-review is in flight against the current head - no verdict yet labels Oct 9, 2026
@github-actions

github-actions Bot commented Oct 9, 2026

Copy link
Copy Markdown
Contributor

Coverage: 90.08% (unit tier, Python 3.14, e274fd866, Unit Tests run success)

Package Statements Missed Branches Partial Cover
pcapkit (top level) 104 4 20 4 93.55%
pcapkit/const 18795 843 2356 756 92.08%
pcapkit/corekit 2161 66 700 33 96.26%
pcapkit/dumpkit 242 2 88 2 98.79%
pcapkit/foundation 2665 98 934 42 95.00%
pcapkit/interface 112 7 40 5 92.11%
pcapkit/protocols 17086 238 4628 190 97.98%
pcapkit/toolkit 539 74 168 3 84.58%
pcapkit/utilities 431 4 124 4 98.56%
pcapkit/vendor 4409 2342 1006 157 43.25%

Per-file detail: the coverage-html artifact of this run.

@JarryShaw
JarryShaw merged commit ca7262e into main Oct 10, 2026
41 checks passed
@github-project-automation github-project-automation Bot moved this from In review to Done in PyPCAPKit Oct 10, 2026
@JarryShaw
JarryShaw deleted the test/1534-tshark-crash-retry branch October 10, 2026 01:57
@JarryShaw JarryShaw removed the review: good-to-go Cross-review at the current head says ready; CI state is separate label Oct 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

test Pull requests that add or correct tests (test: subject prefix)

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

test(foundation): engine agreement module flakes on tshark crashes (TSharkCrashException, retcode 255)

1 participant