Skip to content

test(integration): skip upstream-pinned PCAP-NG counts on a stand-in fixture (#1478) - #1480

Merged
JarryShaw merged 1 commit into
mainfrom
fix/1478-stand-in-fixture-counts
Oct 9, 2026
Merged

JarryShaw merged 1 commit into
mainfrom
fix/1478-stand-in-fixture-counts

Conversation

@JarryShaw

Copy link
Copy Markdown
Owner

Please follow the guide below

What is the purpose of your pull request?

Tick the commit type your subject line carries.

  • fix — corrects a defect
  • feat — adds a feature
  • perf — changes performance, not behaviour
  • refactor — changes neither behaviour nor performance
  • test — tests only
  • docs — documentation only
  • ci — workflows or build tooling
  • release — bumps the version or rolls up a distribution
  • chore — anything else

Description of your pull request and other information

Closes #1478. When a download fails, examples/generators/pcapng.py writes a synthesised stand-in under the upstream name, so the counts pinned to upstream went red on a network blip (job 113845878658).

Design. The generator already pins each download's SHA-256 in FIXTURES. That pinned digest is the marker: tests/_support.stand_in_reason() hashes the capture on disk and compares. A sidecar file could get out of step with the bytes it describes; this check can't. skip_if_stand_in() skips only the assertions pinned to upstream, so they still run whenever the real file is present. No stand-in numbers are hard-coded.

Scope. Only pcapng.py downloads. Of its two downloads:

  • many_interfaces.pcapng: guarded at the three failing sites (:70, :88, :96 on main).
  • dhcp_big_endian.pcapng: its stand-in matches upstream's 4-frame inventory, so the integration test still asserts it. The SHB option layout check in test_pcapng_unit.py is pinned to upstream ([9, 12, 15, 7, 0] against the stand-in's [37, 9, 22, 29, 0]), so that check is guarded too.

Proof (stand-ins written by generate() with _download patched to raise URLError):

  • main's tests on the stand-ins: 4 failed. These are the three from the issue plus the unit check.
  • This branch on the stand-ins: 138 passed, 5 skipped. Each skip gives a reason that names the digest.
  • This branch on the real files: the integration test gives 6 passed, 1 skipped (the existing dictdumper skip), so the 64-frame assertions run.
  • New StandInReasonTests cover the helpers, including a simulated failed download through the real generator.
  • tests/project: 419 passed, 1 skipped.

@JarryShaw JarryShaw added review: running A cross-review is in flight against the current head - no verdict yet test Pull requests that add or correct tests (test: subject prefix) labels Oct 9, 2026
@JarryShaw JarryShaw added review: good-to-go Cross-review at the current head says ready; CI state is separate and removed review: running A cross-review is in flight against the current head - no verdict yet labels Oct 9, 2026
@JarryShaw

Copy link
Copy Markdown
Owner Author

Verdict: GOOD TO GO at 34bd7b13b. Cross-reviewed on Sonnet; the author ran on Opus.

  • Real fixtures still assert the upstream counts. Re-run by me: the downloaded many_interfaces.pcapng matches the pinned digest. The integration file plus tests/test_support_helpers.py give 43 passed, 1 skipped (the existing dictdumper skip). In a scratch copy, changing the expected 64 to 65 fails all three guarded tests.
  • The stand-in path. With the download forced to fail, the integration, test_pcapng_unit and test_pcapng_regression files give 112 passed and 5 skipped, with no failures. Only the upstream-pinned assertions are skipped; dhcp_big_endian still checks 4 frames.
  • The skip cannot hide a parser regression. It depends only on the file's SHA-256 against the generator's pin. The digest helper leaves sys.path, os.environ and logging unchanged, and imports nothing from pcapkit.
  • No pinned count is left unguarded. A grep of tests/ for both fixtures found five count assertions, all guarded. tests/project and the tier/docstring guards: 572 passed.

Not audited, out of scope: other downloaded fixtures, such as http6.cap.

…fixture (#1478)

When the download of many_interfaces.pcapng or dhcp_big_endian.pcapng fails,
examples/generators/pcapng.py writes a synthesised stand-in under the same
name, and assertions pinned to the upstream bytes went red on a network blip.

- tests/_support.py: stand_in_reason() compares a capture's SHA-256 against
  the digest the generator already pins in FIXTURES; skip_if_stand_in()
  skips the test or subtest when they differ.
- test_pcapng_end_to_end.py: guard the many_interfaces.pcapng counts; the
  dhcp_big_endian stand-in matches upstream's inventory and stays asserted.
- test_pcapng_unit.py: guard the big-endian SHB option layout check.
- test_support_helpers.py: cover the helpers, including a simulated failed
  download through the real generator.
@JarryShaw

Copy link
Copy Markdown
Owner Author

Verdict carried to 2167d38c5: GOOD TO GO. This is a pure rebase onto main at bbfd4bf64, after #1475 merged. range-diff shows the one commit as =, and the stable patch-id is unchanged (3dbf97b98e02).

@JarryShaw
JarryShaw force-pushed the fix/1478-stand-in-fixture-counts branch from 34bd7b1 to 2167d38 Compare October 9, 2026 14:06
@github-actions

github-actions Bot commented Oct 9, 2026

Copy link
Copy Markdown
Contributor

Coverage: 89.55% (unit tier, Python 3.14, 2167d38c5, Unit Tests run success)

Package Statements Missed Branches Partial Cover
pcapkit (top level) 104 4 20 4 93.55%
pcapkit/const 18795 958 2356 871 90.99%
pcapkit/corekit 2107 93 672 27 94.60%
pcapkit/dumpkit 242 2 88 2 98.79%
pcapkit/foundation 2648 98 930 48 94.80%
pcapkit/interface 112 7 40 5 92.11%
pcapkit/protocols 17049 240 4610 193 97.95%
pcapkit/toolkit 539 74 168 3 84.58%
pcapkit/utilities 429 4 122 4 98.55%
pcapkit/vendor 4409 2342 1006 157 43.25%

Per-file detail: the coverage-html artifact of this run.

@JarryShaw
JarryShaw merged commit f1e4948 into main Oct 9, 2026
36 checks passed
@JarryShaw
JarryShaw deleted the fix/1478-stand-in-fixture-counts branch October 9, 2026 17:41
@JarryShaw JarryShaw removed the review: good-to-go Cross-review at the current head says ready; CI state is separate label Oct 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

test Pull requests that add or correct tests (test: subject prefix)

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

test(integration): pcapng end-to-end counts fail when many_interfaces.pcapng falls back to the synthesised stand-in

1 participant