Skip to content

fix(engines): read each record whole through the scapy engine (#1717) - #1733

Merged
JarryShaw merged 1 commit into
mainfrom
fix/1717-scapy-record-cap
Oct 14, 2026
Merged

JarryShaw merged 1 commit into
mainfrom
fix/1717-scapy-record-cap

Conversation

@JarryShaw

Copy link
Copy Markdown
Owner

Please follow the guide below

What is the purpose of your pull request?

Tick the commit type your subject line carries.

  • fix — corrects a defect
  • feat — adds a feature
  • perf — changes performance, not behaviour
  • refactor — changes neither behaviour nor performance
  • test — tests only
  • docs — documentation only
  • ci — workflows or build tooling
  • release — bumps the version or rolls up a distribution
  • chore — anything else

Description of your pull request and other information

Closes #1717. The scapy engine cut any record longer than 65,535 octets. Behind that cap sit two more, which each scapy version hits in a different mix:

Cap Scapy versions Fix
sniff() reads with recv(), whose size defaults to MTU (0xffff), and every reader slices to it 2.6.0, 2.7.0, 2.8.0 recv() defaults to 0xFFFFFFFF, the most a 32-bit captured length can count
scapy 2.6 reads each PCAP-NG block at MTU 2.6.0 the _read_block() override defaults to the same
scapy 2.8's PCAP reader reads at most MTU * 4 (262,140) octets of a record 2.8.0 the PCAP reader reads the record itself, in 256 KiB chunks

Reading in chunks means a forged captured length costs only the octets the file actually holds.

Tests:

  • Synthesised PCAP and PCAP-NG records up to 600,000 octets, plus frame 13 of options-ipv6.pcap (65,590 octets). They pass on scapy 2.6.0, 2.7.0 and 2.8.0, and 9 of the 11 checks fail on main on each.
  • Across all 23 generated captures, the only frame whose length changes is that 65,590-octet one.

- sniff() reads with recv(), whose size defaults to Scapy's MTU of 65,535
  octets, and every Scapy reader cuts a record to size: the engine's
  readers default recv() to 0xFFFFFFFF, the most a 32-bit captured length
  counts, for PCAP and PCAP-NG alike; an explicit size is still passed on
- Scapy 2.6 reads each PCAP-NG block at MTU whatever the size: the
  _read_block() override defaults to the same 0xFFFFFFFF
- Scapy 2.8's PCAP reader reads at most MTU * 4 (262,140) octets of a
  record and skips the rest: the PCAP reader reads the record itself, in
  256 KiB chunks so a forged captured length still costs only the octets
  the file holds, from a RawPcapReader base after PcapReader, where
  PcapReader.read_packet's super(PcapReader, self) looks

Tests: test_scapy_record_length_1717_{unit,runtime}.py, 5 passed on scapy
2.6.0, 2.7.0 and 2.8.0; 9 of the 11 fail on each against origin/main.

Closes #1717
@JarryShaw JarryShaw added fix Pull requests that fix a defect (fix: subject prefix) review: good-to-go Cross-review at the current head says ready; CI state is separate labels Oct 14, 2026
@JarryShaw

Copy link
Copy Markdown
Owner Author

Verdict: GOOD TO GO at 910d10345. A Sonnet cross-review checked it (author: Opus).

  • Records arrive whole: records of 65,536, 262,144 and 600,000 octets come through intact on scapy 2.6.0, 2.7.0 and 2.8.0, for PCAP and for PCAP-NG Enhanced, obsolete and Simple Packet Blocks. Frame 13 of options-ipv6.pcap keeps its sha1.
  • Forged length: a record claiming 0xFFFFFFFF octets in a 100-octet file gives [60, 100], peaking at about 110 MiB under a 3 GiB address cap.
  • Other captures: across all 23 captures (1,616 frames) on each version, the only length that changes is frame 13, 65,535 → 65,590.
  • Fail before: the tests fail on main (9 failed, 2 passed per version). Reverting each of the three parts on its own fails the scapy version it targets.

@JarryShaw JarryShaw moved this to In review in PyPCAPKit Oct 14, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Coverage: 89.61% (unit tier, Python 3.15, 910d10345, Unit Tests run success)

Package Statements Missed Branches Partial Cover
pcapkit (top level) 104 3 20 3 95.16%
pcapkit/const 18860 954 2368 868 91.06%
pcapkit/corekit 2384 67 764 35 96.51%
pcapkit/dumpkit 275 3 100 3 98.40%
pcapkit/foundation 2907 164 1016 34 93.58%
pcapkit/interface 138 7 52 5 93.68%
pcapkit/protocols 17174 226 4590 190 98.07%
pcapkit/toolkit 1015 173 372 10 80.89%
pcapkit/utilities 508 4 154 4 98.79%
pcapkit/vendor 4417 2342 1010 157 43.38%

Per-file detail: the coverage-html artifact of this run.

@JarryShaw
JarryShaw merged commit 75d6442 into main Oct 14, 2026
54 checks passed
@github-project-automation github-project-automation Bot moved this from In review to Done in PyPCAPKit Oct 14, 2026
@JarryShaw
JarryShaw deleted the fix/1717-scapy-record-cap branch October 14, 2026 17:26
@JarryShaw JarryShaw removed the review: good-to-go Cross-review at the current head says ready; CI state is separate label Oct 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fix Pull requests that fix a defect (fix: subject prefix)

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

fix(engines): the scapy engine cuts a captured record at 65,535 octets

1 participant