Skip to content

fix(toolkit): the pypcapfile engine hands no TCP or IPv4 input over on the NULL, LOOP, RAW, IPV4 and IPV6 link types #1667

Closed

Description

@JarryShaw

Describe the bug
On every link type but ETHERNET, engine='pypcapfile' hands nothing to TCP reassembly, flow tracing or IPv4 reassembly. The default engine dissects NULL, LOOP, RAW, IPV4 and IPV6 (pcapkit/protocols/misc/pcap/frame.py:101-105, RAW by version nibble at :658-665), so it reads TCP and IPv4 fragments there. The pypcapfile engine only warns unrecognised link layer protocol: <LinkType.RAW: 101>; frames will be left undecoded and all analysis functions ignored.

Reproduction
On main (85e4198c8, pcapkit 1.5.0b10), Linux, CPython 3.11.15, pypcapfile 0.12.0. pypcapfile does not run on 3.14. The default engine's cells are the same on 3.14.8. Each frame is a link header, then IPv4 or IPv6. NULL is little-endian, LOOP big-endian, family 2 for IPv4 and 24 for IPv6. Each cell has six parts, separated by ·:

  1. TCP reassembly/trace inputs for IPv4 TCP
  2. the same for IPv6 TCP
  3. IPv4,IPv6 reassembly inputs for two IPv4 fragments
  4. the same for two IPv6 fragments
  5. TCP inputs for 4in6 TCP
  6. IPv4 reassembly inputs for two 4in6 fragments
Link type default pypcapfile
ETHERNET (1), control 1/1 · 1/1 · 2,0 · 0,2 · 1/1 · 2 1/1 · 0/0 · 2,0 · 0,0 · 1/1 · 2
NULL (0), LOOP (108), RAW (101) 1/1 · 1/1 · 2,0 · 0,2 · 1/1 · 2 0/0 · 0/0 · 0,0 · 0,0 · 0/0 · 0
IPV4 (228) 1/1 · – · 2,0 · – · – · – 0/0 · – · 0,0 · – · – · –
IPV6 (229) – · 1/1 · – · 0,2 · 1/1 · 2 – · 0/0 · – · 0,0 · 0/0 · 0
12, LINUX_SLL (113) all 0 all 0

On ETHERNET, where pypcapfile matches the default engine, its IPv6 zeros are by design: it has no IPv6 decoder, and says so with a warning. Neither engine raises on any row.

Additional context
pypcapfile 0.12.0 decodes ETHERNET alone. pcapfile/linklayer.py:13-18 gives NULL a None decoder and lists none of the others, so load_savefile(layers=2) on a RAW file raises TypeError: 'NoneType' object is not callable (pcapfile/savefile.py:226). _get_decoder (pcapkit/foundation/engines/pypcapfile.py:315-319) then leaves every frame undecoded, and each adapter returns None on an undecoded frame (pcapkit/toolkit/pypcapfile.py:517-518, :575-577, :823-826). But the toolkit already decodes IPv4 that pypcapfile left undecoded, with pypcapfile's own IP(data, 0). It does this behind VLAN tags (:534-545, #1537) and inside tunnels (_decode_ipv4, :602-609, #1581, #1592). A bare IPv4 packet is just as easy to decode: IP(raw, 0) reads one correctly. The warning and the "no reassembly or flow tracing is possible" docstring (pcapkit/foundation/engines/pypcapfile.py:305-307) date from #386. That was before #1594 gave the default engine RAW, NULL and LOOP. The engines' gap table (docs/source/pcapkit/foundation/engines/index.rst:95-103) lists no link-type gap for pypcapfile.

No test catches it. tests/foundation/engines/test_pypcapfile_engine.py:291-302 mocks the decoder as None, and the agreement suites build Ethernet frames only.

Expected behavior
The pypcapfile adapters read the IP packet on NULL and LOOP (by Loopback's family table and family_byteorder, pcapkit/protocols/link/loopback.py:129-132, :47), RAW (by version nibble), IPV4 and IPV6, as they read it behind Ethernet. Then the bold cells match the default engine field for field. TCP over IPv6 on these link types is left out with the per-capture warning, as on Ethernet. 12 and LINUX_SLL stay 0, as in the default engine. The link-type warning and the docstrings name only the link types left unread. packet2dict reports an unread frame's octets, not their hex text. Today it gives raw_len 94 for a 47-octet RAW frame (pcapkit/toolkit/pypcapfile.py:1071-1073).

Related: #1548 (dpkt, raw IP) and #1650 (scapy, link types) apply the same rule to the other engines.

Activity

  1. added
    fixPull requests that fix a defect (fix: subject prefix)
    pendingAccepted and unstarted - nobody is on it and nothing blocks it
    wipWork in flight - a covering PR is open or an agent is actively on it
    and removed
    pendingAccepted and unstarted - nobody is on it and nothing blocks it
    on Oct 13, 2026
  2. JarryShaw commented on Oct 13, 2026

    @JarryShaw
    OwnerAuthor

    A worker has this, together with the pypcapfile part of #1609, the 64-layer frame bound, since both live in pcapkit/toolkit/pypcapfile.py. The timestamp half of #1668 waits for this PR, because it edits the same file.

  3. removed
    wipWork in flight - a covering PR is open or an agent is actively on it
    on Oct 13, 2026
  4. added this to the PyPCAPKit v1.5 milestone on Oct 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    fixPull requests that fix a defect (fix: subject prefix)

    Projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions