fix(engines): TCP trace flows disagree between engines on snapped frames and on IPv4 total_length=0

On main 6af87f608, I derived two captures from examples/captures/tcp.pcap and traced them. The engines produce different TCP flows:

Capture	default	dpkt	scapy
unmodified `tcp.pcap`	`(2,6) (3,) (4,5)`	same	same
every frame snapped by 4 octets (PCAP-NG, `orig_len` kept)	`(3,5) (6,)`	`(1,2,6,7) (3,4,5)`	`(1,2,6,7) (3,4,5)`
IPv4 `total_length` zeroed, as TSO captures do	`(3,4,5)`	`(1,2,6,7) (3,4,5)`	`(1,2,6,7) (3,4,5)`

The first row comes from #1508. The other two are my measurements. Two things look wrong:

the default engine drops frames once IP's length no longer matches the captured length;
dpkt and scapy merge flows, giving a different grouping from the unmodified capture.

I have not established which engine is right, or why. That is the first step.

The #1511 review found this. It is blocked until #1511 merges, because that PR is editing the dpkt and scapy toolkits.

Unblocked: #1511 has merged. A read-only investigation is starting on the first step: which engine groups the snapped and zeroed-total_length captures correctly, and why the others differ. Fixes follow once the cause is known.

Investigation done, and I re-derived the rows myself. dpkt, scapy and Wireshark (tcp.stream) are right; the default engine is wrong, for two separate reasons.

Capture	default	dpkt / scapy / Wireshark
`tcp.pcap`	`(1,2,6,7) (3,4,5)`	same
snapped by 4 octets	`(3,5) (6,)`	`(1,2,6,7) (3,4,5)`
IPv4 `total_length` = 0	`(3,4,5)`	`(1,2,6,7) (3,4,5)`

The issue's first row was wrong: (2,6) (3,) (4,5) is the one-direction view with (1,7) left out. That also undoes the "dpkt and scapy merge flows" reading. Every engine follows the documented definition, one flow per endpoint pair (traceflow/tcp.py:50-62).

Root cause A, kept here: snapped frames. The snap cuts into the TCP header itself, which the parser rejects by design (fix(ipv4,tcp): a header length declared past the data zero-fills the missing octets on rebuild #1404). Frame 1 fails at protocols/transport/tcp.py:518 and frames 2, 4 and 7 at corekit/fields/collections.py:511 (a cut Timestamps option). TCP then becomes Raw, and toolkit/pcap.py:211 / toolkit/pcapng.py:217 skip the frame. The "IP length mismatch" theory is ruled out: frames 3, 5 and 6 have the same mismatch and are traced. The fix leaves fix(ipv4,tcp): a header length declared past the data zero-fills the missing octets on rebuild #1404 alone: when the IP payload is Raw with protocol TCP and at least 20 captured octets, the trace reads the fixed header. A worker is on it.
Root cause B, now fix(protocols): an IPv4 header with total_length 0 (TSO) parses its payload at length -20 #1547: an IPv4 total_length of 0 (TSO) gives a payload length of −20.
A third defect, now fix(toolkit): the dpkt engine traces no TCP on a raw-IP link type #1548: on a raw-IP link type the dpkt engine traces nothing.

fix(toolkit): trace and reassemble a TCP segment whose header the sna…

fix(toolkit): key tunnelled TCP by the innermost IP header (#1581)

fix(protocols): bound a frame's dissection at 64 layers (#1610)

fix(protocols): bound a frame's dissection at 64 layers (#1610) (#1620)

Navigation Menu

Repository navigation

fix(engines): TCP trace flows disagree between engines on snapped frames and on IPv4 total_length=0 #1518

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

fix(engines): TCP trace flows disagree between engines on snapped frames and on IPv4 total_length=0 #1518

Description

Activity

JarryShaw commented on Oct 10, 2026

JarryShaw commented on Oct 10, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions