fix(toolkit): dpkt reads IPv6 in IPv4, and nothing behind MPLS or Cisco ISL (#1637, #1638, #1646)

[JarryShaw]

Please follow the guide below

• You will be asked some questions, please read them carefully and answer honestly

• Put an x into all the boxes [ ] relevant to your pull request (like that [x])

• Use Preview tab to see how your pull request will actually look like

• Searched for similar pull requests

• Followed the coding style (make pylint, make mypy, make isort)

• make test passes, and a test case covers the change

• Added a changelog entry under docs/source/changelog/ and regenerated CHANGELOG.md, if the change is user-visible — N/A — centralised in docs(changelog): shared 1.5.0 changelog — long-lived, merges last (#610, #616, #617, #618, #620) #657

What is the purpose of your pull request?

Tick the commit type your subject line carries.

• fix — corrects a defect
• feat — adds a feature
• perf — changes performance, not behaviour
• refactor — changes neither behaviour nor performance
• test — tests only
• docs — documentation only
• ci — workflows or build tooling
• release — bumps the version or rolls up a distribution
• chore — anything else

---

Description of your pull request and other information

#1637. ipv6_reassembly took _network(packet) alone. It now takes _first_ipv6 (pcapkit/toolkit/dpkt.py:935), which walks IPv4 tunnels and the headers the default engine reads past, as _tcp_carrier does, and stops at a rejected IPv4 header, a later IPv4 fragment, or any other protocol, ICMP quotes included. The octets end at each Total Length above, then at the Payload Length, or the Jumbo Payload Length when that is zero (:1157); a zero one with no Jumbo option gives no input. The Fragment header must also be among the first EXTENSION_HEADER_LIMIT (32) extension headers and within the frame's first FRAME_LAYER_LIMIT (64) layers (:1151), the two bounds IPv6._decode_next_layer walks to (pcapkit/protocols/internet/ipv6.py:555, :561). The count starts at _network_depth (:553), the link layer and each VLAN tag, and adds each IPv4 header and each header _walk_extension_headers steps over (pcapkit/toolkit/pypcapfile.py:679, split out of _past_extension_headers to count them). These rules also fix 5 IPv6-in-Ethernet Payload Length rows, and #1609's dpkt IPv6 reassembly rows; its TCP rows remain.

#1638. _network (:525) took any ip/ip6 that dpkt's Ethernet named. _ethernet_accepted (:477) now refuses a payload dpkt reached past MPLS labels, an ISL header, or a TPID other than 0x8100/0x88A8, so legacy QinQ 0x9100/0x9200 too. dpkt overwrites an ISL tag with an inner 802.1Q tag, so ISL is also read from the captured address (:518).

#1646. dpkt reads some tag stacks in two goes, e.g. 802.1Q then 802.1ad, two 802.1ad, or three 802.1Q, and keeps only the second go's tags. So _layer2bytes looked for the IP header 4 octets early per dropped tag, and ProtocolError ended the extraction: on the TCP and IPv4 paths on main, and on the 6in4 path once _first_ipv6 sliced the IPv4. _wire_hdr_len (:381) now adds the dropped tags, counted off the captured octets by _vlan_tpids (:345). _ethernet_accepted reads the TPIDs there too, so a dropped 0x9100 tag still refuses the frame.

Measured on 3.14 and 3.11 (dpkt 1.9.8): every row of both issues' repros, and every stacked-tag row of the gaps repro, matches the default engine. Of the 62 stacks of one to five 802.1Q/802.1ad tags, dpkt raised on 12 and read nothing behind 46 on main. Now the 12 agree and the 46 are unchanged (below). A nesting probe over Ethernet with 0 to 3 tags, raw IP and BSD loopback agrees on all 287 IPv6 rows, where main disagreed on 88 and raised on 123.

New tests/toolkit/test_dpkt_6in4_reassembly_and_mpls_isl_1637_1638_unit.py compares dpkt with the default engine field for field: 32 IPv6-in-IPv4 cases, 6 Payload Length cases, 20 link-layer cases, 12 chain-length cases (3 controls), 20 layer-limit cases (10 controls) and 7 tag stacks. Reverting pcapkit/ to main gives 61 failures. Reverting only the layer bound fails the 10 rows past it and both datagrams checks; reverting the chain-bound and VLAN changes as well fails 21, ProtocolError ending one read. Dropping only the later-fragment stop (or ip.offset, :986) fails the AH and Hop-by-Hop later-fragment rows.

Not fixed here, identical before and after:

• A Fragment header followed by another extension header, 6in4 included. dpkt raises at ip6.py:96 and keeps the frame raw (fix(toolkit): dpkt, scapy and pyshark drop TCP and IPv4 behind some IPv6 extension-header chains #1626).
• A tag stack dpkt does not finish in two goes, e.g. three 802.1ad tags or any five tags (fix(toolkit): dpkt hands over nothing behind a VLAN tag stack it does not finish decoding in two passes #1654). dpkt decodes no payload, so hands nothing over; the default engine reads it.
• TCP over IPv6 with a zero Payload Length and no Jumbo option (fix(toolkit): dpkt reads TCP and IPv6 fragments past the end of an IPv6 packet whose Payload Length is zero #1651). dpkt hands it to TCP reassembly; the default engine does not.
• TCP over an IPv6 jumbogram, 6in4 included (fix(toolkit): dpkt reads TCP and IPv6 fragments past the end of an IPv6 packet whose Payload Length is zero #1651). dpkt's payload runs past the Jumbo Payload Length, into a link-layer trailer or, with none, past a Jumbo Payload Length shorter than the packet. The default engine's ends at it.

Not fixed here, as on main: _tcp_carrier and _first_ipv4 do not stop at FRAME_LAYER_LIMIT (#1609), so dpkt hands TCP and IPv4 reassembly a segment or fragment past layer 64. Behind a stack dpkt reads in two goes, main raised ProtocolError on those frames instead (20 probe rows).

Closes #1637
Closes #1638
Closes #1646

[JarryShaw]

Verdict: GOOD TO GO at 9a01482ec. Sonnet cross-reviewed this over five rounds. Rounds 3 and 4.1 accepted three new disagreements as disclosures; I sent all three back as code changes, because this PR must create no row where main already agrees with the default engine:

• 6in4 with 32 or more extension headers;
• 6in4 under a stacked VLAN, which raised ProtocolError (fix(toolkit): dpkt ends the extraction with ProtocolError on 802.1Q then 802.1ad, two 802.1ad, or three VLAN tags #1646, root-caused here);
• 6in4 behind 62 or more layers, now bounded at FRAME_LAYER_LIMIT, including the frame's link layer and VLAN tags.

Round 4.2 checked about 5,000 of its own captures and mutants and 77,000 direct-call mutants on 3.14 and 3.11. It found no row where main agreed and this branch disagrees, and no new exception. With the fix reverted, 61 tests fail on main.

I re-ran the new test, test_dpkt_unit and the TCP-carrier test myself on both Pythons: all green. The branch merges cleanly with main, #1652, #1642 and #1653. The commit message's Closes #VLAN placeholder is now Closes #1646. Left open: #1609, #1651 and #1654.

[github-actions]

Coverage: 89.68% (unit tier, Python 3.14, 9a01482ec, Unit Tests run success)

Package	Statements	Missed	Branches	Partial	Cover
pcapkit (top level)	104	3	20	3	95.16%
pcapkit/const	18858	954	2366	868	91.06%
pcapkit/corekit	2366	67	760	35	96.48%
pcapkit/dumpkit	275	3	100	3	98.40%
pcapkit/foundation	2884	145	1010	35	94.14%
pcapkit/interface	138	7	52	5	93.68%
pcapkit/protocols	17166	226	4580	191	98.06%
pcapkit/toolkit	1124	170	432	8	83.55%
pcapkit/utilities	494	4	146	4	98.75%
pcapkit/vendor	4409	2342	1006	157	43.25%

Per-file detail: the coverage-html artifact of this run.