[JarryShaw]

Please follow the guide below

• Searched for similar pull requests
• Followed the coding style (make pylint, make mypy, make isort)
• make test passes, and a test case covers the change. Targeted selections only; see below.
• Added a changelog entry: N/A — centralised in docs(changelog): shared 1.5.0 changelog — long-lived, merges last (#610, #616, #617, #618, #620) #657

What is the purpose of your pull request?

• fix — corrects a defect

---

Description of your pull request and other information

This makes the pypcapfile engine's TCP agree with the default engine's. All of it is in pcapkit/toolkit/pypcapfile.py.

• fix(toolkit): pypcapfile engine silently drops TCP in VLAN-tagged or tunnelled frames #1537, VLAN/QinQ, read through. _network() steps over 0x8100/0x88A8 tags and decodes the IPv4 behind them with pcapfile's own IP decoder. A tagged IPv4 that will not decode warns in the same words as the engine's warning for an untagged one, once per frame.
• fix(toolkit): pypcapfile engine silently drops TCP in VLAN-tagged or tunnelled frames #1537, warnings. TCP over IPv6 gets fix(toolkit): the pypcapfile engine warns that it leaves TCP over IPv6 out (#1513) #1535's warn-once, and TCP tunnelled in IP (4in4, 6in4, 6in6, 4in6) gets one of its own. Tunnels are not read through, because the default engine keys a tunnelled flow by the outer addresses.
• fix(toolkit): pypcapfile engine silently drops TCP in VLAN-tagged or tunnelled frames #1537, when the warnings fire. The decision is made from the raw headers, by the default engine's own rules: IP lengths and Jumbo Payload, later fragments, a Data Offset of at least 5 words, and, in a tunnel, a TCP header that fits within the inner packet. Nothing is dissected, and neither the warnings filters nor the logger are touched.
• Residual, by design. Malformed tunnelled TCP options, or outer IPv4 options, that the default engine rejects still warn. On 40,000 fuzzed frames (two seeds) that is 2,070 warnings without TCP: 2,066 rejected TCP options and 4 rejected outer IPv4 headers. The fuzzer gives a third of its TCP headers malformed options by design. It found no missed warning.
• fix(toolkit): the pypcapfile engine parses a later IPv4 fragment's data as a TCP header #1576. A later IPv4 fragment is never read as a TCP header.
• fix(toolkit): the pypcapfile engine's TCP payload includes the frame's Ethernet padding #1577. The TCP segment ends at the IPv4 Total Length. A Total Length of 0 (TSO) keeps the rest of the frame.

Tests (3.11 parity venv / 3.14 repo venv):

• The pypcapfile toolkit and engine files give 76 passed and 59 passed, 17 skipped.
• On origin/main's toolkit the test file fails 56 times: 11 tests and 45 subtests. Each fix's tests also fail with that fix reverted alone.
• The pypcapfile agreement rows are unchanged: 207 labels, 0 failures, 92 skips.
• pcapkit/toolkit/pypcapfile.py coverage goes from 100% to 100% on 3.11 and from 98% to 99% on 3.14.
• 10k snaplen-cut tunnelled SYNs take 0.13–0.17 s, against 0.10–0.14 s on main.

Closes #1537
Closes #1576
Closes #1577

[JarryShaw]

Verdict: NEEDS CHANGES at 98e8606e4. The changes are small and confined to behaviour this PR introduces. Cross-reviewed on Sonnet; the author ran Opus.

Needed:

1. Malformed tagged IPv4 is dropped silently. A VLAN-tagged frame with IHL below 5 is dropped without a word, while an untagged one gets the "decoding failed" warning. The _network docstring says the tagged path behaves "as an untagged one is". Make the two paths behave the same.
2. The tunnel warning can fire when the default engine sees no TCP. It also fires for tunnelled TCP whose header the default parser rejects, for example a Data Offset past the captured data, or malformed options. That was about 0.3% of fuzzed frames. Either tighten the check or correct the _upper_layer docstring, which says it "fires on the frames the default engine finds TCP in".

Confirmed:

• fix(toolkit): pypcapfile engine silently drops TCP in VLAN-tagged or tunnelled frames #1537: tagged IPv4 matches the default engine field for field on tcp/trace/ipv4 inputs, datagrams and flows, across eight tag stacks. Re-tagging http.pcap (1,117 TCP frames) agrees under all four tag modes. ARP behind a VLAN, unknown EtherTypes and 20 mid-tag truncations don't crash. 35,000 fuzzed frames show no unexplained divergence.
• fix(toolkit): the pypcapfile engine parses a later IPv4 fragment's data as a TCP header #1576: a later fragment is never parsed as TCP. Across 26 fragment scenarios the head matches the default engine, while main differs on 16.
• fix(toolkit): the pypcapfile engine's TCP payload includes the frame's Ethernet padding #1577: about 60 Total Length cases agree with the default engine. The padding in the default engine's IPv4 reassembly input is clipped by the reassembler (foundation/reassembly/ip.py:178-186), so its output is padding-free. That is not a defect.
• Fail-before:
  • reverting fix(toolkit): the pypcapfile engine parses a later IPv4 fragment's data as a TCP header #1576 alone gives 4 failures;
  • reverting fix(toolkit): the pypcapfile engine's TCP payload includes the frame's Ethernet padding #1577 alone gives 6;
  • main's toolkit gives 38 (31 subtests and 7 tests), against the 28 in the body.
• Merges: cleanly with fix(engines): the pypcapfile engine reads the PCAP records pypcapfile drops (#1512) #1569.

Not this PR's: the review also found pypcapfile divergences that main already has. They are being verified, and will be filed separately.

[github-actions]

Coverage: 89.64% (unit tier, Python 3.14, 1de91947b, Unit Tests run success)

Package	Statements	Missed	Branches	Partial	Cover
pcapkit (top level)	104	4	20	4	93.55%
pcapkit/const	18858	957	2366	871	91.03%
pcapkit/corekit	2334	67	746	35	96.43%
pcapkit/dumpkit	275	3	100	3	98.40%
pcapkit/foundation	2792	154	984	42	93.43%
pcapkit/interface	138	7	52	5	93.68%
pcapkit/protocols	16988	230	4540	191	98.02%
pcapkit/toolkit	770	96	266	6	86.68%
pcapkit/utilities	431	4	124	4	98.56%
pcapkit/vendor	4409	2342	1006	157	43.25%

Per-file detail: the coverage-html artifact of this run.

[JarryShaw]

Verdict: NEEDS CHANGES at a01fc0d4d. Sonnet delta review; the author ran Opus.

Blocking: the new tunnel check silences warnings process-wide. _default_finds_tcp (toolkit/pypcapfile.py:591-608) wraps a trial dissection in warnings.catch_warnings() + simplefilter('ignore') and sets the pcapkit root logger to CRITICAL+1. I confirmed this in the code. Measured:

• Dedup broken: a user warning under the default filter reappears after every trial. This is the v1.4.1 registry defect that BaseWarning documents (utilities/warnings.py:146).
• Other threads' output lost: another thread's warnings.warn and pcapkit log record vanish while a trial runs, and a filter installed mid-trial is wiped on exit.
• Logger stuck silent: the logger level is read without a lock. In 17 of 30 two-thread rounds it stayed at 51 permanently.
• Unbounded cost: a candidate the default engine rejects is re-dissected on every frame, and twice per frame. 10k snaplen-cut tunnelled SYNs went from 0.129 s to 9.52 s (74×).

Decision: drop the trial dissection. Detect from the header walk, plus a cheap TCP sanity check (Data Offset ≥ 5 and doff*4 within the captured segment). Say in the docstring that a malformed tunnelled header the default engine would reject can still warn.

Confirmed:

• The tagged IHL < 5 warning matches the untagged one across 8100/88A8/QinQ/4×.
• 10,000 fresh fuzz frames show no warn-without-TCP case.
• The IPv6 Payload/Jumbo bound agrees 75 of 75.
• The PR merges cleanly with fix(engines): the pypcapfile engine reads the PCAP records pypcapfile drops (#1512) #1569.

[JarryShaw]

Verdict: GOOD TO GO at 1de91947b. Round 3 was a Sonnet delta review; the author ran Opus. It merges cleanly with #1569.

• No process-global state left. The diff adds no catch_warnings, simplefilter, setLevel or logger use. I checked that myself.
  • The reviewer's probes reproduce every round-2 defect on a01fc0d4d and none on this head: dedup intact, concurrent warnings and logs all seen, a mid-detection filter kept, and the logger level unchanged in 30 of 30 two-thread rounds.
  • warnings._filters_mutated is called 0 times over 50 detections; the old head called it 150 times.
• Performance is back to normal. The 10k snaplen-cut tunnelled SYNs take 0.16 s, against 0.13 s on main and 9.4 s on the old head. http.pcap and a 50k-frame capture are flat.
• The residual is exactly as documented. All 744 warn-without-TCP cases in a malformed-options fuzz are option rejections, and 0 are a capture-cut header. That fuzz covered 10,000 frames, seeds 11–14. No warnings were missed; main drops 617 frames silently on the same fuzz.
• The IPv6 fit-check skip holds. Directly over IPv6, the default engine still finds TCP through its raw fallback, so warning there is right. Inside tunnels it doesn't, so not warning is right.
• Fail-before: the new dedup test fails on the old head, and main's toolkit fails 56.

Nit: walk.call_count == 16 pins a private call count, so assertLessEqual would be sturdier.

Filed as #1596 (not this PR's): pypcapfile accepts IPv4 options the default engine rejects. On main, a malformed option length gives 1 TCP datagram on pypcapfile and 0 on the default engine.