fix(toolkit): scapy walks link types, layer bounds and IPv6 extension headers as the default engine does (#1609, #1626, #1650)

docs/source/pcapkit/protocols/internet/internet.rst

@@ -37,8 +37,8 @@ A chain of IPv6 extension headers is dissected to at most
 a row, whether it follows IPv6 or another layer such as IPv4. What follows is
 kept as :class:`~pcapkit.protocols.misc.raw.Raw`, with one
 :exc:`~pcapkit.utilities.warnings.ProtocolWarning` (:issue:`1604`). The
-``pypcapfile`` toolkit stops at the same depth. The ``dpkt``, ``scapy`` and
-``pyshark`` engines leave the walk to their own parsers, which do not stop
-there, so they may still find the upper layer past it (:issue:`1609`).
+``pypcapfile``, ``dpkt`` and ``scapy`` toolkits stop at the same depth. The
+``pyshark`` engine leaves the walk to its own parser, which does not stop
+there, so it may still find the upper layer past it (:issue:`1609`).
 
 .. autodata:: pcapkit.protocols.internet.internet.EXTENSION_HEADER_LIMIT

docs/source/pcapkit/protocols/protocol.rst

@@ -116,8 +116,9 @@ there back out, as they do one the TCP parser rejected (:issue:`1518`). A
 protocol constructed from its own octets is not counted itself, so the count
 starts at the first layer it dissects. One chain of extension headers is also bounded on
 its own, by :data:`~pcapkit.protocols.internet.internet.EXTENSION_HEADER_LIMIT`.
-The ``dpkt``, ``scapy``, ``pyshark`` and ``pypcapfile`` engines dissect with
-their own parsers, which do not apply this bound (:issue:`1609`).
+The ``pyshark`` and ``pypcapfile`` engines dissect with their own parsers,
+which do not apply this bound (:issue:`1609`). The ``dpkt`` and ``scapy``
+toolkits count the layers they walk as the default engine does, and stop at it.
 
 .. autodata:: pcapkit.protocols.protocol.FRAME_LAYER_LIMIT
 

tests/foundation/engines/test_ext_chain_limit_agreement_1604_unit.py

@@ -14,11 +14,13 @@
 over IPv6 out, so it is compared over IPv4 only, and ``declined`` checks that
 it warns of leaving out TCP over IPv6 exactly where ``default`` finds some. It
 imports on Python 3.11 only, and is skipped elsewhere. ``dpkt``'s adapters walk
-the chain off the octets, and stop at the same depth too (#1609).
+the chain off the octets, and stop at the same depth too. ``scapy`` walks the
+chain over the octets by the default engine's rules, rather than through Scapy's
+own layers, so stops there as well (#1609).
 
-The other parsers walk the chain themselves, and cannot be told where to stop,
-so where they find the TCP past the limit is recorded in :attr:`KNOWN_FAILURES`
-under #1609 rather than hidden. Measured on scapy 2.7.0 and tshark 4.6.9.
+The other parser, ``pyshark``, walks the chain through tshark, and cannot be
+told where to stop, so where it finds the TCP past the limit is recorded in
+:attr:`KNOWN_FAILURES` under #1609 rather than hidden. Measured on tshark 4.6.9.
 
 Every capture is built in a temporary directory, so no fixture is read.
 
@@ -156,10 +158,6 @@ class TestExtensionHeaderChainAgreement(harness.RoundTripBase):
 
     STATUSES = ('ERROR', 'INPUTS', 'DECLINED')
     KNOWN_FAILURES = (
-        Gap(1609, 'scapy dissects each extension header behind the last, with no limit of its own, '
-                  'so finds the TCP past EXTENSION_HEADER_LIMIT; it gives up near 245 layers at the '
-                  'default recursion limit, and so agrees at 1,000',
-            'INPUTS', 'extra, default has none', ('ipv6-33/scapy/*',)),
         Gap(1609, 'tshark dissects up to about 496 extension headers, so finds the TCP past '
                   'EXTENSION_HEADER_LIMIT, carried in IPv6; at 1,000 it gives up and agrees',
             'INPUTS', 'extra, default has none', ('ipv6-33/pyshark/trace',)),

tests/toolkit/test_scapy_ipv6_fragment_and_encap_walk_1635_1636_unit.py

@@ -14,18 +14,16 @@
 and its engine handed those to TCP reassembly and flow tracing, and to IPv4 and
 IPv6 reassembly (#1636).
 
-The first layer is the one the link type selects. scapy takes it to carry
-whatever `Scapy`_ dissects in it, and a layer behind it to carry IP only if it is
-a VLAN tag, IP or an extension header. So at link types the default engine
-leaves raw, where `Scapy`_ dissects two layers or more before IP -- HDLC-like
-framing then PPP, 802.11 then LLC -- scapy reads no IP either. At those where it
-dissects IP straight behind the first layer, such as LINUX_SLL, scapy still
-reads what the default engine leaves raw, so no case here is one of those.
-
-Only scapy is compared with the default engine: dpkt gives no IPv6 reassembly
-input for a 6in4 fragment (#1637). Behind more than 32 extension headers scapy
-still reads what the default engine leaves raw, which #1609 tracks, so no case
-here is that deep.
+The first layer is the one the link type selects, and a layer behind it carries
+IP only if it is a VLAN tag, IP or an extension header. So at link types the
+default engine leaves raw, where `Scapy`_ dissects two layers or more before IP
+-- HDLC-like framing then PPP, 802.11 then LLC -- scapy reads no IP either. Nor
+does it at those where `Scapy`_ dissects IP straight behind the first layer,
+such as LINUX_SLL, which
+:mod:`tests.toolkit.test_scapy_linktype_and_layer_bounds_1609_1626_1650_unit`
+covers with the frame and extension header bounds (#1650, #1609).
+
+Only scapy is compared with the default engine.
 
 Every capture is built here, so the module is in the unit tier.
 

tests/toolkit/test_scapy_ipv6_payload_and_icmp_quote_1623_1627_unit.py

@@ -14,9 +14,7 @@
 and that IPv4 to IPv4 reassembly. ``packet['IP']`` also reached the IPv4 inside
 the IPv6 packet a Redirected Header option holds, in a field rather than a payload.
 
-dpkt is compared with the default engine as well. Behind more than 32 extension
-headers, scapy still hands IPv4 reassembly the IPv4 the default engine leaves
-raw, which #1609 tracks, so that case is left out here.
+dpkt is compared with the default engine as well.
 
 Every capture is built here, so the module is in the unit tier.
 
@@ -67,9 +65,8 @@ def redirect(packet: bytes) -> bytes:
 
 
 def ipv4_cases() -> 'dict[str, list[bytes]]':
-    """#1615/#1616's frames by case, less #1609's, then those of #1623 and the ICMP quotes."""
+    """#1615/#1616's frames by case, then those of #1623 and the ICMP quotes."""
     built = ipv4_input.cases()
-    del built['4in6 behind 33 extension headers']   # #1609
     ethernet, ipv6 = ipv4_input.ethernet, ipv4_input.ipv6
 
     def first(ident: int) -> bytes: