From 0c8ec5772f9a42164c23223f1f29a0ce2bba4157 Mon Sep 17 00:00:00 2001 From: Jarry Shaw Date: Sun, 25 Oct 2026 17:47:39 -0400 Subject: [PATCH] docs(changelog): sixth whole-repo pass over changelog (#1103) - Add 1.5.0 entries for the round-5 merges with user-visible effect: the breaking PCAPNG.context rename to section_context (#2172, #2205) with its migration line; the registrar code check (#2186); the corekit exports and ListField/SchemaField copy fixes (#2174, #2173); and the ARP, Quick-Start/Fast Open, PCAP-NG, IPX, HIP and message fixes (#2166-#2171). - Correct the systemd journal entries (#678, #704, #723, #794) to the strict walk: a malformed entry keeps the block as captured (#1406). - Restate entries without review/CI process narrative or "unfixed tree". - Extend the intro's cited range to #2205; regenerate CHANGELOG.md. --- CHANGELOG.md | 34 +++++--- docs/source/changelog/1.5.0.rst | 141 +++++++++++++++++++++----------- 2 files changed, 114 insertions(+), 61 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index e916cb5984..a2e3d1fb84 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,6 +1,6 @@ ## 1.5.0 -- unreleased -The largest release since 1.0, and the first recorded here as it happened rather than reconstructed. New extraction engines, ESP with payload decryption, SCTP and NGAP over SCTP, a library logger that no longer hijacks the consumer's, and a defect programme run through the issue tracker, citing issues and pull requests from [#340](https://github.com/JarryShaw/PyPCAPKit/pull/340) to [#2151](https://github.com/JarryShaw/PyPCAPKit/pull/2151). +The largest release since 1.0, and the first recorded here as it happened rather than reconstructed. New extraction engines, ESP with payload decryption, SCTP and NGAP over SCTP, a library logger that no longer hijacks the consumer's, and a defect programme run through the issue tracker, citing issues and pull requests from [#340](https://github.com/JarryShaw/PyPCAPKit/pull/340) to [#2205](https://github.com/JarryShaw/PyPCAPKit/pull/2205). Preceded by `1.5.0a1` (2026-09-15), `1.5.0b1` to `1.5.0b3` (all 2026-09-19) and `1.5.0b4` to `1.5.0b14` (2026-09-20 to 2026-10-22), dated in UTC and all published as prereleases and so resolved only by `pip install --pre`. `1.5.0b1` half-shipped: the tag, the GitHub release and the Conda deployments landed, but PyPI rejected the wheel because `twine check` found a Sphinx-only `:mod:` role in `README.rst`, which `pyproject.toml` declares as the dynamic long description. `1.5.0b2` reshipped it -- an existing version cannot republish -- and `1.5.0b3` followed the CI change that stops a TestPyPI outage from costing a release its wheels ([#497](https://github.com/JarryShaw/PyPCAPKit/pull/497), [#498](https://github.com/JarryShaw/PyPCAPKit/pull/498)). @@ -83,7 +83,7 @@ Preceded by `1.5.0a1` (2026-09-15), `1.5.0b1` to `1.5.0b3` (all 2026-09-19) and - **a breaking change to** `EnumRegistry.register()`: it now raises `ValueError` naming the existing member and pointing at `register_alias()`, where it used to silently alias an already-registered value under the caller's new name -- `aenum` treats a taken value as an alias request, so `register(existing_value, 'TOTALLY_NEW_NAME')` returned the existing member, minted nothing and raised nothing, contradicting its docstring. `pcapkit.corekit.enum.EnumRegistry` is a new mixin implementing `get`, `get_all`, `register`, `register_alias`, `register_aliases` and `_unregistered_member` once, per the owner's ruling on [#842](https://github.com/JarryShaw/PyPCAPKit/issues/842); it first converts registries with no bespoke `__new__` (`ipv6.extension_header.ExtensionHeader`, `tcp.flags.Flags` and the `mh.*_flag` modules), and serves `IntEnum`, `IntFlag` and `StrEnum` alike. The guard reaches only those: the other const registries still carry the old generator's bare `extend_enum(cls, name, value)`, so `pcapkit.const` ships contradictory `register` contracts until [#858](https://github.com/JarryShaw/PyPCAPKit/pull/858) (below). These also gain the generated template's `KeyError` catching on a missing name, so `Flags.get('NOPE', 2048)` returns `Flags.ACK` rather than raising; and a key that is neither `int` nor `str`, which raised `TypeError` on the `IntFlag` registries (`Flags.get(3.5)`), now raises `ValueError`, tried as a value. On `ExtensionHeader` it raised `KeyError`, the shift [#858](https://github.com/JarryShaw/PyPCAPKit/pull/858)'s entry attributes to the other registries ([#855](https://github.com/JarryShaw/PyPCAPKit/pull/855)). - **a breaking change to** `EnumRegistry.register()`, extended: the const registries still on the generated template's bare `extend_enum` now inherit the guard [#855](https://github.com/JarryShaw/PyPCAPKit/pull/855) (above) added, so `TransType.register(6, 'TOTALLY_NEW_NAME')` raises `ValueError` naming the value and the member that holds it and pointing at `register_alias()`, where it used to mint and raise nothing. `pcapkit/vendor/default.py`'s template now emits `class {NAME}(EnumRegistry, IntEnum)` and drops its own `get`/`register`/`_unregistered_member`. The modules with a bespoke `__new__` (`ftp/command`, `ftp/return_code`, `http/method`, `http/status_code`, `pcapng/option_type` and `reg/apptype` among them) are left alone, so **every other class** inherits `EnumRegistry` ([#855](https://github.com/JarryShaw/PyPCAPKit/pull/855)). `get()`'s dispatch shifts only for a key that is neither `int` nor `str` (`KeyError`, tried as a name; now `ValueError`, tried as a value), a key type nothing in the library passes. `pcapkit/corekit/enums.py` is renamed `enum.py`, the owner's ruling ([#858](https://github.com/JarryShaw/PyPCAPKit/pull/858)). - `EnumRegistry.get()`'s `str`-key branch tried only a name lookup (`cls._member_map_[key]`) and on a miss raised or returned `cls(default)`, never the *value* path, so on a `StrEnum` registry a key that resolved through the constructor (`_Str('known-value')`) raised `KeyError` through `get()`, and a supplied `default` overrode a good value match the code never attempted. It now falls back to a plain `_value2member_map_` lookup once the name lookup misses and before consulting `default`. Deliberately not `cls(key)`: `FEATCode._missing_()` (`pcapkit/const/ftp/command.py`) mints via `extend_enum` for any unrecognised string, so routing through the constructor would let a failed name lookup mint a permanent member once a registry like it converts; a dict lookup never reaches `_missing_`. A name still wins when a string is both a name and a different member's value. Step 1 of [#860](https://github.com/JarryShaw/PyPCAPKit/issues/860); step 2, converting the bespoke registries, follows ([#869](https://github.com/JarryShaw/PyPCAPKit/pull/869), [#874](https://github.com/JarryShaw/PyPCAPKit/pull/874), below). The non-`str` path is unchanged. Widens rather than breaks: a call that raised `KeyError` now resolves, and no caller that got a correct result gets a different one ([#863](https://github.com/JarryShaw/PyPCAPKit/pull/863)). -- **a breaking change to** `EnumRegistry.get()`: its docstring claimed "It never mints", but each `cls(default)` call site reached `_missing_` for a `default` inside a still-minting registry's range, growing the registry as a side effect of resolving `default` rather than `key`. Per the owner's ruling that only `register()` mints, and `get()` only through the ranges `_missing_` mints, each site now resolves `default` through a plain `_value2member_map_` lookup; `key` resolution is unchanged, so `EtherType.get(0x0888)` still mints `Xyplex_0x0888` through its own `_missing_`, as [#775](https://github.com/JarryShaw/PyPCAPKit/issues/775)'s deliberate exception permits. A `default` naming no registered member now falls through to the error `key` itself would have raised, rather than a fresh error about the default -- the accepted cost of the ruling. The docstring's "It never mints" is qualified to `default` only. So `Hardware.get('Definitely-Not-A-Member', 40)`, with `40` in `Hardware`'s declared-but-unassigned range, now raises `KeyError` naming the original key; its test waited on [#865](https://github.com/JarryShaw/PyPCAPKit/pull/865) (above), then landed once [#865](https://github.com/JarryShaw/PyPCAPKit/pull/865) merged. Closes [#864](https://github.com/JarryShaw/PyPCAPKit/issues/864) ([#868](https://github.com/JarryShaw/PyPCAPKit/pull/868)). +- **a breaking change to** `EnumRegistry.get()`: its docstring claimed "It never mints", but each `cls(default)` call site reached `_missing_` for a `default` inside a still-minting registry's range, growing the registry as a side effect of resolving `default` rather than `key`. Per the owner's ruling that only `register()` mints, and `get()` only through the ranges `_missing_` mints, each site now resolves `default` through a plain `_value2member_map_` lookup; `key` resolution is unchanged, so `EtherType.get(0x0888)` still mints `Xyplex_0x0888` through its own `_missing_`, as [#775](https://github.com/JarryShaw/PyPCAPKit/issues/775)'s deliberate exception permits. A `default` naming no registered member now falls through to the error `key` itself would have raised, rather than a fresh error about the default -- the accepted cost of the ruling. The docstring's "It never mints" is qualified to `default` only. So `Hardware.get('Definitely-Not-A-Member', 40)`, with `40` in `Hardware`'s declared-but-unassigned range, now raises `KeyError` naming the original key. Closes [#864](https://github.com/JarryShaw/PyPCAPKit/issues/864) ([#868](https://github.com/JarryShaw/PyPCAPKit/pull/868)). - `ProtoChain.__add__()` returned the left operand's cached `protocols` and `aliases`: once `a.protocols` had been read, `a + ProtoChain(IPv4)` reported `chain == 'Ethernet:IPv4'` but `protocols == (Ethernet,)`. `copy.copy(self)` carried the cache across; the result is now built fresh, as `from_list` builds one, and `+=` follows, since it falls back to `__add__` ([#1099](https://github.com/JarryShaw/PyPCAPKit/issues/1099)). - a `SchemaField` with no declared length handed its `-1` placeholder to the nested schema as that schema's length, so every item of such a list warned `packet length < 0`: a HIP `LOCATOR_SET` and an MH CGA Parameters item among them. `SchemaField.unpack()` now passes what is left in the buffer as a declared length -- not `None`, which would turn a truncated item's `FieldValueError` into `prepare()`'s quiet `StreamEOFError`. Fixed-length items are unchanged; the HTTP/2 `SETTINGS` pair, whose size is fixed, is given it in its own schema instead ([#1140](https://github.com/JarryShaw/PyPCAPKit/issues/1140), below) ([#1148](https://github.com/JarryShaw/PyPCAPKit/issues/1148)). - a signed `NumberField` decoded as unsigned, a zero-length `PayloadField` misread, and an empty `Raw` did not rebuild ([#1331](https://github.com/JarryShaw/PyPCAPKit/pull/1331)). @@ -103,6 +103,8 @@ Preceded by `1.5.0a1` (2026-09-15), `1.5.0b1` to `1.5.0b3` (all 2026-09-19) and - `EnumLookup.get()` answers a value miss quietly, whether `default` answers it or the miss is raised: a registry's `_validate_value` rejection logs nothing and installs no exception hook there, while `EnumRegistry.register()` still reports the same rejection loudly ([#1845](https://github.com/JarryShaw/PyPCAPKit/issues/1845), [#1873](https://github.com/JarryShaw/PyPCAPKit/pull/1873)). - each sentinel is a singleton, as `None` is: calling its type, `copy`, `deepcopy` and `pickle` at every protocol give back the one object, and `NO_VALUE` reprs as its own name rather than as an object address. Their new common base, `Sentinel`, is public; a sentinel's type cannot be subclassed, and neither it nor the sentinel takes an attribute write. A pickle written before this change at protocol 0 or 1 still loads, though not as the singleton ([#1841](https://github.com/JarryShaw/PyPCAPKit/issues/1841), [#1842](https://github.com/JarryShaw/PyPCAPKit/issues/1842), [#1876](https://github.com/JarryShaw/PyPCAPKit/pull/1876)). - defects off CPython on Linux. On GraalPy, `SeekableReader`'s reads aliased a buffer a later slide rewrote, so they copy out of it, and a `MultiDict` nested each value list, so it is built from a `dict` ([#1963](https://github.com/JarryShaw/PyPCAPKit/issues/1963), [#1974](https://github.com/JarryShaw/PyPCAPKit/pull/1974)); and a sentinel type refused the cache writes `_py_abc` makes, so `isinstance` against it raised `TypeError` ([#1960](https://github.com/JarryShaw/PyPCAPKit/issues/1960), [#1966](https://github.com/JarryShaw/PyPCAPKit/pull/1966)). `Field.pack` hands `struct` a `bytearray` as `bytes` for the `s`, `p` and `c` formats, which PyPy refuses for `s` and `p` and GraalPy accepts for `c` where CPython does not, so it packs alike on every interpreter ([#2004](https://github.com/JarryShaw/PyPCAPKit/issues/2004), [#2007](https://github.com/JarryShaw/PyPCAPKit/pull/2007), [#2015](https://github.com/JarryShaw/PyPCAPKit/issues/2015), [#2018](https://github.com/JarryShaw/PyPCAPKit/pull/2018)). On Windows, `SeekableReader(buffer_save=True)` with no `buffer_path` could not reopen its temporary file; it creates one it can, and deletes it on `close()` or collection ([#2098](https://github.com/JarryShaw/PyPCAPKit/issues/2098), [#2105](https://github.com/JarryShaw/PyPCAPKit/pull/2105)). +- `PortEnumField` and `SwitchField` are exported from `pcapkit.corekit`, `pcapkit.corekit.fields` and `pcapkit.all`, and `SchemaField`, `nested_packet_context` and `BytesField` from the modules defining them, so `import *` binds them ([#2174](https://github.com/JarryShaw/PyPCAPKit/issues/2174), [#2191](https://github.com/JarryShaw/PyPCAPKit/pull/2191)). +- a `ListField` callback is handed the per-item copy it configures, not the shared field, and a `SchemaField` with a callable length sizes its template from the resolved length rather than `1024s`. No parse in the library reached either ([#2173](https://github.com/JarryShaw/PyPCAPKit/issues/2173), [#2190](https://github.com/JarryShaw/PyPCAPKit/pull/2190)). ### pcapkit.dumpkit @@ -167,6 +169,7 @@ Preceded by `1.5.0a1` (2026-09-15), `1.5.0b1` to `1.5.0b3` (all 2026-09-19) and - an `Extractor` whose constructor raises closes what it owns, the input it opened and then the engine it chose, so a PyShark capture's `tshark` does not outlive it; the original exception still propagates, and a caller's stream stays open ([#2100](https://github.com/JarryShaw/PyPCAPKit/issues/2100), [#2106](https://github.com/JarryShaw/PyPCAPKit/pull/2106), [#2107](https://github.com/JarryShaw/PyPCAPKit/issues/2107), [#2119](https://github.com/JarryShaw/PyPCAPKit/pull/2119)). - `engine='pypcap'` warns and falls back to the default engine when the `pcap` module is installed but cannot load, as `engine='pcap_ct'` already did, where it raised the loader's `OSError` ([#2132](https://github.com/JarryShaw/PyPCAPKit/issues/2132), [#2133](https://github.com/JarryShaw/PyPCAPKit/pull/2133)). - TCP reassembly bounded its holes with `sys.maxsize`, which on GraalPy and 32-bit builds is `2**31 - 1`, inside TCP's sequence space, so a stream past it never completed. The bound is `HOLE_OPEN_BOUND`, `2**63 - 1`, the value 64-bit builds already used ([#1959](https://github.com/JarryShaw/PyPCAPKit/issues/1959), [#1965](https://github.com/JarryShaw/PyPCAPKit/pull/1965)). +- the method-name registrars, such as `register_ipv4_option`, and `register_linktype`, `register_pcap`, `register_pcapng`, `register_ethertype` and `register_transtype` refuse a `code` that is not an enum member with `RegistryError` before touching a registry, where they registered it and then raised `AttributeError`, leaving the entry behind ([#2186](https://github.com/JarryShaw/PyPCAPKit/issues/2186), [#2193](https://github.com/JarryShaw/PyPCAPKit/pull/2193)). ### pcapkit.interface @@ -221,6 +224,7 @@ Layer-limited extraction: `layer='link'` no longer sets the termination flag on - **a breaking change to** the HOPOPT, IPv6-Opts and Mobility Header padding: two octets are one `PadN` with no data (`01 00`) rather than two `Pad1` options, as [RFC 8200 Section 4.2](https://datatracker.ietf.org/doc/html/rfc8200#section-4.2) and [RFC 6275 Section 6.2.2](https://datatracker.ietf.org/doc/html/rfc6275#section-6.2.2) ask. Only a freshly built header with exactly two octets of padding changes; a parsed one that carried two `Pad1` options still rebuilds byte for byte ([#1793](https://github.com/JarryShaw/PyPCAPKit/issues/1793), [#1812](https://github.com/JarryShaw/PyPCAPKit/pull/1812)). - **a breaking change to** `NTPTimestamp`: it and `PMIPv6Timestamp` are `Info` classes in `pcapkit.protocols.data.internet.mh` rather than tuples, and `NTPTimestamp` keeps no alias at its 1.4.x home in `pcapkit.protocols.internet.mh`. As Mappings they no longer index, unpack, compare or hash as a tuple does; `.seconds` and `.fraction` still work, and the dumpers write both as keys ([#1886](https://github.com/JarryShaw/PyPCAPKit/issues/1886), [#1887](https://github.com/JarryShaw/PyPCAPKit/pull/1887)). - **a breaking change to** rebuilding from `to_dict()`: the payload is dispatched only from the `__next_name__` and `__next_type__` keys, and the IPv6 extension-header chain only from `__exthdr__`, never from key order, which on PyPy follows an earlier instance's attribute layout and so picked the wrong layer. `to_dict()`, iteration and `len()` carry the dunder keys a new `Info.__included__` names, and `Info.__excluded__` only keeps a key out of `repr`, `str` and the dumps. A hand-written dict needs those keys, `__short_read__` among them, to rebuild, and a custom `Info` lists its own dunder keys in `__included__` ([#2003](https://github.com/JarryShaw/PyPCAPKit/issues/2003), [#2014](https://github.com/JarryShaw/PyPCAPKit/pull/2014)). +- **a breaking change to** `PCAPNG.context`: the block's section `Context`, which it returned from 1.0.0 through 1.4.x, is `PCAPNG.section_context`, with no alias, and `context` is the caller supplied `ContextRegistry` every protocol inherits from `ProtocolBase`, on a Section Header Block too. `PCAPNG.section_context` raises `UnsupportedCall` on a Section Header Block, as `context` did. Code reading `block.context` for the section reads `block.section_context` ([#2172](https://github.com/JarryShaw/PyPCAPKit/issues/2172), [#2205](https://github.com/JarryShaw/PyPCAPKit/pull/2205)). #### Fixed @@ -247,7 +251,7 @@ This resolves [#548](https://github.com/JarryShaw/PyPCAPKit/issues/548), which r - as a consequence of the above, the unhandled `MemoryError` in `pcapkit/protocols/protocol.py`'s `self._file.read()` call on a truncated PCAP-NG capture. `dhcp_little_endian.pcapng` cut to 161, 641 or 1389 octets left a one-octet read of a little-endian 32-bit block length, which `rjust()` turned into `0x78000000` or `0x84000000` (1.88 to 2.06 GiB) and passed straight to `self._file.read()` as an allocation size, from a 1772-octet file. Under a 1 GiB address-space cap all three raise `MemoryError`; with more space the parse fails anyway. With `ljust()` the same reads report 120 and 132, no large allocation is attempted, and all three end in the ordinary handled parse failure ([#604](https://github.com/JarryShaw/PyPCAPKit/issues/604)). - reading a big-endian classic PCAP byte-swapped every record header field, and then crashed. `Frame.unpack()` seeded the file's byte order under the key `bytesorder` where the schema's `byteorder_callback` reads `byteorder`, so the lookup never found it and fell back to `sys.byteorder`, the host's order rather than the file's. On a little-endian host reading a little-endian capture that is right by coincidence, and every capture in this repository was little-endian. Frame 1 of `big_endian.pcap` read `incl_len=1241513984` for a record whose real value is `74`. `incl_len` is the payload length, so that record consumed the whole file and the second was read with a negative length, raising `ValueError: read length must be non-negative or -1` -- the reported crash, and the *second* symptom. The sibling `Frame.pack()` spelled the key correctly, marking this a slip rather than a second key, and the fallback made a misspelled key indistinguishable from an absent one; `byteorder_callback` now records that it defines the key and why the fallback hides a typo ([#605](https://github.com/JarryShaw/PyPCAPKit/issues/605)). - documentation. `mptcp_dss_ack_selector`'s note said a corrected field-width lambda "would not have worked" and that fixing it belonged to `pcapkit.corekit.fields.numbers`, which is where [#598](https://github.com/JarryShaw/PyPCAPKit/pull/598) then fixed it; the same paragraph sat in `test_tcp_mptcp_length_arithmetic_unit.py`'s docstring, whose other stale claim was that MP_JOIN "cannot be built through the public `TCP()` constructor at all", true only until [#587](https://github.com/JarryShaw/PyPCAPKit/issues/587). A callable-length `NumberField` packs and unpacks both DSS widths now, and wire *absence* was never the obstacle: `MPTCPDSS.ssn`, `dl_len` and `checksum` have always been `ConditionalField` on the sibling `M` flag. The `SwitchField` form is kept for the narrower reason the note now gives: `ConditionalField`'s `length` forwards to the wrapped field without consulting the condition, whereas a `SwitchField` always resolves to a concrete field ([#603](https://github.com/JarryShaw/PyPCAPKit/issues/603)). -- the assertion [#604](https://github.com/JarryShaw/PyPCAPKit/issues/604) left pinning the old padding side, which [#621](https://github.com/JarryShaw/PyPCAPKit/pull/621) turned red on `main`. `TCPUDPUnitTests.test_a_truncated_option_still_parses_its_declared_length` expected a truncated TCP option's `data` as the synthesised zero octets *followed by* the real ones, as `rjust()` produced: `Reserved_79` declaring `length=12` over 6 real octets reports `aabbccddeeff00000000` where the test expected `00000000aabbccddeeff`. [#621](https://github.com/JarryShaw/PyPCAPKit/pull/621) made the padding `ljust()` everywhere but could not retarget this file, since [#612](https://github.com/JarryShaw/PyPCAPKit/pull/612) owned it. The real octets now come first, and the docstring is corrected. Test-only; the sibling case in `tests/protocols/internet/test_ipv4_unit.py` was retargeted in [#621](https://github.com/JarryShaw/PyPCAPKit/pull/621) ([#604](https://github.com/JarryShaw/PyPCAPKit/issues/604)). +- the assertion [#604](https://github.com/JarryShaw/PyPCAPKit/issues/604) left pinning the old padding side, which fails once [#621](https://github.com/JarryShaw/PyPCAPKit/pull/621) pads with `ljust()`. `TCPUDPUnitTests.test_a_truncated_option_still_parses_its_declared_length` expected a truncated TCP option's `data` as the synthesised zero octets *followed by* the real ones, as `rjust()` produced: `Reserved_79` declaring `length=12` over 6 real octets reports `aabbccddeeff00000000` where the test expected `00000000aabbccddeeff`. The real octets now come first, and the docstring is corrected. Test-only; the sibling case in `tests/protocols/internet/test_ipv4_unit.py` was retargeted in [#621](https://github.com/JarryShaw/PyPCAPKit/pull/621) ([#604](https://github.com/JarryShaw/PyPCAPKit/issues/604)). - the HIP `SOLUTION` builder sized the parameter with `4 + math.ceil(max(random.bit_length(), solution.bit_length()) / 4)`, an invalid shorthand for two fields. [RFC 7401 Section 5.2.5](https://datatracker.ietf.org/doc/html/rfc7401#section-5.2.5) gives `Length` as `4 + RHASH_len / 4` over a `Random #I` and a `Puzzle solution #J` of `RHASH_len / 8` octets **each**, and `/ 4` equals twice `/ 8` only because `RHASH_len` is a whole number of octets. For an arbitrary `bit_length()` the builder emitted an *odd* contents width, which its own reader refused: `_read_param_solution`'s `(schema.len - 4) % 2` guard exists because `SolutionParameter` splits that width into two equal halves. `random=0x1` with `solution=0xfff` declared `len=7` and raised `ProtocolError: HIPv2: [ParamNo 321] invalid format` on input the library had produced. The undersized length was worse: both fields take their width from that `len`, so `solution=0xfff` was packed into one octet and came back as `0xff`, silently. Fixed to `4 + 2 * math.ceil(max(...) / 8)`, the form the sibling `PUZZLE` builder uses, which is even by construction. Loosening the reader was rejected: an odd contents width has no meaning in the wire format (the RFC makes the fields equal-width and does not say which takes an extra octet), and a laxer guard would still have truncated the value ([#608](https://github.com/JarryShaw/PyPCAPKit/issues/608)). - `TCP.read()` seeded its connection-flag accumulator with `cast('Enum_Flags', 0)`. `typing.cast` is a runtime no-op, so the accumulator began as the plain `int` `0`, and only the `|=` promoting it to a `pcapkit.const.tcp.flags.Flags` member ever changed that. A segment with an all-zero flags octet (an nmap NULL scan) left `self._flags` an `int`, so a membership test raised `TypeError: argument of type 'int' is not a container or iterable`, and `TCP.connection` returned an `int` where it annotates `Flags`. Any flag masked it, which is how it survived 100% statement and branch coverage. The seed is `Flags(0)`, as [#597](https://github.com/JarryShaw/PyPCAPKit/pull/597) already did for the sibling accumulator in `make`; `Flags` declares no `_missing_`, so `Flags(0)` is an ordinary `aenum.IntFlag` pseudo-member and does not meet the `RecursionError` the same construction hits on the `pcapkit.const.mh` flag enumerations ([#623](https://github.com/JarryShaw/PyPCAPKit/issues/623)). The MP_JOIN path was latent: `mptcp_data_selector` rejects a flagless MP_JOIN before [RFC 8684](https://datatracker.ietf.org/doc/html/rfc8684) section 3.2's three layouts are chosen. A dump-format change is observable: a flagless segment's `connection` renders as a string where it rendered as the number `0`, so it no longer switches JSON type with the flags. That string read `Flags::None [0]` until the `pcapkit.dumpkit` fix above ([#616](https://github.com/JarryShaw/PyPCAPKit/issues/616)). - **a breaking change to a public attribute.** `Frame.len` and `Frame.cap_len` were filled from *opposite* wire fields depending on the container format, so `frame.len` meant the captured length out of a `.pcap` and the on-wire length out of a `.pcapng`. The PCAP reader moved to match PCAP-NG: `len` is the on-wire length (the record header's `orig_len`) and `cap_len` the octets stored (`incl_len`). **Code reading** `frame.len` **or** `frame.cap_len` **from a** `.pcap` **gets the other field's value than before**, and for a truncated frame that is a different number rather than a relabelling. Seniority did not settle it: the PCAP spelling is the *older* (`c43892af`, 2022-01-11) and PCAP-NG's arrived in `25f216f4` (2023-04-27), both internally consistent. The *names* settle it, being Wireshark's: `frame.len` is "Frame length on the wire" and `frame.cap_len` "Frame length stored into the capture file", and `frame.len_lt_caplen` expert info flags `frame_len < cap_len` as malformed, which it could not be if `len` were the smaller, captured one. `Frame.read()` hands `_decode_next_layer` the octets *present*, now `frame.cap_len`; handing over the on-wire length would be the declared-length-exceeds-available-octets fault of [#554](https://github.com/JarryShaw/PyPCAPKit/issues/554), [#573](https://github.com/JarryShaw/PyPCAPKit/issues/573) and [#594](https://github.com/JarryShaw/PyPCAPKit/issues/594) on every snapped frame. The dumpers are untouched. The two lengths differ only for a snapshot-cut frame, and no such fixture existed until [#614](https://github.com/JarryShaw/PyPCAPKit/pull/614) ([#618](https://github.com/JarryShaw/PyPCAPKit/issues/618)). @@ -255,21 +259,21 @@ This resolves [#548](https://github.com/JarryShaw/PyPCAPKit/issues/548), which r - every HIP parameter pcapkit emitted was `4 (mod 8)` octets long, because the padding aligned the *contents* rather than the record. Corrected for **every HIP parameter but** `LOCATOR_SET`, excluded below. **This changes the octets those parameters write, and the** `length` **their data models report.** [RFC 7401 Section 5.2.1](https://datatracker.ietf.org/doc/html/rfc7401#section-5.2.1) requires every TLV parameter to have "a length (that includes the Type and Length fields), which is a multiple of 8 bytes" and gives `Total Length = 11 + Length - (Length + 3) % 8`; every padding site computed `(8 - (Length % 8)) % 8`, which aligns the contents alone. For every `Length` from 0 to 63 the result was never a multiple of eight and never the RFC's value, erring both ways: at `Length = 4` (a whole `SEQ`, which [RFC 7401 Section 5.3.5](https://datatracker.ietf.org/doc/html/rfc7401#section-5.3.5) puts on every `UPDATE`) pcapkit appended four octets that must not be there, and at `Length = 8` it appended none and went out four short. A conformant peer lands mid-parameter; pcapkit did not, because its reader consumed the same wrong count its writer wrote, which is why no round-trip test could see it and why the fix is asserted against the RFC's arithmetic. Every site but `LOCATOR_SET`'s (the `PaddingField` callbacks in `pcapkit/protocols/schema/internet/hip.py` and the record lengths in `pcapkit/protocols/internet/hip.py`) now goes through one `parameter_total_len` and one `parameter_padding_len` ([#651](https://github.com/JarryShaw/PyPCAPKit/issues/651)). `LOCATOR_SET` kept the old expression because defects there cancelled exactly: its padding callback never received the parameter's `len` (the nested `Locator` schemas share a packet context whose own `len` shadowed it, so it saw 4 for an IPv6 locator), and the parameter's `len` was written in 4-octet units where the RFC's `Length` is a byte count; the result, `4 + 24n + 4`, equalled the RFC total `24n + 8`. [#679](https://github.com/JarryShaw/PyPCAPKit/issues/679) later fixed both sites. `EncryptedParameter`'s `data` length callback is fixed in the same change: it subtracted the sixteen `iv` octets but not the four `reserved` ones, which cancelled the padding's four at `Length % 8` in `{0, 5, 6, 7}`, so correcting the padding alone would have made `ENCRYPTED` four octets too long at all eight residues. `HIP.make()`'s `len = total_length // 8 + 4` is not part of the defect: [RFC 7401 Section 5.1.3](https://datatracker.ietf.org/doc/html/rfc7401#section-5.1.3) defines Header Length in 8-byte units excluding the first 8 bytes, so the floor division is exact once each parameter is a multiple of eight. Migration: a `SEQ` parameter's `length` is now 8 where it was 12, and the other parameters move likewise, so code comparing stored output byte for byte or asserting on `Data_*Parameter.length` sees the RFC's values. `LOCATOR_SET` is unchanged in both respects until [#679](https://github.com/JarryShaw/PyPCAPKit/issues/679) (below). `examples/generators/options.py`'s `HIP_COPIES` stayed at two for `R1_COUNTER`'s four-octet `counter` where [RFC 7401 Section 5.2.3](https://datatracker.ietf.org/doc/html/rfc7401#section-5.2.3) requires eight, which this defect had masked, until [#672](https://github.com/JarryShaw/PyPCAPKit/issues/672) widened `counter` and [#679](https://github.com/JarryShaw/PyPCAPKit/issues/679) fixed `LOCATOR_SET`'s `Length` unit, after which [#689](https://github.com/JarryShaw/PyPCAPKit/issues/689) dropped `HIP_COPIES` to one ([#651](https://github.com/JarryShaw/PyPCAPKit/issues/651)). - HTTP/2 flag defects on each side of a round trip. **This changes the octets a reconstructed DATA frame emits, and the dumped value of a flagless frame's flags.** `_make_http_data` was the only `_make_http_*` method that never read `frame.flags`, so a DATA frame parsed with `END_STREAM` set rebuilt with the bit clear (flags octet `0x01` to `0x00`): a well-formed frame saying the stream continues where the capture said it ended [[RFC 9113 Section 6.1](https://datatracker.ietf.org/doc/html/rfc9113#section-6.1)]. `PADDED` was re-derived from `pad_len`, and the sibling builders already restored theirs, so this was a missing read-back ([#652](https://github.com/JarryShaw/PyPCAPKit/issues/652)). Separately, `FrameType.post_process()` seeded its flag accumulator with a bare `0`, which `|=` promotes only as a side effect, so a frame with **no** bit set left `__flags__` a plain `int` where the schema and data model declare a `Flags`, and `END_STREAM in __value__` raised `TypeError: argument of type 'int' is not a container or iterable` rather than answering `False`. That is the common case: a `0x00` flags octet is every opening `SETTINGS`, every non-final `DATA` and every non-ACK `PING`. It now seeds `self.Flags(0)`, as the construct path already did at each of its sites. In the dump, `__value__` was a JSON *number* for a flagless frame and a *string* for every other frame in the same capture; it is consistently a string now, `"Flags::None [0]"` where it read `0`, the same change [#616](https://github.com/JarryShaw/PyPCAPKit/issues/616) made to TCP's `connection`. The seed is guarded because `FrameType.Flags` declares no members and, from Python 3.11, a memberless `enum.Flag` subclass refuses `Flags(0)`, so the frame schemas that inherit it (`UnassignedFrame`, `PriorityFrame`, `RSTStreamFrame`, `GoawayFrame`, `WindowUpdateFrame`) keep the plain `int`, which nothing observes since each passes `flags=None`. A DATA round trip is **still** lossy for an unrelated reason found while measuring: payload length callbacks are mis-parenthesised, so an unpadded `DATA`, `HEADERS` or `PUSH_PROMISE` frame parses with its whole payload dropped ([#668](https://github.com/JarryShaw/PyPCAPKit/issues/668), below) ([#652](https://github.com/JarryShaw/PyPCAPKit/issues/652), [#650](https://github.com/JarryShaw/PyPCAPKit/issues/650)). - MPTCP error messages carried a doubled separator, rendering as `TCP: : [OptNo 30] 1: invalid flags combination`. Cosmetic: the exception type, option number and subtype were right and nothing parses these strings, but an empty field reads as a value that failed to interpolate. The sites were `_read_mptcp_join` and `_make_mptcp_join` for the invalid flag combination, and both guards of `_make_mptcp_dss` for missing required fields; every other message in the file was already spelled correctly, and none is doubled now. Pre-existing since 2023-04-10. The new test asserts each message in full, since the existing `assertIn` on the body passes either side of the change ([#649](https://github.com/JarryShaw/PyPCAPKit/issues/649)). -- names appeared in string annotations their own module never imported, so a type checker or documentation build could not resolve them while every test passed: `Any` in `pcapkit/utilities/logging.py`, and in `pcapkit/protocols/schema/internet/ipv6_route.py` both `Protocol` in a `payload:` stub and `Optional` inside a `typing.cast`. Each is now in its module's `TYPE_CHECKING` block, `Protocol` spelled `ProtocolBase as Protocol` as the sibling schema modules already do. The `cast` case is the one worth naming: `typing.cast` never evaluates its first argument, so no test that runs the line can see a bad name in it. Nothing resolves at runtime that did not before (`TYPE_CHECKING` is `False`); making these annotations resolve at runtime is a separate decision about the idiom. A new `tests/project/test_annotation_names.py` pins the invariant without a type checker: it resolves every string annotation in the package against the names its own module binds, following nested forward references such as `'list["Nested"]'` and treating `Literal` members and `Annotated` metadata as values, and reports exactly those findings on the unfixed tree. The same change corrects the `MANIFEST.in` comment asserting that `include README.md` was the only thing putting the README in a source distribution and that an sdist without it could not be installed; both halves are false, and the [#619](https://github.com/JarryShaw/PyPCAPKit/pull/619) entry above carried the same claim and is corrected with it ([#642](https://github.com/JarryShaw/PyPCAPKit/issues/642)). +- names appeared in string annotations their own module never imported, so a type checker or documentation build could not resolve them while every test passed: `Any` in `pcapkit/utilities/logging.py`, and in `pcapkit/protocols/schema/internet/ipv6_route.py` both `Protocol` in a `payload:` stub and `Optional` inside a `typing.cast`. Each is now in its module's `TYPE_CHECKING` block, `Protocol` spelled `ProtocolBase as Protocol` as the sibling schema modules already do. The `cast` case is the one worth naming: `typing.cast` never evaluates its first argument, so no test that runs the line can see a bad name in it. Nothing resolves at runtime that did not before (`TYPE_CHECKING` is `False`); making these annotations resolve at runtime is a separate decision about the idiom. A new `tests/project/test_annotation_names.py` pins the invariant without a type checker: it resolves every string annotation in the package against the names its own module binds, following nested forward references such as `'list["Nested"]'` and treating `Literal` members and `Annotated` metadata as values; without the fix, it reports exactly those findings. The same change corrects the `MANIFEST.in` comment asserting that `include README.md` was the only thing putting the README in a source distribution and that an sdist without it could not be installed; both halves are false, and the [#619](https://github.com/JarryShaw/PyPCAPKit/pull/619) entry above carried the same claim and is corrected with it ([#642](https://github.com/JarryShaw/PyPCAPKit/issues/642)). - every PCAP-NG packet block now carries the captured octets it declares. Every Enhanced Packet Block of the committed `examples/captures/dhcp.pcapng` reported `len(packet) == 0` against a `captured_len` of 314, 342, 314 and 342, as did the Simple Packet Block and obsolete Packet Block (synthesised; no committed fixture has either). Those block types, exactly `PCAPNG.PACKET_TYPES`, are the whole blast radius. The octets were read correctly and thrown away: `PCAPNG.unpack()` extracted them from the block schema, and `ProtocolBase.__init__()` overwrote the result with `self.packet.payload`. The inherited `packet` splits a protocol at `self.length` octets of header and takes the rest as payload, which holds only for a header followed by its payload; `PCAPNG.length` is the *Block Total Length*, so the split consumed the whole block as header (`len(frame.packet.header) == 348` on a 348-octet block), and a block's trailer (option list, repeated total length) means no `length` could make it right. The fix is at the computation rather than the injection: `PCAPNG.packet` is overridden to take the payload from the block schema's `__payload__` field and the header from the octets ahead of it, summed out of the schema buffers so the packet types' payload offsets (28, 12, 28) are not hard-coded. Teaching `ProtocolBase.__init__()` to skip a protocol that had set `packet` was rejected: it heals `frame.info.packet` while `frame.packet` still reports the whole block as header. The override is a plain `property`, not the `cached_property` it replaces: the inherited one caches because it *reads the stream*, where this one only walks buffers already filled, and caching it would let a second `unpack` on one instance return the *first* call's octets. This moves the parse output of every PCAP-NG capture on the success path, so it ships labelled breaking: `frame.info.packet` goes from `b''` to hundreds of octets, `frame.packet.header` from the whole block to the pre-payload prefix. It was also a wire-format defect: `PCAPIO` writes `value.packet` after each record header, so a dump declared hundreds of octets per record and delivered none, desynchronising any reader that walks by `incl_len`. It round-trips now. The uncaught `ValueError` on an EOF-truncated PCAP-NG ([#678](https://github.com/JarryShaw/PyPCAPKit/issues/678)) is untouched, and [#685](https://github.com/JarryShaw/PyPCAPKit/issues/685) (below) later removed the stale `examples/captures/pcapng.txt` from the index ([#646](https://github.com/JarryShaw/PyPCAPKit/issues/646)). - an unpadded HTTP/2 `DATA`, `HEADERS` or `PUSH_PROMISE` frame parsed with its whole payload silently discarded. The payload length callbacks for those frames in `pcapkit/protocols/schema/application/httpv2.py` put the conditional in the wrong place: a conditional binds looser than `-`, so `pkt['__length__'] - pkt['pad_len'] if pkt['flags']['bit_3'] else 0` groups as `(pkt['__length__'] - pkt['pad_len']) if ... else 0`, and the `else` arm returned `0` ("read no octets") where it meant to subtract `0`. Padding is rare in HTTP/2, so the broken arm was the common one, and nothing raised or warned: `info.data` was `b''`. An unpadded `DATA` frame declaring `b'{"ok":true}\n'` now parses as those twelve octets, and unpadded `HEADERS` and `PUSH_PROMISE` keep their header block fragment. Padded frames are unaffected, since `(A - B) if T else 0` and `A - (B if T else 0)` are both `A - B` for truthy `T`; the padded arm has no off-by-one ([#668](https://github.com/JarryShaw/PyPCAPKit/issues/668) asked that be checked), because `Schema.unpack()` decrements `packet['__length__']` by each field's width, so `pad_len`'s own octet is already out by the time the payload field runs. The same lambda also governs **packing**: an unpadded `DATA` frame carrying twelve octets packed to a nine-octet header whose length field declared **21**, body absent, so pcapkit was *emitting* malformed HTTP/2; a `make` -> parse -> `make` cycle of a non-empty unpadded payload now closes byte-for-byte. The only other conditional of this shape, `TSOption.remainder` in `pcapkit/protocols/schema/internet/ipv4.py`, is correct: its `else 0` is intentional because `ts_data` consumes the whole option data area in that arm. The new tests use non-zero padding rather than the zeros [RFC 9113 Section 6.1](https://datatracker.ietf.org/doc/html/rfc9113#section-6.1) tells a sender to use, so failing to subtract it yields a different byte string. Ships labelled breaking: restoring a dropped payload moves the parse output of essentially every HTTP/2 capture, and constructed frames change byte-for-byte, from malformed to correct ([#668](https://github.com/JarryShaw/PyPCAPKit/issues/668)). - the schema half of every option-like registrar overwrote silently while the parser half of the same call warned. `EnumSchema.register()` was a bare `cls.__enum__[code] = schema`, and is the schema half of the public option registrars, so one `register_ipv4_option` call replacing a built-in named the parser it displaced and said nothing about the schema. `EnumSchema.__init_subclass__()` reaches the same registry without calling `register`, so a subclass declared with a `code=` keyword (the documented way to add a schema) stayed silent too; both are guarded, and the declaration hook's branches are folded into one loop. Presence is a faithful test only because `_EnumRegistry.__missing__()` returns a miss without recording it, so parsing an unknown code cannot make the next legitimate registration warn. The code-keyed parser registrars (on `ProtocolBase`, `Link`, `Internet`, `Frame`, `PCAPNG`, `Transport` and `SCTP`) now name the displaced entry and its replacement, where they reported only that something was overwritten. Their firing condition is harmonised onto the narrower guard [#681](https://github.com/JarryShaw/PyPCAPKit/pull/681) gave `register_protocol` ([#718](https://github.com/JarryShaw/PyPCAPKit/issues/718), [#726](https://github.com/JarryShaw/PyPCAPKit/pull/726)) rather than presence alone: that key is derived from the value it stores, whereas these take a caller-supplied `code`. Where a table is pre-seeded with an unresolved `ModuleDescriptor`, the guard's `is` comparison measures the resolved incoming class against the descriptor, so re-registering the same class under its own pre-seeded code (`Internet.register(TransType.TCP, TCP)`) still warns, a false positive the harmonised guard does not close. `ContextRegistry.register()` already raises on a duplicate, and the reassembly and ESP registrars append to lists with no key, so none of them takes a guard. `import pcapkit` emits no `RegistryWarning`: none of its registry writes (`R1CounterParameter`'s second code, from [#690](https://github.com/JarryShaw/PyPCAPKit/issues/690), among them) lands on a key already present ([#692](https://github.com/JarryShaw/PyPCAPKit/issues/692)). -- every EOF-truncated PCAP-NG file raised a bare `ValueError` out of `Extractor`, losing the whole extraction rather than the one truncated block. Block Total Length is cross-checked against its trailing copy and never against the file, so a last block declaring more than the file holds left `PCAPNG.read()` seeking *past* the end, and the next block read measured a **negative** remainder that `io.RawIOBase.read` refused. The seek is now clamped to the octets `_read_fileng` returned, with a `ProtocolWarning` naming the overrun, and a tail under the twelve octets a block needs is the quiet `StreamEOFError` that `Extractor.record_frames()` already catches (deliberately not a clamp, which would fabricate a frame out of zero padding). Another route to the same shape is closed by a new `nonnegative()` helper, composed into `bounded_option` and `bounded_area` and applied to the unwrapped `length - N` spans and the `__option_padding__` fields: `_TextField.__call__()` builds `f'{length}s'` unconditionally, so a negative became `'-8s'` and `struct.calcsize` raised an exception caught nowhere a frame loop looks. Both of [#678](https://github.com/JarryShaw/PyPCAPKit/issues/678)'s failure families are gone; others remain: `ValueError: N is not a valid BlockType` (filed as [#701](https://github.com/JarryShaw/PyPCAPKit/issues/701)) and `MemoryError` from [#593](https://github.com/JarryShaw/PyPCAPKit/pull/593)'s 32-bit band. Bare exceptions in `SystemdJournalExportBlock.post_process()`, reachable from valid input, are fixed with it: the body's NUL alignment padding was read as a field *name* (`struct.error`; a NUL-only line now ends the entry); a binary field's declared length was unbounded against its entry (`OverflowError`; now clamped and reported); and a non-UTF-8 name, key or value raised `UnicodeDecodeError` (now `errors='replace'` and reported). The silent loss of every field after a binary one is filed as [#704](https://github.com/JarryShaw/PyPCAPKit/issues/704). The unclamped non-packet option areas keep the per-block framing assumption [#676](https://github.com/JarryShaw/PyPCAPKit/pull/676)'s note describes. Also here, the last registrar with no overwrite guard: `Option.register()` now reports a displaced option schema as a `RegistryWarning` naming every namespace the `ns='opt'` fan-out displaced something in, tested by membership (subscripting the per-namespace `defaultdict` would insert `UnknownOption` for a code nobody registered). The condition is identity-based like the parser registrars and [#681](https://github.com/JarryShaw/PyPCAPKit/pull/681)'s guard for `register_protocol` ([#718](https://github.com/JarryShaw/PyPCAPKit/issues/718), [#726](https://github.com/JarryShaw/PyPCAPKit/pull/726)); a namespace the call itself creates is exempt. Flooring the decryption-secrets payloads that read `__length__` *whole* was tried and reverted, since `Schema.pack()` leaves that key at `-1` for "unknown". Ships labelled breaking: well-formed captures are byte-identical, but any truncated PCAP-NG now yields the frames before the cut where it raised, the last of which may carry zero-padded octets, and some short-file depths move from `ProtocolError: unknown byteorder magic` to `StreamEOFError`. The [#683](https://github.com/JarryShaw/PyPCAPKit/pull/683) drift [#685](https://github.com/JarryShaw/PyPCAPKit/issues/685) tracks is unchanged ([#678](https://github.com/JarryShaw/PyPCAPKit/issues/678)). +- every EOF-truncated PCAP-NG file raised a bare `ValueError` out of `Extractor`, losing the whole extraction rather than the one truncated block. Block Total Length is cross-checked against its trailing copy and never against the file, so a last block declaring more than the file holds left `PCAPNG.read()` seeking *past* the end, and the next block read measured a **negative** remainder that `io.RawIOBase.read` refused. The seek is now clamped to the octets `_read_fileng` returned, with a `ProtocolWarning` naming the overrun, and a tail under the twelve octets a block needs is the quiet `StreamEOFError` that `Extractor.record_frames()` already catches (deliberately not a clamp, which would fabricate a frame out of zero padding). Another route to the same shape is closed by a new `nonnegative()` helper, composed into `bounded_option` and `bounded_area` and applied to the unwrapped `length - N` spans and the `__option_padding__` fields: `_TextField.__call__()` builds `f'{length}s'` unconditionally, so a negative became `'-8s'` and `struct.calcsize` raised an exception caught nowhere a frame loop looks. Both of [#678](https://github.com/JarryShaw/PyPCAPKit/issues/678)'s failure families are gone; others remain: `ValueError: N is not a valid BlockType` (filed as [#701](https://github.com/JarryShaw/PyPCAPKit/issues/701)) and `MemoryError` from [#593](https://github.com/JarryShaw/PyPCAPKit/pull/593)'s 32-bit band. Bare exceptions in `SystemdJournalExportBlock.post_process()`, reachable from valid input, are fixed with it: the body's NUL alignment padding was read as a field *name* (`struct.error`; a NUL-only line now ends the entry); a binary field's declared length was unbounded against its entry (`OverflowError`; now a `ProtocolError` that keeps the block's journal data as captured, with a `ProtocolWarning`, [#1406](https://github.com/JarryShaw/PyPCAPKit/issues/1406)); and a non-UTF-8 name, key or value raised `UnicodeDecodeError` (now `errors='replace'` and reported). The silent loss of every field after a binary one is filed as [#704](https://github.com/JarryShaw/PyPCAPKit/issues/704). The unclamped non-packet option areas keep the per-block framing assumption [#676](https://github.com/JarryShaw/PyPCAPKit/pull/676)'s note describes. Also here, the last registrar with no overwrite guard: `Option.register()` now reports a displaced option schema as a `RegistryWarning` naming every namespace the `ns='opt'` fan-out displaced something in, tested by membership (subscripting the per-namespace `defaultdict` would insert `UnknownOption` for a code nobody registered). The condition is identity-based like the parser registrars and [#681](https://github.com/JarryShaw/PyPCAPKit/pull/681)'s guard for `register_protocol` ([#718](https://github.com/JarryShaw/PyPCAPKit/issues/718), [#726](https://github.com/JarryShaw/PyPCAPKit/pull/726)); a namespace the call itself creates is exempt. Flooring the decryption-secrets payloads that read `__length__` *whole* was tried and reverted, since `Schema.pack()` leaves that key at `-1` for "unknown". Ships labelled breaking: well-formed captures are byte-identical, but any truncated PCAP-NG now yields the frames before the cut where it raised, the last of which may carry zero-padded octets, and some short-file depths move from `ProtocolError: unknown byteorder magic` to `StreamEOFError`. The [#683](https://github.com/JarryShaw/PyPCAPKit/pull/683) drift [#685](https://github.com/JarryShaw/PyPCAPKit/issues/685) tracks is unchanged ([#678](https://github.com/JarryShaw/PyPCAPKit/issues/678)). - HIP's `R1_COUNTER`/`R1_Counter` parameter packed 12 octets where [RFC 7401 Section 5.2.3](https://datatracker.ietf.org/doc/html/rfc7401#section-5.2.3) requires 16, and `LOCATOR_SET` declared its `Length` in 4-octet units where [RFC 7401 Section 5.2.1](https://datatracker.ietf.org/doc/html/rfc7401#section-5.2.1)'s `Length` is a byte count -- independent defects, fixed together because the evidence for either needed the other out of the way. `R1CounterParameter`'s `counter` was a `UInt32Field` where the RFC states the width twice, "8 bytes" in the diagram and "a 64-bit unsigned integer" in prose; both versions whose *packing* reaches the class, `R1_Counter` (128) and `R1_COUNTER` (129), packed a 12-octet record at `4 (mod 8)`. It is a `UInt64Field` now and the record is 16 octets ([#672](https://github.com/JarryShaw/PyPCAPKit/issues/672)). `LocatorSetParameter`'s `Length` was `sum(Locator.len)`, in the 4-octet units [RFC 8046 Section 4](https://datatracker.ietf.org/doc/html/rfc8046#section-4) gives `Locator Length`; a set of *n* plain IPv6 locators declared `4n` octets against `24n` present, and `Schema.unpack()` handed the nested `ListField` only the declared octets, so `n = 2` and `n = 5` parsed one truncated locator and left the rest unconsumed. `HIP._make_param_locator_set()` now sums `8 + Locator.len * 4` per locator ([#679](https://github.com/JarryShaw/PyPCAPKit/issues/679)). Another defect in the same parameter's padding, which [#651](https://github.com/JarryShaw/PyPCAPKit/issues/651) left alone because it cancelled this one, is fixed alongside: the nested `Locator` schemas share the parameter's packet context and pack their own `len` over it, so `padding`, evaluated after the list, saw 4 rather than the real length. A new `locator_set_len_callback` snapshots `Length` under a private key before any locator packs, and `locator_set_padding_len` reads that snapshot. The `counter` defaulted to `0` and the width defect could not be told apart from it, so the fixture now overrides both `R1_COUNTER` codes with `counter=0xaabbccdd`; new tests are `test_hip_r1_counter_width_unit.py` and `test_hip_locator_set_length_unit.py`. The `hip-parameter/R1_Counter` `EXPECTED_FAILURES` entry (code 128 registered no schema of its own, since `R1CounterParameter` declared only `code=129`) was deleted once that registry defect was fixed as [#690](https://github.com/JarryShaw/PyPCAPKit/issues/690). `HIP_COPIES` dropped to one as [#689](https://github.com/JarryShaw/PyPCAPKit/issues/689), once [#672](https://github.com/JarryShaw/PyPCAPKit/issues/672) and [#679](https://github.com/JarryShaw/PyPCAPKit/issues/679) left it routing around nothing. - `httpv1.HTTP` raised a bare exception for any payload that is not an HTTP/1 message, at these sites: the header/body split in `read`, the header's CRLF split and the start-line split in `_read_http_header`, and `item[1]` on a field line with no colon. `HTTP._guess_version()` falls through on `ProtocolError` alone, so the HTTP/1 attempt aborted the guess and the HTTP/2 arm below was dead code for each of those inputs. Each site now raises `ProtocolError`, which each method's `Raises:` section documented, and a test pins the invariant (nothing escapes `httpv1.HTTP` that is not a `ProtocolError`), so any further bare-raising site would fail there. An obs-fold continuation line [[RFC 9112 Section 5.2](https://datatracker.ietf.org/doc/html/rfc9112#section-5.2)] is legal HTTP/1 and is now unfolded, the RFC's remedy, rather than read as a field of its own (with no colon it raised that `IndexError`; with one it parsed into a spurious extra field). Reaching the HTTP/2 arm exposed a payload under nine octets to `httpv2.HTTP`, where it usually fails inside the schema with a bare `struct.error` (not a `ValueError`, `ProtocolError` or pcapkit `StructError`, so uncatchable); `read` converts it and `_guess_version` suppresses it on the last arm only, since an arm that is not last hands the payload onward when it swallows. Separately, `PayloadField.protocol` looked a `str` up in `__proto__` as given while the registry is keyed on the upper-cased class name, so a name not already upper-case resolved to `None` and yielded `Raw`; it now folds case and warns with `RegistryWarning` for a genuinely unregistered name, and `__init__` assigns through that property instead of writing `_protocol` past it ([#787](https://github.com/JarryShaw/PyPCAPKit/issues/787), [#789](https://github.com/JarryShaw/PyPCAPKit/pull/789)). -- a systemd journal entry whose last field lacks the format's mandatory trailing newline let the PCAP-NG block's 32-bit alignment padding be read as part of that field's value, with no exception or warning: `b'MESSAGE=hello'` plus three NULs returned `MESSAGE == 'hello\x00\x00\x00'`. The padding-only-line guard from [#678](https://github.com/JarryShaw/PyPCAPKit/issues/678) catches padding only on a line of its own, which needs the preceding field to have ended with a real newline; without one `readline()` runs through the value into the padding. Reachable on text fields only (a binary field's length-prefixed read is never in play, and a binary field *name* on such a line fails its own length-prefix read regardless). The fix is tolerant: block padding is 0-3 octets, so at most the trailing three octets of a terminator-less line are stripped, with a `SchemaWarning`, and anything past that is left as data. The gate reads the *raw* line, since a real trailing newline, or any other raw last octet including ASCII whitespace, proves the padding is zero ([#794](https://github.com/JarryShaw/PyPCAPKit/issues/794)). `examples/generators/pcapng.py`'s `_journal_entry` cited a bare line number for the defect its alignment workaround avoids and mis-attributed the fix; it now names [#678](https://github.com/JarryShaw/PyPCAPKit/issues/678)'s `not line.strip(b'\x00')` guard and the mechanism ([#791](https://github.com/JarryShaw/PyPCAPKit/issues/791)). +- a systemd journal entry whose last field lacks the format's mandatory trailing newline let the PCAP-NG block's 32-bit alignment padding be read as part of that field's value, with no exception or warning: `b'MESSAGE=hello'` plus three NULs returned `MESSAGE == 'hello\x00\x00\x00'`. The padding-only-line guard from [#678](https://github.com/JarryShaw/PyPCAPKit/issues/678) catches padding only on a line of its own, which needs the preceding field to have ended with a real newline; without one `readline()` runs through the value into the padding. Reachable on text fields only (a binary field's length-prefixed read is never in play, and a binary field *name* on such a line fails its own length-prefix read regardless). A field line with no terminating newline raises `ProtocolError`, so the block's journal data is kept as captured, padding included, with a `ProtocolWarning` ([#794](https://github.com/JarryShaw/PyPCAPKit/issues/794), [#1406](https://github.com/JarryShaw/PyPCAPKit/issues/1406)). `examples/generators/pcapng.py`'s `_journal_entry` cited a bare line number for the defect its alignment workaround avoids and mis-attributed the fix; it now names [#678](https://github.com/JarryShaw/PyPCAPKit/issues/678)'s `not line.strip(b'\x00')` guard and the mechanism ([#791](https://github.com/JarryShaw/PyPCAPKit/issues/791)). - `httpv2.HTTP.read()` tested only the 24-bit declared length off the wire (`schema.length < 9`), never how many octets the buffer held, so a frame backed by far fewer real octets than declared parsed and reported the declared, attacker-controlled length. Sweeping the declared length against a four-octet buffer gave non-monotone accept/reject (9, 15, 65535 and 16777215 parsed, 10 and 100 did not), an artifact of `_read_http_settings`'s own `(declared - 9) % 6` check. `HTTP.unpack` now rejects a non-empty buffer under nine octets before the schema layer runs, preserving `StreamEOFError` for a genuinely exhausted implicit-length stream, and `read`'s guard also requires the declared length not to exceed the buffer. A buffer that clears nine octets can still carry a frame type whose fixed fields exceed what is left (GOAWAY at 9-16 octets, PUSH_PROMISE at 9-12, padded DATA, HEADERS and PUSH_PROMISE) and crash the schema layer with a bare `struct.error`; that root cause is in the generic `Schema.unpack()`/ `FieldBase.length` machinery the schema modules share, out of scope and filed as [#805](https://github.com/JarryShaw/PyPCAPKit/issues/805), which is why `_guess_version`'s last arm keeps suppressing `struct.error` alongside `ProtocolError` ([#802](https://github.com/JarryShaw/PyPCAPKit/pull/802)). -- `SystemdJournalExportBlock.post_process()` skipped a binary field's one trailing newline with a bare `entry_data.read()`, which reads to **EOF**: the next `readline()` returned `b''` and ended the entry, discarding every field behind the first binary one. The skip is bound to `entry_data.read(1)`; a missing or non-`b'\n'` octet ends the entry with a `SchemaWarning` instead of resynchronising at the wrong offset. `test_journal_fields_following_a_binary_field_are_not_discarded` fails against the unfixed code with `MissingKeyError: 'SECOND'` ([#704](https://github.com/JarryShaw/PyPCAPKit/issues/704)). -- the same block's entries were split on a bare `b'\n\n'`, which shreds a binary field whose value contains that pair. Walking the entry instead of splitting it introduced a regression on first review: a `malformed` flag broke the *outer* per-block loop on a bad terminator, ending the whole block and dropping every well-formed entry behind it. Fixed by dropping `malformed`, leaving the per-entry `break` as the only effect. Another gap closed in the same change: a trailing separator on the block's last octet was indistinguishable from EOF and swallowed, so a rebuild wrote `length` one octet short ([#723](https://github.com/JarryShaw/PyPCAPKit/issues/723)). +- `SystemdJournalExportBlock.post_process()` skipped a binary field's one trailing newline with a bare `entry_data.read()`, which reads to **EOF**: the next `readline()` returned `b''` and ended the entry, discarding every field behind the first binary one. The skip is bound to `entry_data.read(1)`, and a value not followed by its newline raises `ProtocolError` rather than resynchronising at the wrong offset, so the block's journal data is kept as captured, as one `bytes` item, with a `ProtocolWarning` ([#1406](https://github.com/JarryShaw/PyPCAPKit/issues/1406)). Without the fix, `test_journal_fields_following_a_binary_field_are_not_discarded` fails with `MissingKeyError: 'SECOND'` ([#704](https://github.com/JarryShaw/PyPCAPKit/issues/704)). +- the same block's entries were split on a bare `b'\n\n'`, which shreds a binary field whose value contains that pair. The entries are walked instead, each binary value counted out by its length prefix; a bad terminator or length raises `ProtocolError`, so the block's journal data is kept as captured, with a `ProtocolWarning` ([#1406](https://github.com/JarryShaw/PyPCAPKit/issues/1406)). Another gap closed in the same change: a trailing separator on the block's last octet was indistinguishable from EOF and swallowed, so a rebuild wrote `length` one octet short ([#723](https://github.com/JarryShaw/PyPCAPKit/issues/723)). - **a breaking change to** `.get()`-backed enum fields: TCP/UDP/SCTP's `PortEnumField` and PCAP-NG's `OptionEnumField` each minted a member through `aenum.extend_enum` for every value no row or documented span covers (111 calls on `http.pcap`, all ephemeral ports). Each now peeks at what `.get()` would consult and call it only once the value resolves; a genuine miss gets `EnumField._unregistered_member()`, a real member of the same registry built through its storage base's `__new__`, so `isinstance` holds and the lookup table stops growing. Riding along: PCAP-NG option namespaces stop cross-contaminating: once a code was minted under `opt`, `OptionType.get()` merged `opt` over the requested namespace and most other namespaces answered `opt_unknown` for it; each now gets its own `_unknown`. **Breaking** because [#575](https://github.com/JarryShaw/PyPCAPKit/issues/575) asked for byte-identical output across the sample captures, as [#420](https://github.com/JarryShaw/PyPCAPKit/pull/420) and [#427](https://github.com/JarryShaw/PyPCAPKit/pull/427) did, and this deliberately misses that bar: some sample captures change, entirely from the new `` rendering plus the PLIST escaping above; normalising both leaves them byte-identical. So that a resolved unassigned member still pickles, `_unregistered_member` installs a `__reduce_ex__` rebuilding an equivalent unregistered member ([#575](https://github.com/JarryShaw/PyPCAPKit/issues/575)). - `httpv2._guess_version` tried to parse a stream as HTTP/2 and read failure as "not HTTP/2", rather than identifying the version first, so a genuine connection preface followed by a real `SETTINGS` frame raised `ProtocolError: unknown HTTP version`. It now identifies before it parses: a preface is recognised as HTTP/2 outright, skipped rather than fed to `httpv2`, and counted as header so `info` stays byte-identical to reading the following frame alone (otherwise the injected `packet=self.packet.payload`, sliced from octet 9 of a buffer whose frame starts at 24, reported preface remnants as payload). A preface with no frame behind it raises `ProtocolError: HTTP/2: connection preface with no frame`; a bare mid-stream `SETTINGS` frame, an `Upgrade: h2c` request and non-HTTP input are unchanged, the frame and the request left undecidable on purpose (`Upgrade: h2c` needs per-connection state a single payload cannot carry, and a mid-stream frame has no safe heuristic, since a `type <= 9` guess misfires on binary HTTP/1 bodies). Protochain over the sample captures is unchanged ([#800](https://github.com/JarryShaw/PyPCAPKit/issues/800)). - `TCP.__proto__` bound `httpv1.HTTP` directly for ports 80 and 8080, so a segment on either port was HTTP/1 by assertion of the port alone; both versions share those ports, so the payload has to decide. Repointed to the generic `pcapkit.protocols.application.http.HTTP` proxy, whose identification became positive only once [#800](https://github.com/JarryShaw/PyPCAPKit/issues/800) and [#814](https://github.com/JarryShaw/PyPCAPKit/pull/814) landed, which is why this waited. `udp.py` already bound the proxy for both ports; this removes the asymmetry its docstring and `docs/source/contributing/pep.rst` documented as an open request. Over the sample captures every HTTP/1.1 frame keeps its chain, and the genuine HTTP/2 frames in `options-transport.pcap` change `Ethernet:IPv4:TCP:Raw` to `Ethernet:IPv4:TCP:HTTP/2`. Not labelled breaking, but not free: `TCP:80/8080` traffic that is neither valid HTTP/1 nor preface-carrying is now exposed to the fall-through arm, where the direct `httpv1` binding left it `Raw` regardless; a caller depending on that is the one who would notice ([#682](https://github.com/JarryShaw/PyPCAPKit/issues/682)). -- **a breaking change to** more bespoke registries: step 2 of [#860](https://github.com/JarryShaw/PyPCAPKit/issues/860), ahead of `AppType` ([#874](https://github.com/JarryShaw/PyPCAPKit/pull/874), above), bringing `StatusCode`, `ReturnCode`, `ResponseKind`, `GroupingInformation`, `OptionType`, `FEATCode`, `Command` and `Method` onto `EnumRegistry` and applying [#775](https://github.com/JarryShaw/PyPCAPKit/issues/775)'s mint/unmint ruling to their `get()` as well as `_missing_`. `StatusCode`, `ReturnCode`, `ResponseKind`, `GroupingInformation` and `OptionType` convert their unambiguous placeholder branches (`Unassigned`, `Unknown`, `opt_unknown`) to `_unregistered_member`; those with a custom `__new__` (`StatusCode`, `ReturnCode`, `OptionType`) get an override reconstructing the attributes the base's generic helper would leave unset, and `StatusCode`/`ReturnCode`'s hand-written `get()`, still on the retired `default == -1` convention, is replaced by the base's (no caller relied on the old form). `OptionType` keeps its own `get()` for its multi-namespace dispatch, but a later review found it still minted on both its int/namespace and `str` paths, and the live pcapng parse path (`PCAPNG._make_pcapng_options()`) calls it with wire bytes, so parsing an undeclared option code still registered a permanent member; both paths now build an unregistered member. `FEATCode`, `Command` and `Method` mint the literal wire value as its own name rather than any placeholder; per the owner's ruling that `get()` lacks the information to create new members (`Command` needs `feat`/`desc`/`type`/`conf` and `Method` needs `safe`/`idempotent`, which a bare wire string lacks), both `_missing_` and each class's own `get()` (an independent mint site of its own) now build an unregistered member. `FEATCode`'s crawler now declares every real FEAT-code value from the live IANA table instead of minting some of them as a side effect of evaluating `Command`'s rows at import time, the import-time-mutation shape [#861](https://github.com/JarryShaw/PyPCAPKit/pull/861) removed from `FilterType`; pinned count-agnostically so a future IANA update cannot fail a correct regeneration. Untouched, per rulings: `CommandType` stays `IntFlag` (real `A|P` composites in the generated data), and `TransportProtocol`'s `auto()` renumbering and `AppType` are [#874](https://github.com/JarryShaw/PyPCAPKit/pull/874)'s. `pcapkit/protocols/schema/misc/pcapng.py`'s `OptionEnumField.post_process` docstring is corrected: it still bypasses `OptionType.get()` directly, now for `pickle` round-trip safety ([#860](https://github.com/JarryShaw/PyPCAPKit/issues/860) already stops the minting that used to be the reason), since neither of `OptionType`'s paths gets both pickling and correct rendering right at once ([#869](https://github.com/JarryShaw/PyPCAPKit/pull/869)). +- **a breaking change to** more bespoke registries: step 2 of [#860](https://github.com/JarryShaw/PyPCAPKit/issues/860), ahead of `AppType` ([#874](https://github.com/JarryShaw/PyPCAPKit/pull/874), above), bringing `StatusCode`, `ReturnCode`, `ResponseKind`, `GroupingInformation`, `OptionType`, `FEATCode`, `Command` and `Method` onto `EnumRegistry` and applying [#775](https://github.com/JarryShaw/PyPCAPKit/issues/775)'s mint/unmint ruling to their `get()` as well as `_missing_`. `StatusCode`, `ReturnCode`, `ResponseKind`, `GroupingInformation` and `OptionType` convert their unambiguous placeholder branches (`Unassigned`, `Unknown`, `opt_unknown`) to `_unregistered_member`; those with a custom `__new__` (`StatusCode`, `ReturnCode`, `OptionType`) get an override reconstructing the attributes the base's generic helper would leave unset, and `StatusCode`/`ReturnCode`'s hand-written `get()`, still on the retired `default == -1` convention, is replaced by the base's (no caller relied on the old form). `OptionType` keeps its own `get()` for its multi-namespace dispatch, and both its int/namespace and `str` paths build an unregistered member, where each minted one; the live pcapng parse path (`PCAPNG._make_pcapng_options()`) calls it with wire bytes, so parsing an undeclared option code registered a permanent member. `FEATCode`, `Command` and `Method` mint the literal wire value as its own name rather than any placeholder; per the owner's ruling that `get()` lacks the information to create new members (`Command` needs `feat`/`desc`/`type`/`conf` and `Method` needs `safe`/`idempotent`, which a bare wire string lacks), both `_missing_` and each class's own `get()` (an independent mint site of its own) now build an unregistered member. `FEATCode`'s crawler now declares every real FEAT-code value from the live IANA table instead of minting some of them as a side effect of evaluating `Command`'s rows at import time, the import-time-mutation shape [#861](https://github.com/JarryShaw/PyPCAPKit/pull/861) removed from `FilterType`; pinned count-agnostically so a future IANA update cannot fail a correct regeneration. Untouched, per rulings: `CommandType` stays `IntFlag` (real `A|P` composites in the generated data), and `TransportProtocol`'s `auto()` renumbering and `AppType` are [#874](https://github.com/JarryShaw/PyPCAPKit/pull/874)'s. `pcapkit/protocols/schema/misc/pcapng.py`'s `OptionEnumField.post_process` docstring is corrected: it still bypasses `OptionType.get()` directly, now for `pickle` round-trip safety ([#860](https://github.com/JarryShaw/PyPCAPKit/issues/860) already stops the minting that used to be the reason), since neither of `OptionType`'s paths gets both pickling and correct rendering right at once ([#869](https://github.com/JarryShaw/PyPCAPKit/pull/869)). - `register` on `ProtocolBase`, `Frame`, `PCAPNG`, `SCTP`, `Link`, `Internet` and the concrete `Transport` subclasses (`TCP`, `UDP`) now raises `RegistryError` for a non-class argument, where it raised a bare `TypeError` ("issubclass() arg 1 must be a class") from inside `abc`. Each guard gains an explicit `isinstance(protocol, type)` test ahead of its `issubclass`, after the `ModuleDescriptor` is unwrapped, so a descriptor naming a non-class attribute is rejected too. Every such `Raises:` clause already promised `RegistryError` if `protocol` "is not a ... subclass", which a non-class satisfied in spirit but not in fact; they now read "is not a class, or not a ... subclass". No guard's target class changes, and a wrong class still raises `RegistryError` as before. `Transport.register()` on `Transport` itself still raises `UnsupportedCall` first; only its subclasses reach the guard. Not breaking for exception handling: `RegistryError` subclasses `TypeError` through `BaseError`, so `except TypeError` still catches it, and the type and message text are the only things a handler can key on that change. Observable side effects do come with it, because `BaseError` is loud by default: the non-class path now emits pcapkit's one `CRITICAL` log record, and outside devmode it installs `sys.excepthook` and `threading.excepthook`, neither of which the bare `TypeError` from `abc` did. That is the wrong-class path's existing behaviour, so the fix makes the two consistent rather than introducing something new. Follows the same fix for the foundation registrars ([#1021](https://github.com/JarryShaw/PyPCAPKit/issues/1021)) ([#1026](https://github.com/JarryShaw/PyPCAPKit/issues/1026)). - `PCAPNG`'s `ns_dnsname`, `ns_dnsIP4addr` and `ns_dnsIP6addr` scope guards, on both the read and the make side, now say the option "must be in Name Resolution Block", which is the block they check; they used to name the systemd(1) Journal Export Block, with a literal `:manpage:` role in the text. `register_record`'s `RegistryWarning` likewise now says "name resolution record already registered". The make-side `ns_dnsIP4addr`/`ns_dnsIP6addr` messages also drop the lowercase `ns_dnsip4addr`/`ns_dnsip6addr` spelling for the `OptionType` member name. Other guards named the wrong option too: `_make_option_if_rxspeed`'s scope and duplicate guards now say `[if_rxspeed]` rather than `[if_txspeed]`, and `_read_option_epb_queue`'s length guard says `[epb_queue]` rather than `[epb_packetid]`. Only message text changes; the exception and warning types are unchanged ([#1038](https://github.com/JarryShaw/PyPCAPKit/issues/1038)). - `httpv1.HTTP` refused every HTTP/1 message with no header fields, which [[RFC 9112 Section 2.1](https://datatracker.ietf.org/doc/html/rfc9112#section-2.1)] allows (`*( field-line CRLF )`): `b'GET / HTTP/1.0\r\n\r\n'` raised `ProtocolError: HTTP: invalid format`, because `_read_http_header` treated a header with no CRLF as malformed. So `HTTPv1(method='GET', uri='/', http_version='1.1')` failed to read back the bytes its own `make` built, and `HTTP._guess_version()` answered `unknown HTTP version`, since `_test_start_line` mirrored the same check. A CRLF-less header is now a start line with zero fields, in the parser and the predicate alike. That check was also what refused the HTTP/2 connection preface, whose header is exactly `PRI * HTTP/2.0`; it is now refused by name instead, to the same message ([#1041](https://github.com/JarryShaw/PyPCAPKit/issues/1041)). @@ -336,6 +340,12 @@ This resolves [#548](https://github.com/JarryShaw/PyPCAPKit/issues/548), which r - a declared length above `sys.maxsize`, such as a PCAP `incl_len` of `2**31 - 16` on GraalPy or 32-bit CPython, raised `OverflowError` from a read or seek. Every length-driven read in `Schema.unpack()`, `Frame.read()` and `PCAPNG.read()` is bounded at `sys.maxsize`, so such a claim gets the overrun handling it gets on 64-bit CPython ([#1975](https://github.com/JarryShaw/PyPCAPKit/issues/1975), [#1979](https://github.com/JarryShaw/PyPCAPKit/pull/1979)). - PCAP-NG blocks cut short by the file or by their own framing no longer abort an extraction. A Decryption Secrets Block whose `secrets_length` runs past the block keeps the declared length and warns, its secrets capped at what the block holds ([#1980](https://github.com/JarryShaw/PyPCAPKit/issues/1980), [#1983](https://github.com/JarryShaw/PyPCAPKit/pull/1983)); an unknown, Custom or systemd journal export block the file truncates gets the "block truncated" warning the other block types give ([#1984](https://github.com/JarryShaw/PyPCAPKit/issues/1984), [#1988](https://github.com/JarryShaw/PyPCAPKit/pull/1988)); and a malformed TLS or WireGuard key-log line warns with `ProtocolWarning` and is left out of `entries`, where it raised `FieldValueError` ([#1985](https://github.com/JarryShaw/PyPCAPKit/issues/1985), [#1989](https://github.com/JarryShaw/PyPCAPKit/pull/1989)). A ZigBee NWK or APS key off its fixed width rebuilds as captured, where it was zero-padded or cut to that width ([#1986](https://github.com/JarryShaw/PyPCAPKit/issues/1986), [#2001](https://github.com/JarryShaw/PyPCAPKit/pull/2001)). - capture timestamps convert without the platform's `fromtimestamp` limits. Windows' C runtime covers only 1970 to 3000, so a PCAP-NG timestamp outside that range fell back to 1970 with `invalid timestamp`, and the Mobility Header's NTP and PMIPv6 timestamps had the same gap. PCAP, PCAP-NG and MH convert through `pcapkit.utilities.datetime.fromtimestamp()`, which gives the stdlib's result on Linux ([#2099](https://github.com/JarryShaw/PyPCAPKit/issues/2099), [#2111](https://github.com/JarryShaw/PyPCAPKit/pull/2111)), and encode through `pcapkit.utilities.datetime.totimestamp()`, since GraalPy's `timestamp()` put a pre-1970 instant with a fraction one second late ([#2134](https://github.com/JarryShaw/PyPCAPKit/issues/2134), [#2136](https://github.com/JarryShaw/PyPCAPKit/pull/2136)). +- ARP's address makers refuse what they cannot pack: a `bool` hardware or protocol address raises `FieldValueError` whatever `htype` or `ptype` is; a hardware address that is none of `str`, `bytes` and `bytearray`, or a protocol address not packed as IPv4 or IPv6 that is none of those and not an `ipaddress` address, raises `ProtocolError`, where both failed later with a bare `TypeError`. A `bytearray` packs as `bytes` ([#2166](https://github.com/JarryShaw/PyPCAPKit/issues/2166), [#2182](https://github.com/JarryShaw/PyPCAPKit/pull/2182)). +- the Quick-Start rate parses as the `int` the IPv4, HOPOPT, IPv6-Opts and TCP data models declare, where it was a `float` such as `80.0`, and a TCP Fast Open cookie request, the option of length 2, parses with `cookie` as `b''` rather than `None` ([#2167](https://github.com/JarryShaw/PyPCAPKit/issues/2167), [#2185](https://github.com/JarryShaw/PyPCAPKit/pull/2185)). +- PCAP-NG's `ns_dnsIP6addr` option and IPv6 name resolution record build with their defaults, `2001:4860:4860::8888` and `::1`, where IPv4 defaults made them raise `FieldValueError` ([#2168](https://github.com/JarryShaw/PyPCAPKit/issues/2168), [#2179](https://github.com/JarryShaw/PyPCAPKit/pull/2179)). +- message text: the `epb_flags` and `pack_flags` length errors say `expected 4`, the length they check, rather than `expected 8` ([#2168](https://github.com/JarryShaw/PyPCAPKit/issues/2168), [#2179](https://github.com/JarryShaw/PyPCAPKit/pull/2179)), and the IPv4 TS option's unknown-flag warning and the Mobility Header's prefix error spell `timestamp` and `mobile` ([#2171](https://github.com/JarryShaw/PyPCAPKit/issues/2171), [#2189](https://github.com/JarryShaw/PyPCAPKit/pull/2189)). +- IPX dissects its payload from a registry of its own, `IPX.__proto__`, keyed by IPX Packet Type, where it looked the Packet Type up as an IP protocol number, so a type 17 (NCP) payload parsed as `IPX:UDP` and type 4 as `IPX:IPv4`. No IPX-native protocol ships, so the payload is `Raw` unless one is registered ([#2169](https://github.com/JarryShaw/PyPCAPKit/issues/2169), [#2187](https://github.com/JarryShaw/PyPCAPKit/pull/2187)). +- HIP's `ECHO_RESPONSE_UNSIGNED` maker builds an `EchoResponseUnsignedParameter` schema, where it built the request's; the octets were already right ([#2170](https://github.com/JarryShaw/PyPCAPKit/issues/2170), [#2188](https://github.com/JarryShaw/PyPCAPKit/pull/2188)). ### pcapkit.toolkit @@ -349,7 +359,7 @@ This resolves [#548](https://github.com/JarryShaw/PyPCAPKit/issues/548), which r - the engine adapters, which were quietly wrong rather than loud. The `dpkt` toolkit split TCP and IPv4 headers at their fixed struct size instead of their real length, so option octets overwrote payload in the sequence-indexed reassembly buffer; it also read an `ipv6_frag.nh` that `dpkt` does not have, and passed fragment offsets unscaled ([#351](https://github.com/JarryShaw/PyPCAPKit/issues/351), [#370](https://github.com/JarryShaw/PyPCAPKit/issues/370), [#385](https://github.com/JarryShaw/PyPCAPKit/pull/385), [#395](https://github.com/JarryShaw/PyPCAPKit/pull/395)). `scapy` never loaded its layer registry, so that engine did not dissect at all ([#409](https://github.com/JarryShaw/PyPCAPKit/pull/409)), and its IPv4 fragment offset reached reassembly unscaled ([#483](https://github.com/JarryShaw/PyPCAPKit/issues/483), [#484](https://github.com/JarryShaw/PyPCAPKit/pull/484)). The IPv6 adapters disagreed about whether the 8-octet Fragment header belongs to `ihl`, `header` and `tl`; per [RFC 8200](https://datatracker.ietf.org/doc/html/rfc8200) section 4.5 it belongs to none, and they now agree ([#415](https://github.com/JarryShaw/PyPCAPKit/issues/415), [#424](https://github.com/JarryShaw/PyPCAPKit/pull/424)). - `pypcapfile`'s `IP.src`/`.dst` are dotted-decimal ASCII text in a `ctypes.c_char_p` (per `pypcapfile` 0.12.0's `ip.py`), not packed 4-byte values, so `ipaddress.IPv4Address(ipv4.src)` raised `AddressValueError` and `ipv4_header()`'s `struct.pack('...II', ipv4.src, ...)` raised `struct.error` on every call. A single `_parse_ipv4_address()` chokepoint replaces the separate call sites, also accepting packed `bytes` or an `int`. The same engine's `IP.opt`/`.payload` carry hex-ASCII rather than raw bytes once decoding stops short of the transport layer; a new `_maybe_unhex()` covers the sites that read them. Not `breaking`: nothing on this path worked on 3.10/3.11, so correct behaviour cannot regress a working caller ([#743](https://github.com/JarryShaw/PyPCAPKit/issues/743)). - `pypcapfile`'s `_read_a_packet(layers=0)` hexlifies a whole frame into ASCII text and stops, but `_decode()` fed that into `Ethernet(packet.packet, layers=1)`, whose `__init__` unpacks the header with `struct.unpack`, so every field came out garbage without raising. Fixed with `binascii.unhexlify(packet.packet)` ahead of the decoder call, in preference to decoding eagerly at `layers>0` inside the lazy generator, which would lose `_decode()`'s per-frame `AttributeWarning` fallback. Link-layer dispatch resolves from the header's `ll_type` and is unaffected. Landed with [#743](https://github.com/JarryShaw/PyPCAPKit/issues/743): standalone, this turns a non-crashing extraction into a crashing one, since `Extractor.run()` catches only `EOFError`, `StopIteration` and `KeyboardInterrupt` and the `AddressValueError`/`struct.error` [#743](https://github.com/JarryShaw/PyPCAPKit/issues/743) fixes escape until both land ([#746](https://github.com/JarryShaw/PyPCAPKit/issues/746)). -- **a breaking change to** pyshark link-type resolution in `tcp_traceflow`: an unrecognised encapsulation or filter name now raises `MissingKeyError` instead of substituting a plausible DLT. `pcapkit/toolkit/pyshark.py` resolved a frame's link type from `packet.layers[0].layer_name`, a Wireshark display-filter name, with a blanket `Enum_LinkType.get(name.upper())` fallback. One filter name serves several encapsulations, so that fallback returned *wrong DLTs silently* (`null` for a `DLT_LOOP` capture, `101` (`RAW`) for `rawip6`). Re-keyed onto `frame.encap_type`, which is distinct per encapsulation, and the fallback is gone. Both tables were measured rather than transcribed (Wireshark's source is not on the build host): each entry round-tripped through `editcap -F pcap -T ` and read back with `tshark`. `ENCAP_TYPE_TO_LINKTYPE` grows to cover every measured encapsulation whose DLT has a `LinkType` member, and `FILTER_NAME_TO_LINKTYPE` grows from [#838](https://github.com/JarryShaw/PyPCAPKit/pull/838)'s hand-built entries to every root filter name, pseudo-protocol roots aside, that resolved to a single `LinkType` member across the encapsulations this host's Wireshark can write. `pcapkit/toolkit/pyshark.py` reaches full coverage under new tests that fail against the unfixed tree; live pyshark parsing stays unexercised, since `pyshark.FileCapture` does not run on Python 3.14 ([#850](https://github.com/JarryShaw/PyPCAPKit/pull/850)). +- **a breaking change to** pyshark link-type resolution in `tcp_traceflow`: an unrecognised encapsulation or filter name now raises `MissingKeyError` instead of substituting a plausible DLT. `pcapkit/toolkit/pyshark.py` resolved a frame's link type from `packet.layers[0].layer_name`, a Wireshark display-filter name, with a blanket `Enum_LinkType.get(name.upper())` fallback. One filter name serves several encapsulations, so that fallback returned *wrong DLTs silently* (`null` for a `DLT_LOOP` capture, `101` (`RAW`) for `rawip6`). Re-keyed onto `frame.encap_type`, which is distinct per encapsulation, and the fallback is gone. Both tables were measured rather than transcribed (Wireshark's source is not on the build host): each entry round-tripped through `editcap -F pcap -T ` and read back with `tshark`. `ENCAP_TYPE_TO_LINKTYPE` grows to cover every measured encapsulation whose DLT has a `LinkType` member, and `FILTER_NAME_TO_LINKTYPE` grows from [#838](https://github.com/JarryShaw/PyPCAPKit/pull/838)'s hand-built entries to every root filter name, pseudo-protocol roots aside, that resolved to a single `LinkType` member across the encapsulations this host's Wireshark can write. `pcapkit/toolkit/pyshark.py` reaches full coverage under new tests, which fail without the fix; live pyshark parsing stays unexercised, since `pyshark.FileCapture` does not run on Python 3.14 ([#850](https://github.com/JarryShaw/PyPCAPKit/pull/850)). - `pypcapfile`'s `tcp_reassembly` set `last` one past the segment. `TCP_Packet.last` is inclusive, and the other adapters set `seqnum + len(payload) - 1`, as this one now does too ([#1100](https://github.com/JarryShaw/PyPCAPKit/issues/1100)). - scapy's `packet2dict` dumped nested packets as `{}` and mutated its input ([#1287](https://github.com/JarryShaw/PyPCAPKit/pull/1287)); the dpkt reassembly adapters build their data from the record's own octets rather than dpkt's re-serialised bytes, and leave the packet alone ([#1297](https://github.com/JarryShaw/PyPCAPKit/issues/1297), [#1335](https://github.com/JarryShaw/PyPCAPKit/pull/1335)). - the TCP adapters: pyshark accepts tshark's `True`/`False` TCP flags ([#1514](https://github.com/JarryShaw/PyPCAPKit/issues/1514), [#1527](https://github.com/JarryShaw/PyPCAPKit/pull/1527)); a TCP segment whose header the snapshot cut is traced and reassembled ([#1518](https://github.com/JarryShaw/PyPCAPKit/issues/1518), [#1556](https://github.com/JarryShaw/PyPCAPKit/pull/1556)); dpkt reads SYN, FIN and RST whatever the NS bit ([#1618](https://github.com/JarryShaw/PyPCAPKit/issues/1618), [#1621](https://github.com/JarryShaw/PyPCAPKit/pull/1621)); and the scapy adapters stop at the IP datagram's declared length ([#1584](https://github.com/JarryShaw/PyPCAPKit/issues/1584), [#1589](https://github.com/JarryShaw/PyPCAPKit/pull/1589)). @@ -385,7 +395,7 @@ This resolves [#548](https://github.com/JarryShaw/PyPCAPKit/issues/548), which r #### Fixed - `re.sub` sites under `pcapkit/vendor/` passed `re.MULTILINE` as the fourth *positional* argument, which is `count`, not `flags`: substitution capped at 8 matches, and a `DeprecationWarning` on Python 3.13+ (a future `TypeError`). Found by an AST sweep rather than grep, since most put the flag on a continuation line. The flag moves to `flags=` at every site, in `default.py`, `hip/eddsa_curve.py`, `http/frame.py`, `http/method.py`, `http/status_code.py`, `ipv4/router_alert.py`, `ipv6/option.py`, `ipv6/router_alert.py`, `ipv6/tagger_id.py`, `reg/ethertype.py`, `tcp/flags.py`, `tcp/mp_tcp_option.py` and `tcp/option.py`. Behaviour preserving: the shared pattern (`r'\r*\n'`) has no MULTILINE-sensitive anchor, so no `const/` regeneration was needed ([#796](https://github.com/JarryShaw/PyPCAPKit/issues/796)). -- `main` was red ([#866](https://github.com/JarryShaw/PyPCAPKit/issues/866)) with `RecursionError` in `tests/const/test_const_enum_lookup.py`, `tests/const/test_const_enum_no_mint.py` and `tests/protocols/misc/test_pcapng_unit.py`. `pcapkit/vendor/pcapng/record_type.py` and `secrets_type.py` rendered a `_missing_` body whose first line did not return: `cls._unregistered_member(value, 'Unassigned')` followed by `return cls(value)`. That was harmless while the first line was `extend_enum(...)`, which registers the member so `cls(value)` found it; [#861](https://github.com/JarryShaw/PyPCAPKit/pull/861) (above) replaced it with `_unregistered_member`, which deliberately does not register, so `cls(value)` missed and re-entered `_missing_`. [#861](https://github.com/JarryShaw/PyPCAPKit/pull/861) fixed the *generated* modules but not these *crawlers*, so [#861](https://github.com/JarryShaw/PyPCAPKit/pull/861)'s CI stayed green until a live regeneration reintroduced the defect; [#861](https://github.com/JarryShaw/PyPCAPKit/pull/861)'s review had sampled only some regenerations, and neither crawler's was among them. Both crawlers now collapse to the single returning line the other registries use, and their const files are regenerated. A new guard at the crawler-emission layer requires a hard-coded `_missing_` body to end in a return and no emitted line to re-enter `_missing_` for the same value -- deliberately narrower than "every emitted line must return", which `Vendor.process()`'s shape legitimately breaks. The guard sees only some crawlers; the rest are covered behaviourally. Closes [#866](https://github.com/JarryShaw/PyPCAPKit/issues/866) ([#867](https://github.com/JarryShaw/PyPCAPKit/pull/867)). +- `main` was red ([#866](https://github.com/JarryShaw/PyPCAPKit/issues/866)) with `RecursionError` in `tests/const/test_const_enum_lookup.py`, `tests/const/test_const_enum_no_mint.py` and `tests/protocols/misc/test_pcapng_unit.py`. `pcapkit/vendor/pcapng/record_type.py` and `secrets_type.py` rendered a `_missing_` body whose first line did not return: `cls._unregistered_member(value, 'Unassigned')` followed by `return cls(value)`. That was harmless while the first line was `extend_enum(...)`, which registers the member so `cls(value)` found it; [#861](https://github.com/JarryShaw/PyPCAPKit/pull/861) (above) replaced it with `_unregistered_member`, which deliberately does not register, so `cls(value)` missed and re-entered `_missing_`. [#861](https://github.com/JarryShaw/PyPCAPKit/pull/861) fixed the *generated* modules but not these *crawlers*, so a live regeneration reintroduced the defect. Both crawlers now collapse to the single returning line the other registries use, and their const files are regenerated. A new guard at the crawler-emission layer requires a hard-coded `_missing_` body to end in a return and no emitted line to re-enter `_missing_` for the same value -- deliberately narrower than "every emitted line must return", which `Vendor.process()`'s shape legitimately breaks. The guard sees only some crawlers; the rest are covered behaviourally. Closes [#866](https://github.com/JarryShaw/PyPCAPKit/issues/866) ([#867](https://github.com/JarryShaw/PyPCAPKit/pull/867)). - `python -m pcapkit.vendor ` could not fail: `run()` swallowed every crawler exception as a filterable `VendorRuntimeWarning` and `main()` always returned `0`, so a broken crawler regenerating nothing looked like success to a human and to the cron job. Per the owner's ruling -- discard only what a failing sub-vendor changed, and let a cron job warn about failed sub-vendors without failing as a whole -- the pieces below land. **Exit code** (`pcapkit/vendor/__main__.py`): `run()` returns whether its target succeeded; `main()` attempts every target and returns `1` if any raised, writing the failing target's qualified name and exception `repr` to stderr unconditionally via `print()` rather than the filterable warning. **Per-target snapshot and restore**: a new `_snapshot_and_restore` context manager backs up a target's const file with `tempfile.mkstemp` before it runs (resolving the destination through `Vendor._dest_path()`, now a `classmethod` in `pcapkit/vendor/default.py` so it can be called before `Vendor.__init__` fetches, renders or writes), restores the backup via `os.replace` on any exception before re-raising, and discards it on success. A target whose destination cannot be resolved (a crawler outside the real tree) runs unprotected rather than failing the snapshot step, since `Vendor.__init__` hits the same resolution failure itself; a destination that does not exist yet is left alone. **Workflow** (`.github/workflows/cron-vendor.yml`): the "Update Vendor" step no longer aborts under `bash -e` when `pcapkit-vendor` exits non-zero. The exit code is captured around that one command, so targets that succeeded are still sorted by `isort`, diffed, committed and pushed, while a "Warning" block naming the failed target(s) is appended to `$GITHUB_STEP_SUMMARY`, a `::warning::` annotation is emitted per failure, and the step's own exit code is always `0`. A per-target `git checkout -- ` restore step was dropped: with snapshot/restore, a failed target's const file is never touched. Closes [#872](https://github.com/JarryShaw/PyPCAPKit/issues/872) ([#873](https://github.com/JarryShaw/PyPCAPKit/pull/873)). - follow-ups from [#873](https://github.com/JarryShaw/PyPCAPKit/pull/873)'s review, none changing merged behaviour: an unpinned invariant and inaccurate prose and lint. **1.** `except BaseException:` in `_snapshot_and_restore` is pinned: a test asserts that a `KeyboardInterrupt` still restores the file and propagates. **2.** A false docstring clause is corrected: a crawler whose `_dest_path` is an instance-method-only override fails the *class* call `_snapshot_and_restore` makes but not the *instance* call inside `__init__`, so that target runs unprotected and **succeeds silently**. **3.** The `# pylint: disable=no-else-raise` pragma is **kept**: the belief that R1720 never applies to `try`/`except`/`else` was a message-control artefact (`--disable=all --enable=no-else-raise` does not re-enable the check the way `--enable=R` does), and under the `Makefile`'s flags, which CI runs, pylint flags the `else:`, which is deliberate because it makes the backup cleanup unreachable from the `except`. Left out of scope: an untested `os.close(fd)` leak (harmless, since `copy2` reopens the path) and the semantically identical unindented form of the same `else:`. Closes [#875](https://github.com/JarryShaw/PyPCAPKit/issues/875) ([#876](https://github.com/JarryShaw/PyPCAPKit/pull/876)). - `TLSKeyLabel` tracked the Mozilla NSS key-log wiki, but `draft-ietf-opsawg-pcapng-06` section 4.7 now points at `[I-D.ietf-tls-keylogfile]` (draft-05), published as [RFC 9850](https://datatracker.ietf.org/doc/html/rfc9850), "The SSLKEYLOGFILE Format for TLS", whose section 4.2 creates the IANA "TLS SSLKEYLOGFILE Labels" registry. Against it, `TLSKeyLabel` was missing `ECH_SECRET` and `ECH_CONFIG` ([RFC 9850](https://datatracker.ietf.org/doc/html/rfc9850) sections 2.3 and 4.2), so `TLSKeyLabel('ECH_SECRET')` raised `ValueError` on a legitimate Decryption Secrets Block; both are added, `ECH_SECRET` keeping the file's `# nosec B105` bandit convention for `*_SECRET` members. `RSA` has no row in [RFC 9850](https://datatracker.ietf.org/doc/html/rfc9850)'s registry (it is an NSS-only label the NSS wiki records as removed in NSS 3.34) and is kept, since removing a public member is breaking, with a docstring comment marking it the historical exception. `pcapkit/vendor/pcapng/secrets_type.py`'s stale `# NSS Key Log Format` comment is repointed at RFC 9850, and `TLSKeyLabel`'s docstring cites it. A new test pins the member set against RFC 9850 plus the `RSA` exception. Closes [#882](https://github.com/JarryShaw/PyPCAPKit/issues/882) ([#883](https://github.com/JarryShaw/PyPCAPKit/pull/883)). @@ -407,7 +417,7 @@ This resolves [#548](https://github.com/JarryShaw/PyPCAPKit/issues/548), which r - an end-to-end test tier ([#376](https://github.com/JarryShaw/PyPCAPKit/pull/376)), sample-capture generators so a fresh clone can rebuild every fixture ([#340](https://github.com/JarryShaw/PyPCAPKit/pull/340)), a Dockerised engine benchmark covering every supported Python version ([#410](https://github.com/JarryShaw/PyPCAPKit/pull/410)), and registry round-trip coverage that records the entries which cannot close the cycle rather than skipping them ([#440](https://github.com/JarryShaw/PyPCAPKit/pull/440), [#504](https://github.com/JarryShaw/PyPCAPKit/pull/504)). - coverage for the untested half of the [#431](https://github.com/JarryShaw/PyPCAPKit/issues/431) accommodation: a TCP or IPv4 option whose declared length asks for more data than the capture holds, pinning that it still parses with the short read zero-padded rather than rejected. The test [#431](https://github.com/JarryShaw/PyPCAPKit/issues/431) left covers only an option area with no data behind it; a candidate fix for [#554](https://github.com/JarryShaw/PyPCAPKit/issues/554) turned the untested half into an unwrapped `FieldValueError` while the rest of the suite stayed green ([#571](https://github.com/JarryShaw/PyPCAPKit/pull/571), [#572](https://github.com/JarryShaw/PyPCAPKit/issues/572)). - `CITATION.cff`, citation metadata in Citation File Format 1.2.0, which GitHub renders as the "Cite this repository" button and which citation managers and dependency inventories read directly. It is the machine-readable half of the attribution BSD-3-Clause already asks for. Validated with `cffconvert --validate` and against the published 1.2.0 schema; `doi` and `orcid` are omitted rather than invented, since neither exists for this project and both are checked formats, so a wrong value would still validate. The licence is unchanged (BSD-3-Clause, no `NOTICE`). Alongside it the copyright line moves from `2018-2023` to `2018-2026` (`LICENSE` was its only occurrence; `docs/source/conf.py` derives its own from the current year), and a stray `s` after the closing `DAMAGE.` of the licence text, present since the Mozilla-to-BSD relicence, is removed so the wording matches canonical BSD-3-Clause ([#615](https://github.com/JarryShaw/PyPCAPKit/pull/615)). -- `examples/generators/endian.py`, and the byte-order tests that read what it writes. There was no big-endian `.pcap` in the repository, which is why [#605](https://github.com/JarryShaw/PyPCAPKit/issues/605) survived its own code review: the fix to a misspelled key left the corrected path as untested as the broken one. The generator writes `big_endian.pcap` (magic `a1 b2 c3 d4`), `big_endian_nanosecond.pcap` (`a1 b2 3c 4d`, the first fixture to take that branch of the magic-number table) and `little_endian.pcap` (`d4 c3 b2 a1`), carrying the *same records* in each, so tests can assert that byte order makes no difference to what is read rather than only matching numbers written in a test. Frame 3 is captured short (1200 octets on the wire, 96-octet `snaplen`), so `incl_len` and `orig_len` differ and cannot both be satisfied by one byte-swapped value. `test_frame_endian_runtime.py` drives each file through `extract()` and derives expectations by walking its record chain with `struct`; a unit-tier case in `test_header_frame_unit.py` builds a big-endian capture in memory, so the fixture-free selection CI runs on every push catches it too. Each fails on the unfixed tree (the fixture-backed ones by that `ValueError`, the in-memory one by `AssertionError: 3106905 != 1500000000`) while the little-endian twin passes on both, showing the records themselves are not the variable ([#605](https://github.com/JarryShaw/PyPCAPKit/issues/605)). +- `examples/generators/endian.py`, and the byte-order tests that read what it writes. There was no big-endian `.pcap` in the repository, which is why [#605](https://github.com/JarryShaw/PyPCAPKit/issues/605) survived its own code review: the fix to a misspelled key left the corrected path as untested as the broken one. The generator writes `big_endian.pcap` (magic `a1 b2 c3 d4`), `big_endian_nanosecond.pcap` (`a1 b2 3c 4d`, the first fixture to take that branch of the magic-number table) and `little_endian.pcap` (`d4 c3 b2 a1`), carrying the *same records* in each, so tests can assert that byte order makes no difference to what is read rather than only matching numbers written in a test. Frame 3 is captured short (1200 octets on the wire, 96-octet `snaplen`), so `incl_len` and `orig_len` differ and cannot both be satisfied by one byte-swapped value. `test_frame_endian_runtime.py` drives each file through `extract()` and derives expectations by walking its record chain with `struct`; a unit-tier case in `test_header_frame_unit.py` builds a big-endian capture in memory, so the fixture-free selection CI runs on every push catches it too. Without the fix, each fails (the fixture-backed ones by that `ValueError`, the in-memory one by `AssertionError: 3106905 != 1500000000`) while the little-endian twin passes with and without it, showing the records themselves are not the variable ([#605](https://github.com/JarryShaw/PyPCAPKit/issues/605)). - local type stubs under `typings/` for dpkt, pyshark, `pcap` and pypcapfile, declaring what pcapkit uses of each, so mypy checks the engines against them. `mypy.ini` reads them through `mypy_path`, and neither the sdist nor the wheel ships them ([#1878](https://github.com/JarryShaw/PyPCAPKit/issues/1878), [#1882](https://github.com/JarryShaw/PyPCAPKit/pull/1882)). - a glossary page for the terms with several meanings -- registry, schema, data model, info and default engine -- each sense linked to the page that explains it ([#1949](https://github.com/JarryShaw/PyPCAPKit/issues/1949), [#1955](https://github.com/JarryShaw/PyPCAPKit/pull/1955)). - the engine benchmark on GitHub Actions: `benchmark.yml`, weekly and on dispatch, measures every engine on the runner's own interpreters through `examples/benchmark/native.sh` and posts the report to the *CI reports* discussions ([#2125](https://github.com/JarryShaw/PyPCAPKit/issues/2125), [#2128](https://github.com/JarryShaw/PyPCAPKit/pull/2128)). A `pcap` engine whose module cannot load is an explained row rather than a fatal error ([#2129](https://github.com/JarryShaw/PyPCAPKit/issues/2129), [#2131](https://github.com/JarryShaw/PyPCAPKit/pull/2131)), and the report is reST with footnotes, so it pastes unedited onto `docs/source/engines.rst` ([#2142](https://github.com/JarryShaw/PyPCAPKit/pull/2142)), the page the engine comparison and prerequisites move to from `index.rst`, carrying the first run's figures (`f9de043d6`, `d7aa41535`). The tables had first been regenerated with Python 3.15 measured ([#2109](https://github.com/JarryShaw/PyPCAPKit/pull/2109)). diff --git a/docs/source/changelog/1.5.0.rst b/docs/source/changelog/1.5.0.rst index 69ab6d469e..ca0a4ef147 100644 --- a/docs/source/changelog/1.5.0.rst +++ b/docs/source/changelog/1.5.0.rst @@ -5,7 +5,7 @@ The largest release since 1.0, and the first recorded here as it happened rather than reconstructed. New extraction engines, ESP with payload decryption, SCTP and NGAP over SCTP, a library logger that no longer hijacks the consumer's, and a defect programme run through the issue tracker, citing issues and pull -requests from :pr:`340` to :pr:`2151`. +requests from :pr:`340` to :pr:`2205`. Preceded by ``1.5.0a1`` (2026-09-15), ``1.5.0b1`` to ``1.5.0b3`` (all 2026-09-19) and ``1.5.0b4`` to ``1.5.0b14`` (2026-09-20 to 2026-10-22), dated in UTC and all @@ -698,8 +698,8 @@ Fixed never mints" is qualified to ``default`` only. So ``Hardware.get('Definitely-Not-A-Member', 40)``, with ``40`` in :class:`~pcapkit.const.arp.hardware.Hardware`'s declared-but-unassigned - range, now raises ``KeyError`` naming the original key; its test waited on :pr:`865` - (above), then landed once :pr:`865` merged. Closes :issue:`864` (:pr:`868`). + range, now raises ``KeyError`` naming the original key. Closes :issue:`864` + (:pr:`868`). * :meth:`.ProtoChain.__add__` returned the left operand's cached ``protocols`` and ``aliases``: once ``a.protocols`` had been read, ``a + ProtoChain(IPv4)`` reported ``chain == 'Ethernet:IPv4'`` but ``protocols == (Ethernet,)``. ``copy.copy(self)`` @@ -782,6 +782,14 @@ Fixed Windows, ``SeekableReader(buffer_save=True)`` with no ``buffer_path`` could not reopen its temporary file; it creates one it can, and deletes it on ``close()`` or collection (:issue:`2098`, :pr:`2105`). +* :class:`.PortEnumField` and :class:`.SwitchField` are exported from + :mod:`pcapkit.corekit`, :mod:`pcapkit.corekit.fields` and ``pcapkit.all``, and + :class:`.SchemaField`, ``nested_packet_context`` and :class:`.BytesField` from the + modules defining them, so ``import *`` binds them (:issue:`2174`, :pr:`2191`). +* a :class:`.ListField` callback is handed the per-item copy it configures, not the + shared field, and a :class:`.SchemaField` with a callable length sizes its template + from the resolved length rather than ``1024s``. No parse in the library reached either + (:issue:`2173`, :pr:`2190`). pcapkit.dumpkit --------------- @@ -1192,6 +1200,11 @@ Fixed completed. The bound is :data:`~pcapkit.foundation.reassembly.data.tcp.HOLE_OPEN_BOUND`, ``2**63 - 1``, the value 64-bit builds already used (:issue:`1959`, :pr:`1965`). +* the method-name registrars, such as ``register_ipv4_option``, and + ``register_linktype``, ``register_pcap``, ``register_pcapng``, ``register_ethertype`` + and ``register_transtype`` refuse a ``code`` that is not an enum member with + :exc:`.RegistryError` before touching a registry, where they registered it and then + raised ``AttributeError``, leaving the entry behind (:issue:`2186`, :pr:`2193`). pcapkit.interface ----------------- @@ -1467,6 +1480,14 @@ Changed hand-written dict needs those keys, ``__short_read__`` among them, to rebuild, and a custom :class:`.Info` lists its own dunder keys in ``__included__`` (:issue:`2003`, :pr:`2014`). +* **a breaking change to** ``PCAPNG.context``: the block's section + :class:`~pcapkit.foundation.engines.pcapng.Context`, which it returned from 1.0.0 + through 1.4.x, is :attr:`.PCAPNG.section_context`, with no alias, and ``context`` is + the caller supplied :class:`~pcapkit.corekit.context.ContextRegistry` every protocol + inherits from :class:`.ProtocolBase`, on a Section Header Block too. + :attr:`.PCAPNG.section_context` raises :exc:`.UnsupportedCall` on a Section Header + Block, as ``context`` did. Code reading ``block.context`` for the section reads + ``block.section_context`` (:issue:`2172`, :pr:`2205`). Fixed ~~~~~ @@ -1756,15 +1777,14 @@ Fixed gives: :class:`.ConditionalField`'s ``length`` forwards to the wrapped field without consulting the condition, whereas a :class:`.SwitchField` always resolves to a concrete field (:issue:`603`). -* the assertion :issue:`604` left pinning the old padding side, which :pr:`621` - turned red on ``main``. +* the assertion :issue:`604` left pinning the old padding side, which fails once + :pr:`621` pads with ``ljust()``. ``TCPUDPUnitTests.test_a_truncated_option_still_parses_its_declared_length`` expected a truncated TCP option's ``data`` as the synthesised zero octets *followed by* the real ones, as ``rjust()`` produced: ``Reserved_79`` declaring ``length=12`` over 6 real octets reports ``aabbccddeeff00000000`` where the test expected - ``00000000aabbccddeeff``. :pr:`621` made the padding ``ljust()`` everywhere but - could not retarget this file, since :pr:`612` owned it. The real octets now come - first, and the docstring is corrected. Test-only; the sibling case in + ``00000000aabbccddeeff``. The real octets now come first, and the docstring is + corrected. Test-only; the sibling case in ``tests/protocols/internet/test_ipv4_unit.py`` was retargeted in :pr:`621` (:issue:`604`). * the HIP ``SOLUTION`` builder sized the parameter with @@ -1961,8 +1981,8 @@ Fixed ``tests/project/test_annotation_names.py`` pins the invariant without a type checker: it resolves every string annotation in the package against the names its own module binds, following nested forward references such as ``'list["Nested"]'`` - and treating ``Literal`` members and ``Annotated`` metadata as values, and - reports exactly those findings on the unfixed tree. The same change corrects the + and treating ``Literal`` members and ``Annotated`` metadata as values; without the + fix, it reports exactly those findings. The same change corrects the ``MANIFEST.in`` comment asserting that ``include README.md`` was the only thing putting the README in a source distribution and that an sdist without it could not be installed; both halves are false, and the :pr:`619` entry above carried the same @@ -2076,15 +2096,16 @@ Fixed in :meth:`.SystemdJournalExportBlock.post_process`, reachable from valid input, are fixed with it: the body's NUL alignment padding was read as a field *name* (``struct.error``; a NUL-only line now ends the entry); a binary field's declared - length was unbounded against its entry (``OverflowError``; now clamped and - reported); and a non-UTF-8 name, key or value raised ``UnicodeDecodeError`` (now - ``errors='replace'`` and reported). The silent loss of every field after a binary - one is filed as :issue:`704`. The unclamped non-packet option areas keep the - per-block framing assumption :pr:`676`'s note describes. Also here, the last - registrar with no overwrite guard: :meth:`.Option.register` now reports a displaced - option schema as a :exc:`.RegistryWarning` naming every namespace the ``ns='opt'`` - fan-out displaced something in, tested by membership (subscripting the per-namespace - ``defaultdict`` would insert + length was unbounded against its entry (``OverflowError``; now a + :exc:`.ProtocolError` that keeps the block's journal data as captured, with a + :exc:`.ProtocolWarning`, :issue:`1406`); and a non-UTF-8 name, key or value raised + ``UnicodeDecodeError`` (now ``errors='replace'`` and reported). The silent loss of + every field after a binary one is filed as :issue:`704`. The unclamped non-packet + option areas keep the per-block framing assumption :pr:`676`'s note describes. Also + here, the last registrar with no overwrite guard: :meth:`.Option.register` now + reports a displaced option schema as a :exc:`.RegistryWarning` naming every namespace + the ``ns='opt'`` fan-out displaced something in, tested by membership (subscripting + the per-namespace ``defaultdict`` would insert :class:`~pcapkit.protocols.schema.misc.pcapng.UnknownOption` for a code nobody registered). The condition is identity-based like the parser registrars and :pr:`681`'s guard for ``register_protocol`` (:issue:`718`, :pr:`726`); a namespace the @@ -2160,11 +2181,9 @@ Fixed field to have ended with a real newline; without one ``readline()`` runs through the value into the padding. Reachable on text fields only (a binary field's length-prefixed read is never in play, and a binary field *name* on such a line fails - its own length-prefix read regardless). The fix is tolerant: block padding is 0-3 - octets, so at most the trailing three octets of a terminator-less line are stripped, - with a :exc:`.SchemaWarning`, and anything past that is left as data. The gate reads - the *raw* line, since a real trailing newline, or any other raw last octet including - ASCII whitespace, proves the padding is zero (:issue:`794`). + its own length-prefix read regardless). A field line with no terminating newline + raises :exc:`.ProtocolError`, so the block's journal data is kept as captured, + padding included, with a :exc:`.ProtocolWarning` (:issue:`794`, :issue:`1406`). ``examples/generators/pcapng.py``'s ``_journal_entry`` cited a bare line number for the defect its alignment workaround avoids and mis-attributed the fix; it now names :issue:`678`'s ``not line.strip(b'\x00')`` guard and the mechanism (:issue:`791`). @@ -2187,19 +2206,19 @@ Fixed * :meth:`.SystemdJournalExportBlock.post_process` skipped a binary field's one trailing newline with a bare ``entry_data.read()``, which reads to **EOF**: the next ``readline()`` returned ``b''`` and ended the entry, discarding every field behind - the first binary one. The skip is bound to ``entry_data.read(1)``; a missing or - non-``b'\n'`` octet ends the entry with a :exc:`.SchemaWarning` instead of - resynchronising at the wrong offset. - ``test_journal_fields_following_a_binary_field_are_not_discarded`` fails against the - unfixed code with ``MissingKeyError: 'SECOND'`` (:issue:`704`). + the first binary one. The skip is bound to ``entry_data.read(1)``, and a value not + followed by its newline raises :exc:`.ProtocolError` rather than resynchronising at + the wrong offset, so the block's journal data is kept as captured, as one ``bytes`` + item, with a :exc:`.ProtocolWarning` (:issue:`1406`). Without the fix, + ``test_journal_fields_following_a_binary_field_are_not_discarded`` fails with + ``MissingKeyError: 'SECOND'`` (:issue:`704`). * the same block's entries were split on a bare ``b'\n\n'``, which shreds a binary - field whose value contains that pair. Walking the entry instead of splitting it - introduced a regression on first review: a ``malformed`` flag broke the *outer* - per-block loop on a bad terminator, ending the whole block and dropping every - well-formed entry behind it. Fixed by dropping ``malformed``, leaving the per-entry - ``break`` as the only effect. Another gap closed in the same change: a trailing - separator on the block's last octet was indistinguishable from EOF and swallowed, so - a rebuild wrote ``length`` one octet short (:issue:`723`). + field whose value contains that pair. The entries are walked instead, each binary + value counted out by its length prefix; a bad terminator or length raises + :exc:`.ProtocolError`, so the block's journal data is kept as captured, with a + :exc:`.ProtocolWarning` (:issue:`1406`). Another gap closed in the same change: a + trailing separator on the block's last octet was indistinguishable from EOF and + swallowed, so a rebuild wrote ``length`` one octet short (:issue:`723`). * **a breaking change to** ``.get()``-backed enum fields: TCP/UDP/SCTP's :class:`.PortEnumField` and PCAP-NG's ``OptionEnumField`` each minted a member through ``aenum.extend_enum`` for every value no row or documented span covers (111 calls on @@ -2267,12 +2286,12 @@ Fixed hand-written ``get()``, still on the retired ``default == -1`` convention, is replaced by the base's (no caller relied on the old form). :class:`~pcapkit.const.pcapng.option_type.OptionType` keeps its own ``get()`` for its - multi-namespace dispatch, but a later review found it still minted on both its - int/namespace and ``str`` paths, and the live pcapng parse path + multi-namespace dispatch, and both its int/namespace and ``str`` paths build an + unregistered member, where each minted one; the live pcapng parse path (:meth:`.PCAPNG._make_pcapng_options`) calls it with wire bytes, so parsing an - undeclared option code still registered a permanent member; both paths now build an - unregistered member. :class:`.FEATCode`, :class:`~pcapkit.const.ftp.command.Command` - and :class:`~pcapkit.const.http.method.Method` mint the literal wire value as its own + undeclared option code registered a permanent member. :class:`.FEATCode`, + :class:`~pcapkit.const.ftp.command.Command` and + :class:`~pcapkit.const.http.method.Method` mint the literal wire value as its own name rather than any placeholder; per the owner's ruling that ``get()`` lacks the information to create new members (:class:`~pcapkit.const.ftp.command.Command` needs ``feat``/``desc``/``type``/``conf`` and :class:`~pcapkit.const.http.method.Method` @@ -2679,6 +2698,31 @@ Fixed Linux (:issue:`2099`, :pr:`2111`), and encode through :func:`pcapkit.utilities.datetime.totimestamp`, since GraalPy's ``timestamp()`` put a pre-1970 instant with a fraction one second late (:issue:`2134`, :pr:`2136`). +* ARP's address makers refuse what they cannot pack: a ``bool`` hardware or protocol + address raises :exc:`.FieldValueError` whatever ``htype`` or ``ptype`` is; a hardware + address that is none of ``str``, ``bytes`` and ``bytearray``, or a protocol address + not packed as IPv4 or IPv6 that is none of those and not an ``ipaddress`` address, + raises :exc:`.ProtocolError`, where both failed later with a bare ``TypeError``. A + ``bytearray`` packs as ``bytes`` (:issue:`2166`, :pr:`2182`). +* the Quick-Start rate parses as the ``int`` the IPv4, HOPOPT, IPv6-Opts and TCP data + models declare, where it was a ``float`` such as ``80.0``, and a TCP Fast Open cookie + request, the option of length 2, parses with ``cookie`` as ``b''`` rather than + ``None`` (:issue:`2167`, :pr:`2185`). +* PCAP-NG's ``ns_dnsIP6addr`` option and IPv6 name resolution record build with their + defaults, ``2001:4860:4860::8888`` and ``::1``, where IPv4 defaults made them raise + :exc:`.FieldValueError` (:issue:`2168`, :pr:`2179`). +* message text: the ``epb_flags`` and ``pack_flags`` length errors say ``expected 4``, + the length they check, rather than ``expected 8`` (:issue:`2168`, :pr:`2179`), and + the IPv4 TS option's unknown-flag warning and the Mobility Header's prefix error + spell ``timestamp`` and ``mobile`` (:issue:`2171`, :pr:`2189`). +* IPX dissects its payload from a registry of its own, :attr:`.IPX.__proto__`, keyed by + IPX Packet Type, where it looked the Packet Type up as an IP protocol number, so a + type 17 (NCP) payload parsed as ``IPX:UDP`` and type 4 as ``IPX:IPv4``. No IPX-native + protocol ships, so the payload is :class:`~pcapkit.protocols.misc.raw.Raw` unless one + is registered (:issue:`2169`, :pr:`2187`). +* HIP's ``ECHO_RESPONSE_UNSIGNED`` maker builds an ``EchoResponseUnsignedParameter`` + schema, where it built the request's; the octets were already right (:issue:`2170`, + :pr:`2188`). pcapkit.toolkit --------------- @@ -2759,7 +2803,7 @@ Fixed root filter name, pseudo-protocol roots aside, that resolved to a single :class:`~pcapkit.const.reg.linktype.LinkType` member across the encapsulations this host's Wireshark can write. ``pcapkit/toolkit/pyshark.py`` reaches full coverage under - new tests that fail against the unfixed tree; live pyshark parsing stays unexercised, + new tests, which fail without the fix; live pyshark parsing stays unexercised, since ``pyshark.FileCapture`` does not run on Python 3.14 (:pr:`850`). * ``pypcapfile``'s ``tcp_reassembly`` set ``last`` one past the segment. ``TCP_Packet.last`` is inclusive, and the other adapters set @@ -2889,9 +2933,8 @@ Fixed member so ``cls(value)`` found it; :pr:`861` (above) replaced it with ``_unregistered_member``, which deliberately does not register, so ``cls(value)`` missed and re-entered ``_missing_``. :pr:`861` fixed the *generated* modules but - not these *crawlers*, so :pr:`861`'s CI stayed green until a live regeneration - reintroduced the defect; :pr:`861`'s review had sampled only some regenerations, - and neither crawler's was among them. Both crawlers now collapse to the single + not these *crawlers*, so a live regeneration reintroduced the defect. Both crawlers + now collapse to the single returning line the other registries use, and their const files are regenerated. A new guard at the crawler-emission layer requires a hard-coded ``_missing_`` body to end in a return and no emitted line to re-enter ``_missing_`` for the same value -- @@ -3034,10 +3077,10 @@ Added file through ``extract()`` and derives expectations by walking its record chain with ``struct``; a unit-tier case in ``test_header_frame_unit.py`` builds a big-endian capture in memory, so the fixture-free selection CI runs on - every push catches it too. Each fails on the unfixed tree (the - fixture-backed ones by that ``ValueError``, the in-memory one by - ``AssertionError: 3106905 != 1500000000``) while the little-endian twin passes on - both, showing the records themselves are not the variable (:issue:`605`). + every push catches it too. Without the fix, each fails (the fixture-backed ones by + that ``ValueError``, the in-memory one by ``AssertionError: 3106905 != 1500000000``) + while the little-endian twin passes with and without it, showing the records + themselves are not the variable (:issue:`605`). * local type stubs under ``typings/`` for dpkt, pyshark, ``pcap`` and pypcapfile, declaring what pcapkit uses of each, so mypy checks the engines against them. ``mypy.ini`` reads them through ``mypy_path``, and neither the sdist nor the wheel