diff --git a/CHANGELOG.md b/CHANGELOG.md index 7b60853ec..8082a1257 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,6 +1,6 @@ ## 1.5.0 -- unreleased -The largest release since 1.0, and the first recorded here as it happened rather than reconstructed. Three more extraction engines, ESP with payload decryption, SCTP and NGAP over SCTP, a library logger that no longer hijacks the consumer's, and a defect programme run through the issue tracker spanning 254 issues and pull requests, from [#121](https://github.com/JarryShaw/PyPCAPKit/pull/121) to [#883](https://github.com/JarryShaw/PyPCAPKit/pull/883) as of this revision. The programme is still running, so those figures are a snapshot rather than a total -- they are re-derived from the entries below each time this file is regenerated. +The largest release since 1.0, and the first recorded here as it happened rather than reconstructed. Three more extraction engines, ESP with payload decryption, SCTP and NGAP over SCTP, a library logger that no longer hijacks the consumer's, and a defect programme run through the issue tracker, citing 266 issues and pull requests from [#121](https://github.com/JarryShaw/PyPCAPKit/pull/121) to [#1050](https://github.com/JarryShaw/PyPCAPKit/issues/1050) as of this revision. The programme is still running, so those figures are a snapshot rather than a total. Preceded by `1.5.0a1` (2026-09-15), `1.5.0b1` and `1.5.0b2` (both 2026-09-18) and `1.5.0b3` (2026-09-19), all published as prereleases and so resolved only by `pip install --pre`. `1.5.0b1` half-shipped: the tag, the GitHub release and the Conda deployments landed, but PyPI rejected the wheel because `twine check` found a Sphinx-only `:mod:` role in `README.rst`, which `pyproject.toml` declares as the dynamic long description. `1.5.0b2` reshipped it -- an existing version cannot republish -- and `1.5.0b3` followed the CI change that stops a TestPyPI outage from costing a release its wheels ([#497](https://github.com/JarryShaw/PyPCAPKit/pull/497), [#498](https://github.com/JarryShaw/PyPCAPKit/pull/498)). @@ -9,66 +9,66 @@ Preceded by `1.5.0a1` (2026-09-15), `1.5.0b1` and `1.5.0b2` (both 2026-09-18) an #### Changed - **a breaking change to** `pcapkit.const.reg.apptype`. `AppType` becomes a memberless base over one registry per transport protocol -- `TCP`, `UDP`, `SCTP` and `DCCP` -- because `aenum` refuses to subclass an enumeration that has members. `isinstance(TCP.http, AppType)` still holds and `AppType.get(port, proto=...)` proxies to the registry, so nothing under `pcapkit.protocols` changes. **The 1,004 rows IANA assigns no port, and the 704 it assigns no transport protocol, stop being members**; they become list-table docstrings on the registry that owns them. `__members_proto__` is now a per-registry `MultiDict` keyed on port, so all three services IANA registers on port 80 survive. `get()` answers with the canonical service, curated from `/etc/services` because IANA names no precedence, and `get_all()` reaches the aliases ([#754](https://github.com/JarryShaw/PyPCAPKit/pull/754)). -- the six `%`-formatted `raise ValueError(...)` calls in the generated `pcapkit.const.reg.apptype.apptype`, left by [#759](https://github.com/JarryShaw/PyPCAPKit/issues/759) so its diff over a 12,391-member file stayed reviewable, are f-strings, matching the other 605 tree-wide ([#792](https://github.com/JarryShaw/PyPCAPKit/issues/792)). The remainder followed: three `_missing_` guards (`pcapkit.vendor.tcp.flags`, `.ftp.command`, `.http.method`), `AppType`'s `__new__`/`__repr__`/`__str__`, and the `extend_enum` fallbacks in `get()` and in the span-handling tail. The `consider-using-f-string` disable drops from `tcp.flags` and `reg.apptype.apptype` and stays on `ftp.command` and `http.method`, whose `__repr__` still used `%`. Edited in the vendor templates and applied to the generated const files by hand, since a crawl would fetch IANA's live registry and rewrite unrelated rows ([#798](https://github.com/JarryShaw/PyPCAPKit/issues/798)). -- the three `__repr__` methods [#798](https://github.com/JarryShaw/PyPCAPKit/issues/798) left on `%` (`pcapkit/const/ftp/command.py:40,119`, `pcapkit/const/http/method.py:42`) are f-strings, so the disable drops from both files and both `pcapkit/vendor/` templates. The other bespoke templates (`{const,vendor}/ftp/return_code.py`, `{const,vendor}/http/status_code.py`, `vendor/http/frame.py`) and `vendor/default.py` still carry it, so [#804](https://github.com/JarryShaw/PyPCAPKit/issues/804)'s claim that nothing in `pcapkit/{const,vendor}/{ftp,http}/` needs it holds for this pair only. `test_the_disable_drops_only_where_nothing_else_needs_percent_formatting` asserted the opposite for this pair and is flipped; `repr()` is asserted member by member against the old `%` output ([#804](https://github.com/JarryShaw/PyPCAPKit/issues/804)). -- `test_const_enum_no_mint.py`'s `is_manufactured()` exemption for [#878](https://github.com/JarryShaw/PyPCAPKit/pull/878)'s (above) 53 preserved hex-suffixed names moves from a per-*file* to a per-*name-shape* one. The per-file `HEX_SUFFIXED_NAME_EXEMPT_PATHS` was shape-blind: rewriting a branch to `'Xyplex_%d' % value` -- exactly the value-derived shape the predicate exists to flag -- left the sweep green. The check now matches an AST `%`-`BinOp` whose left side is a string ending in `'_0x%s'` and whose right side equals a parsed `hex(value)[2:].upper().zfill(4)` (via `ast.dump()`, immune to quote-style or whitespace drift). The exact `exempt_hits == 53` bound stays, so adding or removing a branch still forces a human look. Only `tests/const/test_const_enum_no_mint.py` changed. Closes [#879](https://github.com/JarryShaw/PyPCAPKit/issues/879) ([#881](https://github.com/JarryShaw/PyPCAPKit/pull/881)). +- the six `%`-formatted `raise ValueError(...)` calls in the generated `pcapkit.const.reg.apptype.apptype`, left by [#759](https://github.com/JarryShaw/PyPCAPKit/issues/759), are f-strings ([#792](https://github.com/JarryShaw/PyPCAPKit/issues/792)). The remainder followed: three `_missing_` guards (`pcapkit.vendor.tcp.flags`, `.ftp.command`, `.http.method`), `AppType`'s `__new__`/`__repr__`/`__str__`, and the `extend_enum` fallbacks in `get()` and in the span-handling tail. The `consider-using-f-string` disable drops from `tcp.flags` and `reg.apptype.apptype` and stays on `ftp.command` and `http.method`, whose `__repr__` still used `%`. Edited in the vendor templates and applied to the generated const files by hand, since a crawl would fetch IANA's live registry and rewrite unrelated rows ([#798](https://github.com/JarryShaw/PyPCAPKit/issues/798)). +- the three `__repr__` methods [#798](https://github.com/JarryShaw/PyPCAPKit/issues/798) left on `%` (`pcapkit/const/ftp/command.py:40,119`, `pcapkit/const/http/method.py:42`) are f-strings, so the disable drops from both files and both `pcapkit/vendor/` templates. The other bespoke templates (`{const,vendor}/ftp/return_code.py`, `{const,vendor}/http/status_code.py`, `vendor/http/frame.py`) and `vendor/default.py` still carry it, so [#804](https://github.com/JarryShaw/PyPCAPKit/issues/804)'s claim that nothing in `pcapkit/{const,vendor}/{ftp,http}/` needs it holds for this pair only; `repr()` output is unchanged ([#804](https://github.com/JarryShaw/PyPCAPKit/issues/804)). +- `test_const_enum_no_mint.py`'s `is_manufactured()` exemption for [#878](https://github.com/JarryShaw/PyPCAPKit/pull/878)'s (above) 53 preserved hex-suffixed names moves from a per-*file* to a per-*name-shape* one. The per-file `HEX_SUFFIXED_NAME_EXEMPT_PATHS` was shape-blind: rewriting a branch to `'Xyplex_%d' % value` -- exactly the value-derived shape the predicate exists to flag -- left the sweep green. The check now exempts only a `'..._0x%s' % hex(value)[2:].upper().zfill(4)` expression, and the exact `exempt_hits == 53` bound stays. Test-only. Closes [#879](https://github.com/JarryShaw/PyPCAPKit/issues/879) ([#881](https://github.com/JarryShaw/PyPCAPKit/pull/881)). #### Fixed - constant lookups that rejected a value the registry defines. `RouterAlert(0)` is the only value [RFC 2113](https://datatracker.ietf.org/doc/html/rfc2113) defines and the one IGMP, RSVP and MLD actually send, and it was discarded because the vendor crawler skipped a header row IANA's CSV does not have; IPX `Socket(0)` is that protocol's own default, so `bytes(IPX(...))` crashed on its own defaults; and two FTP `_missing_` overrides were plain methods rather than classmethods, so every unregistered value raised `TypeError` instead of extending the enumeration ([#492](https://github.com/JarryShaw/PyPCAPKit/issues/492), [#503](https://github.com/JarryShaw/PyPCAPKit/pull/503)). - the string-keyed `get()` in `pcapkit.const.ftp.command` and `pcapkit.const.http.method` tested membership with the raw key but registered `key.upper()`, so the *first* lowercase or mixed-case token raised `TypeError: 'RETR' already in use`. Reachable from wire data: `pcapkit.protocols.application.ftp` matches with `re.I` and passes the match verbatim, and [RFC 959 Section 5](https://datatracker.ietf.org/doc/html/rfc959#section-5) makes FTP commands case-insensitive, so `retr file.txt` was a valid request the library could not parse. `get()` and `_missing_` now look the key up under the canonical upper-case name, so every casing resolves to the existing member; a duplicate `GET` would have lacked the real one's `safe` and `idempotent` attributes ([#582](https://github.com/JarryShaw/PyPCAPKit/issues/582), [#583](https://github.com/JarryShaw/PyPCAPKit/issues/583)). -- `get()`'s documented `default` was ignored on the integer path across the generated `pcapkit.const` tree, because `_missing_` cannot see the caller's `default` -- `Hardware.get(99999, 0)` raised `ValueError` instead of returning `0`. The integer path now consults `default` before letting the error escape. `-1`, the placeholder the generated signature already carried, still separates "no default" from "a default"; it is to be replaced by an identity sentinel in [#859](https://github.com/JarryShaw/PyPCAPKit/pull/859) (not yet landed) once the shared base [#858](https://github.com/JarryShaw/PyPCAPKit/pull/858) (below) puts `get()` behind one call site. The sweep covers 110 of the 118 integer registries, not the three [#584](https://github.com/JarryShaw/PyPCAPKit/issues/584) named; `pcapng` `OptionType` and `reg` `AppType` carry a bespoke integer fallback and are deliberately left alone, since neither drops a default by raising. Not reachable from wire data, so a contract fix rather than a parse fix. Applied to the nine vendor templates and the 113 generated modules; a new test renders the shared template and compares it with the generated module, so a regeneration cannot undo it ([#584](https://github.com/JarryShaw/PyPCAPKit/issues/584)). -- `_missing_` in the four Mobility Header flag enumerations (`BindingACKFlag`, `BindingUpdateFlag`, `HandoverACKFlag`, `HandoverInitiateFlag`) ended in `return cls(value)`, the constructor that had just failed, so every in-range value that is not already a member re-entered `_missing_` unbounded and raised `RecursionError` -- `F(0)`, no flags set, and every composite such as `BindingACKFlag(0x06)`. Defining `_missing_` at all caused it, by shadowing the `aenum` `Flag` machinery that resolves exactly those values; `pcapkit/const/tcp/flags.py` had none and so never had the defect (it has one now, from [#647](https://github.com/JarryShaw/PyPCAPKit/issues/647) below, with the same tail). All four now end in `return super()._missing_(value)`, as `pcapkit/vendor/default.py` emits and 75 of the 117 modules under `pcapkit/const/` already did (77 with [#647](https://github.com/JarryShaw/PyPCAPKit/issues/647)'s two new modules). `F(0)` is an empty flag and `BindingACKFlag(0x06)` is `S|D`. The `extend_enum` mint used by `pcapng/record_type.py` and `secrets_type.py` was rejected for a flag type: naming `0x06` `Unassigned_0x06` would hide the composite and add a `_member_map_` entry per bit pattern, up to 65536 for `BindingUpdateFlag`. The range guard is untouched. Fixed in the four `pcapkit/vendor/mh/` templates and regenerated ([#623](https://github.com/JarryShaw/PyPCAPKit/issues/623)). -- six of the 123 constant registries rejected an invalid value differently from the built-in `enum`, and three did not reject it at all. `Flags`, `ftp.command.CommandType` and `reg.apptype.TransportProtocol` are `IntFlag` types with no `_missing_`, so `aenum` composed a pseudo-member for any integer: `Flags(-1)` returned `65520` (every TCP flag set, from a value no 16-bit field can hold) and `Flags(-65536)` none. `ftp.command.Command`, `FEATCode` and `http.method.Method` are `StrEnum` types whose `_missing_` called `value.upper()` before checking the type, so an integer raised `AttributeError`. All six now raise a bare `ValueError`, as `enum.IntEnum` does and 113 of the 117 modules already did. [#647](https://github.com/JarryShaw/PyPCAPKit/issues/647)'s proposal to replace those 113 with `pcapkit.utilities.exceptions.EnumError` was deliberately rejected: `EnumError` is `(BaseError, TypeError)`, not a `ValueError`, so it would diverge from the built-in, walk past the `except ValueError` in all 113 generated `get()` bodies (silently undoing [#584](https://github.com/JarryShaw/PyPCAPKit/issues/584)), and log CRITICAL from `BaseError.__init__` once per discarded default. The one deliberate divergence is kept and pinned: mutable registries look up, miss, and `extend_enum` rather than raising, so `Method('FROBNICATE')`, `ProtectionAuthority(1 << 70)` and `AppType.get(65000, proto=tcp)` still register. The three flag guards end in `return super()._missing_(value)`, so in-range composites still decompose and [#623](https://github.com/JarryShaw/PyPCAPKit/issues/623) stays fixed. `pcapkit/vendor/tcp/flags.py` carried the value checker all along as `FLAG = 'isinstance(value, int) and 4 <= value <= 15'` without interpolating it; those are *bit offsets*, so emitting it unchanged would have rejected every composite and `Flags(0)`. It now reads `0 <= value <= 0xFFFF`. `TransportProtocol` reads its bound from `cls.__members__`, because `get` grows the registry at runtime and a literal would reject the member it had just added. Fixed in the four bespoke templates under `pcapkit/vendor/` and regenerated from live IANA data (42 insertions, no deletions, in 4 of 134 files). A new `tests/const/test_const_enum_builtin_parity.py` asserts that every one of the 123 registries raises the same exception type as a stdlib `enum.IntEnum`; it keys on `aenum.Enum` because an `IntFlag` is not an `IntEnum` subclass, which is how three of the six escaped the existing sweeps ([#647](https://github.com/JarryShaw/PyPCAPKit/issues/647)). +- `get()`'s documented `default` was ignored on the integer path across the generated `pcapkit.const` tree, because `_missing_` cannot see the caller's `default` -- `Hardware.get(99999, 0)` raised `ValueError` instead of returning `0`. The integer path now consults `default` before letting the error escape. `-1`, the placeholder the generated signature already carried, still separates "no default" from "a default"; [#859](https://github.com/JarryShaw/PyPCAPKit/pull/859) replaced it with an identity sentinel once the shared base [#858](https://github.com/JarryShaw/PyPCAPKit/pull/858) (below) put `get()` behind one call site. The sweep covers 110 of the 118 integer registries, not the three [#584](https://github.com/JarryShaw/PyPCAPKit/issues/584) named; `pcapng` `OptionType` and `reg` `AppType` carry a bespoke integer fallback and are deliberately left alone, since neither drops a default by raising. Not reachable from wire data, so a contract fix rather than a parse fix ([#584](https://github.com/JarryShaw/PyPCAPKit/issues/584)). +- `_missing_` in the four Mobility Header flag enumerations (`BindingACKFlag`, `BindingUpdateFlag`, `HandoverACKFlag`, `HandoverInitiateFlag`) ended in `return cls(value)`, the constructor that had just failed, so every in-range value that is not already a member re-entered `_missing_` unbounded and raised `RecursionError` -- `F(0)`, no flags set, and every composite such as `BindingACKFlag(0x06)`. Defining `_missing_` at all caused it, by shadowing the `aenum` `Flag` machinery that resolves exactly those values; `pcapkit/const/tcp/flags.py` had none and so never had the defect (it has one now, from [#647](https://github.com/JarryShaw/PyPCAPKit/issues/647) below, with the same tail). All four now end in `return super()._missing_(value)`, as `pcapkit/vendor/default.py` emits and most const modules ([#647](https://github.com/JarryShaw/PyPCAPKit/issues/647)'s two new ones included) already did, so `F(0)` is an empty flag and `BindingACKFlag(0x06)` is `S|D`. Minting through `extend_enum` was rejected for a flag type: naming `0x06` `Unassigned_0x06` would hide the composite and add a member per bit pattern. The range guard is untouched ([#623](https://github.com/JarryShaw/PyPCAPKit/issues/623)). +- six of the 123 constant registries rejected an invalid value differently from the built-in `enum`, and three did not reject it at all. `Flags`, `ftp.command.CommandType` and `reg.apptype.TransportProtocol` are `IntFlag` types with no `_missing_`, so `aenum` composed a pseudo-member for any integer: `Flags(-1)` returned `65520` (every TCP flag set, from a value no 16-bit field can hold) and `Flags(-65536)` none. `ftp.command.Command`, `FEATCode` and `http.method.Method` are `StrEnum` types whose `_missing_` called `value.upper()` before checking the type, so an integer raised `AttributeError`. All six now raise a bare `ValueError`, as `enum.IntEnum` does and 113 of the 117 modules already did. [#647](https://github.com/JarryShaw/PyPCAPKit/issues/647)'s proposal to replace those 113 with `pcapkit.utilities.exceptions.EnumError` was deliberately rejected: `EnumError` is `(BaseError, TypeError)`, not a `ValueError`, so it would diverge from the built-in, walk past the `except ValueError` in all 113 generated `get()` bodies (silently undoing [#584](https://github.com/JarryShaw/PyPCAPKit/issues/584)), and log CRITICAL from `BaseError.__init__` once per discarded default. The one deliberate divergence is kept and pinned: mutable registries look up, miss, and `extend_enum` rather than raising, so `Method('FROBNICATE')`, `ProtectionAuthority(1 << 70)` and `AppType.get(65000, proto=tcp)` still register. The three flag guards end in `return super()._missing_(value)`, so in-range composites still decompose and [#623](https://github.com/JarryShaw/PyPCAPKit/issues/623) stays fixed; `Flags` accepts `0 <= value <= 0xFFFF`. A new `tests/const/test_const_enum_builtin_parity.py` asserts that every one of the 123 registries raises the same exception type as a stdlib `enum.IntEnum`; it keys on `aenum.Enum` because an `IntFlag` is not an `IntEnum` subclass, which is how three of the six escaped the existing sweeps ([#647](https://github.com/JarryShaw/PyPCAPKit/issues/647)). - **a breaking change to** `AppType.get`: an out-of-range port is refused rather than minted. Its `except ValueError` caught the range rejection from `_missing_` and minted regardless, so `AppType.get(-1, proto='tcp')` returned an unreachable `PORT_-1_tcp` while `TCP(-1)` raised. It now tests `_missing_` for `None` ("no row for this port") and lets the rejection through; a valid but unassigned port still mints. **`Transport._make_port` passes an unvalidated `int`, so `TCP.make(srcport=99999)` now raises where it minted junk.** Separately, the crawler rendered one `_missing_` branch per registry row, so the two spans IANA registers on more than one transport emitted the same condition twice and every registry answered with the first: `AppType.get(6010, proto='udp')` gave a member whose `proto` read `tcp`, `6666/udp` answered `ircu` rather than `reserved`, and `proto='sctp'` minted `x11` where IANA assigns nothing. A branch naming a transport now tests `cls.__transport__`; the 762 naming none are unchanged ([#764](https://github.com/JarryShaw/PyPCAPKit/pull/764)). -- **a breaking change to** `AppType._dispatch`: it resolved a `proto` naming several transports by its lowest set bit, and `tcp` is lowest, so every composite containing it dispatched into the TCP registry -- silently, and undetectably to a caller checking equality, since `AppType.__eq__` compares on `port` alone. `TransportProtocol` is an `aenum.IntFlag` and a member carries the whole set IANA assigned the service, so `tcp | udp` is an ordinary value to pass back in. Sweeping all 10,625 multi-transport members through `get(m.port, proto=m.proto)` gave 46 answers naming another service at 20 ports; 44 differ only in not being the port's canonical, which a single-bit lookup does too, leaving two real defects: port 888 (`accessbuilder` resolved to TCP's `cddbp`) and port 999 (`puprouter` resolved to TCP's `garcon`). It now raises `ProtocolError`, a `ValueError` subclass, so the documented contract holds. No caller is affected: the ten library call sites each name one transport, and `register_apptype`, the one reader of a composite `proto`, tests it with `in` ([#759](https://github.com/JarryShaw/PyPCAPKit/issues/759)). -- `get()`'s string-key miss and every generated registry's `_missing_` bounded-range branch called `aenum.extend_enum()` on an unrecognised value, so e.g. `Hardware(40)` minted a permanent member. Fixed at the two shared sites in `pcapkit/vendor/default.py`, plus a new `register(value, name)` base classmethod for the caller-named path that still mints. All 105 generated registries are updated; the non-minting `get()` miss applies to all 105, but the `_missing_` fix changes behaviour only for the **21** with a bounded-range `extend_enum` branch. `pcapkit/toolkit/scapy.py` and `pcapkit/toolkit/pyshark.py` fed `get()` a string key with no default and now raise `MissingKeyError` instead of receiving a placeholder; four reader sites in `hopopt.py`/`ipv6_opts.py` convert the `KeyError` into `ProtocolError`. Riding along: `pcapkit/toolkit/pyshark.py` gains a two-entry `FILTER_NAME_TO_LINKTYPE` (`eth` to `ETHERNET`, `tr` to `IEEE802_5`), consulted ahead of `LinkType.get` since pyshark reports a filter name, not a DLT name. Out of scope and deferred: 1026 minting call sites (1007 in `_missing_`, 7 in `get`, 12 hand-written in `pcapkit/protocols/internet/mh.py` and `pcapkit/protocols/application/ngap.py`); the 106 `register`/`register_alias` sites are the by-design caller-named path ([#838](https://github.com/JarryShaw/PyPCAPKit/pull/838)). -- **a breaking change to** `pcapkit.const.ipx.socket.Socket`'s `_missing_`: three of its five archived range branches were strict subsets of a range tested earlier -- `(0x0020, 0x003F)` after `(0x0001, 0x0BB8)`, and `(0x4000, 0x4FFF)` and `(0x8000, 0xFFFF)` after `(0x0BB9, 0xFFFF)` -- so they never fired. The generator `pcapkit/vendor/ipx/socket.py` now lists each subset range ahead of the range containing it. **36891 of the 65521 `_missing_`-resolved values (56.3%) change name**: 32 from `Registered by Xerox` to `Experimental`, 4095 from `Dynamically Assigned` to `Dynamically Assigned Socket Numbers` and 32764 from `Dynamically Assigned` to `Statically Assigned Socket Numbers`. No defined member changes and every value still resolves. The prior tests had pinned the shadowing as intended (`# masks 'Experimental'`); they are corrected rather than dropped, and a new test pins all three ranges as reachable ([#847](https://github.com/JarryShaw/PyPCAPKit/pull/847)). -- `LinkType(209).name` returned `'IPMB_LINUX'`, which tcpdump's own table marks "Legacy names (do not use)", because a 2024-05-04 regeneration emitted that row above the current `I2C_LINUX = 209` and `aenum` gives a value to the member defined first. `pcapkit/vendor/reg/linktype.py` now sinks any row whose note mentions "legacy" after every current row, so the current name is defined first; today that affects value 209 alone. `IPMB_LINUX` stays a reachable alias (`LinkType['IPMB_LINUX']` still resolves to 209). Not breaking: no name disappears, no value or membership changes, and `LinkType.IPMB_LINUX is LinkType.I2C_LINUX` holds before and after; only the canonical name for the shared value moves ([#848](https://github.com/JarryShaw/PyPCAPKit/pull/848)). -- **a breaking change to** 82 more const registries' `_missing_`: applies the owner's mint/unmint ruling (settled on [#847](https://github.com/JarryShaw/PyPCAPKit/pull/847), confirmed on [#775](https://github.com/JarryShaw/PyPCAPKit/issues/775) -- *"a final concrete assigned name -> mint; a notation for the readers -> unmint"*) across the 89 `_missing_` files still calling `extend_enum`. 80 registries convert wholly (164 branches), each minting a bare status word (`Unassigned`, `Reserved [...]`, `Reserved for Private/Experimental Use`, `Unspecified in the IANA registry`, `Deprecated`), which the ruling treats as a notation rather than an assignment. Two are mixed: `EtherType`'s `DEC Unassigned` (3 branches) and "Old Xerox Experimental values. Invalid as an Ethertype since 1983." (1) convert, while 46 vendor-company names across 50 range blocks (Xyplex, Motorola, Walker Richer & Quinn and 43 more) keep minting, since a proprietary protocol has no public name beyond the company's; `IEEE802_3_Length_Field` and `Berkeley_Trailer_encap_IP` also keep minting but are not company names. `Socket` (ipx)'s allocation-policy names (`Experimental`, `Dynamically Assigned Socket Numbers`, `Statically Assigned Socket Numbers`, `Dynamically Assigned`) convert, while `Registered by Xerox` keeps minting, a real ownership fact. 7 of the 89 are left alone: `CGAType`'s `Tag_` mint is not an IANA-style range, and the other 6 files sit on 9 classes not yet on `EnumRegistry` (`AppType`, `StatusCode`, `ftp.return_code`, `ftp.command`, `Method`, `OptionType`), blocked on [#859](https://github.com/JarryShaw/PyPCAPKit/pull/859)/[#860](https://github.com/JarryShaw/PyPCAPKit/issues/860). Each conversion changes both the vendor crawler and the generated file, per [#838](https://github.com/JarryShaw/PyPCAPKit/pull/838)'s and [#858](https://github.com/JarryShaw/PyPCAPKit/pull/858)'s precedent. `docs/source/conventions.rst`'s worked examples already agreed with keeping `Registered by Xerox` under Mint ([#861](https://github.com/JarryShaw/PyPCAPKit/pull/861)). -- **a breaking change to** `EtherType`'s `_missing_` ordering: values `0x0101`-`0x01FF` now resolve to their Old Xerox Experimental name instead of an `IEEE802_3_Length_Field_0x0101`-style one. The generated method tested `0x0000 <= value <= 0x05DC` (IEEE802.3 Length Field) first, and the Old Xerox range sits wholly inside it, so that branch was unreachable. Unlike [#841](https://github.com/JarryShaw/PyPCAPKit/issues/841)'s IPX fix, these bounds come from the live IANA CSV, so the fix is general: `pcapkit/vendor/reg/ethertype.py`'s `process()` collects range rows and a new `EtherType._insert_range` places each ahead of the first already-placed range that fully contains it, leaving non-overlapping pairs in CSV order. Of all 56 range tests, this pair is the only containment. `tests/const/test_const_ethertype_862_unit.py` pins the symptom, the root cause and the ordering rule. Closes [#862](https://github.com/JarryShaw/PyPCAPKit/issues/862) ([#865](https://github.com/JarryShaw/PyPCAPKit/pull/865)). -- **a breaking change to** `Method`: `__new__` called `str.__new__(cls)` with no argument, so all 40 registered members' `str` payload was empty regardless of value -- `str(Method.GET) == ''` and `Method.GET == 'GET'` was `False`; `.value` was always correct. [#869](https://github.com/JarryShaw/PyPCAPKit/pull/869) (above) had fixed the *unregistered* path only, leaving `str(Method('frob')) == 'frob'` beside `str(Method.GET) == ''`. Fixed to `str.__new__(cls, value)`, mirroring `Command.__new__`, in the generator (`pcapkit/vendor/http/method.py`) and the generated module. No sibling hand-rolled `__new__` has the defect (`OptionType`/`AppType` deliberately store a display string). Consequences, all corrections: `Method.GET == 'GET'` is `True`; `bool(Method.GET)` flips `False` to `True`; `json.dumps(Method.GET)` emits `"GET"` not `""`; `sorted()` is lexicographic rather than input-order-stable; and `{m: ... for m in Method}` gains 40 entries where it had 1, because all 40 members previously hashed as `''` and compared equal. Nothing under `pcapkit/protocols` relied on the old behaviour (`httpv1.py`/`httpv2.py` read only `.value`). Closes [#870](https://github.com/JarryShaw/PyPCAPKit/issues/870) ([#871](https://github.com/JarryShaw/PyPCAPKit/pull/871)). -- **a breaking change to** `TransportProtocol` and `AppType`: step 2 of [#860](https://github.com/JarryShaw/PyPCAPKit/issues/860), PR 2 of 2 (the 8 non-`AppType` bespoke registries were [#869](https://github.com/JarryShaw/PyPCAPKit/pull/869), above). `AppType` and its four transport subclasses move onto `EnumRegistry` and both mint sites -- `_missing_`'s 766 range branches and `get()`'s second mint (`PORT_{port}_{transport}` -> `'unknown'`) -- now build through a new `_unregistered_member` override that reconstructs `svc`/`port`/`proto`, per the owner's ruling that *"only IANA registered ones are legit values... get will not have sufficient information to create new ones"*. All 766+1 branches convert, including 8 naming a real service assigned to a whole port span (7 distinct names: `x11` twice, `active-net`, `satvid-datalnk`, `vrml-multi-use`, `ircu`, `swx`, `flex-lm`); unlike `FEATCode`'s fix ([#869](https://github.com/JarryShaw/PyPCAPKit/pull/869), above) there is no import-time self-mutation to cure by declaring members statically, so these stay deliberately unregistered lookups. `AppType` gains a working `register()`; the base's would have built a member with `svc=''`. **The breaking part**: `TransportProtocol` moves from power-of-two values (`tcp=1, udp=2, sctp=4, dccp=8`) to sequential `auto()` (`sctp=3, dccp=4`), per the owner's follow-up ruling once `|`-joined composites stopped being parsed. `AppType.get` treats whatever `int` it is given as a whole, so the renumbering changes what specific integers *mean*: on `AppType.get(80, proto=...)`, a bare `4` resolved to `SCTP.http` on `main` and now **silently** resolves to `DCCP.unknown`, because a real member sits at `4` under both numberings; a bare `8` used to mint `DCCP.PORT_80_dccp` and now raises `ValueError`. The silent reroute is disclosed here, and the composed-value case is deliberately untested, per the owner's ruling that it is "an obsoleted path from a breaking change". ([#874](https://github.com/JarryShaw/PyPCAPKit/pull/874)). -- closes the remaining scope of [#775](https://github.com/JarryShaw/PyPCAPKit/issues/775): `EtherType` (52 branches) and `Socket` (1), the last registries whose `_missing_` minted a permanent member, now return a non-registering `cls._unregistered_member(value, name)`. The issue's AST census of `extend_enum` in `_missing_`/`get` goes from 54 sites to the single `CGAType` carve-out (its mint is not an IANA-style range), from the original 1,169. `EtherType(0x0888)` used to grow `__members__` from 160 to 161 and return the identical object on a repeat; it now returns `` with `__members__` unchanged, absent from `_value2member_map_`, equal but not identical across lookups -- likewise `Socket(0x0010)`. Weaker verification, disclosed: `vendor/ipx/socket.py` regenerates byte-identically offline, but `vendor/reg/ethertype.py` needs the live IANA CSV, so it rests on a mechanical substitution matching exactly 52 lines plus a run against the committed CSV fixture. Keeping each branch's hex-suffixed name reintroduces the value-derived shape `test_const_enum_no_mint.py`'s `is_manufactured()` sweep forbids; rather than weaken the predicate, a narrow `HEX_SUFFIXED_NAME_EXEMPT_PATHS` covers exactly these two files, since the collision the check guards against cannot occur once `_unregistered_member` never registers. `pcapkit/corekit/enum.py`'s docstring now names `CGAType` alone as still minting. `tests/vendor/test_ipx_socket_unit.py` is unaffected (`EXPECTED_MISSING_NAMES` still carries `0x0010: 'Registered by Xerox_0x0010'`) ([#878](https://github.com/JarryShaw/PyPCAPKit/pull/878)). +- **a breaking change to** `AppType._dispatch`: it resolved a `proto` naming several transports by its lowest set bit, and `tcp` is lowest, so every composite containing it dispatched into the TCP registry -- silently, and undetectably to a caller checking equality, since `AppType.__eq__` compares on `port` alone. `TransportProtocol` is an `aenum.IntFlag` and a member carries the whole set IANA assigned the service, so `tcp | udp` is an ordinary value to pass back in. Over every multi-transport member this misrouted two: port 888 (`accessbuilder` resolved to TCP's `cddbp`) and port 999 (`puprouter` resolved to TCP's `garcon`). It now raises `ProtocolError`, a `ValueError` subclass, so the documented contract holds. No library caller passes a composite ([#759](https://github.com/JarryShaw/PyPCAPKit/issues/759)). +- `get()`'s string-key miss and every generated registry's `_missing_` bounded-range branch called `aenum.extend_enum()` on an unrecognised value, so e.g. `Hardware(40)` minted a permanent member. Fixed at the two shared sites in `pcapkit/vendor/default.py`, plus a new `register(value, name)` base classmethod for the caller-named path that still mints. All 105 generated registries are updated; the non-minting `get()` miss applies to all 105, but the `_missing_` fix changes behaviour only for the **21** with a bounded-range `extend_enum` branch. `pcapkit/toolkit/scapy.py` and `pcapkit/toolkit/pyshark.py` fed `get()` a string key with no default and now raise `MissingKeyError` instead of receiving a placeholder; four reader sites in `hopopt.py`/`ipv6_opts.py` convert the `KeyError` into `ProtocolError`. Riding along: `pcapkit/toolkit/pyshark.py` gains a two-entry `FILTER_NAME_TO_LINKTYPE` (`eth` to `ETHERNET`, `tr` to `IEEE802_5`), consulted ahead of `LinkType.get` since pyshark reports a filter name, not a DLT name. The other minting call sites in `_missing_`, `get` and the hand-written `mh.py` and `ngap.py` were deferred; the 106 `register`/`register_alias` sites are not deferred but the by-design caller-named path ([#838](https://github.com/JarryShaw/PyPCAPKit/pull/838)). +- **a breaking change to** `pcapkit.const.ipx.socket.Socket`'s `_missing_`: three of its five archived range branches were strict subsets of a range tested earlier -- `(0x0020, 0x003F)` after `(0x0001, 0x0BB8)`, and `(0x4000, 0x4FFF)` and `(0x8000, 0xFFFF)` after `(0x0BB9, 0xFFFF)` -- so they never fired. The generator `pcapkit/vendor/ipx/socket.py` now lists each subset range ahead of the range containing it. **36891 of the 65521 `_missing_`-resolved values (56.3%) change name**: 32 from `Registered by Xerox` to `Experimental`, 4095 from `Dynamically Assigned` to `Dynamically Assigned Socket Numbers` and 32764 from `Dynamically Assigned` to `Statically Assigned Socket Numbers`. No defined member changes and every value still resolves ([#847](https://github.com/JarryShaw/PyPCAPKit/pull/847)). +- `LinkType(209).name` returned `'IPMB_LINUX'`, which tcpdump's own table marks "Legacy names (do not use)", because a 2024-05-04 regeneration emitted that row above the current `I2C_LINUX = 209` and `aenum` gives a value to the member defined first. `pcapkit/vendor/reg/linktype.py` now sinks any row whose note mentions "legacy" after every current row, so the current name is defined first; today that affects value 209 alone. `IPMB_LINUX` stays a reachable alias (`LinkType['IPMB_LINUX']` still resolves to 209). Not breaking: only the canonical name for the shared value moves ([#848](https://github.com/JarryShaw/PyPCAPKit/pull/848)). +- **a breaking change to** 82 more const registries' `_missing_`: applies the owner's mint/unmint ruling (settled on [#847](https://github.com/JarryShaw/PyPCAPKit/pull/847), confirmed on [#775](https://github.com/JarryShaw/PyPCAPKit/issues/775) -- *"a final concrete assigned name -> mint; a notation for the readers -> unmint"*) across the 89 `_missing_` files still calling `extend_enum`. 80 registries convert wholly (164 branches), each minting a bare status word (`Unassigned`, `Reserved [...]`, `Reserved for Private/Experimental Use`, `Unspecified in the IANA registry`, `Deprecated`), which the ruling treats as a notation rather than an assignment. Two are mixed: `EtherType`'s `DEC Unassigned` (3 branches) and "Old Xerox Experimental values. Invalid as an Ethertype since 1983." (1) convert, while 46 vendor-company names across 50 range blocks (Xyplex, Motorola, Walker Richer & Quinn and 43 more) keep minting, since a proprietary protocol has no public name beyond the company's; `IEEE802_3_Length_Field` and `Berkeley_Trailer_encap_IP` also keep minting but are not company names. `Socket` (ipx)'s allocation-policy names (`Experimental`, `Dynamically Assigned Socket Numbers`, `Statically Assigned Socket Numbers`, `Dynamically Assigned`) convert, while `Registered by Xerox` keeps minting, a real ownership fact. 7 of the 89 are left alone: `CGAType`'s `Tag_` mint is not an IANA-style range, and the other 6 files sit on 9 classes not yet on `EnumRegistry` (`AppType`, `StatusCode`, `ftp.return_code`, `ftp.command`, `Method`, `OptionType`), left to [#859](https://github.com/JarryShaw/PyPCAPKit/pull/859)/[#860](https://github.com/JarryShaw/PyPCAPKit/issues/860). Each conversion changes both the vendor crawler and the generated file, per [#838](https://github.com/JarryShaw/PyPCAPKit/pull/838)'s and [#858](https://github.com/JarryShaw/PyPCAPKit/pull/858)'s precedent. The worked examples in `docs/source/contributing/conventions/mint-criterion.rst` keep `Registered by Xerox` under Mint ([#861](https://github.com/JarryShaw/PyPCAPKit/pull/861)). +- **a breaking change to** `EtherType`'s `_missing_` ordering: values `0x0101`-`0x01FF` now resolve to their Old Xerox Experimental name instead of an `IEEE802_3_Length_Field_0x0101`-style one. The generated method tested `0x0000 <= value <= 0x05DC` (IEEE802.3 Length Field) first, and the Old Xerox range sits wholly inside it, so that branch was unreachable. Unlike [#841](https://github.com/JarryShaw/PyPCAPKit/issues/841)'s IPX fix, these bounds come from the live IANA CSV, so the fix is general: `pcapkit/vendor/reg/ethertype.py`'s `process()` collects range rows and a new `EtherType._insert_range` places each ahead of the first already-placed range that fully contains it, leaving non-overlapping pairs in CSV order; this pair is the only containment among the 56 ranges. Closes [#862](https://github.com/JarryShaw/PyPCAPKit/issues/862) ([#865](https://github.com/JarryShaw/PyPCAPKit/pull/865)). +- **a breaking change to** `Method`: `__new__` called `str.__new__(cls)` with no argument, so all 40 registered members' `str` payload was empty regardless of value -- `str(Method.GET) == ''` and `Method.GET == 'GET'` was `False`; `.value` was always correct. [#869](https://github.com/JarryShaw/PyPCAPKit/pull/869) (above) had fixed the *unregistered* path only, leaving `str(Method('frob')) == 'frob'` beside `str(Method.GET) == ''`. Fixed to `str.__new__(cls, value)`, mirroring `Command.__new__`, in the generator (`pcapkit/vendor/http/method.py`) and the generated module. No sibling hand-rolled `__new__` has the defect (`OptionType`/`AppType` deliberately store a display string). Consequences, all corrections: `Method.GET == 'GET'` is `True`; `bool(Method.GET)` flips `False` to `True`; `json.dumps(Method.GET)` emits `"GET"` not `""`; `sorted()` is lexicographic rather than input-order-stable; and `{m: ... for m in Method}` gains 40 entries where it had 1, because all 40 members previously hashed as `''` and compared equal. Nothing under `pcapkit/protocols` relied on the old behaviour. Closes [#870](https://github.com/JarryShaw/PyPCAPKit/issues/870) ([#871](https://github.com/JarryShaw/PyPCAPKit/pull/871)). +- **a breaking change to** `TransportProtocol` and `AppType`: step 2 of [#860](https://github.com/JarryShaw/PyPCAPKit/issues/860), PR 2 of 2 (the 8 non-`AppType` bespoke registries were [#869](https://github.com/JarryShaw/PyPCAPKit/pull/869), above). `AppType` and its four transport subclasses move onto `EnumRegistry` and both mint sites -- `_missing_`'s 766 range branches and `get()`'s second mint (`PORT_{port}_{transport}` -> `'unknown'`) -- now build through a new `_unregistered_member` override that reconstructs `svc`/`port`/`proto`, per the owner's ruling that *"only IANA registered ones are legit values... get will not have sufficient information to create new ones"*. All 766+1 branches convert, including 8 naming a real service assigned to a whole port span (7 distinct names: `x11` twice, `active-net`, `satvid-datalnk`, `vrml-multi-use`, `ircu`, `swx`, `flex-lm`); unlike `FEATCode`'s fix ([#869](https://github.com/JarryShaw/PyPCAPKit/pull/869), above) there is no import-time self-mutation to cure by declaring members statically, so these stay deliberately unregistered lookups. `AppType` gains a working `register()`; the base's would have built a member with `svc=''`. **The breaking part**: `TransportProtocol` moves from power-of-two values (`tcp=1, udp=2, sctp=4, dccp=8`) to sequential `auto()` (`sctp=3, dccp=4`), per the owner's follow-up ruling once `|`-joined composites stopped being parsed. `AppType.get` treats whatever `int` it is given as a whole, so the renumbering changes what specific integers *mean*: on `AppType.get(80, proto=...)`, a bare `4` resolved to `SCTP.http` on `main` and now **silently** resolves to `DCCP.unknown`, because a real member sits at `4` under both numberings; a bare `8` used to mint `DCCP.PORT_80_dccp` and now raises `ValueError`. The composed-value case is deliberately untested, per the owner's ruling that it is "an obsoleted path from a breaking change" ([#874](https://github.com/JarryShaw/PyPCAPKit/pull/874)). +- closes the remaining scope of [#775](https://github.com/JarryShaw/PyPCAPKit/issues/775): `EtherType` (52 branches) and `Socket` (1), the last registries whose `_missing_` minted a permanent member, now return a non-registering `cls._unregistered_member(value, name)`. The issue's AST census of `extend_enum` in `_missing_`/`get` goes from 54 sites to the single `CGAType` carve-out (its mint is not an IANA-style range), from the original 1,169. `EtherType(0x0888)` used to grow `__members__` from 160 to 161 and return the identical object on a repeat; it now returns `` with `__members__` unchanged, absent from `_value2member_map_`, equal but not identical across lookups -- likewise `Socket(0x0010)`. The regenerated `EtherType` was not checked against the live IANA CSV, only against the committed fixture. Keeping each branch's hex-suffixed name reintroduces the value-derived shape `test_const_enum_no_mint.py`'s `is_manufactured()` sweep forbids; rather than weaken the predicate, a narrow exemption covers exactly these two files, since the collision the check guards against cannot occur once `_unregistered_member` never registers ([#878](https://github.com/JarryShaw/PyPCAPKit/pull/878)). ### pcapkit.corekit #### Added -- `tests/corekit/test_fields_numbers_width_repair.py`, covering each octet boundary in its own method, since the defect is a pattern and a single case would pass against a fix that special-cased the reported width. Each boundary is asserted as a pair (the value below, which always packed, and the value above, which did not), so a width shifted by one in the other direction fails too. It asserts widths and octets rather than exception types, because the exception depends on whether the mis-sized width has a native `struct` code ([#599](https://github.com/JarryShaw/PyPCAPKit/issues/599)). -- `FieldBaseShortReadPaddingSideTests`, ten cases over 133 subtests covering both byte orders at 2, 4 and 8 octets and every truncation point, including the two figures [#604](https://github.com/JarryShaw/PyPCAPKit/issues/604) reports as literals. Eight fail on the unfixed tree; the other two are guard rails -- a full read at every width and order, which may not move, and a read against an empty buffer, which pads to zeros either way and is the [#431](https://github.com/JarryShaw/PyPCAPKit/issues/431) behaviour the option and list loops depend on ([#604](https://github.com/JarryShaw/PyPCAPKit/issues/604)). +- `tests/corekit/test_fields_numbers_width_repair.py`, covering each octet boundary in its own method, since the defect is a pattern and a single case would pass against a fix that special-cased the reported width ([#599](https://github.com/JarryShaw/PyPCAPKit/issues/599)). +- `FieldBaseShortReadPaddingSideTests`, covering both byte orders at 2, 4 and 8 octets and every truncation point, including the two figures [#604](https://github.com/JarryShaw/PyPCAPKit/issues/604) reports, plus guard rails for a full read and for a read against an empty buffer, the [#431](https://github.com/JarryShaw/PyPCAPKit/issues/431) behaviour the option and list loops depend on ([#604](https://github.com/JarryShaw/PyPCAPKit/issues/604)). #### Changed - `Probe`, `CipherSuite` and `IntegritySuite` are `Info` subclasses rather than `typing.NamedTuple`, and no `NamedTuple` remains in the package. They are Mappings now, so `len()` and iteration yield field names rather than values. -- `ModuleDescriptor.klass` reads an already-imported module out of `sys.modules` rather than re-entering `importlib.import_module`, since next layer dispatch resolves a descriptor there on a per-frame path. A registry *hit* holding a `ModuleDescriptor` is resolved once and written back; a *miss* deliberately is not -- recording a miss in a class-level `collections.defaultdict` is the defect [#421](https://github.com/JarryShaw/PyPCAPKit/issues/421)/[#426](https://github.com/JarryShaw/PyPCAPKit/pull/426) fixed at this layer, [#425](https://github.com/JarryShaw/PyPCAPKit/issues/425)/[#428](https://github.com/JarryShaw/PyPCAPKit/pull/428) for the option, chunk and block registries, and [#555](https://github.com/JarryShaw/PyPCAPKit/issues/555)/[#560](https://github.com/JarryShaw/PyPCAPKit/pull/560) at the schema layer. So every unrecognised frame resolved the fallback descriptor again (48 of the 52 resolutions in an extraction of `many_interfaces.pcapng`); a resolution now costs ~117 ns rather than ~436 ns. **This is not measurable in** `extract()` **wall clock**: 48 avoided calls is ~17 us against a ~37 ms extraction, inside run-to-run variance, and the "~40% of cumulative time" reading that prompted the work was an artifact of `_import_next_layer` being a recursive dispatcher. The change is taken for its shape: `sys.modules` stays the only module cache, so its invalidation is the interpreter's, whereas a memo of the class would serve the pre-reload class after an `importlib.reload` forever and fail `isinstance` against the live one. Proposed by `@Ts-Boom` in [#563](https://github.com/JarryShaw/PyPCAPKit/pull/563), whose profiling found the miss path; the implementation differs because that one added a second, never-invalidated cache of resolved classes ([#574](https://github.com/JarryShaw/PyPCAPKit/issues/574)). -- `SeekableReader.truncate()` raises instead of returning a size, and the misspelled `writeable()` is now `writable()`. Both break external callers; nothing inside the package calls either. `io.IOBase` documents one gate over two methods -- "If False, write() and truncate() will raise OSError" -- and this reader's `writable()` answers False, yet `truncate` returned its new size. Every ordinary read-only file object in CPython raises here (`open(path, 'rb').truncate()` gives `io.UnsupportedOperation`), so this is not a pure-Python nicety that the C path skips. The refusal is `UnsupportedOperation('truncate')`, the same one `write` raises; pcapkit's subclasses `io.UnsupportedOperation`, so the house exception *is* the one the contract names. The old resizing is not deleted, only made private as `_truncate_buffer`: it resizes a private lookback window, not the stream (the issue's own counter-argument), and it is the only route to the "position sits before the window" state that `seek` and the four buffered read paths must refuse, for which [#643](https://github.com/JarryShaw/PyPCAPKit/issues/643) and [#644](https://github.com/JarryShaw/PyPCAPKit/issues/644) landed tests. `writeable()` was never an override of anything -- `SeekableReader.writable is io.IOBase.writable` was True -- so `io`, `shutil` and third parties never saw it; both answered False, which hid it, and the value reported is unchanged. The new test asserts the property, that a `SeekableReader` raises the same exception type as a read-only `io.BufferedReader`, capturing that type by running the call so it tracks CPython across 3.10--3.15. ([#645](https://github.com/JarryShaw/PyPCAPKit/issues/645)). -- **a breaking change to** `@final`, now enforced at runtime on `Info` and `Schema`. `info_final` and `schema_final` both end `return final(cls)`, so every finalised class carried `typing.final`'s `__final__` marker and nothing read it. **`@final` alone on an `Info` or `Schema` subclass -- without `@info_final` or `@schema_final` -- now raises `InfoError` or `SchemaError` at first construction**, the class being marked final but never finalised and so carrying no generated `__init__`. `__init_subclass__` cannot catch that, since `final` applies after class creation, so the check sits in the existing one-shot `FinalisedState.NONE` branch and a finalised class pays nothing. **Deriving from a finalised class raises too**, from new `Info.__init_subclass__` and `Schema.__init_subclass__` hooks (before, only `EnumSchema` had one). **`SchemaError` is a `ValueError`**, so code catching `TypeError` around a subclass declaration will not see it. `@info_final @final` and `@final @info_final` both finalise silently; the re-entry check keys on `__finalised__` rather than `__final__`, because only the former records *the decorator* having run and decorators apply bottom-up, so a `__final__` test would read a class marked an instant earlier as already finalised. `@info_final` twice warns and returns the class unchanged. Every check reads its marker from the class's own `__dict__`, since the markers are ordinary inheriting attributes. `pcapkit.utilities.compat` takes `final` from `typing_extensions` below 3.11 rather than 3.8, because `typing.final` only records `__final__` from 3.11 and the guards were otherwise a silent no-op on the 3.10 leg. `EnumSchema.__init_subclass__` calls the base hook first so a refused declaration cannot leave `__enum__` pointing at a discarded class, and `pcapkit.protocols.schema.misc.pcapng`'s `Option.__init_subclass__` is reordered to match. Additive in tree: no `Info` or `Schema` class is subclassed after finalising or carries `__final__` without `FinalisedState.FINAL`, so the guard cannot fire on the library itself ([#778](https://github.com/JarryShaw/PyPCAPKit/issues/778)). +- `ModuleDescriptor.klass` reads an already-imported module out of `sys.modules` rather than re-entering `importlib.import_module`, since next layer dispatch resolves a descriptor there on a per-frame path. A registry *hit* holding a `ModuleDescriptor` is resolved once and written back; a *miss* deliberately is not -- recording a miss in a class-level `collections.defaultdict` is the defect [#421](https://github.com/JarryShaw/PyPCAPKit/issues/421)/[#426](https://github.com/JarryShaw/PyPCAPKit/pull/426) fixed at this layer, [#425](https://github.com/JarryShaw/PyPCAPKit/issues/425)/[#428](https://github.com/JarryShaw/PyPCAPKit/pull/428) for the option, chunk and block registries, and [#555](https://github.com/JarryShaw/PyPCAPKit/issues/555)/[#560](https://github.com/JarryShaw/PyPCAPKit/pull/560) at the schema layer. A resolution now costs ~117 ns rather than ~436 ns, which is **not measurable in** `extract()` **wall clock**. The change is taken for its shape: `sys.modules` stays the only module cache, whereas a memo of the class would serve the pre-reload class after an `importlib.reload` forever and fail `isinstance` against the live one. Proposed by `@Ts-Boom` in [#563](https://github.com/JarryShaw/PyPCAPKit/pull/563); the implementation differs because that one added a second, never-invalidated cache of resolved classes ([#574](https://github.com/JarryShaw/PyPCAPKit/issues/574)). +- `SeekableReader.truncate()` raises instead of returning a size, and the misspelled `writeable()` is now `writable()`. Both break external callers; nothing inside the package calls either. `io.IOBase` documents one gate over two methods -- "If False, write() and truncate() will raise OSError" -- and this reader's `writable()` answers False, yet `truncate` returned its new size, where `open(path, 'rb').truncate()` raises. The refusal is `UnsupportedOperation('truncate')`, the same one `write` raises; pcapkit's subclasses `io.UnsupportedOperation`. The old resizing is kept as the private `_truncate_buffer`: it resizes a private lookback window, not the stream, and it is the only route to the "position sits before the window" state that `seek` and the four buffered read paths must refuse, for which [#643](https://github.com/JarryShaw/PyPCAPKit/issues/643) and [#644](https://github.com/JarryShaw/PyPCAPKit/issues/644) landed tests. `writeable()` was never an override of anything, so `io`, `shutil` and third parties never saw it, and the value reported is unchanged ([#645](https://github.com/JarryShaw/PyPCAPKit/issues/645)). +- **a breaking change to** `@final`, now enforced at runtime on `Info` and `Schema`. `info_final` and `schema_final` both end `return final(cls)`, so every finalised class carried `typing.final`'s `__final__` marker and nothing read it. **`@final` alone on an `Info` or `Schema` subclass -- without `@info_final` or `@schema_final` -- now raises `InfoError` or `SchemaError` at first construction**, the class being marked final but never finalised and so carrying no generated `__init__`. `__init_subclass__` cannot catch that, since `final` applies after class creation, so the check sits in the existing one-shot `FinalisedState.NONE` branch and a finalised class pays nothing. **Deriving from a finalised class raises too**, from new `Info.__init_subclass__` and `Schema.__init_subclass__` hooks (before, only `EnumSchema` had one). **`SchemaError` is a `ValueError`**, so code catching `TypeError` around a subclass declaration will not see it. `@info_final @final` and `@final @info_final` both finalise silently; the re-entry check keys on `__finalised__` rather than `__final__`, because only the former records *the decorator* having run. `@info_final` twice warns and returns the class unchanged. `pcapkit.utilities.compat` takes `final` from `typing_extensions` below 3.11 rather than 3.8, because `typing.final` only records `__final__` from 3.11. The guard cannot fire on the library's own classes ([#778](https://github.com/JarryShaw/PyPCAPKit/issues/778)). #### Fixed - stdlib exceptions leaking out where the library's own were promised: a malformed IP field value raised a bare `ValueError` instead of `FieldValueError` ([#465](https://github.com/JarryShaw/PyPCAPKit/issues/465)); a `bool` address was silently packed as `0.0.0.1` or `0.0.0.0`, `bool` being an `int` subclass ([#491](https://github.com/JarryShaw/PyPCAPKit/issues/491), [#500](https://github.com/JarryShaw/PyPCAPKit/pull/500)); and `@prepare` discarded extra arguments silently and treated a *declared* zero length as end of stream, which is now distinguished from a genuinely exhausted one and raises `StreamEOFError` ([#454](https://github.com/JarryShaw/PyPCAPKit/issues/454), [#458](https://github.com/JarryShaw/PyPCAPKit/issues/458)). - `FieldBase.unpack` zero-padded up to a field's declared `length` with `rjust()` however little data `buffer` held; a ~40-octet PCAP-NG Decryption Secrets Block with a bogus inner length forced a multi-gigabyte allocation, since `length` is often wire-derived. A declared length past 262144 octets -- libpcap's `MAXIMUM_SNAPLEN` and this package's default `snaplen` -- that the buffer cannot back now raises `FieldValueError`; the option and list loops' tolerance for a short read past a truncated area ([#431](https://github.com/JarryShaw/PyPCAPKit/issues/431)) is far under that ceiling and untouched ([#554](https://github.com/JarryShaw/PyPCAPKit/issues/554)). -- that ceiling bounds one field, and a packet holds many, so the *sum* of a parse's zero padding was still unbounded: 200 minimal PCAP-NG Decryption Secrets Blocks (4,800 wire octets, each declaring `secrets_length` 262,142 against two supplied octets) retained 50.0 MiB, 10,922x per block, with the per-field guard never firing. `FieldBase.unpack` now keeps a per-context ledger of octets supplied against octets synthesised, and raises `FieldValueError` once the total of shortfalls *past 65,536 octets* passes `262144 + 16 * supplied`. The same 200 blocks retain 0.2 MiB (54.6x). A shortfall of 65,536 octets or fewer is padded unconditionally and charged to nothing, and that band is load-bearing. 65,536 is the span of a 16-bit wire length (IP header, IPv6 payload, TCP/IPv4 option, PCAP-NG option), so no shortfall from a snapshot-truncated capture is subject to the budget. Without the band, a running budget alone made the *same* legitimate 54-octet frame declaring an IPv4 total length of 65,535 parse one way on 37 of 40 identical calls and another on the rest, depending on what had been parsed before; a guard whose answer moves with history is worse than the amplification it bounds. **What this does not close**: a length declared by a 16-bit field can still be repeated without limit. A crafted 80,048-octet PCAP-NG file of 2,000 Enhanced Packet Blocks, each with one option declaring 65,535 octets against four real ones, retains 125.00 MiB of zeros (1,637x its size), before and after. That is not an oversight: a bare 40-octet IPv4 header declaring total length 65,535, which is what a legitimate capture of offload-sized segments truncated to its snapshot length looks like, amplifies by **the same 1,637x**, and no budget at this layer separates them. Separating them needs the frame's own `incl_len`/`orig_len`, knowable at the protocol layer and not here. Nor is the budget per file: nothing resets the ledger, so it bounds everything a context has parsed, which is still proportionate to the octets that context was given; tightening it is a one-line change at whichever layer owns a run. Padding tallies and each parse's outcome are identical to before across every capture in `examples/captures/` truncated at some 8,700 offsets ([#573](https://github.com/JarryShaw/PyPCAPKit/issues/573)). -- which exception a malformed TCP SACK option raised depended on unrelated process state: a clean interpreter raised `ProtocolError` as documented, but a process that had popped `pcapkit.corekit.fields.misc` from `sys.modules` (as the `#439` ABC-cache regression tests do in every `setUp`/`tearDown`) raised `FieldValueError` from a different layer, before the documented check ([#525](https://github.com/JarryShaw/PyPCAPKit/issues/525)). `ListField.unpack` resolved `SchemaField` through a function-local import re-run on every call; a module reimported mid-process is a second, distinct class, so `isinstance` misclassified the field and billed each item by its declared length instead of what it consumed. **Any caller relying on the previously-observed** `FieldValueError` **now gets** `ProtocolError`, deterministically, matching the docstring. Fixed by importing at module level. +- that ceiling bounds one field, and a packet holds many, so the *sum* of a parse's zero padding was still unbounded: 200 minimal PCAP-NG Decryption Secrets Blocks (4,800 wire octets, each declaring `secrets_length` 262,142 against two supplied octets) retained 50.0 MiB, 10,922x per block, with the per-field guard never firing. `FieldBase.unpack` now keeps a per-context ledger of octets supplied against octets synthesised, and raises `FieldValueError` once the total of shortfalls *past 65,536 octets* passes `262144 + 16 * supplied`. The same 200 blocks retain 0.2 MiB (54.6x). A shortfall of 65,536 octets or fewer, the span of a 16-bit wire length, is padded unconditionally and charged to nothing, so no snapshot-truncated capture is subject to the budget; without that band the same legitimate frame parsed differently depending on what had been parsed before. **What this does not close**: a length declared by a 16-bit field can still be repeated without limit, and at this layer a crafted file is indistinguishable from a legitimate snapshot-truncated capture of offload-sized segments; separating them needs the frame's own `incl_len`/`orig_len`. Nothing resets the ledger, so it bounds everything a context has parsed rather than one file ([#573](https://github.com/JarryShaw/PyPCAPKit/issues/573)). +- which exception a malformed TCP SACK option raised depended on unrelated process state: a clean interpreter raised `ProtocolError` as documented, but a process that had popped `pcapkit.corekit.fields.misc` from `sys.modules` (as the `#439` ABC-cache regression tests do in every `setUp`/`tearDown`) raised `FieldValueError` from a different layer, before the documented check ([#525](https://github.com/JarryShaw/PyPCAPKit/issues/525)). `ListField.unpack` resolved `SchemaField` through a function-local import re-run on every call; a module reimported mid-process is a second, distinct class, so `isinstance` misclassified the field and billed each item by its declared length instead of what it consumed. **Any caller relying on the previously-observed** `FieldValueError` **now gets** `ProtocolError`, deterministically, matching the docstring. - a `NumberField` whose `length` was a callable could not pack or parse at any width `struct` has a native integer code for. `length` is `-1` until the callable resolves, `-1` has no native code, and the template builder raised `_need_process` for it and never cleared it -- a latch. Resolving the real width rebuilt the template but left the latch set, so `pre_process` handed `bytes` to a template that had become `>Q` (`struct.error: required argument is not an integer`), and parsing failed in mirror by calling `int.from_bytes` on an integer `struct.unpack` had produced. The flag is now recomputed from the width in force, which also keeps a callable resolving to a width with no native code (3 octets) byte-packed. **All four native widths were affected, not only the 8 reported**, on `NumberField` and `EnumField` alike. This made every extended 8-octet MPTCP DSS form unbuildable, those widths being chosen at runtime from the DSS flags; [#585](https://github.com/JarryShaw/PyPCAPKit/pull/585) worked around it in the TCP schema alone ([#591](https://github.com/JarryShaw/PyPCAPKit/issues/591)). -- `NumberField.pre_process` sized a value with floor division dressed as a ceiling. When a field is packed while `length` is still the `-1` placeholder, the width came from `math.ceil(value.bit_length() // 8)`; the `//` had already floored, so `math.ceil` did nothing and the width was one octet short: `256` at one octet, `65536` at two, `16777216` at three. **The reach is wider than "just past a boundary"**: every bit length that is not a multiple of eight is wrong, so `1` (bit length 1, floored to *zero* octets) and every value from 1 to 127 failed too. It is now the ceiling it was meant to be, matching the `math.ceil(n / 8)` idiom used elsewhere. The repair is reached only by packing a field the caller never resolved (a schema resolves every field first), which is why it survived [#591](https://github.com/JarryShaw/PyPCAPKit/issues/591)'s suite: its five repair-path values (`0xFF`, `0xFFFF`, `0xFFFFFFFF`, `0xFFFFFFFFFFFFFFFF`, `0x800001`) have bit lengths of 8, 16, 32, 64 and 24, where floor and ceiling agree. [#591](https://github.com/JarryShaw/PyPCAPKit/issues/591)'s fix neither caused nor masked this, but it changed the failure: a mis-sized 1, 2 or 4 octets now surfaces from `struct.pack` as `'B' format requires 0 <= number <= 255` rather than `OverflowError` from `int.to_bytes`. Deliberately left alone: a signed field is sized without room for its sign bit, so an unresolved signed field still cannot pack `128`; and an unresolved field's bit mask is `-1`, making the masking and sign remap no-ops. The same `math.ceil(x.bit_length() // 8)` survives at the two ILNP nonce option builders in `hopopt.py` and `ipv6_opts.py`, a separate change ([#599](https://github.com/JarryShaw/PyPCAPKit/issues/599)). -- `FieldBase.unpack` padded a short read on the wrong side. When a buffer falls short of the declared length -- the accommodation that lets a snapshot-truncated capture parse ([#431](https://github.com/JarryShaw/PyPCAPKit/issues/431)) -- the shortfall was zero-filled with `rjust()`, placing the zeros at the *front*, as if the octets never read were the leading ones; a short read has lost the trailing ones. It is now `ljust()`, correct for **both** byte orders. The two directions failed differently and only one was loud: one octet of a four-octet little-endian `120` read as `2013265920` (inflated by `2 ** 24`), while three octets of a four-octet big-endian `0x01020304` read as `0x10203` (scaled *down* by 256). The second is the dangerous one, since a value smaller than the truth passes a sanity check where an inflated one overruns, which is why the big-endian half went unnoticed. A full read is untouched ([#604](https://github.com/JarryShaw/PyPCAPKit/issues/604)). -- `SeekableReader.truncate` put its padding where the reader's bookkeeping says the content is, and `read` then returned one octet fewer than the buffer held. Same family as [#604](https://github.com/JarryShaw/PyPCAPKit/issues/604), but two defects. The buffer keeps content at `[0:_buffer_cur]` with padding behind it, so a reduction must keep the octets it has and an extension must append at the *tail*; `temp[-size:]` sliced the buffer rather than the content, keeping trailing padding and discarding octets read, and `temp.rjust(size)` prefixed the zeros. Separately, `read` capped at `min(size, self._buffer_cur - 1)`, one short at the start and reaching into padding further in, and made up the shortfall from the stream *past* the skipped octet. The reported symptom needed both: `read(4)`, `truncate(8)`, `seek(0)`, `read(8)` over `b'abcde'` returned `b'\x00\x00\x00e'` and now returns `b'abcde'`. Three more contract faults are fixed with it: the position was reset to the buffer start rather than left alone; an omitted `size` resized to `0` rather than the current position; and `_buffer_cur` was left addressing octets a reduced buffer no longer had, so the next read raised `ValueError: memoryview assignment: lvalue and rvalue have different structures` from `_write_buffer`. `truncate(0)` raised the same by a second route (`buf[-self._buffer_size:]` is `buf[-0:]`, the whole buffer), so `_write_buffer` now counts from the front; that state is reachable only through `truncate`. A reduction also advances `_buffer_set` past the dropped octets, keeping `_buffer_set + _buffer_cur` equal to how far the stream has been consumed, which `seek` reads as its licence to fetch more; clamping `_buffer_cur` alone made that sum under-report, and the next forward `seek` silently spliced in octets from the wrong offset (`read(8)`, `truncate(3)`, `seek(6)`, `read(1)` over `b'abcdefghijklmnop'` returned `b'l'` where `b'g'` sits at offset 6). All three came from fuzzing random operation sequences against the buffer's invariants: of 3000 rounds, 2372 left the bookkeeping inconsistent and 683 raised an undocumented exception, and none do now. Latent rather than live -- nothing here calls `truncate`, though it is public -- and invisible to existing tests, which asserted the return value and never the content ([#622](https://github.com/JarryShaw/PyPCAPKit/issues/622)). -- the 16-bit band the [#573](https://github.com/JarryShaw/PyPCAPKit/issues/573) entry above records as deliberately left open: a length declared by a 16-bit wire field could still be repeated without limit. The crafted 80,048-octet PCAP-NG of 2,000 Enhanced Packet Blocks that entry describes synthesised 131,070,000 octets of zeros (1637.393x its size) and now synthesises none, with all 2,000 frames and options still parsed. The bound is taken one layer up, where the missing information exists: a PCAP-NG block's Block Total Length is authoritative and cross-checked against its trailing copy, so the option area is that length less the fixed fields, `captured_len` and its padding, and an option declaring more payload than the area has left is malformed. A snapshot-truncated capture says so through `captured_len` and leaves its options whole, so it never trips this -- which the [#571](https://github.com/JarryShaw/PyPCAPKit/pull/571) `len(buffer) < length` rejection could not distinguish, and why it was declined. A new `bounded_option` clamps the payload to the octets the area has left and warns with `SchemaWarning`; it is applied to all fifteen variable-width option and record payloads in the PCAP-NG schema, and deliberately not to the block-level payload fields, the 32-bit band the running ledger already budgets. A second, `bounded_area`, clamps a packet block's option area to the octets the block itself holds less the trailing Block Total Length: the area was otherwise sized from a declared length nothing checks against the file, so a block declaring 1,000,000 octets while holding 36 had a 999,964-octet area and 65,535 zeros were synthesised from 36 (1,820x) with no warning. The cross-review found that one. Both are no-ops on a well-formed block. The five non-packet blocks' option areas keep the framing assumption; the general fix for a declared length reaching a read is [#678](https://github.com/JarryShaw/PyPCAPKit/issues/678). Clamping rather than refusing keeps the [#431](https://github.com/JarryShaw/PyPCAPKit/issues/431) accommodation, since a PCAP-NG block read has no catch point above `FieldBase.unpack` and one refusal would abort a whole extraction rather than one block; and the clamp reads only the block's own declared framing, never a running total, so identical input answers identically whatever preceded it. The skip for a remainder already past zero is load-bearing on the unpacking path (a ten-octet area leaves `__length__` at -2 before a payload sizes itself, and clamping would give the field a `'-2s'` template); it is *not* what protects packing, since `BytesField` and `StringField` repair a negative width in `pre_process`. No clamp fires across the six PCAP-NG sample captures (338 options), and 501 truncation levels of `dhcp.pcapng` give byte-identical outcomes, most being the uncaught `ValueError` that [#678](https://github.com/JarryShaw/PyPCAPKit/issues/678) records. The bound is pinned as a property, not one input: across every combination of one to three options and nine declared lengths the options never report more octets than the area, and the ratio is flat across 1, 8, 64 and 512 blocks, which separates a bound from a smaller constant. The worst ratio over five adversarial shapes is now 0.862x, against 1,637x, 960x and 224x for the same three before ([#594](https://github.com/JarryShaw/PyPCAPKit/issues/594)). +- `NumberField.pre_process` sized a value with floor division dressed as a ceiling. When a field is packed while `length` is still the `-1` placeholder, the width came from `math.ceil(value.bit_length() // 8)`; the `//` had already floored, so `math.ceil` did nothing and the width was one octet short: `256` at one octet, `65536` at two, `16777216` at three. **The reach is wider than "just past a boundary"**: every bit length that is not a multiple of eight is wrong, so `1` (bit length 1, floored to *zero* octets) and every value from 1 to 127 failed too. It is now the ceiling it was meant to be. The repair is reached only by packing a field the caller never resolved (a schema resolves every field first), which is why [#591](https://github.com/JarryShaw/PyPCAPKit/issues/591)'s suite, whose values all had byte-aligned bit lengths, missed it; [#591](https://github.com/JarryShaw/PyPCAPKit/issues/591)'s fix neither caused nor masked it. Deliberately left alone: a signed field is sized without room for its sign bit, so an unresolved signed field still cannot pack `128`. The same typo at the two ILNP nonce option builders is a separate change ([#599](https://github.com/JarryShaw/PyPCAPKit/issues/599)). +- `FieldBase.unpack` padded a short read on the wrong side. When a buffer falls short of the declared length -- the accommodation that lets a snapshot-truncated capture parse ([#431](https://github.com/JarryShaw/PyPCAPKit/issues/431)) -- the shortfall was zero-filled with `rjust()`, placing the zeros at the *front*, as if the octets never read were the leading ones; a short read has lost the trailing ones. It is now `ljust()`, correct for **both** byte orders: one octet of a four-octet little-endian `120` read as `2013265920`, and three octets of a four-octet big-endian `0x01020304` as `0x10203`. A full read is untouched ([#604](https://github.com/JarryShaw/PyPCAPKit/issues/604)). +- `SeekableReader.truncate` put its padding where the reader's bookkeeping says the content is, and `read` then returned one octet fewer than the buffer held. Same family as [#604](https://github.com/JarryShaw/PyPCAPKit/issues/604), but two defects. The buffer keeps content at `[0:_buffer_cur]` with padding behind it, so a reduction must keep the octets it has and an extension must append at the *tail*; `temp[-size:]` sliced the buffer rather than the content, keeping trailing padding and discarding octets read, and `temp.rjust(size)` prefixed the zeros. Separately, `read` capped at `min(size, self._buffer_cur - 1)`, one short at the start and reaching into padding further in, and made up the shortfall from the stream *past* the skipped octet. The reported symptom needed both: `read(4)`, `truncate(8)`, `seek(0)`, `read(8)` over `b'abcde'` returned `b'\x00\x00\x00e'` and now returns `b'abcde'`. Three more contract faults are fixed with it: the position was reset to the buffer start rather than left alone; an omitted `size` resized to `0` rather than the current position; and `_buffer_cur` was left addressing octets a reduced buffer no longer had, so the next read raised `ValueError: memoryview assignment: lvalue and rvalue have different structures` from `_write_buffer`. `truncate(0)` raised the same by a second route (`buf[-self._buffer_size:]` is `buf[-0:]`, the whole buffer), so `_write_buffer` now counts from the front; that state is reachable only through `truncate`. A reduction also advances `_buffer_set` past the dropped octets, keeping `_buffer_set + _buffer_cur` equal to how far the stream has been consumed, which `seek` reads as its licence to fetch more; clamping `_buffer_cur` alone made that sum under-report, and the next forward `seek` silently spliced in octets from the wrong offset (`read(8)`, `truncate(3)`, `seek(6)`, `read(1)` over `b'abcdefghijklmnop'` returned `b'l'` where `b'g'` sits at offset 6). Latent rather than live: nothing here calls `truncate`, though it is public ([#622](https://github.com/JarryShaw/PyPCAPKit/issues/622)). +- the 16-bit band the [#573](https://github.com/JarryShaw/PyPCAPKit/issues/573) entry above records as deliberately left open: a length declared by a 16-bit wire field could still be repeated without limit. The crafted PCAP-NG of 2,000 Enhanced Packet Blocks, each with an option declaring 65,535 octets against four real ones, synthesised 1,637x its size in zeros and now synthesises none, with every frame and option still parsed. The bound is taken one layer up, where the missing information exists: a PCAP-NG block's Block Total Length is authoritative and cross-checked against its trailing copy, so the option area is that length less the fixed fields, `captured_len` and its padding, and an option declaring more payload than the area has left is malformed. A snapshot-truncated capture says so through `captured_len` and leaves its options whole, so it never trips this -- which the [#571](https://github.com/JarryShaw/PyPCAPKit/pull/571) `len(buffer) < length` rejection could not distinguish, and why it was declined. A new `bounded_option` clamps the payload to the octets the area has left and warns with `SchemaWarning`; it is applied to all fifteen variable-width option and record payloads in the PCAP-NG schema, and deliberately not to the block-level payload fields, the 32-bit band the running ledger already budgets. A second, `bounded_area`, clamps a packet block's option area to the octets the block itself holds less the trailing Block Total Length: the area was otherwise sized from a declared length nothing checks against the file, so a block declaring 1,000,000 octets while holding 36 synthesised 65,535 zeros with no warning. Both are no-ops on a well-formed block. The five non-packet blocks' option areas keep the framing assumption; the general fix for a declared length reaching a read is [#678](https://github.com/JarryShaw/PyPCAPKit/issues/678). Clamping rather than refusing keeps the [#431](https://github.com/JarryShaw/PyPCAPKit/issues/431) accommodation, since a PCAP-NG block read has no catch point above `FieldBase.unpack` and one refusal would abort a whole extraction rather than one block; and the clamp reads only the block's own declared framing, never a running total, so identical input answers identically whatever preceded it. The skip for a remainder already past zero is load-bearing on the unpacking path, where clamping would give the field a negative template such as `'-2s'`. No clamp fires on the sample captures, and [#678](https://github.com/JarryShaw/PyPCAPKit/issues/678)'s uncaught `ValueError` on truncated input is unchanged ([#594](https://github.com/JarryShaw/PyPCAPKit/issues/594)). - **a breaking change to** `FieldBase.length`: it called `struct.calcsize` on a template built from a negative resolved length (e.g. `'-5s'`, from a `length=` callback whose running counter had been overdrawn), raising a bare `struct.error` that ordinary caller code does not catch. The chokepoint every affected schema shares now raises `ProtocolError`, covering `application/{ftp,httpv1,httpv2,ngap}.py`, `internet/{hip,ipv6_route,mh}.py`, `link/ethernet.py`, `misc/pcapng.py` and `transport/sctp.py` with no other file changed; a non-negative length is unaffected. This closes the follow-on the `httpv2` length-guard entry above filed as out of scope ([#805](https://github.com/JarryShaw/PyPCAPKit/issues/805)). -- **a breaking change to** `EnumRegistry.register()`: it now raises `ValueError` naming the existing member and pointing at `register_alias()`, where it used to silently alias an already-registered value under the caller's new name -- `aenum` treats a taken value as an alias request, so `register(existing_value, 'TOTALLY_NEW_NAME')` returned the existing member, minted nothing and raised nothing, contradicting its docstring. `pcapkit.corekit.enums.EnumRegistry` is a new mixin implementing `get`, `get_all`, `register`, `register_alias`, `register_aliases` and `_unregistered_member` once, per the maintainer's ruling on [#842](https://github.com/JarryShaw/PyPCAPKit/issues/842); it converts the first six registries with no bespoke `__new__` (`ipv6.extension_header.ExtensionHeader`, `tcp.flags.Flags` and four `mh.*_flag` modules), and serves `IntEnum`, `IntFlag` and `StrEnum` alike. The guard reaches only those six: the other 105 const registries still carry the old generator's bare `extend_enum(cls, name, value)`, so `pcapkit.const` ships two contradictory `register` contracts until [#858](https://github.com/JarryShaw/PyPCAPKit/pull/858) (below). The six also gain the generated template's `KeyError` catching on a missing name, so `Flags.get('NOPE', 0)` returns `0` rather than raising (a widening, undisclosed until this PR); and a key that is neither `int` nor `str`, which raised `TypeError` on the five `IntFlag` registries' hand-copied `get` (`Flags.get(3.5)`, since `cls[key]` is a containment test on a `Flag`), now raises `ValueError`, tried as a value. `ExtensionHeader`, the only `IntEnum` among them, already raised `KeyError` there, the shift [#858](https://github.com/JarryShaw/PyPCAPKit/pull/858)'s entry attributes to the other 105 ([#855](https://github.com/JarryShaw/PyPCAPKit/pull/855)). -- **a breaking change to** `EnumRegistry.register()`, extended: the 105 const registries still on the generated template's bare `extend_enum` now inherit the guard [#855](https://github.com/JarryShaw/PyPCAPKit/pull/855) (above) added, so `TransType.register(6, 'TOTALLY_NEW_NAME')` raises `ValueError` naming the value and the member that holds it and pointing at `register_alias()`, where it used to mint and raise nothing. `pcapkit/vendor/default.py`'s template now emits `class {NAME}(EnumRegistry, IntEnum)` and drops its own `get`/`register`/`_unregistered_member`. Across 121 const modules holding 127 enum classes: 6 already used `EnumRegistry` ([#855](https://github.com/JarryShaw/PyPCAPKit/pull/855)); of the other 115 modules, 105 share the template byte-for-byte and are converted, and the remaining 10 keep their bespoke `__new__` and are left alone: `ftp/command` (4 classes), `ftp/return_code` (3), `http/method`, `http/status_code`, `pcapng/option_type`, `reg/apptype/apptype.py` (`AppType` and `TransportProtocol`) and its four transport subclasses. That makes **111 of the 127 classes** inherit `EnumRegistry`, up from 6. `get()`'s dispatch shifts only for a key that is neither `int` nor `str` (`KeyError`, tried as a name; now `ValueError`, tried as a value), a key type nothing in the library passes. `pcapkit/corekit/enums.py` is renamed **`enum.py`**, the maintainer's ruling ([#858](https://github.com/JarryShaw/PyPCAPKit/pull/858)). -- `EnumRegistry.get()`'s `str`-key branch tried only a name lookup (`cls._member_map_[key]`) and on a miss raised or returned `cls(default)`, never the *value* path, so on a `StrEnum` registry a key that resolved through the constructor (`_Str('known-value')`) raised `KeyError` through `get()`, and a supplied `default` overrode a good value match the code never attempted. It now falls back to a plain `_value2member_map_` lookup once the name lookup misses and before consulting `default`. Deliberately not `cls(key)`: `FEATCode._missing_` (`pcapkit/const/ftp/command.py`) mints via `extend_enum` for any unrecognised string, so routing through the constructor would let a failed name lookup mint a permanent member once a registry like it converts; a dict lookup never reaches `_missing_`. A name still wins when a string is both a name and a different member's value. Step 1 of [#860](https://github.com/JarryShaw/PyPCAPKit/issues/860); step 2, converting the bespoke registries, follows ([#869](https://github.com/JarryShaw/PyPCAPKit/pull/869), [#874](https://github.com/JarryShaw/PyPCAPKit/pull/874), below). The `IntEnum`/`IntFlag` non-`str` path is unchanged, measured on `TransType` and `HandoverACKFlag`. Widens rather than breaks: a call that raised `KeyError` now resolves, and no caller that got a correct result gets a different one ([#863](https://github.com/JarryShaw/PyPCAPKit/pull/863)). -- **a breaking change to** `EnumRegistry.get()`: its docstring claimed "It never mints", but both `cls(default)` call sites reached `_missing_` for a `default` inside a still-minting registry's range, growing the registry as a side effect of resolving `default` rather than `key`. Per the owner's ruling (*"Only register can mint. get should not mint unless it falls through the _missing_'s minted ranges"*), both sites now resolve `default` through a plain `_value2member_map_` lookup; `key` resolution is unchanged, so `EtherType.get(0x0888)` still mints `Xyplex_0x0888` through its own `_missing_`, as [#775](https://github.com/JarryShaw/PyPCAPKit/issues/775)'s deliberate exception permits. A `default` naming no registered member now falls through to the error `key` itself would have raised, rather than a fresh error about the default -- the accepted cost of the ruling. The docstring's "It never mints" is qualified to `default` only. New `GetDefaultNoMintTests` pin the issue's repro on the real `EtherType` registry, the preserved key-path mint, and default resolution on an `int` and a `str` registry. Four assertions in `tests/const/test_const_enum_get.py` and `tests/const/test_const_enum_builtin_parity.py` that encoded the old contract are retargeted rather than weakened; a fifth, `Hardware.get('Definitely-Not-A-Member', 40)` (`40` sits in `Hardware`'s declared-but-unassigned range and now raises `KeyError` naming the original key), waited on [#865](https://github.com/JarryShaw/PyPCAPKit/pull/865) because its test file was contended, then landed once [#865](https://github.com/JarryShaw/PyPCAPKit/pull/865) (above) merged, with `Hardware`'s member count unchanged at 42. Closes [#864](https://github.com/JarryShaw/PyPCAPKit/issues/864) ([#868](https://github.com/JarryShaw/PyPCAPKit/pull/868)). +- **a breaking change to** `EnumRegistry.register()`: it now raises `ValueError` naming the existing member and pointing at `register_alias()`, where it used to silently alias an already-registered value under the caller's new name -- `aenum` treats a taken value as an alias request, so `register(existing_value, 'TOTALLY_NEW_NAME')` returned the existing member, minted nothing and raised nothing, contradicting its docstring. `pcapkit.corekit.enums.EnumRegistry` is a new mixin implementing `get`, `get_all`, `register`, `register_alias`, `register_aliases` and `_unregistered_member` once, per the maintainer's ruling on [#842](https://github.com/JarryShaw/PyPCAPKit/issues/842); it converts the first six registries with no bespoke `__new__` (`ipv6.extension_header.ExtensionHeader`, `tcp.flags.Flags` and four `mh.*_flag` modules), and serves `IntEnum`, `IntFlag` and `StrEnum` alike. The guard reaches only those six: the other 105 const registries still carry the old generator's bare `extend_enum(cls, name, value)`, so `pcapkit.const` ships two contradictory `register` contracts until [#858](https://github.com/JarryShaw/PyPCAPKit/pull/858) (below). The six also gain the generated template's `KeyError` catching on a missing name, so `Flags.get('NOPE', 2048)` returns `Flags.ACK` rather than raising; and a key that is neither `int` nor `str`, which raised `TypeError` on the five `IntFlag` registries (`Flags.get(3.5)`), now raises `ValueError`, tried as a value. On `ExtensionHeader` it raised `KeyError`, the shift [#858](https://github.com/JarryShaw/PyPCAPKit/pull/858)'s entry attributes to the other 105 ([#855](https://github.com/JarryShaw/PyPCAPKit/pull/855)). +- **a breaking change to** `EnumRegistry.register()`, extended: the 105 const registries still on the generated template's bare `extend_enum` now inherit the guard [#855](https://github.com/JarryShaw/PyPCAPKit/pull/855) (above) added, so `TransType.register(6, 'TOTALLY_NEW_NAME')` raises `ValueError` naming the value and the member that holds it and pointing at `register_alias()`, where it used to mint and raise nothing. `pcapkit/vendor/default.py`'s template now emits `class {NAME}(EnumRegistry, IntEnum)` and drops its own `get`/`register`/`_unregistered_member`. The 10 modules with a bespoke `__new__` (`ftp/command`, `ftp/return_code`, `http/method`, `http/status_code`, `pcapng/option_type` and `reg/apptype`) are left alone, so **111 of the 127 classes** inherit `EnumRegistry`, up from 6 ([#855](https://github.com/JarryShaw/PyPCAPKit/pull/855)). `get()`'s dispatch shifts only for a key that is neither `int` nor `str` (`KeyError`, tried as a name; now `ValueError`, tried as a value), a key type nothing in the library passes. `pcapkit/corekit/enums.py` is renamed **`enum.py`**, the maintainer's ruling ([#858](https://github.com/JarryShaw/PyPCAPKit/pull/858)). +- `EnumRegistry.get()`'s `str`-key branch tried only a name lookup (`cls._member_map_[key]`) and on a miss raised or returned `cls(default)`, never the *value* path, so on a `StrEnum` registry a key that resolved through the constructor (`_Str('known-value')`) raised `KeyError` through `get()`, and a supplied `default` overrode a good value match the code never attempted. It now falls back to a plain `_value2member_map_` lookup once the name lookup misses and before consulting `default`. Deliberately not `cls(key)`: `FEATCode._missing_` (`pcapkit/const/ftp/command.py`) mints via `extend_enum` for any unrecognised string, so routing through the constructor would let a failed name lookup mint a permanent member once a registry like it converts; a dict lookup never reaches `_missing_`. A name still wins when a string is both a name and a different member's value. Step 1 of [#860](https://github.com/JarryShaw/PyPCAPKit/issues/860); step 2, converting the bespoke registries, follows ([#869](https://github.com/JarryShaw/PyPCAPKit/pull/869), [#874](https://github.com/JarryShaw/PyPCAPKit/pull/874), below). The non-`str` path is unchanged. Widens rather than breaks: a call that raised `KeyError` now resolves, and no caller that got a correct result gets a different one ([#863](https://github.com/JarryShaw/PyPCAPKit/pull/863)). +- **a breaking change to** `EnumRegistry.get()`: its docstring claimed "It never mints", but both `cls(default)` call sites reached `_missing_` for a `default` inside a still-minting registry's range, growing the registry as a side effect of resolving `default` rather than `key`. Per the owner's ruling (*"Only register can mint. get should not mint unless it falls through the _missing_'s minted ranges"*), both sites now resolve `default` through a plain `_value2member_map_` lookup; `key` resolution is unchanged, so `EtherType.get(0x0888)` still mints `Xyplex_0x0888` through its own `_missing_`, as [#775](https://github.com/JarryShaw/PyPCAPKit/issues/775)'s deliberate exception permits. A `default` naming no registered member now falls through to the error `key` itself would have raised, rather than a fresh error about the default -- the accepted cost of the ruling. The docstring's "It never mints" is qualified to `default` only. So `Hardware.get('Definitely-Not-A-Member', 40)`, with `40` in `Hardware`'s declared-but-unassigned range, now raises `KeyError` naming the original key; its test waited on [#865](https://github.com/JarryShaw/PyPCAPKit/pull/865) (above), then landed once [#865](https://github.com/JarryShaw/PyPCAPKit/pull/865) merged. Closes [#864](https://github.com/JarryShaw/PyPCAPKit/issues/864) ([#868](https://github.com/JarryShaw/PyPCAPKit/pull/868)). ### pcapkit.dumpkit #### Fixed -- a flag value with no declared bits no longer dumps as `Type::None [0]`. **This changes user-visible output in every textual dump format.** `make_dumper`'s `object_hook` renders each enumeration member as `Type::name [value]` and interpolated the name unguarded, but a `Flag` value composed entirely of undeclared bits has `name is None`, so the literal `None` landed in the name half: `Flags(0)` rendered as `Flags::None [0]` in `json`, `tree`, `text`, `txt`, `plist` and `xml`, out of both `Extractor` and `TraceFlow`. A nameless member now renders with the value's decimal spelling, so `Flags(0)` gives `Flags::0 [0]` and `Flags(8)` gives `Flags::8 [8]`. That is the spelling the enumeration libraries use for an undeclared residue (`Flags(2057).name` is `ACK|9`), and it cannot be mistaken for a member name, since an identifier may not begin with a digit, whereas `NONE` is a real declared name elsewhere. Three sites carried the identical interpolation (the `OrderedMultiDict` key path, the `addon` branch and the scalar return) and now share one `render_enum` helper. The guard is on `name is None` and not on the value, because the defect was never about zero (`Flags(1)`, `Flags(8)`, `Flags(9)` and `Flags(65536)` are equally nameless) and is not an `aenum` quirk: stdlib `enum.IntFlag` answers `name is None` at the same values. Five of the seven flag registries are nameless at zero (`Flags` and the four Mobility Header flags; the other two declare `undefined = 0`). Pre-existing since 2023-04-28 and surfaced by [#634](https://github.com/JarryShaw/PyPCAPKit/pull/634), which made a flagless TCP segment reach this path. The committed example dumps do not move ([#648](https://github.com/JarryShaw/PyPCAPKit/issues/648)). -- `tests/dumpkit/test_nameless_enum_rendering_unit.py` (added by [#670](https://github.com/JarryShaw/PyPCAPKit/pull/670)) carried two probes assuming every flag-enum registry is 16 bits wide: `test_scalar_return_renders_a_nameless_member_as_its_value` used one fixed tuple ending in `65536`, and `test_no_flag_registry_renders_the_literal_none` swept every registry but only against `registry(0)`. Widths differ: `ftp.command.CommandType` 3 bits, `reg.apptype.TransportProtocol` 4 (computed at runtime), the three `mh` ack/initiate flags 8, and `mh.binding_update_flag.BindingUpdateFlag` and `tcp.flags.Flags` 16. `tcp.flags.Flags(65536)` correctly raises (`0 <= value <= 0xFFFF`), so the sweep was failing on correct behaviour. A new `_field_mask` derives each registry's own bound from its declared members, `_nameless_values` returns the values that bound admits but no member names (zero, each undeclared bit alone, all combined), and `test_a_value_past_the_field_is_refused_rather_than_rendered` pins the boundary: the widest in-field value is accepted, one past it is refused. A cross-review found all seven guard `raise` lines it reaches (`tcp/flags.py`'s own included) were already covered by a test predating the fix, so none was newly reached. Nothing under `pcapkit/` changed ([#702](https://github.com/JarryShaw/PyPCAPKit/issues/702)). -- the plist writer emitted mapping keys unescaped, so a key carrying `&`, `<` or `>` produced a report that is not well-formed XML. `dictdumper`'s `_append_dict` interpolates the key into `'{item}'` and calls `_encode_value` on the value only, so both branches of `pcapkit.dumpkit.common` that build a mapping now escape their keys through one `escape_key` helper -- a `MultiDict`, where [#575](https://github.com/JarryShaw/PyPCAPKit/issues/575) escaped an enum-derived key inline and nothing else, and a plain dict, where nothing was escaped. The json, tree and text writers are untouched (`escape_key` is a no-op for them and a plain dict is not rebuilt). Across 6 captures and 5 formats, exactly 2 of 30 reports changed (`test.pcapng`'s plist and xml, one writer under two names), by one line each; that capture is why a non-`str` key is handled at all, its decryption secrets block keying TLS key log entries by a raw bytes client random whose repr carries all three characters. The upstream half is `JarryShaw/DictDumper#125` (`_append_dict` should call `_encode_value` on a key); this does not wait on it. `PcapngUnescapedKeyTests`' docstrings, which claimed both the json and plist reports were unparseable, are reworded: only the json writer's quoting defect stands, upstream as `JarryShaw/DictDumper#121` ([#772](https://github.com/JarryShaw/PyPCAPKit/issues/772)). +- a flag value with no declared bits no longer dumps as `Type::None [0]`. **This changes user-visible output in every textual dump format.** `make_dumper`'s `object_hook` renders each enumeration member as `Type::name [value]` and interpolated the name unguarded, but a `Flag` value composed entirely of undeclared bits has `name is None`, so the literal `None` landed in the name half: `Flags(0)` rendered as `Flags::None [0]` in `json`, `tree`, `text`, `txt`, `plist` and `xml`, out of both `Extractor` and `TraceFlow`. A nameless member now renders with the value's decimal spelling, so `Flags(0)` gives `Flags::0 [0]` and `Flags(8)` gives `Flags::8 [8]`. That is the spelling the enumeration libraries use for an undeclared residue (`Flags(2057).name` is `ACK|9`), and it cannot be mistaken for a member name, since an identifier may not begin with a digit, whereas `NONE` is a real declared name elsewhere. The guard is on `name is None` and not on the value, because the defect was never about zero (`Flags(1)` and `Flags(9)` are equally nameless). Surfaced by [#634](https://github.com/JarryShaw/PyPCAPKit/pull/634), which made a flagless TCP segment reach this path ([#648](https://github.com/JarryShaw/PyPCAPKit/issues/648)). +- `tests/dumpkit/test_nameless_enum_rendering_unit.py` (added by [#670](https://github.com/JarryShaw/PyPCAPKit/pull/670)) carried two probes assuming every flag-enum registry is 16 bits wide: `test_scalar_return_renders_a_nameless_member_as_its_value` used one fixed tuple ending in `65536`, and `test_no_flag_registry_renders_the_literal_none` swept every registry but only against `registry(0)`. Widths differ: `ftp.command.CommandType` 3 bits, `reg.apptype.TransportProtocol` 4 (computed at runtime), the three `mh` ack/initiate flags 8, and `mh.binding_update_flag.BindingUpdateFlag` and `tcp.flags.Flags` 16. `tcp.flags.Flags(65536)` correctly raises (`0 <= value <= 0xFFFF`), so the sweep was failing on correct behaviour. The probes now derive each registry's bound from its declared members, and a new test pins that one past the widest in-field value is refused. Test-only ([#702](https://github.com/JarryShaw/PyPCAPKit/issues/702)). +- the plist writer emitted mapping keys unescaped, so a key carrying `&`, `<` or `>` produced a report that is not well-formed XML. `dictdumper`'s `_append_dict` interpolates the key into `'{item}'` and calls `_encode_value` on the value only, so both branches of `pcapkit.dumpkit.common` that build a mapping now escape their keys through one `escape_key` helper -- a `MultiDict`, where [#575](https://github.com/JarryShaw/PyPCAPKit/issues/575) escaped an enum-derived key inline and nothing else, and a plain dict, where nothing was escaped. The json, tree and text writers are untouched. A non-`str` key is escaped too: a PCAP-NG decryption secrets block keys TLS key log entries by a raw bytes client random whose repr can carry all three characters. The upstream half is `JarryShaw/DictDumper#125`; this does not wait on it. The json writer's quoting defect stands, upstream as `JarryShaw/DictDumper#121` ([#772](https://github.com/JarryShaw/PyPCAPKit/issues/772)). ### pcapkit.foundation @@ -84,24 +84,24 @@ Preceded by `1.5.0a1` (2026-09-15), `1.5.0b1` and `1.5.0b2` (both 2026-09-18) an - two reassembly and flow-tracing defaults moved ([#435](https://github.com/JarryShaw/PyPCAPKit/pull/435)), both visible to a caller. `Datagram.completed` widened from `bool` to a `Completion` enumeration (`COMPLETE`, `PARTIAL`, `TIMEOUT`); only `COMPLETE` is truthy, so `if datagram.completed:` is unaffected but `datagram.completed == True` no longer holds. TCP flow tracing is **bidirectional by default**, which merges each flow's two halves and closes one only once both have FINed -- 331 flows become 111 on the sample HTTP capture, the difference being single-frame stray tails; pass `trace_bidirectional=False` for the old behaviour. IP reassembly also gained the 60-second timeout [[RFC 1122](https://datatracker.ietf.org/doc/html/rfc1122), [RFC 8200](https://datatracker.ietf.org/doc/html/rfc8200)], clocked off the capture's own timestamps, tunable with `reasm_timeout=`; TCP reassembly gets no timeout by default. `trace_analyse=` is new, and reassembles each traced flow's application layer. - conflicting TCP overlaps resolve first-write-wins, per [RFC 9293](https://datatracker.ietf.org/doc/html/rfc9293) section 3.10, where they had silently resolved last-write-wins ([#443](https://github.com/JarryShaw/PyPCAPKit/issues/443), [#478](https://github.com/JarryShaw/PyPCAPKit/pull/478)). A deliberate, narrow behaviour break: a conforming retransmission carries identical bytes. IP fragment reassembly keeps last-write-wins, because [RFC 791](https://datatracker.ietf.org/doc/html/rfc791) specifies the opposite resolution, and records the disagreement instead ([#482](https://github.com/JarryShaw/PyPCAPKit/pull/482)). - renames with no compatibility alias left behind: `HoleDiscriptor` is spelled `HoleDescriptor` and its package alias `TCP_HoleDiscriptor` is `TCP_HoleDescriptor` ([#350](https://github.com/JarryShaw/PyPCAPKit/pull/350)). -- subclass registration is **opt-in** for `Engine`, `Reassembly`, `TraceFlow` and `dumpkit`'s `Dumper` ([#514](https://github.com/JarryShaw/PyPCAPKit/issues/514)). Each registers if and only if its registry keyword is given -- `engine=` for `Engine`, `protocol=` for `Reassembly` and `TraceFlow`, `fmt=` for `Dumper`. Previously an absent keyword fell back to the class' own name, so *every* subclass of the public class was registered, and declining meant subclassing the parallel `*Base` class under an alias, which every built-in does (hence the public classes had **0** subclasses against the `*Base` classes' 9, 5, 2 and 3). **This breaks out-of-tree code that subclasses one of the four and relies on the derived key**; pass the keyword, or call the matching `register_*` function. Nothing the library ships is affected, and the `*Base` classes remain importable. Two silent failures are now loud: an unrecognised class keyword raises `UnsupportedCall` instead of being swallowed by `**kwargs` (passing `name=` to a `Reassembly` subclass silently ignored the key, since `protocol=` is the real one), and so does `Dumper`'s `ext=` without `fmt=`. A class attribute is not an opt-in: `__engine_name__` and `__protocol_name__` still set the name a class reports, registered or not. Each metaclass gained a class-level `registry` property mirroring `EnumSchema.registry`, and a `Dumper` subclass no longer touches the filesystem while its `class` statement runs (inferring `fmt` from `kind` meant instantiating it against a `NamedTemporaryFile`). `Engine`'s keyword is `engine=` rather than the `name=` first shipped, because `name` cannot be a class keyword on Python 3.10: `mcls`, `name`, `bases` and `namespace` collide with `abc.ABCMeta.__new__`'s parameters (positional-or-keyword before 3.11, positional-only from 3.11), raising `TypeError` before the hook runs. Those four are the whole collision surface (measured on 3.10.21, 3.11.15 and 3.14.7), and `engine=`, `protocol=` and `fmt=` are outside it. There is no `name=` alias: a keyword that works on some interpreters and not others is the trap being removed. -- **a breaking change to** `Extractor.register_engine`, `register_reassembly` and `register_traceflow`, and the `register_extractor_*` wrappers over them: each now accepts only a subclass of the public `Engine`, `Reassembly` or `TraceFlow` ([#1016](https://github.com/JarryShaw/PyPCAPKit/issues/1016)). They had accepted a subclass of the matching `*Base` class since [#513](https://github.com/JarryShaw/PyPCAPKit/issues/513), which was a tolerance for pcapkit's own built-ins (all 13 derive from the base, none from the public class) rather than a contract: `*Base` is documented as internal, and the public class is the one carrying the registration hook. Third-party code that subclasses `EngineBase`, `ReassemblyBase` or `TraceFlowBase` directly and registers through these functions now gets `RegistryError` naming the public class; subclass the public class instead. The error messages are unchanged. The built-ins are unaffected because they are declared directly in `Extractor`'s class-body mappings as `ModuleDescriptor` literals and never pass through a registrar at all; a new internal path, `Extractor._register_internal_engine` and its two siblings, checks the base for programmatic internal registration and is what the [#513](https://github.com/JarryShaw/PyPCAPKit/issues/513) regression test now exercises. `register_protocol` is untouched: every in-house protocol derives from `ProtocolBase` and none from `Protocol`. +- subclass registration is **opt-in** for `Engine`, `Reassembly`, `TraceFlow` and `dumpkit`'s `Dumper` ([#514](https://github.com/JarryShaw/PyPCAPKit/issues/514)). Each registers if and only if its registry keyword is given -- `engine=` for `Engine`, `protocol=` for `Reassembly` and `TraceFlow`, `fmt=` for `Dumper`. Previously an absent keyword fell back to the class' own name, so *every* subclass of the public class was registered, and declining meant subclassing the parallel `*Base` class under an alias, which every built-in does (hence the public classes had **0** subclasses against the `*Base` classes' 9, 5, 2 and 3). **This breaks out-of-tree code that subclasses one of the four and relies on the derived key**; pass the keyword, or call the matching `register_*` function. Nothing the library ships is affected, and the `*Base` classes remain importable. Two silent failures are now loud: an unrecognised class keyword raises `UnsupportedCall` instead of being swallowed by `**kwargs` (passing `name=` to a `Reassembly` subclass silently ignored the key, since `protocol=` is the real one), and so does `Dumper`'s `ext=` without `fmt=`. A class attribute is not an opt-in: `__engine_name__` and `__protocol_name__` still set the name a class reports, registered or not. Each metaclass gained a class-level `registry` property mirroring `EnumSchema.registry`, and a `Dumper` subclass no longer touches the filesystem while its `class` statement runs. `Engine`'s keyword is `engine=` rather than the `name=` first shipped, because `mcls`, `name`, `bases` and `namespace` collide with `abc.ABCMeta.__new__`'s parameters on Python 3.10 and raise `TypeError` before the hook runs. There is no `name=` alias: a keyword that works on some interpreters and not others is the trap being removed. +- **a breaking change to** `Extractor.register_engine`, `register_reassembly` and `register_traceflow`, and the `register_extractor_*` wrappers over them: each now accepts only a subclass of the public `Engine`, `Reassembly` or `TraceFlow` ([#1016](https://github.com/JarryShaw/PyPCAPKit/issues/1016)). They had accepted a subclass of the matching `*Base` class since [#513](https://github.com/JarryShaw/PyPCAPKit/issues/513), which was a tolerance for pcapkit's own built-ins (all 13 derive from the base, none from the public class) rather than a contract: `*Base` is documented as internal, and the public class is the one carrying the registration hook. Third-party code that subclasses `EngineBase`, `ReassemblyBase` or `TraceFlowBase` directly and registers through these functions now gets `RegistryError` naming the public class; subclass the public class instead. The error messages are unchanged. The built-ins never pass through a registrar; `Extractor._register_internal_engine` and its two siblings check the base for internal registration, and are what the [#513](https://github.com/JarryShaw/PyPCAPKit/issues/513) regression test now exercises. `register_protocol` is untouched. - extraction is around 46% faster on a 1,117-frame HTTP capture, with byte-identical output ([#420](https://github.com/JarryShaw/PyPCAPKit/pull/420)). A reassembled datagram's payload is analysed on first read rather than eagerly, cutting IP reassembly's own cost by 90.7% and TCP's by 23.7% (IP reassembly submits a datagram for every frame, fragmented or not) ([#424](https://github.com/JarryShaw/PyPCAPKit/pull/424)). Flow tracing over the same capture went from 1416.6 ms to 744.0 ms, because the flow dumper handed each record to a `Frame` constructor that re-dissected the whole stack to return the bytes it had just been given; options are no longer parsed twice either ([#427](https://github.com/JarryShaw/PyPCAPKit/pull/427)). -- **a breaking change to** `Extractor.engine` and `Extractor.record_header`: both are now declared to return `EngineBase[_P]` rather than `Engine`, and the private `_exeng` attribute follows. Both built-in engines subclass `EngineBase` directly, so neither was ever an `Engine`; the declaration now names the class the object has, and the `cast` calls that hid the gap now launder only the frame type parameter. Nothing is lost: `Engine` adds only the `__init_subclass__` registration hook, and the frame type parameter is preserved. Two things are visible to a type checker. Code passing the result to an `Engine`-typed parameter must now accept `EngineBase`; and because the old declarations were *unparameterised*, `engine.read_frame()` and `record_header().read_frame()` used to be `Any` and are now the extractor's own frame type, so an assignment that relied on `Any` there no longer type-checks ([#1022](https://github.com/JarryShaw/PyPCAPKit/issues/1022)). -- the documentation of the engine base classes: the member stubs (`name`, `module`, `registry`, `extractor`, `unsupported_reason`, `run`, `read_frame`, `close` and the rest) moved from `Engine` to `EngineBase`, where they are defined, and `Engine` now documents only the `__init_subclass__` registration hook it adds. `Extractor.engine` declares `EngineBase[_P]` as its return type, but that class was documented `:no-members:` under *Internal Definitions*, so the type a reader followed led to no API. Cross-references to `Engine.run`, `Engine.read_frame`, `Engine.close` and `Engine.unsupported_reason` now point at `EngineBase`, and the `PCAP` engine's `run` docstring lost the verbless sentence "will also be save the current `PCAP` engine instance". `EngineBase` is still not exported in `__all__`; that is left to [#1016](https://github.com/JarryShaw/PyPCAPKit/issues/1016) ([#1024](https://github.com/JarryShaw/PyPCAPKit/issues/1024)). +- **a breaking change to** `Extractor.engine` and `Extractor.record_header`: both are now declared to return `EngineBase[_P]` rather than `Engine`, and the private `_exeng` attribute follows. The built-in engines subclass `EngineBase` directly, so none was ever an `Engine`, which adds only the `__init_subclass__` registration hook. Two things are visible to a type checker. Code passing the result to an `Engine`-typed parameter must now accept `EngineBase`; and because the old declarations were *unparameterised*, `engine.read_frame()` and `record_header().read_frame()` used to be `Any` and are now the extractor's own frame type, so an assignment that relied on `Any` there no longer type-checks ([#1022](https://github.com/JarryShaw/PyPCAPKit/issues/1022)). +- the documentation of the engine base classes: the member stubs (`name`, `module`, `registry`, `extractor`, `unsupported_reason`, `run`, `read_frame`, `close` and the rest) moved from `Engine` to `EngineBase`, where they are defined, and `Engine` now documents only the `__init_subclass__` registration hook it adds, so the `EngineBase[_P]` that `Extractor.engine` returns leads to an API. Cross-references to `Engine.run`, `Engine.read_frame`, `Engine.close` and `Engine.unsupported_reason` now point at `EngineBase`. Exporting `EngineBase` in `__all__` is left to [#1016](https://github.com/JarryShaw/PyPCAPKit/issues/1016) ([#1024](https://github.com/JarryShaw/PyPCAPKit/issues/1024)). #### Fixed - TCP reassembly mixed absolute sequence numbers with buffer-relative slicing, so on any capture carrying a SYN with a realistic initial sequence number incomplete datagrams were dropped silently, and the `completed=False` branch of the public API was unreachable ([#349](https://github.com/JarryShaw/PyPCAPKit/issues/349), [#376](https://github.com/JarryShaw/PyPCAPKit/pull/376)). - `format='text'` raised `AttributeError` before writing anything, naming a `dictdumper.Text` that has never existed. It now points at `Tree`, as the `'txt'` alias beside it already did. -- `Extractor` had the ownership of its input stream inverted: a handle it opened itself, from `fin` given as a path, was **never closed**, while a stream the *caller* supplied and still needed **was**. `_cleanup` closed under `not self._flag_s` (the flag that gated the `open()`), and the `SeekableReader` wrapping a non-seekable input asked for `stream_closing=not self._flag_s` in the same wrong direction. The leak was one descriptor per extraction for the life of the process, and the production root cause of the `ResourceWarning` that reddened [#577](https://github.com/JarryShaw/PyPCAPKit/pull/577), [#596](https://github.com/JarryShaw/PyPCAPKit/pull/596) and [#600](https://github.com/JarryShaw/PyPCAPKit/pull/600); [#606](https://github.com/JarryShaw/PyPCAPKit/issues/606) made `tests/utilities/test_stacklevel.py` immune to foreign warnings, which repaired the CI signal but not the leak. Closing the caller's stream was the more dangerous half: silent data loss for anyone passing an open file they meant to keep reading. Both conditions now follow ownership via a single `Extractor._owns_input`, and a finaliser releases the handle of an extraction *abandoned* before end of file (`auto=False`, iterated part way and dropped), which reaches `_cleanup` by no route and was the second warning. A path-given extraction used to leave 1 descriptor open and a stream-given one came back closed and unreadable; the two files [#610](https://github.com/JarryShaw/PyPCAPKit/issues/610) names emit 0 such warnings where they emitted 2. `__exit__` deliberately still closes unconditionally, since a caller who scopes an `Extractor` with `with` asked for that ([#610](https://github.com/JarryShaw/PyPCAPKit/issues/610)). -- `extract(..., no_eof=True)` never returned. End of stream was detected and `ExtractionWarning: EOF reached` fired, but all three loops that handle it (`record_frames`, `__next__`, `__call__`) read `if self._flag_n: continue`, so the flag suppressed the error that *ended* the loop without supplying another ending and an exhausted input was retried forever. The flag is not pointless -- `__main__` sets `no_eof` for `fin='-'`, and the end of what has arrived on a live capture is not the end of the capture -- so this is a termination condition rather than a deletion. The input's own position tells a paused capture from a finished one: `prepare` raises end of stream when zero bytes remain and restores the position first, so two consecutive ends of stream at the same position mean nothing arrived between them (`Extractor._note_eof_progress`). A pipe is unaffected, since a read there *blocks* while the writer is open but idle. Also fixed, not in the issue: `pcapkit -` hung on **any** finished stdin; `cat in.pcap | python -m pcapkit -` went from killed at 25s to exiting 0 with a byte-identical dump. **One deliberate narrowing**: a *seekable* input does not block, so its two probes fall microseconds apart and a file still being appended to ends at the data present when the extraction reached it. The previous behaviour was unbounded, which is the defect; a timed grace period would make the cut-off intermittent rather than absent, so following a growing file is left to a policy of its own and the decision is pinned by a test. Pre-existing and untouched: an append landing *mid-record* raises `ValueError: read length must be non-negative or -1`, so `no_eof` over a growing file only ever worked for a writer flushing whole records. Both `no_eof` docstrings say all of this. The end-to-end regression tests are bounded by a **child process** rather than `tests._support.time_limit`, because the in-process deadline proved *intermittent* on this loop (once it escaped entirely, past ten minutes and 13.2 GB RSS), and an intermittent guard against a hang leaves a wedged suite rather than a red test. Two guesses at the cause are ruled out: the parse path's two `except Exception` handlers are never entered during the spin (`prepare` raises before any next-layer decode), and the memory is the retry loop emitting tens of thousands of `EOF reached` records a second, which the runner retains ([#620](https://github.com/JarryShaw/PyPCAPKit/issues/620)). -- `register_protocol` displaced one protocol class with another and said nothing. The registry behind `pcapkit.protocols.__proto__` is keyed on `cls.__name__.upper()`, and three dispatchable classes are all named `HTTP` (the base `pcapkit.protocols.application.http.HTTP` and the `httpv1` and `httpv2` implementations), so a bare assignment decided the winner: `register_protocol(httpv2.HTTP)` moved `__proto__['HTTP']` to the `httpv2` class with no exception, log line or `RegistryWarning`, and `ProtocolBase.expand_comp('HTTP')` then resolved to whichever registered last. The overwrite now raises a `RegistryWarning` naming the displaced and the replacing class. The sharpest path is not a call to the registrar: `Protocol.__init_subclass__` registers unconditionally, so merely *defining* a subclass of the public `Protocol` under a name a built-in holds displaced that built-in. A plain `import pcapkit` emits zero new warnings: the three built-in `HTTP` classes derive from `ProtocolBase`, so `__init_subclass__` never fires for them, and import-time seeding keys off the distinct names `HTTP`, `HTTPv1` and `HTTPv2`. The issue called this the one registrar in `pcapkit/foundation/registry/` with no warning; more accurately, none warned by itself, the siblings only by delegating to a classmethod with an `if code in cls.__xxx__: warn(...)` guard ([#726](https://github.com/JarryShaw/PyPCAPKit/pull/726) later gave all seven the identity guard described elsewhere), and this is the only `__name__.upper()`-keyed registry in `pcapkit/` that mutates its target with no guard and no delegate able to supply one. The new guard deliberately reads "key present and the incumbent is a different class" where siblings warned on mere presence then: this key is derived from the class, and this function is the funnel all nine wrapper registrars end in, so registering one class under two codes (`register_tcp` then `register_udp`, supported and documented) reaches it twice with nothing displaced, and warning there would put noise on a documented path whose wholesale filter would then hide the real collision. The collision is reported, not resolved: re-keying is a registry-format change that none of the bare-name readers can absorb (each degrades *silently* to a string or `Raw` on a miss), so it belongs to the registry redesign in [#514](https://github.com/JarryShaw/PyPCAPKit/issues/514), which this is sequenced ahead of ([#675](https://github.com/JarryShaw/PyPCAPKit/issues/675)). -- two `#:` autodoc comments in `pcapkit/foundation/traceflow/traceflow.py` named a bare `Type`, which Sphinx resolves against every class named `Type` in the project (five) rather than `typing.Type`, and silently linked to `pcapkit.const.l2tp.type.Type`, an L2TP field-type enum. Line 424 (`#: ~typing.Type[Dumper]: Dumper class.`, spelled out since [#709](https://github.com/JarryShaw/PyPCAPKit/issues/709) fixed it) is the live case: once [#684](https://github.com/JarryShaw/PyPCAPKit/issues/684) rendered `TraceFlow._foutio`, the built docs pointed at the wrong class with no warning. Both sites now spell it `~typing.Type[Dumper]`, as eight other files already do. Line 146 (the first line of `__output__`'s `#:` block) is fixed for the same reason but is currently inert, since the `# type:` comment at line 162 spells the same bare `Type[Dumper]`; it is insurance for whatever eventually renders that type. This pair is part of [#709](https://github.com/JarryShaw/PyPCAPKit/issues/709). Four more bare `Type` sites, the hand-written `:type:` fields at `docs/source/pcapkit/foundation/engines/engine.rst:40`, `.../reassembly/reassembly.rst:33` and `:43`, and `.../traceflow/traceflow.rst:40`, are fixed separately by [#714](https://github.com/JarryShaw/PyPCAPKit/pull/714) ([#709](https://github.com/JarryShaw/PyPCAPKit/issues/709)). -- `register_protocol`'s overwrite warning could claim a protocol was replaced with itself. The guard was correct (`incumbent is not protocol`, an identity check from [#681](https://github.com/JarryShaw/PyPCAPKit/pull/681)), but the message built both operands with a bare `repr()`. A factory defining a same-named closure-local class on every call (as `tests/protocols/test_construction_keyword_check_unit.py`'s `_protocol_class` does) produces two distinct classes sharing one `__module__` and `__qualname__`, so a real overwrite read "overwriting X with X." The message now compares the two reprs and, only when they coincide, appends each object's `id()`. `__module__`/`__qualname__` was rejected as the disambiguator: for the reported shape they are exactly what the coinciding repr already renders. The new `test_register_protocol_disambiguates_classes_sharing_a_repr` fails against the unfixed message ([#710](https://github.com/JarryShaw/PyPCAPKit/issues/710)). -- five more registrars warned on mere key presence rather than an actual overwrite, outside the wording of [#718](https://github.com/JarryShaw/PyPCAPKit/issues/718)'s identity guard (`incumbent is not None and incumbent is not new`, landed by [#726](https://github.com/JarryShaw/PyPCAPKit/pull/726)), whose issue named only code-keyed registrars: `register_engine`, `register_reassembly` and `register_traceflow` on `Extractor`, and `register_dumper` on both `Extractor` and `TraceFlow`. All five now compare the incumbent by identity before warning; both `register_dumper` sites compare only the stored dumper, so re-registering with just a new file extension stays silent too, judged defensible rather than comparing the full `(dumper, ext)` pair. Non-breaking: a correct caller sees strictly fewer warnings and no change to return value or exception. Five new tests pin the silent/warns-anyway split ([#739](https://github.com/JarryShaw/PyPCAPKit/issues/739)). -- **six registrars** now raise `RegistryError` for a non-class argument, where a bare `TypeError` ("issubclass() arg 1 must be a class") used to escape -- from the guard itself for the two `register_dumper` methods, and from inside `abc` for the other four. They are `register_dumper`, `register_engine`, `register_reassembly` and `register_traceflow` on `Extractor`, `TraceFlow.register_dumper` (reached through `register_traceflow_dumper`), and `pcapkit.foundation.registry.protocols.register_protocol`. Each guard gains an explicit `isinstance(x, type)` test ahead of its `issubclass`; in the five that accept a `ModuleDescriptor` it runs after the descriptor is unwrapped, so a descriptor naming a non-class attribute is rejected too, while `register_protocol` takes no descriptor. No guard's target class changes, and a wrong class still raises `RegistryError` as before. `RegistryError` subclasses `TypeError`, so `except TypeError` still catches it; a caller matching the exact type, or the old message, does not. **Not every site is covered**: six of the thirteen bare `issubclass` guards in the package are fixed here, and the same guard remains in the `register` classmethods of `ProtocolBase`, `Frame`, `PCAPNG`, `SCTP`, `Link`, `Internet` and `Transport`, all of which still leak `TypeError` for a non-class. `Transport.register` is reachable despite its `UnsupportedCall`, which is gated on `cls is Transport` and so fires only for the abstract class -- the guard below it leaks through `TCP.register` and `UDP.register`, which is the only way it is ever called. [#1026](https://github.com/JarryShaw/PyPCAPKit/issues/1026) tracks all seven ([#1021](https://github.com/JarryShaw/PyPCAPKit/issues/1021)). -- a missing extraction engine package now gives one `EngineWarning` rather than two ([#1045](https://github.com/JarryShaw/PyPCAPKit/issues/1045), [#1046](https://github.com/JarryShaw/PyPCAPKit/pull/1046)). `Extractor.import_test` warned that the package was absent, then `Extractor.run` warned again when it fell back to the default engine; `run` now stays silent there. The surviving message is the one `run` already emitted, `engine () is not installed; using default engine instead` (the module name is in backticks in the real message, which this changelog's Markdown generator cannot show literally), so `import_test` now says the same. Only code that matched `import_test`'s old wording, `extraction engine '' not available; using default engine instead`, or that expects two warnings, breaks. +- `Extractor` had the ownership of its input stream inverted: a handle it opened itself, from `fin` given as a path, was **never closed**, while a stream the *caller* supplied and still needed **was**. `_cleanup` closed under `not self._flag_s` (the flag that gated the `open()`), and the `SeekableReader` wrapping a non-seekable input asked for `stream_closing=not self._flag_s` in the same wrong direction. The leak was one descriptor per extraction, the root cause of the `ResourceWarning` that reddened [#577](https://github.com/JarryShaw/PyPCAPKit/pull/577), [#596](https://github.com/JarryShaw/PyPCAPKit/pull/596) and [#600](https://github.com/JarryShaw/PyPCAPKit/pull/600) ([#606](https://github.com/JarryShaw/PyPCAPKit/issues/606) had only made the test immune). Both conditions now follow ownership via a single `Extractor._owns_input`, and a finaliser releases the handle of an extraction *abandoned* before end of file (`auto=False`, iterated part way and dropped); the two files [#610](https://github.com/JarryShaw/PyPCAPKit/issues/610) names emit no such warning. `__exit__` deliberately still closes unconditionally, since a caller who scopes an `Extractor` with `with` asked for that ([#610](https://github.com/JarryShaw/PyPCAPKit/issues/610)). +- `extract(..., no_eof=True)` never returned. End of stream was detected and `ExtractionWarning: EOF reached` fired, but all three loops that handle it (`record_frames`, `__next__`, `__call__`) read `if self._flag_n: continue`, so the flag suppressed the error that *ended* the loop without supplying another ending and an exhausted input was retried forever. The flag is not pointless -- `__main__` sets `no_eof` for `fin='-'`, and the end of what has arrived on a live capture is not the end of the capture -- so this is a termination condition rather than a deletion. The input's own position tells a paused capture from a finished one: `prepare` raises end of stream when zero bytes remain and restores the position first, so two consecutive ends of stream at the same position mean nothing arrived between them (`Extractor._note_eof_progress`). A pipe is unaffected, since a read there *blocks* while the writer is open but idle. This also fixes `pcapkit -`, which hung on **any** finished stdin. **One deliberate narrowing**: a *seekable* input does not block, so a file still being appended to ends at the data present when the extraction reached it; a timed grace period would make the cut-off intermittent rather than absent, so following a growing file is left to a policy of its own. Pre-existing and untouched: an append landing *mid-record* raises `ValueError: read length must be non-negative or -1`. The end-to-end tests are bounded by a **child process** rather than `tests._support.time_limit`, because the in-process deadline proved intermittent on this loop, and an intermittent guard against a hang leaves a wedged suite rather than a red test. Two guesses at the cause are ruled out: the parse path's two `except Exception` handlers are never entered during the spin, and the memory is the retry loop's `EOF reached` records, which the runner retains ([#620](https://github.com/JarryShaw/PyPCAPKit/issues/620)). +- `register_protocol` displaced one protocol class with another and said nothing. The registry behind `pcapkit.protocols.__proto__` is keyed on `cls.__name__.upper()`, and three dispatchable classes are all named `HTTP` (the base `pcapkit.protocols.application.http.HTTP` and the `httpv1` and `httpv2` implementations), so a bare assignment decided the winner: `register_protocol(httpv2.HTTP)` moved `__proto__['HTTP']` to the `httpv2` class with no exception, log line or `RegistryWarning`, and `ProtocolBase.expand_comp('HTTP')` then resolved to whichever registered last. The overwrite now raises a `RegistryWarning` naming the displaced and the replacing class. The sharpest path is not a call to the registrar: `Protocol.__init_subclass__` registers unconditionally, so merely *defining* a subclass of the public `Protocol` under a name a built-in holds displaced that built-in. A plain `import pcapkit` emits zero new warnings: the three built-in `HTTP` classes derive from `ProtocolBase`, so `__init_subclass__` never fires for them, and import-time seeding keys off the distinct names `HTTP`, `HTTPv1` and `HTTPv2`. The guard deliberately reads "key present and the incumbent is a different class", not mere presence ([#726](https://github.com/JarryShaw/PyPCAPKit/pull/726) later gave all seven siblings the same identity guard): this function is the funnel all nine wrapper registrars end in, so registering one class under two codes (`register_tcp` then `register_udp`, a documented path) reaches it twice with nothing displaced. The collision is reported, not resolved: re-keying is a registry-format change that none of the bare-name readers can absorb (each degrades *silently* to a string or `Raw` on a miss), so it belongs to the registry redesign in [#514](https://github.com/JarryShaw/PyPCAPKit/issues/514). The issue called this the one registrar with no warning; in fact none warned by itself, the siblings only by delegating to a guarded classmethod ([#675](https://github.com/JarryShaw/PyPCAPKit/issues/675)). +- two `#:` autodoc comments in `pcapkit/foundation/traceflow/traceflow.py` named a bare `Type`, which Sphinx resolves against every class named `Type` in the project (five) rather than `typing.Type`, and silently linked to `pcapkit.const.l2tp.type.Type`, an L2TP field-type enum; once [#684](https://github.com/JarryShaw/PyPCAPKit/issues/684) rendered `TraceFlow._foutio`, the built docs pointed at the wrong class with no warning. Both sites now spell it `~typing.Type[Dumper]`, one pair of the sites [#709](https://github.com/JarryShaw/PyPCAPKit/issues/709) lists ([#709](https://github.com/JarryShaw/PyPCAPKit/issues/709)). Four more bare `Type` sites, in hand-written `:type:` fields under `docs/source/pcapkit/foundation/`, are fixed separately by [#714](https://github.com/JarryShaw/PyPCAPKit/pull/714) ([#709](https://github.com/JarryShaw/PyPCAPKit/issues/709)). +- `register_protocol`'s overwrite warning could claim a protocol was replaced with itself. The guard was correct (`incumbent is not protocol`, an identity check from [#681](https://github.com/JarryShaw/PyPCAPKit/pull/681)), but the message built both operands with a bare `repr()`. A factory defining a same-named closure-local class on every call (as `tests/protocols/test_construction_keyword_check_unit.py`'s `_protocol_class` does) produces two distinct classes sharing one `__module__` and `__qualname__`, so a real overwrite read "overwriting X with X." The message now compares the two reprs and, only when they coincide, appends each object's `id()`; `__module__`/`__qualname__` would not disambiguate, being what the repr already renders ([#710](https://github.com/JarryShaw/PyPCAPKit/issues/710)). +- five more registrars warned on mere key presence rather than an actual overwrite, outside the wording of [#718](https://github.com/JarryShaw/PyPCAPKit/issues/718)'s identity guard (`incumbent is not None and incumbent is not new`, landed by [#726](https://github.com/JarryShaw/PyPCAPKit/pull/726)), whose issue named only code-keyed registrars: `register_engine`, `register_reassembly` and `register_traceflow` on `Extractor`, and `register_dumper` on both `Extractor` and `TraceFlow`. All five now compare the incumbent by identity before warning; both `register_dumper` sites compare only the stored dumper, so re-registering with just a new file extension stays silent too. Non-breaking: a correct caller sees strictly fewer warnings ([#739](https://github.com/JarryShaw/PyPCAPKit/issues/739)). +- **six registrars** now raise `RegistryError` for a non-class argument, where a bare `TypeError` ("issubclass() arg 1 must be a class") used to escape -- from the guard itself for the two `register_dumper` methods, and from inside `abc` for the other four. They are `register_dumper`, `register_engine`, `register_reassembly` and `register_traceflow` on `Extractor`, `TraceFlow.register_dumper` (reached through `register_traceflow_dumper`), and `pcapkit.foundation.registry.protocols.register_protocol`. Each guard gains an explicit `isinstance(x, type)` test ahead of its `issubclass`; in the five that accept a `ModuleDescriptor` it runs after the descriptor is unwrapped, so a descriptor naming a non-class attribute is rejected too, while `register_protocol` takes no descriptor. No guard's target class changes, and a wrong class still raises `RegistryError` as before. `RegistryError` subclasses `TypeError`, so `except TypeError` still catches it; a caller matching the exact type, or the old message, does not. The same guard in the seven protocol `register` classmethods is fixed separately by [#1026](https://github.com/JarryShaw/PyPCAPKit/issues/1026) ([#1021](https://github.com/JarryShaw/PyPCAPKit/issues/1021)). +- a missing extraction engine package now gives one `EngineWarning` rather than two ([#1045](https://github.com/JarryShaw/PyPCAPKit/issues/1045), [#1046](https://github.com/JarryShaw/PyPCAPKit/pull/1046)). `Extractor.import_test` warned that the package was absent, then `Extractor.run` warned again when it fell back to the default engine; `run` now stays silent there, and `import_test` says what `run` used to: `engine () is not installed; using default engine instead`, the module name in backticks. Only code that matched `import_test`'s old wording, or that expects two warnings, breaks. ### pcapkit.protocols @@ -112,12 +112,12 @@ Preceded by `1.5.0a1` (2026-09-15), `1.5.0b1` and `1.5.0b2` (both 2026-09-18) an - NGAP over SCTP (3GPP TS 38.413), decoding aligned PER through `pycrate` (`pip install pypcapkit[NGAP]`) ([#251](https://github.com/JarryShaw/PyPCAPKit/discussions/251), [#417](https://github.com/JarryShaw/PyPCAPKit/pull/417)). Decoding is generic by ASN.1 shape rather than per-procedure, so all 81 elementary procedures and 438 protocol IEs work and a new 3GPP release needs no code change. Registered as a default on PPID 60 and 66, but only 60 decodes: PPID 66 is an NGAP PDU inside a DTLS record, and there is no DTLS dissector. `pycrate` is deliberately excluded from the `all` extra -- it is LGPL-2.1+ where this package is BSD-3-Clause, and lands some 238 MB to obtain one module. - the Mobility Header registry, completed ([#383](https://github.com/JarryShaw/PyPCAPKit/pull/383), [#437](https://github.com/JarryShaw/PyPCAPKit/pull/437)): the [RFC 5568](https://datatracker.ietf.org/doc/html/rfc5568) fast-handover messages and options, then all 24 registered message data types, all 71 registered options (with nested sub-option registries for the flow identification, access network identifier, quality-of-service and LMA-controlled MAG parameter families) and all 4 CGA extensions. The CGA Parameters option, the only option that can carry a CGA extension on the wire, was the last to come off the generic handler, which is what let the four extensions round-trip end to end. - dispatch entries for dissectors that existed but were reachable from no registry ([#436](https://github.com/JarryShaw/PyPCAPKit/pull/436)): FTP-DATA on TCP 20, HTTP/1 on TCP 8080, HTTP on UDP 8080, `L2TPv2` on UDP 1701 and OSPF at `TransType` 89. `VLAN` became an abstract base with `C_Tag` (802.1Q) and `S_Tag` (802.1ad) as concrete subclasses, so a Q-in-Q frame no longer collapses into one opaque `Raw`; `L2TP` likewise became a base, with `L2TPv2` carrying the [RFC 2661](https://datatracker.ietf.org/doc/html/rfc2661) implementation. -- `Protocol`/`ProtocolBase` gain a `code=` class keyword for the next-layer dispatch registries (`Link.__proto__`, `Internet.__proto__`, `TCP.__proto__`, `UDP.__proto__`, `SCTP.__proto__`, `Frame.__proto__`, `PCAPNG.__proto__`), the opt-in treatment [#514](https://github.com/JarryShaw/PyPCAPKit/issues/514) gave `Engine`, `Reassembly`, `TraceFlow` and `Dumper` above, extended to the one family that needed a registration key invented. Omitting `code` leaves a subclass unregistered, as before; the built-in tables are still filled by literal assignment in each layer module, so nothing the library ships moves. `code` accepts a bare enum member, whose *type* infers the destination (`EtherType` means `Link`, `TransType` means `Internet`, `PayloadProtocolIdentifier` means `SCTP`, and `LinkType` means *both* `Frame` **and** `PCAPNG`, mirroring `register_linktype`), or a `{destination: key}` mapping, required for a raw `int` such as a port number, which cannot say which transport it belongs to. Either form may appear in an iterable, so one declaration can register a class into several registries (e.g. a `L2TP` subclass reachable by IP protocol number and by a UDP port), and the mapping form is accepted even where inference would work. Inference refuses rather than guesses: an enum member whose type names no known destination raises `RegistryError`, and an unrecognised class keyword raises `UnsupportedCall`, matching the other four families. Backed by `pcapkit.foundation.registry.protocols.register_protocol_code`, also callable directly for a class that declined at definition time. It was written up as the mechanism [#548](https://github.com/JarryShaw/PyPCAPKit/issues/548) (`TransType.L2TP` registered nowhere) needs, "now a one-declaration change"; investigating [#548](https://github.com/JarryShaw/PyPCAPKit/issues/548) found otherwise -- 115 is an [RFC 3931](https://datatracker.ietf.org/doc/html/rfc3931) L2TPv3-over-IP header with no class to dispatch to, so the declaration would have pointed the [RFC 2661](https://datatracker.ietf.org/doc/html/rfc2661) parser at it (see the corresponding **Fixed** entry below). The mechanism is unaffected, and its worked example now names `L2TPv3` rather than `L2TPv2` ([#570](https://github.com/JarryShaw/PyPCAPKit/pull/570)). -- `tests/protocols/test_dispatch_reachability_unit.py`, the coverage [#548](https://github.com/JarryShaw/PyPCAPKit/issues/548) asked for: every `ProtocolBase` descendant whose `__index__` returns an enum member must be reachable under that code in the registry its enum *type* designates, read from the same `_CODE_DESTINATIONS` table backing `code=` so the two cannot drift. Where `test_dispatch_registry_unit.py` walks the 38 entries that exist and checks each parses, this walks the classes and catches one nothing registered, the shape in which `OSPF` once shipped reachable from no table. 23 claims verified, no gaps; a companion case injects a gap and confirms the audit reports it ([#548](https://github.com/JarryShaw/PyPCAPKit/issues/548)). -- `tests/protocols/transport/test_tcp_mptcp_join_flag_ordering_unit.py`, covering all three MP_JOIN layouts through the *public* constructor, the construct-pack-parse cycle for each, the stale-flags case that rules out a zero-valued default, the statement order, and controls for the parse path and the flag-independent options. It closes the gap that let 100% statement and branch coverage of the two changed modules coexist with a broken public path: the old cases reached `_make_mptcp_join` by assigning a Python `set` to `_flags` on a bare `TCP.__new__(TCP)`, which executes every branch while bypassing both the ordering and the accumulator's type. The stale `tcp-mptcp/MP_JOIN` entry is deleted from `EXPECTED_FAILURES` and the MP_JOIN exclusion in `test_tcp_mptcp_subtype_unit.py` is lifted ([#587](https://github.com/JarryShaw/PyPCAPKit/issues/587)). -- `tests/protocols/test_option_generator_tcp_base_unit.py`, asserting that every `TCP_BASE` key is a parameter `TCP.make` declares (derived from `inspect.signature`, so it catches a fourth misspelling nobody has made yet) and that the built segment carries the stated header in both data model and packed octets. Two of the three wrong names were invisible to any assertion about a *value*, the value asked for equalling the default used instead ([#602](https://github.com/JarryShaw/PyPCAPKit/issues/602)). -- ILNP nonce sizing coverage in `tests/protocols/internet/test_ipv6_extension_unit.py`, one test per protocol, asserting the declared length, the exact packed octets and the construct-pack-parse cycle over ten nonces. The existing suite missed the defect because its only ILNP nonce was `0xFFFFFF` (bit length 24, a multiple of eight, where floor division and the ceiling agree), the blind spot that hid the same typo in `numbers.py` behind bit lengths 8, 16, 24, 32 and 64. Every new case bar two controls therefore has a bit length that is *not* a multiple of eight, several below 256, and the test asserts that at least six stay non-byte-aligned and one stays below 256, so rounding them off later cannot disarm it ([#601](https://github.com/JarryShaw/PyPCAPKit/issues/601)). -- `SOLUTION` parameter width coverage in `tests/protocols/internet/test_hip_unit.py`: eight widths asserting the declared length, the reader's acceptance and the construct-pack-parse cycle, plus a 57-bit case pinning the 20-octet length [RFC 5201 Section 5.2.5](https://datatracker.ietf.org/doc/html/rfc5201#section-5.2.5) requires of HIPv1. Five widths (1, 9, 12, 17 and 25 bits) are deliberately *not* multiples of eight, because at a multiple of eight the defective `ceil(bits / 4)` coincides with the correct width, which is why every fixture reaching this builder passed; the round-trip case in `examples/generators/options.py` supplies no `random` or `solution`, so it exercised the formula at zero bits. 57 bits discriminates on the HIPv1 path where 64 does not: both formulas give 20 at a full-width value ([#608](https://github.com/JarryShaw/PyPCAPKit/issues/608)). +- `Protocol`/`ProtocolBase` gain a `code=` class keyword for the next-layer dispatch registries (`Link.__proto__`, `Internet.__proto__`, `TCP.__proto__`, `UDP.__proto__`, `SCTP.__proto__`, `Frame.__proto__`, `PCAPNG.__proto__`), the opt-in treatment [#514](https://github.com/JarryShaw/PyPCAPKit/issues/514) gave `Engine`, `Reassembly`, `TraceFlow` and `Dumper` above, extended to the one family that needed a registration key invented. Omitting `code` leaves a subclass unregistered, as before; the built-in tables are still filled by literal assignment in each layer module, so nothing the library ships moves. `code` accepts a bare enum member, whose *type* infers the destination (`EtherType` means `Link`, `TransType` means `Internet`, `PayloadProtocolIdentifier` means `SCTP`, and `LinkType` means *both* `Frame` **and** `PCAPNG`, mirroring `register_linktype`), or a `{destination: key}` mapping, required for a raw `int` such as a port number, which cannot say which transport it belongs to. Either form may appear in an iterable, so one declaration can register a class into several registries (e.g. a `L2TP` subclass reachable by IP protocol number and by a UDP port), and the mapping form is accepted even where inference would work. Inference refuses rather than guesses: an enum member whose type names no known destination raises `RegistryError`, and an unrecognised class keyword raises `UnsupportedCall`, matching the other four families. Backed by `pcapkit.foundation.registry.protocols.register_protocol_code`, also callable directly for a class that declined at definition time. It was written up as the mechanism [#548](https://github.com/JarryShaw/PyPCAPKit/issues/548) needs; [#548](https://github.com/JarryShaw/PyPCAPKit/issues/548) turned out to need a missing class instead (115 is [RFC 3931](https://datatracker.ietf.org/doc/html/rfc3931) L2TPv3 over IP, not [RFC 2661](https://datatracker.ietf.org/doc/html/rfc2661); see the corresponding **Fixed** entry below), so its worked example names `L2TPv3` ([#570](https://github.com/JarryShaw/PyPCAPKit/pull/570)). +- `tests/protocols/test_dispatch_reachability_unit.py`, the coverage [#548](https://github.com/JarryShaw/PyPCAPKit/issues/548) asked for: every `ProtocolBase` descendant whose `__index__` returns an enum member must be reachable under that code in the registry its enum *type* designates, read from the same `_CODE_DESTINATIONS` table backing `code=` so the two cannot drift. It walks the classes rather than the registry entries, so it catches a class nothing registered, the shape in which `OSPF` once shipped ([#548](https://github.com/JarryShaw/PyPCAPKit/issues/548)). +- `tests/protocols/transport/test_tcp_mptcp_join_flag_ordering_unit.py`, covering all three MP_JOIN layouts through the *public* constructor, the construct-pack-parse cycle for each, the stale-flags case that rules out a zero-valued default, the statement order, and controls for the parse path and the flag-independent options. The old cases bypassed the public path, so full coverage coexisted with a broken one. The stale `tcp-mptcp/MP_JOIN` entry is deleted from `EXPECTED_FAILURES` ([#587](https://github.com/JarryShaw/PyPCAPKit/issues/587)). +- `tests/protocols/test_option_generator_tcp_base_unit.py`, asserting that every `TCP_BASE` key is a parameter `TCP.make` declares (derived from `inspect.signature`) and that the built segment carries the stated header in both data model and packed octets ([#602](https://github.com/JarryShaw/PyPCAPKit/issues/602)). +- ILNP nonce sizing coverage in `tests/protocols/internet/test_ipv6_extension_unit.py`, one test per protocol, asserting the declared length, the exact packed octets and the construct-pack-parse cycle over ten nonces, most with a bit length that is *not* a multiple of eight, where floor division and the ceiling disagree ([#601](https://github.com/JarryShaw/PyPCAPKit/issues/601)). +- `SOLUTION` parameter width coverage in `tests/protocols/internet/test_hip_unit.py`: eight widths asserting the declared length, the reader's acceptance and the construct-pack-parse cycle, plus a 57-bit case pinning the 20-octet length [RFC 5201 Section 5.2.5](https://datatracker.ietf.org/doc/html/rfc5201#section-5.2.5) requires of HIPv1. Five widths are deliberately *not* multiples of eight, where the defective `ceil(bits / 4)` coincides with the correct width ([#608](https://github.com/JarryShaw/PyPCAPKit/issues/608)). #### Changed @@ -134,10 +134,10 @@ Import paths, before to after: `layer` changes from `'Link'` to `'Application'` for all three. `OSPF` now takes `Application` as its only base, and `RARP` becomes `class RARP(Application, ARP)` -- layer base first, because `ARP`'s chain reaches `Link`, which owns `__layer__`. `ARP` and `InARP` stay `'Link'`. Dispatch keys are unchanged: OSPF is still reached from `Internet.__proto__` at `TransType.OSPFIGP` and RARP from `Link.__proto__` by EtherType. Leaving `Link` repoints `OSPF.__proto__` at `ProtocolBase.__proto__`, so OSPF no longer sees Link's EtherType entries -- which nothing used, since it is reached by protocol number, and a `-1` lookup falls back to `Raw` in either registry without inserting. `register` and `_read_protos` still resolve, on `ProtocolBase` rather than `Link`. `RARP` keeps Link's through `ARP`. -Layer-limited extraction: `layer='link'` no longer sets the termination flag on OSPF and `layer='application'` now does -- the same inversion applies to RARP -- but the extracted protochain is unchanged at every `layer=` value, including `'internet'` (`IPv4:OSPFIGP` before and after, as IPv4 ends the chain itself). Measured: `IPv4:OSPFv2:Raw` for a headerless `IPV4` capture, `Ethernet:RARP:Raw` for a padded RARP frame. +Layer-limited extraction: `layer='link'` no longer sets the termination flag on OSPF and `layer='application'` now does -- the same inversion applies to RARP -- but the extracted protochain is unchanged at every `layer=` value, including `'internet'` (`IPv4:OSPFIGP` before and after, as IPv4 ends the chain itself). - renames with no compatibility alias left behind: PCAP-NG `Option` subclasses spell the namespace class keyword `ns=` instead of `namespace=` ([#439](https://github.com/JarryShaw/PyPCAPKit/issues/439)). -- `tests/protocols/transport/test_tcp_udp_unit.py` now reaches the MP_JOIN dispatchers through `TCP()` itself, instead of assigning a Python `set` to `_flags` on a bare `TCP.__new__(TCP)`. A `set` answers the membership tests `_make_mptcp_join` and `_read_mptcp_join` use, so every flag branch ran and both TCP modules read 100% coverage, while the attribute had neither the `aenum.IntFlag` type production assigns nor the ordering that governs when it exists -- how [#587](https://github.com/JarryShaw/PyPCAPKit/issues/587) stayed invisible behind that number, and the `cast('Enum_Flags', 0)` no-op behind it. Reverting [#587](https://github.com/JarryShaw/PyPCAPKit/issues/587)'s hoist now fails two of the file's 17 tests with `AttributeError: 'TCP' object has no attribute '_flags'`, and restoring the `cast` fails two with `TypeError: argument of type 'int' is not a container or iterable`; all 17 passed before. The library is unchanged and coverage does not move -- the point is what the same numbers are now worth ([#603](https://github.com/JarryShaw/PyPCAPKit/issues/603)). -- building a protocol through its constructor with a keyword that names nothing now raises `UnsupportedCall` instead of discarding it. **This is a behaviour change to a public API**: every `make` in the tree ends its signature with `**kwargs` and reads nothing out of it, so a misspelled keyword was accepted, dropped, and the field it named kept its default -- wrong octets, nothing said. That is what [#602](https://github.com/JarryShaw/PyPCAPKit/issues/602) cost: `examples/generators/options.py` asked for `seq=1` where `TCP.make` spells it `seq_no`, and 25 generated fixture frames carried sequence number `0`. [#541](https://github.com/JarryShaw/PyPCAPKit/issues/541) and [#556](https://github.com/JarryShaw/PyPCAPKit/issues/556) were the same silence. The schema layer was never so permissive (`Schema.__update__` warns `UnknownFieldWarning`), and that asymmetry is what this closes. The check sits in `ProtocolBase.__init__` rather than `make`, because `__post_init__` passes one `**kwargs` to the construction *and* to the parse of what it has just constructed, so a keyword declared only by `read` legitimately travels through `make` (`HIP.read` declares `extension`, which `HIP.make` does not, and the option generator depends on it). The accepted set is the union of every keyword-taking parameter of `make`, `read`, `pack`, `unpack`, `__post_init__` and `__init__` across the MRO, computed once per class from `inspect.signature`. Parsing is deliberately untouched: there the keywords are whatever the engines and the four `_import_next_layer` implementations forward, a protocol cannot know which its parent passed on, and a dropped parse keyword changes how a packet is read, not what its octets say. Two opt-in escapes exist per class through a new `__keywords__`: a set, for a keyword read out of `**kwargs` by name (`ESP.read` with `packet`); and `None`, for a dispatcher whose real signature belongs to a class chosen at call time (only `HTTP.make` forwarding to `HTTPv1`/`HTTPv2`). The message names the near neighbour (`seq` reports *did you mean 'seq_no'?*). `from_data` warns `UnknownFieldWarning` rather than raising, because its keywords are whatever `_make_data` returned, so the defect is a key of that mapping disagreeing with the signature it is spread into, and the person who meets it cannot fix it. This surfaced four latent bugs in this repository, all residue of [#602](https://github.com/JarryShaw/PyPCAPKit/issues/602) and fixed here: the `_TCP_BASE` of `examples/generators/dispatch.py` and three stale copies under `tests/protocols/transport/`, each building segments with sequence number `0`. Three more are reported, not fixed, being a defect per protocol: `Frame._make_data` returns `ts_src` where `make` declares `ts_sec`, `L2TPv2._make_data` returns `prio` where it declares `priority`, and `Header._make_data` returns a `magic_number` that `Header.make` does not take, so `from_data` has been dropping a frame's timestamp, an L2TPv2 priority bit and a capture's byte order, and now says so. One limitation: a *direct* `SomeProtocol.make(...)` call is not checked and still discards silently, since the check sits where every producer's keywords converge rather than inside each of the 30 `make` implementations; `object.__new__(cls).make(**kwargs)`, the idiom `HTTP.make` uses, reaches it ([#617](https://github.com/JarryShaw/PyPCAPKit/issues/617)). +- `tests/protocols/transport/test_tcp_udp_unit.py` now reaches the MP_JOIN dispatchers through `TCP()` itself, instead of assigning a Python `set` to `_flags` on a bare `TCP.__new__(TCP)`. A `set` answers the same membership tests, so every branch ran while the attribute had neither the type production assigns nor the ordering that governs when it exists -- how [#587](https://github.com/JarryShaw/PyPCAPKit/issues/587) stayed invisible. Reverting [#587](https://github.com/JarryShaw/PyPCAPKit/issues/587)'s hoist, or restoring the `cast('Enum_Flags', 0)` seed, now fails the file. Test-only ([#603](https://github.com/JarryShaw/PyPCAPKit/issues/603)). +- building a protocol through its constructor with a keyword that names nothing now raises `UnsupportedCall` instead of discarding it. **This is a behaviour change to a public API**: every `make` in the tree ends its signature with `**kwargs` and reads nothing out of it, so a misspelled keyword was accepted, dropped, and the field it named kept its default -- wrong octets, nothing said, as in [#602](https://github.com/JarryShaw/PyPCAPKit/issues/602), [#541](https://github.com/JarryShaw/PyPCAPKit/issues/541) and [#556](https://github.com/JarryShaw/PyPCAPKit/issues/556). The check sits in `ProtocolBase.__init__` rather than `make`, because `__post_init__` passes one `**kwargs` to the construction *and* to the parse of what it has just constructed, so a keyword declared only by `read` legitimately travels through `make` (`HIP.read` declares `extension`). The accepted set is the union of every keyword-taking parameter of `make`, `read`, `pack`, `unpack`, `__post_init__` and `__init__` across the MRO. Parsing is deliberately untouched: a protocol cannot know which keywords its parent passed on. A new `__keywords__` opts a class out: a set, for a keyword read out of `**kwargs` by name (`ESP.read` with `packet`); and `None`, for a dispatcher whose real signature belongs to a class chosen at call time (only `HTTP.make`). The message names the near neighbour (`seq` reports *did you mean 'seq_no'?*). `from_data` warns `UnknownFieldWarning` rather than raising, because the defect is then a `_make_data` key the caller cannot fix. Four leftover copies of [#602](https://github.com/JarryShaw/PyPCAPKit/issues/602)'s misspelt TCP base, in `examples/generators/dispatch.py` and `tests/protocols/transport/`, are fixed. Three such `_make_data` keys are reported, not fixed: `Frame._make_data` returns `ts_src` where `make` declares `ts_sec`, `L2TPv2._make_data` returns `prio` where it declares `priority`, and `Header._make_data` returns a `magic_number` that `Header.make` does not take. A *direct* `SomeProtocol.make(...)` call is not checked and still discards silently ([#617](https://github.com/JarryShaw/PyPCAPKit/issues/617)). - **a breaking change to** `Application`: it no longer forbids payload dispatch outright. It encodes "no further protocol layer above this one", which undissected trailer bytes are not, so `_decode_next_layer` and `_import_next_layer` accept the `-1` sentinel -- the rest of the packet, resolving to `Raw`, or to `NoPayload` when nothing remains -- and `__post_init__` keeps a payload that `read` set instead of overwriting it with `NoPayload`. A real protocol number (any `proto` other than `-1`) is still refused with `UnsupportedCall`. A subclass that calls either method with `-1` changes from raising to succeeding; no subclass in the package does ([#719](https://github.com/JarryShaw/PyPCAPKit/issues/719)). #### Fixed @@ -164,30 +164,30 @@ This resolves [#548](https://github.com/JarryShaw/PyPCAPKit/issues/548), which r - `_RE_STATUS` in the same reader had the same unanchored prefix defect, found by auditing `_RE_METHOD`'s siblings, and escaped as the wrong exception type. The pattern is only a guard (the value comes from `int(para2)` on the raw token), so a prefix match let a malformed status past the guard and out of `int()` uncaught, where `_read_http_header` documents `ProtocolError`: `200x` raised `ValueError: invalid literal for int()` and `2000` raised `ValueError: 2000 is not a valid StatusCode`; both are now `ProtocolError`. [RFC 9112 Section 4](https://datatracker.ietf.org/doc/html/rfc9112#section-4) gives `status-code = 3DIGIT`, exactly three, so the anchor is what the grammar said (the production lives in HTTP/1.1 because `status-code` is part of its `status-line`; [RFC 9110 Section 15](https://datatracker.ietf.org/doc/html/rfc9110#section-15) covers code semantics and the registry). `_RE_VERSION` was audited and is safe, since both call sites read the captured group ([#583](https://github.com/JarryShaw/PyPCAPKit/issues/583)). - as a consequence of the above, the unhandled `MemoryError` at `pcapkit/protocols/protocol.py:1411` on a truncated PCAP-NG capture. `dhcp_little_endian.pcapng` cut to 161, 641 or 1389 octets left a one-octet read of a little-endian 32-bit block length, which `rjust()` turned into `0x78000000` or `0x84000000` (1.88 to 2.06 GiB) and passed straight to `self._file.read()` as an allocation size, from a 1772-octet file. Under a 1 GiB address-space cap all three raise `MemoryError`; with more space the parse fails anyway. With `ljust()` the same reads report 120 and 132, no large allocation is attempted, and all three end in the ordinary handled parse failure ([#604](https://github.com/JarryShaw/PyPCAPKit/issues/604)). - reading a big-endian classic PCAP byte-swapped every record header field, and then crashed. `Frame.unpack` seeded the file's byte order under the key `bytesorder` where the schema's `byteorder_callback` reads `byteorder`, so the lookup never found it and fell back to `sys.byteorder`, the host's order rather than the file's. On a little-endian host reading a little-endian capture that is right by coincidence, and every capture in this repository was little-endian. Frame 1 of `big_endian.pcap` read `incl_len=1241513984` for a record whose real value is `74`. `incl_len` is the payload length, so that record consumed the whole file and the second was read with a negative length, raising `ValueError: read length must be non-negative or -1` -- the reported crash, and the *second* symptom. The sibling `Frame.pack` eleven lines earlier spelled the key correctly, marking this a slip rather than a second key, and the fallback made a misspelled key indistinguishable from an absent one; `byteorder_callback` now records that it defines the key and why the fallback hides a typo ([#605](https://github.com/JarryShaw/PyPCAPKit/issues/605)). -- documentation. `mptcp_dss_ack_selector`'s note said a corrected field-width lambda "would not have worked" and that fixing it belonged to `pcapkit.corekit.fields.numbers`, which is where [#598](https://github.com/JarryShaw/PyPCAPKit/pull/598) then fixed it; the same paragraph sat in `test_tcp_mptcp_length_arithmetic_unit.py`'s docstring, whose other stale claim was that MP_JOIN "cannot be built through the public `TCP()` constructor at all", true only until [#587](https://github.com/JarryShaw/PyPCAPKit/issues/587). A callable-length `NumberField` packs and unpacks both DSS widths now, and wire *absence* was never the obstacle: `MPTCPDSS.ssn`, `dl_len` and `checksum` have always been `ConditionalField` on the sibling `M` flag. The `SwitchField` form is kept for the narrower reason the note now gives: `ConditionalField`'s `length` forwards to the wrapped field without consulting the condition, so it is safe here only because `Schema.pack` and `Schema.unpack` special-case that wrapper by name, whereas a `SwitchField` always resolves to a concrete field. Replacing it would be a behaviour change and is not made ([#603](https://github.com/JarryShaw/PyPCAPKit/issues/603)). -- the one assertion [#604](https://github.com/JarryShaw/PyPCAPKit/issues/604) left pinning the old padding side, red on `mainline` since [#621](https://github.com/JarryShaw/PyPCAPKit/pull/621) merged. `TCPUDPUnitTests.test_a_truncated_option_still_parses_its_declared_length` expected a truncated TCP option's `data` as the synthesised zero octets *followed by* the real ones, as `rjust()` produced; [#621](https://github.com/JarryShaw/PyPCAPKit/pull/621) made the padding `ljust()` everywhere but could not retarget this file, since [#612](https://github.com/JarryShaw/PyPCAPKit/pull/612) owned it and editing concurrently risked discarding work that has since landed. The real octets now come first for both parametrised widths, and the docstring, which said the short read was *left*-padded, is corrected. Test-only; the sibling case in `tests/protocols/internet/test_ipv4_unit.py` was retargeted in [#621](https://github.com/JarryShaw/PyPCAPKit/pull/621) (two subtest failures before, none after; [#604](https://github.com/JarryShaw/PyPCAPKit/issues/604)). -- `main` went red the moment [#604](https://github.com/JarryShaw/PyPCAPKit/issues/604)'s `ljust()` landed, because that test still pinned the head-padded short read: `Reserved_79` declaring `length=12` over 6 real octets now reports `aabbccddeeff00000000` where the test expected `00000000aabbccddeeff`, failing both subtests (`declared_length=12` and `=32`) while the 17 other cases stayed green. The inputs discriminate: `trailing` is non-zero and the pad width is 4 and 24. [#621](https://github.com/JarryShaw/PyPCAPKit/pull/621) left the file alone because [#612](https://github.com/JarryShaw/PyPCAPKit/pull/612) owned it at the time, and merged two minutes ahead of the cross-review verdict that named it ([#604](https://github.com/JarryShaw/PyPCAPKit/issues/604), [#621](https://github.com/JarryShaw/PyPCAPKit/pull/621)). +- documentation. `mptcp_dss_ack_selector`'s note said a corrected field-width lambda "would not have worked" and that fixing it belonged to `pcapkit.corekit.fields.numbers`, which is where [#598](https://github.com/JarryShaw/PyPCAPKit/pull/598) then fixed it; the same paragraph sat in `test_tcp_mptcp_length_arithmetic_unit.py`'s docstring, whose other stale claim was that MP_JOIN "cannot be built through the public `TCP()` constructor at all", true only until [#587](https://github.com/JarryShaw/PyPCAPKit/issues/587). A callable-length `NumberField` packs and unpacks both DSS widths now, and wire *absence* was never the obstacle: `MPTCPDSS.ssn`, `dl_len` and `checksum` have always been `ConditionalField` on the sibling `M` flag. The `SwitchField` form is kept for the narrower reason the note now gives: `ConditionalField`'s `length` forwards to the wrapped field without consulting the condition, whereas a `SwitchField` always resolves to a concrete field ([#603](https://github.com/JarryShaw/PyPCAPKit/issues/603)). +- the one assertion [#604](https://github.com/JarryShaw/PyPCAPKit/issues/604) left pinning the old padding side, red on `mainline` since [#621](https://github.com/JarryShaw/PyPCAPKit/pull/621) merged. `TCPUDPUnitTests.test_a_truncated_option_still_parses_its_declared_length` expected a truncated TCP option's `data` as the synthesised zero octets *followed by* the real ones, as `rjust()` produced; [#621](https://github.com/JarryShaw/PyPCAPKit/pull/621) made the padding `ljust()` everywhere but could not retarget this file, since [#612](https://github.com/JarryShaw/PyPCAPKit/pull/612) owned it. The real octets now come first, and the docstring is corrected. Test-only; the sibling case in `tests/protocols/internet/test_ipv4_unit.py` was retargeted in [#621](https://github.com/JarryShaw/PyPCAPKit/pull/621) ([#604](https://github.com/JarryShaw/PyPCAPKit/issues/604)). +- `main` went red the moment [#604](https://github.com/JarryShaw/PyPCAPKit/issues/604)'s `ljust()` landed, because that test still pinned the head-padded short read: `Reserved_79` declaring `length=12` over 6 real octets now reports `aabbccddeeff00000000` where the test expected `00000000aabbccddeeff`. [#621](https://github.com/JarryShaw/PyPCAPKit/pull/621) left the file alone because [#612](https://github.com/JarryShaw/PyPCAPKit/pull/612) owned it at the time ([#604](https://github.com/JarryShaw/PyPCAPKit/issues/604), [#621](https://github.com/JarryShaw/PyPCAPKit/pull/621)). - the HIP `SOLUTION` builder sized the parameter with `4 + math.ceil(max(random.bit_length(), solution.bit_length()) / 4)`, an invalid shorthand for two fields. [RFC 7401 Section 5.2.5](https://datatracker.ietf.org/doc/html/rfc7401#section-5.2.5) gives `Length` as `4 + RHASH_len / 4` over a `Random #I` and a `Puzzle solution #J` of `RHASH_len / 8` octets **each**, and `/ 4` equals twice `/ 8` only because `RHASH_len` is a whole number of octets. For an arbitrary `bit_length()` the builder emitted an *odd* contents width, which its own reader refused: `_read_param_solution`'s `(schema.len - 4) % 2` guard exists because `SolutionParameter` splits that width into two equal halves. `random=0x1` with `solution=0xfff` declared `len=7` and raised `ProtocolError: HIPv2: [ParamNo 321] invalid format` on input the library had produced. The undersized length was worse: both fields take their width from that `len`, so `solution=0xfff` was packed into one octet and came back as `0xff`, silently. Fixed to `4 + 2 * math.ceil(max(...) / 8)`, the form the sibling `PUZZLE` builder uses, which is even by construction. Loosening the reader was rejected: an odd contents width has no meaning in the wire format (the RFC makes the fields equal-width and does not say which takes an extra octet), and a laxer guard would still have truncated the value ([#608](https://github.com/JarryShaw/PyPCAPKit/issues/608)). -- `TCP.read` seeded its connection-flag accumulator with `cast('Enum_Flags', 0)`. `typing.cast` is a runtime no-op, so the accumulator began as the plain `int` `0`, and only the `|=` promoting it to a `pcapkit.const.tcp.flags.Flags` member ever changed that. A segment with an all-zero flags octet (an nmap NULL scan) left `self._flags` an `int`, so a membership test raised `TypeError: argument of type 'int' is not a container or iterable`, and `TCP.connection` returned an `int` where it annotates `Flags`. Any flag masked it, which is how it survived 100% statement and branch coverage. The seed is `Flags(0)`, as [#597](https://github.com/JarryShaw/PyPCAPKit/pull/597) already did for the sibling accumulator in `make`; `Flags` declares no `_missing_`, so `Flags(0)` is an ordinary `aenum.IntFlag` pseudo-member and does not meet the `RecursionError` the same construction hits on the `pcapkit.const.mh` flag enumerations ([#623](https://github.com/JarryShaw/PyPCAPKit/issues/623)). Nothing a caller can reach changed: the three MP_JOIN layouts of [RFC 8684](https://datatracker.ietf.org/doc/html/rfc8684) section 3.2 are chosen from these membership tests, but `mptcp_data_selector` rejects a flagless MP_JOIN with a `FieldError` first, so the `TypeError` was latent, and only by virtue of a guard in another file. Called directly on a flagless parsed segment the dispatcher now raises `ProtocolError` naming an invalid flags combination. One difference *is* observable, a dump-format change rather than a value change: a flagless segment's `connection` renders as the string `Flags::None [0]` where it rendered as the number `0`, in the JSON, tree and PLIST outputs. The value and wire bytes are unchanged; `connection` no longer switches JSON type with the flags (a number for a flagless segment, a string such as `Flags::ACK [2048]` for every other). The literal `None` is a separate rendering defect in the hook `pcapkit.dumpkit.common.make_dumper` installs, which interpolates `o.name` without accounting for a nameless composite member; it is left to its own change and pinned by a test. The committed example dumps are unaffected (`examples/captures/in.pcap` has no flagless segment). Coverage cannot see this fix, since the changed line already executed; the added subtests are the evidence ([#616](https://github.com/JarryShaw/PyPCAPKit/issues/616)). -- **a breaking change to a public attribute.** `Frame.len` and `Frame.cap_len` were filled from *opposite* wire fields depending on the container format, so `frame.len` meant the captured length out of a `.pcap` and the on-wire length out of a `.pcapng`. The PCAP reader moved to match PCAP-NG: `len` is the on-wire length (the record header's `orig_len`) and `cap_len` the octets stored (`incl_len`). **Code reading `frame.len` or `frame.cap_len` from a `.pcap` gets the other field's value than before**, and for a truncated frame that is a different number rather than a relabelling. Which reader to move was a decision, not settled by seniority: the PCAP spelling is the *older* (`c43892af`, 2022-01-11, with the data model's docstrings agreeing a day later), the opposite spelling in `pcapkit.toolkit.pcapng` arrived 15 months later in `25f216f4` (2023-04-27), and both were internally consistent. What settles it is that the *names* are Wireshark's: `epan/dissectors/packet-frame.c` registers `frame.len` as "Frame length on the wire" and `frame.cap_len` as "Frame length stored into the capture file", and raises `frame.len_lt_caplen` expert info (`PI_MALFORMED`/`PI_ERROR`) on `frame_len < cap_len`, which could not be malformed if `len` were the smaller, captured one. Both formats define the underlying fields alike (`pcap-savefile(5)`'s `incl_len`/`orig_len`, `draft-ietf-opsawg-pcapng`'s Captured/Original Packet Length). The internal consumer moved with it: `Frame.read` hands `_decode_next_layer` the octets *present*, now `frame.cap_len`; handing over the on-wire length would be the declared-length-exceeds-available-octets fault of [#554](https://github.com/JarryShaw/PyPCAPKit/issues/554), [#573](https://github.com/JarryShaw/PyPCAPKit/issues/573) and [#594](https://github.com/JarryShaw/PyPCAPKit/issues/594) on every snapped frame. The dumpers are untouched (`PCAPIO` packs its record header from `frame_info.incl_len` and `orig_len`, already right in both readers). It went unnoticed because the two lengths differ only for a snapshot-cut frame and no such fixture existed until [#614](https://github.com/JarryShaw/PyPCAPKit/pull/614) added one (`big_endian.pcap`'s third frame, 1200 octets on the wire against 96 captured), so every earlier assertion compared a value against itself ([#618](https://github.com/JarryShaw/PyPCAPKit/issues/618)). +- `TCP.read` seeded its connection-flag accumulator with `cast('Enum_Flags', 0)`. `typing.cast` is a runtime no-op, so the accumulator began as the plain `int` `0`, and only the `|=` promoting it to a `pcapkit.const.tcp.flags.Flags` member ever changed that. A segment with an all-zero flags octet (an nmap NULL scan) left `self._flags` an `int`, so a membership test raised `TypeError: argument of type 'int' is not a container or iterable`, and `TCP.connection` returned an `int` where it annotates `Flags`. Any flag masked it, which is how it survived 100% statement and branch coverage. The seed is `Flags(0)`, as [#597](https://github.com/JarryShaw/PyPCAPKit/pull/597) already did for the sibling accumulator in `make`; `Flags` declares no `_missing_`, so `Flags(0)` is an ordinary `aenum.IntFlag` pseudo-member and does not meet the `RecursionError` the same construction hits on the `pcapkit.const.mh` flag enumerations ([#623](https://github.com/JarryShaw/PyPCAPKit/issues/623)). The MP_JOIN path was latent: `mptcp_data_selector` rejects a flagless MP_JOIN before [RFC 8684](https://datatracker.ietf.org/doc/html/rfc8684) section 3.2's three layouts are chosen. One dump-format change is observable: a flagless segment's `connection` renders as a string where it rendered as the number `0`, so it no longer switches JSON type with the flags. That string read `Flags::None [0]` until the `pcapkit.dumpkit` fix above ([#616](https://github.com/JarryShaw/PyPCAPKit/issues/616)). +- **a breaking change to a public attribute.** `Frame.len` and `Frame.cap_len` were filled from *opposite* wire fields depending on the container format, so `frame.len` meant the captured length out of a `.pcap` and the on-wire length out of a `.pcapng`. The PCAP reader moved to match PCAP-NG: `len` is the on-wire length (the record header's `orig_len`) and `cap_len` the octets stored (`incl_len`). **Code reading `frame.len` or `frame.cap_len` from a `.pcap` gets the other field's value than before**, and for a truncated frame that is a different number rather than a relabelling. Seniority did not settle it: the PCAP spelling is the *older* (`c43892af`, 2022-01-11) and PCAP-NG's arrived in `25f216f4` (2023-04-27), both internally consistent. The *names* settle it, being Wireshark's: `frame.len` is "Frame length on the wire" and `frame.cap_len` "Frame length stored into the capture file", and `frame.len_lt_caplen` expert info flags `frame_len < cap_len` as malformed, which it could not be if `len` were the smaller, captured one. `Frame.read` hands `_decode_next_layer` the octets *present*, now `frame.cap_len`; handing over the on-wire length would be the declared-length-exceeds-available-octets fault of [#554](https://github.com/JarryShaw/PyPCAPKit/issues/554), [#573](https://github.com/JarryShaw/PyPCAPKit/issues/573) and [#594](https://github.com/JarryShaw/PyPCAPKit/issues/594) on every snapped frame. The dumpers are untouched. The two lengths differ only for a snapshot-cut frame, and no such fixture existed until [#614](https://github.com/JarryShaw/PyPCAPKit/pull/614) ([#618](https://github.com/JarryShaw/PyPCAPKit/issues/618)). - the HIP `PUZZLE` and `SOLUTION` builders derived three wire-format values from the payload value instead of the data model, and all three were wrong. **This changes two public data models and the octets both parameters emit.** The width of `Random #I` and `Puzzle solution #J` came from `int.bit_length()` alone, so every leading zero octet was dropped on re-serialisation: a `SOLUTION` read with `Length = 20` rebuilt as `Length = 6`, a `PUZZLE` with `Length = 12` as `Length = 5`, silently. That width is `RHASH_len / 8` octets [[RFC 7401 Section 5.2.4](https://datatracker.ietf.org/doc/html/rfc7401#section-5.2.4), [RFC 7401 Section 5.2.5](https://datatracker.ietf.org/doc/html/rfc7401#section-5.2.5)], a property of the Responder's HIT Suite, so both data models now carry it as `rhash_len` and both builders prefer it; a new `rhash_len=` keyword states it for a build from scratch, which under HIPv2 is the only place it can come from, since `RSA,DSA/SHA-256` is the REQUIRED HIT Suite [[RFC 7401 Section 5.2.10](https://datatracker.ietf.org/doc/html/rfc7401#section-5.2.10)] and makes the field 32 octets rather than 8. On the mandatory path about one parameter in 256 has a zero top octet and lost it ([#653](https://github.com/JarryShaw/PyPCAPKit/issues/653)). `SOLUTION`'s second contents octet is `Reserved`, "zero when sent, ignored when received" [[RFC 7401 Section 5.2.5](https://datatracker.ietf.org/doc/html/rfc7401#section-5.2.5), and [RFC 5201 Section 5.2.5](https://datatracker.ietf.org/doc/html/rfc5201#section-5.2.5) identically], not the `Lifetime` that only [RFC 7401 Section 5.2.4](https://datatracker.ietf.org/doc/html/rfc7401#section-5.2.4) defines, which pcapkit encoded there as `2^(value - 32)` seconds. It wrote `0x20`, `0x21`, `0x25` or `0x2b` into a field the RFC requires to be zero, and could not write that zero: a conformant `SOLUTION` with `Reserved` `0x00` parsed to `timedelta(0)` and could not be re-serialised, escaping a bare `ValueError` from `math.log2(0.0)` that was not a `BaseError`. The field is renamed `reserved`, defaults to zero, and is carried verbatim across a round trip ([#654](https://github.com/JarryShaw/PyPCAPKit/issues/654)). And neither builder read its own `version` keyword, so `version=1` and `version=2` computed identical lengths, where [RFC 5201 Section 5.2.4](https://datatracker.ietf.org/doc/html/rfc5201#section-5.2.4) and [RFC 5201 Section 5.2.5](https://datatracker.ietf.org/doc/html/rfc5201#section-5.2.5) state both fields as literally 8 bytes and `Length` as literally 12 and 20. Under HIPv1 each builder accepted only a `bit_length()` of 57..64 and built, for anything narrower, a parameter this library's reader rejects -- the shape of [#608](https://github.com/JarryShaw/PyPCAPKit/issues/608), in the `PUZZLE` builder that [#629](https://github.com/JarryShaw/PyPCAPKit/pull/629) never opened ([#655](https://github.com/JarryShaw/PyPCAPKit/issues/655)). The two remaining `math.log2` sites, both in `PUZZLE` where a lifetime is real, raise `ProtocolError` (`(BaseError, ValueError)`, so a caller written around the bare exception still catches it). Migration: `SolutionParameter.lifetime` is now `reserved` and an `int` rather than a `timedelta`; `PuzzleParameter` and `SolutionParameter` gained a required `rhash_len`; and `_make_param_solution` no longer takes `lifetime=`. All three were reachable end to end only once [#608](https://github.com/JarryShaw/PyPCAPKit/issues/608) was fixed in [#629](https://github.com/JarryShaw/PyPCAPKit/pull/629), which removed the parity guard that had failed these rebuilds loudly; the round trip then stopped raising and started quietly emitting a different parameter, which is why they were fixed together ([#653](https://github.com/JarryShaw/PyPCAPKit/issues/653), [#654](https://github.com/JarryShaw/PyPCAPKit/issues/654), [#655](https://github.com/JarryShaw/PyPCAPKit/issues/655)). - every HIP parameter pcapkit emitted was `4 (mod 8)` octets long, because the padding aligned the *contents* rather than the record. Corrected for **45 of the 46** HIP parameters; `LOCATOR_SET` is excluded, below. **This changes the octets those 45 parameters write, and the `length` their data models report.** [RFC 7401 Section 5.2.1](https://datatracker.ietf.org/doc/html/rfc7401#section-5.2.1) requires every TLV parameter to have "a length (that includes the Type and Length fields), which is a multiple of 8 bytes" and gives `Total Length = 11 + Length - (Length + 3) % 8`; all 95 padding sites computed `(8 - (Length % 8)) % 8`, which aligns the contents alone. For every `Length` from 0 to 63 the result was never a multiple of eight and never the RFC's value, erring both ways: at `Length = 4` (a whole `SEQ`, which [RFC 7401 Section 5.3.5](https://datatracker.ietf.org/doc/html/rfc7401#section-5.3.5) puts on every `UPDATE`) pcapkit appended four octets that must not be there, and at `Length = 8` it appended none and went out four short. A conformant peer lands mid-parameter; pcapkit did not, because its reader consumed the same wrong count its writer wrote, which is why no round-trip test could see it and why the fix is asserted against the RFC's arithmetic. 94 of the 95 sites (45 of 46 `PaddingField` callbacks in `pcapkit/protocols/schema/internet/hip.py` and 49 of 49 record lengths in `pcapkit/protocols/internet/hip.py`) are now one `parameter_total_len` and one `parameter_padding_len` ([#651](https://github.com/JarryShaw/PyPCAPKit/issues/651)). `LOCATOR_SET` kept the old expression because two defects there cancelled exactly: its padding callback never received the parameter's `len` (the nested `Locator` schemas share a packet context whose own `len` shadowed it, so it saw 4 for an IPv6 locator), and the parameter's `len` was written in 4-octet units where the RFC's `Length` is a byte count; the result, `4 + 24n + 4`, equalled the RFC total `24n + 8`. [#679](https://github.com/JarryShaw/PyPCAPKit/issues/679) later fixed both sites. `EncryptedParameter`'s `data` length callback is fixed in the same change: it subtracted the sixteen `iv` octets but not the four `reserved` ones, which cancelled the padding's four at `Length % 8` in `{0, 5, 6, 7}`, so correcting the padding alone would have made `ENCRYPTED` four octets too long at all eight residues. `HIP.make`'s `len = total_length // 8 + 4` is not part of the defect: [RFC 7401 Section 5.1.3](https://datatracker.ietf.org/doc/html/rfc7401#section-5.1.3) defines Header Length in 8-byte units excluding the first 8 bytes, so the floor division is exact once each parameter is a multiple of eight. Migration: a `SEQ` parameter's `length` is now 8 where it was 12, and the other 44 move likewise, so code comparing stored output byte for byte or asserting on `Data_*Parameter.length` sees the RFC's values. `LOCATOR_SET` is unchanged in both respects until [#679](https://github.com/JarryShaw/PyPCAPKit/issues/679) (below). `examples/generators/options.py`'s `HIP_COPIES` stayed at two for `R1_COUNTER`'s four-octet `counter` where [RFC 7401 Section 5.2.3](https://datatracker.ietf.org/doc/html/rfc7401#section-5.2.3) requires eight, which this defect had masked, until [#672](https://github.com/JarryShaw/PyPCAPKit/issues/672) widened `counter` and [#679](https://github.com/JarryShaw/PyPCAPKit/issues/679) fixed `LOCATOR_SET`'s `Length` unit, after which [#689](https://github.com/JarryShaw/PyPCAPKit/issues/689) dropped `HIP_COPIES` to one ([#651](https://github.com/JarryShaw/PyPCAPKit/issues/651)). - two HTTP/2 flag defects, one on each side of a round trip. **This changes the octets a reconstructed DATA frame emits, and the dumped value of a flagless frame's flags.** `_make_http_data` was the only one of the six `_make_http_*` methods that never read `frame.flags`, so a DATA frame parsed with `END_STREAM` set rebuilt with the bit clear (flags octet `0x01` to `0x00`): a well-formed frame saying the stream continues where the capture said it ended [[RFC 9113 Section 6.1](https://datatracker.ietf.org/doc/html/rfc9113#section-6.1)]. `PADDED` was re-derived from `pad_len`, and the five sibling builders already restored theirs, so this was a missing read-back ([#652](https://github.com/JarryShaw/PyPCAPKit/issues/652)). Separately, `FrameType.post_process` seeded its flag accumulator with a bare `0`, which `|=` promotes only as a side effect, so a frame with **no** bit set left `__flags__` a plain `int` where the schema and data model declare a `Flags`, and `END_STREAM in __value__` raised `TypeError: argument of type 'int' is not a container or iterable` rather than answering `False`. That is the common case: a `0x00` flags octet is every opening `SETTINGS`, every non-final `DATA` and every non-ACK `PING`. It now seeds `self.Flags(0)`, as the construct path already did at all six of its sites. In the dump, `__value__` was a JSON *number* for a flagless frame and a *string* for every other frame in the same capture; it is consistently a string now, `"Flags::None [0]"` where it read `0`, the same change [#616](https://github.com/JarryShaw/PyPCAPKit/issues/616) made to TCP's `connection`. The seed is guarded because `FrameType.Flags` declares no members and, from Python 3.11, a memberless `enum.Flag` subclass refuses `Flags(0)`, so the five frame schemas that inherit it (`UnassignedFrame`, `PriorityFrame`, `RSTStreamFrame`, `GoawayFrame`, `WindowUpdateFrame`) keep the plain `int`, which nothing observes since all five pass `flags=None`. A DATA round trip is **still** lossy for an unrelated reason found while measuring: three payload length callbacks are mis-parenthesised, so an unpadded `DATA`, `HEADERS` or `PUSH_PROMISE` frame parses with its whole payload dropped ([#668](https://github.com/JarryShaw/PyPCAPKit/issues/668)). ([#652](https://github.com/JarryShaw/PyPCAPKit/issues/652), [#650](https://github.com/JarryShaw/PyPCAPKit/issues/650)). - four MPTCP error messages carried a doubled separator, rendering as `TCP: : [OptNo 30] 1: invalid flags combination`. Cosmetic: the exception type, option number and subtype were right and nothing parses these strings, but an empty field reads as a value that failed to interpolate. Four sites (`_read_mptcp_join` and `_make_mptcp_join` for the invalid flag combination, both guards of `_make_mptcp_dss` for missing required fields), against 28 messages in the same file spelled correctly; now 32 correct against 0 doubled. Pre-existing since 2023-04-10. The new test asserts each message in full, since the existing `assertIn` on the body passes either side of the change ([#649](https://github.com/JarryShaw/PyPCAPKit/issues/649)). - three names appeared in string annotations their own module never imported, so a type checker or documentation build could not resolve them while every test passed: `Any` in `pcapkit/utilities/logging.py`, and in `pcapkit/protocols/schema/internet/ipv6_route.py` both `Protocol` in a `payload:` stub and `Optional` inside a `typing.cast`. All three are now in their module's `TYPE_CHECKING` block, `Protocol` spelled `ProtocolBase as Protocol` as twenty-one sibling schema modules already do. The `cast` case is the one worth naming: `typing.cast` never evaluates its first argument, so no test that runs the line can see a bad name in it. Nothing resolves at runtime that did not before (`TYPE_CHECKING` is `False`); making these annotations resolve at runtime is a separate decision about the idiom. A new `tests/project/test_annotation_names.py` pins the invariant without a type checker: it resolves every string annotation in the package against the names its own module binds, following nested forward references such as `'list["Nested"]'` and treating `Literal` members and `Annotated` metadata as values, and reports exactly those three findings on the unfixed tree. The same change corrects the `MANIFEST.in` comment asserting that `include README.md` was the only thing putting the README in a source distribution and that an sdist without it could not be installed; both halves are false, and the [#619](https://github.com/JarryShaw/PyPCAPKit/pull/619) entry above carried the same claim and is corrected with it ([#642](https://github.com/JarryShaw/PyPCAPKit/issues/642)). -- every PCAP-NG packet block now carries the captured octets it declares. All four Enhanced Packet Blocks of the committed `examples/captures/dhcp.pcapng` reported `len(packet) == 0` against a `captured_len` of 314, 342, 314 and 342, as did the Simple Packet Block and obsolete Packet Block (synthesised; no committed fixture has either). Those three, exactly `PCAPNG.PACKET_TYPES`, are the whole blast radius. The octets were read correctly and thrown away: `PCAPNG.unpack` extracted them from the block schema, and `ProtocolBase.__init__` overwrote the result with `self.packet.payload`. The inherited `packet` splits a protocol at `self.length` octets of header and takes the rest as payload, which holds only for a header followed by its payload; `PCAPNG.length` is the *Block Total Length*, so the split consumed the whole block as header (`len(frame.packet.header) == 348` on a 348-octet block), and a block's trailer (option list, repeated total length) means no `length` could make it right. The fix is at the computation rather than the injection: `PCAPNG.packet` is overridden to take the payload from the block schema's `__payload__` field and the header from the octets ahead of it, summed out of the schema buffers so the three types' payload offsets (28, 12, 28) are not hard-coded. Teaching `ProtocolBase.__init__` to skip a protocol that had set `packet` was rejected: it heals `frame.info.packet` while `frame.packet` still reports the whole block as header. The override is a plain `property`, not the `cached_property` it replaces, which a cross-review on a second model settled: the inherited one caches because it *reads the stream*, where this one only walks buffers already filled, and caching it would let a second `unpack` on one instance return the *first* call's octets; the unit test swaps the block on a live instance and fails `b'payload' != b'cached'` under a cache. `ProtocolBase.packet` gains the contract in its docstring and no executable line. This moves the parse output of every PCAP-NG capture on the success path, so it ships labelled breaking: `frame.info.packet` goes from `b''` to hundreds of octets, `frame.packet.header` from the whole block to the pre-payload prefix. It was also a wire-format defect: `PCAPIO` writes `value.packet` after each 16-octet record header, so dumping those four blocks produced a **104-octet** file whose record headers declared hundreds of octets and delivered none, desynchronising any reader that walks by `incl_len` (pcapkit refused its own output with `ValueError: read length must be non-negative or -1`, and scapy returned 1 packet of 64 octets instead of 5). It round-trips now. `tests/protocols/test_pcapng_regression.py` grows 4 tests to 11 and 3 subtests to 24, with payload expectations derived twice (literals, and a `struct`-only re-parse independent of the code under test) and each synthesised payload a different length modulo 4, so an offset off by a field or a payload that picked up block padding cannot pass by coincidence; shapes no fixture exercised (an option area *after* the captured data, a big-endian section, snapped Enhanced and Simple Packet Blocks) are covered. `pcapkit/protocols/protocol.py` is identical in every coverage column, confirming its change is docstring-only. The uncaught `ValueError` on an EOF-truncated PCAP-NG ([#678](https://github.com/JarryShaw/PyPCAPKit/issues/678)) is untouched: over the 101 whole-percent prefixes `max(1, 1508 * i // 100)` both trees give 96 of that `ValueError`, one `FormatError`, one `ProtocolError` and three parses. `examples/captures/pcapng.txt`, a hand-regenerated legacy reference, still showed the old `packet -> NIL` here; [#685](https://github.com/JarryShaw/PyPCAPKit/issues/685) (below) later removed it from the index rather than regenerating it again ([#646](https://github.com/JarryShaw/PyPCAPKit/issues/646)). -- an unpadded HTTP/2 `DATA`, `HEADERS` or `PUSH_PROMISE` frame parsed with its whole payload silently discarded. Three payload length callbacks in `pcapkit/protocols/schema/application/httpv2.py` put the conditional in the wrong place: a conditional binds looser than `-`, so `pkt['__length__'] - pkt['pad_len'] if pkt['flags']['bit_3'] else 0` groups as `(pkt['__length__'] - pkt['pad_len']) if ... else 0`, and the `else` arm returned `0` ("read no octets") where it meant to subtract `0`. Padding is rare in HTTP/2, so the broken arm was the common one, and nothing raised or warned: `info.data` was `b''`. An unpadded `DATA` frame declaring `b'{"ok":true}\n'` now parses as those twelve octets, and unpadded `HEADERS` and `PUSH_PROMISE` keep their header block fragment. Padded frames are unaffected, since `(A - B) if T else 0` and `A - (B if T else 0)` are both `A - B` for truthy `T`; the padded arm has no off-by-one ([#668](https://github.com/JarryShaw/PyPCAPKit/issues/668) asked that be checked), because `Schema.unpack` decrements `packet['__length__']` by each field's width, so `pad_len`'s own octet is already out by the time the payload field runs. Three sibling sites that were never wrong (`ContinuationFrame.fragment`, `UnassignedFrame.data`, `GoawayFrame.debug`) localise the defect to the grouping and are pinned as controls. Nothing caught it: `test_option_roundtrip_unit` drives every frame type through `make` -> parse -> `make`, where both sides agreed on an empty payload (its generator passes no payload for these frames), and `test_httpv2_frame_readers_cover_successful_frames` uses hand-built schema stubs, so the callback never ran. The same lambda also governs **packing**, which neither the issue nor the first revision noticed and a cross-review on a second model did: `BytesField` consults `length` on both paths, so an unpadded `DATA` frame carrying twelve octets packed to a nine-octet header whose length field declares **21**, body absent (unpadded `HEADERS` and `PUSH_PROMISE` declared 29 against 9 and 33 against 13). pcapkit was *emitting* malformed HTTP/2 that desynchronises any reader walking a stream by that field; a `make` -> parse -> `make` cycle of a non-empty unpadded payload now closes byte-for-byte (`HTTPv2ConstructedFrameDeclaresWhatItWritesUnitTests`). The new `tests/protocols/application/test_httpv2_payload_length_unit.py` reads wire octets (23 tests, 12 subtests over padded, unpadded and padding-only shapes of all three frames plus both `PRIORITY` combinations) and asserts payload *octets*, never length, because a length assertion survives the right width read from the wrong offset; padded cases use non-zero padding `b'\xde\xad\xbe\xef'` rather than the zeros [RFC 9113 Section 6.1](https://datatracker.ietf.org/doc/html/rfc9113#section-6.1) tells a sender to use, so failing to subtract it yields a different byte string. Six wrong fixes written over the schema confirm the tests discriminate, each rejected; the cross-review's own, `max(computed, 1)` on the two `fragment` fields, **passed** the first revision's tests because only `DataFrame` had a padding-only case (a padding-only `HEADERS` frame then leaked one padding octet); the missing padding-only cases and `test_the_padded_arm_reaches_zero_and_is_not_clamped` close it, `0` being a legitimate answer. `EXPECTED_FAILURES` is unmoved (its one HTTP/2 entry `httpv2-frame/PRIORITY` keeps its `CONSTRUCT` status, `PriorityFrame` having no payload field or `PADDED` flag; its cited `httpv2.py:572` is stale, the `header.length != 9` guard now being at `:650`). The construct side needed no change: `_make_http_length` always computed `len(frame.data) + (pad_len + 1 if pad_len else 0)`, and the shared length callback disagreed on both paths. An AST sweep of all 496 package files found one other site of the same shape, `pcapkit/protocols/schema/internet/ipv4.py:336` (`TSOption.remainder`), and it is correct: its `else 0` is intentional because `ts_data` consumes the whole option data area in that arm. Ships labelled breaking: restoring a dropped payload moves the parse output of essentially every HTTP/2 capture, so a golden file or regression baseline that recorded the empty value sees it change, and constructed frames change byte-for-byte, from malformed to correct ([#668](https://github.com/JarryShaw/PyPCAPKit/issues/668)). -- the schema half of every option-like registrar overwrote silently while the parser half of the same call warned. `EnumSchema.register` was a bare `cls.__enum__[code] = schema`, and is the schema half of 14 public registrars, so one `register_ipv4_option` call replacing a built-in named the parser it displaced and said nothing about the schema. `EnumSchema.__init_subclass__` reaches the same registry without calling `register`, so a subclass declared with a `code=` keyword (the documented way to add a schema) stayed silent too; both are guarded, and the declaration hook's two branches are folded into one loop. Presence is a faithful test only because `_EnumRegistry.__missing__` returns a miss without recording it, so parsing an unknown code cannot make the next legitimate registration warn. The seven code-keyed parser registrars (on `ProtocolBase`, `Link`, `Internet`, `Frame`, `PCAPNG`, `Transport` and `SCTP`) now name the displaced entry and its replacement, where they reported only that something was overwritten. Their firing condition is harmonised onto the narrower guard [#681](https://github.com/JarryShaw/PyPCAPKit/pull/681) gave `register_protocol` ([#718](https://github.com/JarryShaw/PyPCAPKit/issues/718), [#726](https://github.com/JarryShaw/PyPCAPKit/pull/726)) rather than presence alone: that key is derived from the value it stores, whereas these take a caller-supplied `code`. Five of the seven (`Link` 7 entries, `Internet` 16, `Frame` 3, `PCAPNG` 3, `SCTP` 2) ship pre-seeded with unresolved `ModuleDescriptor` values, as do `TCP` (4) and `UDP` (3), each keeping its own dict rather than filling the one `ProtocolBase` and abstract `Transport` share and leave at 0 (`Transport.register` raises `UnsupportedCall`). Where a table is pre-seeded, the incumbent may be a two-string descriptor while the replacement is the class it names. Resolution happens earlier, on the incoming argument only (`if isinstance(protocol, ModuleDescriptor): protocol = protocol.klass`), so the guard's `is` comparison measures a resolved class against an unresolved descriptor and warns on re-registering the same class under its own pre-seeded code (`Internet.register(TransType.TCP, TCP)`), a false positive the harmonised guard does not close. `ContextRegistry.register` already raises on a duplicate, and the reassembly and ESP registrars append to lists with no key (duplicate security-association SPIs are designed, resolved by scoring), so none of the three takes a guard. `import pcapkit` holds at one warning, a third-party deprecation, and no `RegistryWarning`, over the 327 registry writes it performs (one being `R1CounterParameter`'s second code, from [#690](https://github.com/JarryShaw/PyPCAPKit/issues/690)), none landing on a key already present ([#692](https://github.com/JarryShaw/PyPCAPKit/issues/692)). -- every EOF-truncated PCAP-NG file raised a bare `ValueError` out of `Extractor`, losing the whole extraction rather than the one truncated block. Block Total Length is cross-checked against its trailing copy and never against the file, so a last block declaring more than the file holds left `PCAPNG.read` seeking *past* the end, and the next block read measured a **negative** remainder that `io.RawIOBase.read` refused. The seek is now clamped to the octets `_read_fileng` returned, with a `ProtocolWarning` naming the overrun, and a tail under the twelve octets a block needs is the quiet `StreamEOFError` that `Extractor.record_frames` already catches (deliberately not a clamp, which would fabricate a frame out of zero padding). A second route to the same shape is closed by a new `nonnegative()` helper, composed into `bounded_option` and `bounded_area` and applied to the eight unwrapped `length - N` spans and eight `__option_padding__` fields: `_TextField.__call__` builds `f'{length}s'` unconditionally, so a negative became `'-8s'` and `struct.calcsize` raised an exception caught nowhere a frame loop looks. Across all 1,509 octet boundaries of `examples/captures/dhcp.pcapng`, 6 levels parsed before and 1,495 do now; the 14 that still raise are twelve files too short to hold a block and two cuts into the Interface Description Block's `if_tsresol` option, so none costs a frame that was in the file. A mutation fuzz removes both of [#678](https://github.com/JarryShaw/PyPCAPKit/issues/678)'s families; two others remain, unchanged: `ValueError: N is not a valid BlockType` (filed as [#701](https://github.com/JarryShaw/PyPCAPKit/issues/701); an unassigned block type raises from `aenum` inside `EnumField.post_process`, so the registry's `UnknownBlock` default is unreachable) and `MemoryError` from [#593](https://github.com/JarryShaw/PyPCAPKit/pull/593)'s 32-bit band. Three bare exceptions in `SystemdJournalExportBlock.post_process`, reachable from valid input, are fixed with it: the body's NUL alignment padding was read as a field *name*, so every journal entry of unaligned length raised `struct.error` (a NUL-only line now ends the entry); a binary field's declared length was unbounded against its entry (`OverflowError` at `2**63` and above, silently wrong below; now clamped and reported); and a non-UTF-8 name, key or value raised `UnicodeDecodeError` (now `errors='replace'` and reported, as `StringField` does). The silent loss of every field after a binary one is filed as [#704](https://github.com/JarryShaw/PyPCAPKit/issues/704). The five unclamped non-packet option areas keep the per-block framing assumption [#676](https://github.com/JarryShaw/PyPCAPKit/pull/676)'s note describes. Also here, the fifth and last registrar with no overwrite guard: `Option.register` now reports a displaced option schema as a `RegistryWarning` naming every namespace the `ns='opt'` fan-out displaced something in, once per registration, tested by membership (subscripting the per-namespace `defaultdict` would insert `UnknownOption` for a code nobody registered). The condition is identity-based like the seven parser registrars and [#681](https://github.com/JarryShaw/PyPCAPKit/pull/681)'s guard for `register_protocol` ([#718](https://github.com/JarryShaw/PyPCAPKit/issues/718), [#726](https://github.com/JarryShaw/PyPCAPKit/pull/726)); a namespace the call itself creates is exempt. Flooring the three decryption-secrets payloads that read `__length__` *whole* was tried and reverted: `Schema.pack` leaves that key at `-1` for "unknown", so it packed nothing and turned `pcapng-secrets/TLS_Key_Log` and `.../WireGuard_Key_Log` from `MISMATCH` to `OK` (an empty payload equals an empty payload). Ships labelled breaking: well-formed captures are byte-identical, but any truncated PCAP-NG now yields the frames before the cut where it raised, so a caller reading "extraction raised" as "this file is unusable" gets a partial result whose last frame may carry zero-padded octets, and eight short-file depths move from `ProtocolError: unknown byteorder magic` to `StreamEOFError`. The [#683](https://github.com/JarryShaw/PyPCAPKit/pull/683) drift [#685](https://github.com/JarryShaw/PyPCAPKit/issues/685) tracks is unchanged ([#678](https://github.com/JarryShaw/PyPCAPKit/issues/678)). +- every PCAP-NG packet block now carries the captured octets it declares. All four Enhanced Packet Blocks of the committed `examples/captures/dhcp.pcapng` reported `len(packet) == 0` against a `captured_len` of 314, 342, 314 and 342, as did the Simple Packet Block and obsolete Packet Block (synthesised; no committed fixture has either). Those three, exactly `PCAPNG.PACKET_TYPES`, are the whole blast radius. The octets were read correctly and thrown away: `PCAPNG.unpack` extracted them from the block schema, and `ProtocolBase.__init__` overwrote the result with `self.packet.payload`. The inherited `packet` splits a protocol at `self.length` octets of header and takes the rest as payload, which holds only for a header followed by its payload; `PCAPNG.length` is the *Block Total Length*, so the split consumed the whole block as header (`len(frame.packet.header) == 348` on a 348-octet block), and a block's trailer (option list, repeated total length) means no `length` could make it right. The fix is at the computation rather than the injection: `PCAPNG.packet` is overridden to take the payload from the block schema's `__payload__` field and the header from the octets ahead of it, summed out of the schema buffers so the three types' payload offsets (28, 12, 28) are not hard-coded. Teaching `ProtocolBase.__init__` to skip a protocol that had set `packet` was rejected: it heals `frame.info.packet` while `frame.packet` still reports the whole block as header. The override is a plain `property`, not the `cached_property` it replaces: the inherited one caches because it *reads the stream*, where this one only walks buffers already filled, and caching it would let a second `unpack` on one instance return the *first* call's octets. This moves the parse output of every PCAP-NG capture on the success path, so it ships labelled breaking: `frame.info.packet` goes from `b''` to hundreds of octets, `frame.packet.header` from the whole block to the pre-payload prefix. It was also a wire-format defect: `PCAPIO` writes `value.packet` after each record header, so a dump declared hundreds of octets per record and delivered none, desynchronising any reader that walks by `incl_len`. It round-trips now. The uncaught `ValueError` on an EOF-truncated PCAP-NG ([#678](https://github.com/JarryShaw/PyPCAPKit/issues/678)) is untouched, and [#685](https://github.com/JarryShaw/PyPCAPKit/issues/685) (below) later removed the stale `examples/captures/pcapng.txt` from the index ([#646](https://github.com/JarryShaw/PyPCAPKit/issues/646)). +- an unpadded HTTP/2 `DATA`, `HEADERS` or `PUSH_PROMISE` frame parsed with its whole payload silently discarded. Three payload length callbacks in `pcapkit/protocols/schema/application/httpv2.py` put the conditional in the wrong place: a conditional binds looser than `-`, so `pkt['__length__'] - pkt['pad_len'] if pkt['flags']['bit_3'] else 0` groups as `(pkt['__length__'] - pkt['pad_len']) if ... else 0`, and the `else` arm returned `0` ("read no octets") where it meant to subtract `0`. Padding is rare in HTTP/2, so the broken arm was the common one, and nothing raised or warned: `info.data` was `b''`. An unpadded `DATA` frame declaring `b'{"ok":true}\n'` now parses as those twelve octets, and unpadded `HEADERS` and `PUSH_PROMISE` keep their header block fragment. Padded frames are unaffected, since `(A - B) if T else 0` and `A - (B if T else 0)` are both `A - B` for truthy `T`; the padded arm has no off-by-one ([#668](https://github.com/JarryShaw/PyPCAPKit/issues/668) asked that be checked), because `Schema.unpack` decrements `packet['__length__']` by each field's width, so `pad_len`'s own octet is already out by the time the payload field runs. The same lambda also governs **packing**: an unpadded `DATA` frame carrying twelve octets packed to a nine-octet header whose length field declared **21**, body absent, so pcapkit was *emitting* malformed HTTP/2; a `make` -> parse -> `make` cycle of a non-empty unpadded payload now closes byte-for-byte. The one other conditional of this shape, `TSOption.remainder` in `pcapkit/protocols/schema/internet/ipv4.py`, is correct: its `else 0` is intentional because `ts_data` consumes the whole option data area in that arm. The new tests use non-zero padding rather than the zeros [RFC 9113 Section 6.1](https://datatracker.ietf.org/doc/html/rfc9113#section-6.1) tells a sender to use, so failing to subtract it yields a different byte string. Ships labelled breaking: restoring a dropped payload moves the parse output of essentially every HTTP/2 capture, and constructed frames change byte-for-byte, from malformed to correct ([#668](https://github.com/JarryShaw/PyPCAPKit/issues/668)). +- the schema half of every option-like registrar overwrote silently while the parser half of the same call warned. `EnumSchema.register` was a bare `cls.__enum__[code] = schema`, and is the schema half of 14 public registrars, so one `register_ipv4_option` call replacing a built-in named the parser it displaced and said nothing about the schema. `EnumSchema.__init_subclass__` reaches the same registry without calling `register`, so a subclass declared with a `code=` keyword (the documented way to add a schema) stayed silent too; both are guarded, and the declaration hook's two branches are folded into one loop. Presence is a faithful test only because `_EnumRegistry.__missing__` returns a miss without recording it, so parsing an unknown code cannot make the next legitimate registration warn. The seven code-keyed parser registrars (on `ProtocolBase`, `Link`, `Internet`, `Frame`, `PCAPNG`, `Transport` and `SCTP`) now name the displaced entry and its replacement, where they reported only that something was overwritten. Their firing condition is harmonised onto the narrower guard [#681](https://github.com/JarryShaw/PyPCAPKit/pull/681) gave `register_protocol` ([#718](https://github.com/JarryShaw/PyPCAPKit/issues/718), [#726](https://github.com/JarryShaw/PyPCAPKit/pull/726)) rather than presence alone: that key is derived from the value it stores, whereas these take a caller-supplied `code`. Where a table is pre-seeded with an unresolved `ModuleDescriptor`, the guard's `is` comparison measures the resolved incoming class against the descriptor, so re-registering the same class under its own pre-seeded code (`Internet.register(TransType.TCP, TCP)`) still warns, a false positive the harmonised guard does not close. `ContextRegistry.register` already raises on a duplicate, and the reassembly and ESP registrars append to lists with no key, so none of the three takes a guard. `import pcapkit` emits no `RegistryWarning`: none of its 327 registry writes (one being `R1CounterParameter`'s second code, from [#690](https://github.com/JarryShaw/PyPCAPKit/issues/690)) lands on a key already present ([#692](https://github.com/JarryShaw/PyPCAPKit/issues/692)). +- every EOF-truncated PCAP-NG file raised a bare `ValueError` out of `Extractor`, losing the whole extraction rather than the one truncated block. Block Total Length is cross-checked against its trailing copy and never against the file, so a last block declaring more than the file holds left `PCAPNG.read` seeking *past* the end, and the next block read measured a **negative** remainder that `io.RawIOBase.read` refused. The seek is now clamped to the octets `_read_fileng` returned, with a `ProtocolWarning` naming the overrun, and a tail under the twelve octets a block needs is the quiet `StreamEOFError` that `Extractor.record_frames` already catches (deliberately not a clamp, which would fabricate a frame out of zero padding). A second route to the same shape is closed by a new `nonnegative()` helper, composed into `bounded_option` and `bounded_area` and applied to the eight unwrapped `length - N` spans and eight `__option_padding__` fields: `_TextField.__call__` builds `f'{length}s'` unconditionally, so a negative became `'-8s'` and `struct.calcsize` raised an exception caught nowhere a frame loop looks. Both of [#678](https://github.com/JarryShaw/PyPCAPKit/issues/678)'s failure families are gone; two others remain: `ValueError: N is not a valid BlockType` (filed as [#701](https://github.com/JarryShaw/PyPCAPKit/issues/701)) and `MemoryError` from [#593](https://github.com/JarryShaw/PyPCAPKit/pull/593)'s 32-bit band. Three bare exceptions in `SystemdJournalExportBlock.post_process`, reachable from valid input, are fixed with it: the body's NUL alignment padding was read as a field *name* (`struct.error`; a NUL-only line now ends the entry); a binary field's declared length was unbounded against its entry (`OverflowError`; now clamped and reported); and a non-UTF-8 name, key or value raised `UnicodeDecodeError` (now `errors='replace'` and reported). The silent loss of every field after a binary one is filed as [#704](https://github.com/JarryShaw/PyPCAPKit/issues/704). The five unclamped non-packet option areas keep the per-block framing assumption [#676](https://github.com/JarryShaw/PyPCAPKit/pull/676)'s note describes. Also here, the fifth and last registrar with no overwrite guard: `Option.register` now reports a displaced option schema as a `RegistryWarning` naming every namespace the `ns='opt'` fan-out displaced something in, tested by membership (subscripting the per-namespace `defaultdict` would insert `UnknownOption` for a code nobody registered). The condition is identity-based like the seven parser registrars and [#681](https://github.com/JarryShaw/PyPCAPKit/pull/681)'s guard for `register_protocol` ([#718](https://github.com/JarryShaw/PyPCAPKit/issues/718), [#726](https://github.com/JarryShaw/PyPCAPKit/pull/726)); a namespace the call itself creates is exempt. Flooring the three decryption-secrets payloads that read `__length__` *whole* was tried and reverted, since `Schema.pack` leaves that key at `-1` for "unknown". Ships labelled breaking: well-formed captures are byte-identical, but any truncated PCAP-NG now yields the frames before the cut where it raised, the last of which may carry zero-padded octets, and eight short-file depths move from `ProtocolError: unknown byteorder magic` to `StreamEOFError`. The [#683](https://github.com/JarryShaw/PyPCAPKit/pull/683) drift [#685](https://github.com/JarryShaw/PyPCAPKit/issues/685) tracks is unchanged ([#678](https://github.com/JarryShaw/PyPCAPKit/issues/678)). - HIP's `R1_COUNTER`/`R1_Counter` parameter packed 12 octets where [RFC 7401 Section 5.2.3](https://datatracker.ietf.org/doc/html/rfc7401#section-5.2.3) requires 16, and `LOCATOR_SET` declared its `Length` in 4-octet units where [RFC 7401 Section 5.2.1](https://datatracker.ietf.org/doc/html/rfc7401#section-5.2.1)'s `Length` is a byte count -- two independent defects, fixed together because the evidence for either needed the other out of the way. `R1CounterParameter`'s `counter` was a `UInt32Field` where the RFC states the width twice, "8 bytes" in the diagram and "a 64-bit unsigned integer" in prose; both versions whose *packing* reaches the class, `R1_Counter` (128) and `R1_COUNTER` (129), packed a 12-octet record at `4 (mod 8)`. It is a `UInt64Field` now and the record is 16 octets ([#672](https://github.com/JarryShaw/PyPCAPKit/issues/672)). `LocatorSetParameter`'s `Length` was `sum(Locator.len)`, in the 4-octet units [RFC 8046 Section 4](https://datatracker.ietf.org/doc/html/rfc8046#section-4) gives `Locator Length`; a set of *n* plain IPv6 locators declared `4n` octets against `24n` present, and `Schema.unpack` handed the nested `ListField` only the declared octets, so `n = 2` and `n = 5` parsed one truncated locator and left the rest unconsumed. `HIP._make_param_locator_set` now sums `8 + Locator.len * 4` per locator ([#679](https://github.com/JarryShaw/PyPCAPKit/issues/679)). A second defect in the same parameter's padding, which [#651](https://github.com/JarryShaw/PyPCAPKit/issues/651) left alone because it cancelled this one, is fixed alongside: the nested `Locator` schemas share the parameter's packet context and pack their own `len` over it, so `padding`, evaluated after the list, saw 4 rather than the real length. A new `locator_set_len_callback` snapshots `Length` under a private key before any locator packs, and `locator_set_padding_len` reads that snapshot. The `counter` defaulted to `0` and the width defect could not be told apart from it, so the fixture now overrides both `R1_COUNTER` codes with `counter=0xaabbccdd`; new tests are `test_hip_r1_counter_width_unit.py` and `test_hip_locator_set_length_unit.py`. The `hip-parameter/R1_Counter` `EXPECTED_FAILURES` entry (code 128 registered no schema of its own, since `R1CounterParameter` declared only `code=129`) was deleted once that registry defect was fixed as [#690](https://github.com/JarryShaw/PyPCAPKit/issues/690), leaving 43 entries. `HIP_COPIES` dropped to one as [#689](https://github.com/JarryShaw/PyPCAPKit/issues/689), once [#672](https://github.com/JarryShaw/PyPCAPKit/issues/672) and [#679](https://github.com/JarryShaw/PyPCAPKit/issues/679) left it routing around nothing. - `httpv1.HTTP` raised a bare exception for any payload that is not an HTTP/1 message, at four sites: the header/body split in `read`, the header's CRLF split and the start-line split in `_read_http_header`, and `item[1]` on a field line with no colon. `HTTP._guess_version` falls through on `ProtocolError` alone, so the HTTP/1 attempt aborted the guess and the HTTP/2 arm below was dead code for all four input classes. All four now raise `ProtocolError`, which each method's `Raises:` section documented, and a test pins the invariant (nothing escapes `httpv1.HTTP` that is not a `ProtocolError`), so a fifth bare-raising site would fail there. An obs-fold continuation line [[RFC 9112 Section 5.2](https://datatracker.ietf.org/doc/html/rfc9112#section-5.2)] is legal HTTP/1 and is now unfolded, the RFC's remedy, rather than read as a field of its own (with no colon it raised that `IndexError`; with one it parsed into a spurious extra field). Reaching the HTTP/2 arm exposed a payload under nine octets to `httpv2.HTTP`, where it usually fails inside the schema with a bare `struct.error` (not a `ValueError`, `ProtocolError` or pcapkit `StructError`, so uncatchable); `read` converts it and `_guess_version` suppresses it on the last arm only, since an arm that is not last hands the payload onward when it swallows. Separately, `PayloadField.protocol` looked a `str` up in `__proto__` as given while the registry is keyed on the upper-cased class name, so a name not already upper-case resolved to `None` and yielded `Raw`; it now folds case and warns with `RegistryWarning` for a genuinely unregistered name, and `__init__` assigns through that property instead of writing `_protocol` past it ([#787](https://github.com/JarryShaw/PyPCAPKit/issues/787)). - a systemd journal entry whose last field lacks the format's mandatory trailing newline let the PCAP-NG block's 32-bit alignment padding be read as part of that field's value, with no exception or warning: `b'MESSAGE=hello'` plus three NULs returned `MESSAGE == 'hello\x00\x00\x00'`. The padding-only-line guard from [#678](https://github.com/JarryShaw/PyPCAPKit/issues/678) catches padding only on a line of its own, which needs the preceding field to have ended with a real newline; without one `readline()` runs through the value into the padding. Reachable on text fields only (a binary field's length-prefixed read is never in play, and a binary field *name* on such a line fails its own length-prefix read regardless). The fix is tolerant: block padding is 0-3 octets, so at most the trailing three octets of a terminator-less line are stripped, with a `SchemaWarning`, and anything past that is left as data. The gate reads the *raw* line, since a real trailing newline, or any other raw last octet including ASCII whitespace, proves the padding is zero ([#794](https://github.com/JarryShaw/PyPCAPKit/issues/794)). `examples/generators/pcapng.py`'s `_journal_entry` cited a bare line number for the defect its alignment workaround avoids and mis-attributed the fix; it now names [#678](https://github.com/JarryShaw/PyPCAPKit/issues/678)'s `not line.strip(b'\x00')` guard and the mechanism ([#791](https://github.com/JarryShaw/PyPCAPKit/issues/791)). - `httpv2.HTTP.read` tested only the 24-bit declared length off the wire (`schema.length < 9`), never how many octets the buffer held, so a frame backed by far fewer real octets than declared parsed and reported the declared, attacker-controlled length. Sweeping the declared length against a four-octet buffer gave non-monotone accept/reject (9, 15, 65535 and 16777215 parsed, 10 and 100 did not), an artifact of `_read_http_settings`'s own `(declared - 9) % 6` check. `HTTP.unpack` now rejects a non-empty buffer under nine octets before the schema layer runs, preserving `StreamEOFError` for a genuinely exhausted implicit-length stream, and `read`'s guard also requires the declared length not to exceed the buffer. A buffer that clears nine octets can still carry a frame type whose fixed fields exceed what is left (GOAWAY at 9-16 octets, PUSH_PROMISE at 9-12, padded DATA, HEADERS and PUSH_PROMISE) and crash the schema layer with a bare `struct.error`; that root cause is in the generic `Schema.unpack`/ `FieldBase.length` machinery shared by ten schema modules, out of scope and filed as [#805](https://github.com/JarryShaw/PyPCAPKit/issues/805), which is why `_guess_version`'s last arm keeps suppressing `struct.error` alongside `ProtocolError` ([#802](https://github.com/JarryShaw/PyPCAPKit/pull/802)). - `SystemdJournalExportBlock.post_process` skipped a binary field's one trailing newline with a bare `entry_data.read()`, which reads to **EOF**: the next `readline()` returned `b''` and ended the entry, discarding every field behind the first binary one. The skip is bound to `entry_data.read(1)`; a missing or non-`b'\n'` octet ends the entry with a `SchemaWarning` instead of resynchronising at the wrong offset. `test_journal_fields_following_a_binary_field_are_not_discarded` fails against the unfixed code with `MissingKeyError: 'SECOND'` ([#704](https://github.com/JarryShaw/PyPCAPKit/issues/704)). - the same block's entries were split on a bare `b'\n\n'`, which shreds a binary field whose value contains that pair. Walking the entry instead of splitting it introduced a regression on first review: a `malformed` flag broke the *outer* per-block loop on a bad terminator, ending the whole block and dropping every well-formed entry behind it. Fixed by dropping `malformed`, leaving the per-entry `break` as the only effect. A second gap closed in the same change: a trailing separator on the block's last octet was indistinguishable from EOF and swallowed, so a rebuild wrote `length` one octet short ([#723](https://github.com/JarryShaw/PyPCAPKit/issues/723)). -- **a breaking change to** four `.get()`-backed enum fields: TCP/UDP/SCTP's `PortEnumField` and PCAP-NG's `OptionEnumField` minted a member through `aenum.extend_enum` for every value no row or documented span covers (111 calls on `http.pcap`, all ephemeral ports). All four now peek at what `.get()` would consult and mint only once the value is unresolvable; a genuine miss gets `EnumField._unregistered_member`, a real member of the same registry built through its storage base's `__new__`, so `isinstance` holds and the lookup table stops growing. `extend_enum` calls drop 111 to 0 on `http.pcap` and 15 to 2 on `many_interfaces.pcapng` (both from the untouched `_missing_`). Riding along: PCAP-NG option namespaces stop cross-contaminating: once a code was minted under `opt`, `OptionType.get` merged `opt` over the requested namespace and 6 of the other 7 namespaces answered `opt_unknown` for it; each now gets its own `_unknown`. **Breaking** because [#575](https://github.com/JarryShaw/PyPCAPKit/issues/575) asked for byte-identical output across the sample captures, as [#420](https://github.com/JarryShaw/PyPCAPKit/pull/420) and [#427](https://github.com/JarryShaw/PyPCAPKit/pull/427) did, and this deliberately misses that bar: 14 of 23 captures change, entirely from the new `` rendering plus the PLIST escaping above; normalising both leaves all 14 byte-identical. A `pickle` round-trip of a resolved unassigned member worked before (it was minted, so registered) and would have broken silently after; `_unregistered_member` now installs a `__reduce_ex__` rebuilding an equivalent unregistered member, checked on protocols 0-5 under both picklers and `copy`/`deepcopy` ([#575](https://github.com/JarryShaw/PyPCAPKit/issues/575)). -- `httpv2._guess_version` tried to parse a stream as HTTP/2 and read failure as "not HTTP/2", rather than identifying the version first, so a genuine connection preface followed by a real `SETTINGS` frame raised `ProtocolError: unknown HTTP version`. It now identifies before it parses: a preface is recognised as HTTP/2 outright, skipped rather than fed to `httpv2`, and counted as header so `info` stays byte-identical to reading the following frame alone (otherwise the injected `packet=self.packet.payload`, sliced from octet 9 of a buffer whose frame starts at 24, reported preface remnants as payload). A preface with no frame behind it raises `ProtocolError: HTTP/2: connection preface with no frame`; a bare mid-stream `SETTINGS` frame, an `Upgrade: h2c` request and non-HTTP input are unchanged, the first two left undecidable on purpose (`Upgrade: h2c` needs per-connection state a single payload cannot carry, and a mid-stream frame has no safe heuristic, since a `type <= 9` guess misfires on binary HTTP/1 bodies). Protochain over all 23 sample captures (1,604 frames, 231 HTTP-bearing) is byte-identical to the unfixed tree ([#800](https://github.com/JarryShaw/PyPCAPKit/issues/800)). -- `TCP.__proto__` bound `httpv1.HTTP` directly for ports 80 and 8080, so a segment on either port was HTTP/1 by assertion of the port alone; both versions share those ports, so the payload has to decide. Repointed to the generic `pcapkit.protocols.application.http.HTTP` proxy, whose identification became positive only once [#800](https://github.com/JarryShaw/PyPCAPKit/issues/800) and [#814](https://github.com/JarryShaw/PyPCAPKit/pull/814) landed, which is why this waited. `udp.py` already bound the proxy for both ports; this removes the asymmetry its docstring and `docs/source/pep.rst` documented as an open request (prose-only on that side). Protochain over all 23 captures (1,604 frames) is *not* byte-identical, and that is the fix: all 231 HTTP/1.1 frames keep their chain, and nine frames in `options-transport.pcap` change `Ethernet:IPv4:TCP:Raw` to `Ethernet:IPv4:TCP:HTTP/2` -- genuine HTTP/2 frames this library's own `httpv2.HTTP.make` built, previously refused by the HTTP/1 parser. `_guess_version`'s entry count over the corpus goes 0 to 252 (231 HTTP/1.1, 9 HTTP/2, 12 fall-throughs that stay `Raw`). Not labelled breaking, but not free: TCP:80/8080 traffic that is neither valid HTTP/1 nor preface-carrying is now exposed to the fall-through arm, where the direct `httpv1` binding left it `Raw` regardless; a caller depending on that is the one who would notice ([#682](https://github.com/JarryShaw/PyPCAPKit/issues/682)). +- **a breaking change to** four `.get()`-backed enum fields: TCP/UDP/SCTP's `PortEnumField` and PCAP-NG's `OptionEnumField` minted a member through `aenum.extend_enum` for every value no row or documented span covers (111 calls on `http.pcap`, all ephemeral ports). All four now peek at what `.get()` would consult and mint only once the value is unresolvable; a genuine miss gets `EnumField._unregistered_member`, a real member of the same registry built through its storage base's `__new__`, so `isinstance` holds and the lookup table stops growing. Riding along: PCAP-NG option namespaces stop cross-contaminating: once a code was minted under `opt`, `OptionType.get` merged `opt` over the requested namespace and 6 of the other 7 namespaces answered `opt_unknown` for it; each now gets its own `_unknown`. **Breaking** because [#575](https://github.com/JarryShaw/PyPCAPKit/issues/575) asked for byte-identical output across the sample captures, as [#420](https://github.com/JarryShaw/PyPCAPKit/pull/420) and [#427](https://github.com/JarryShaw/PyPCAPKit/pull/427) did, and this deliberately misses that bar: 14 of 23 captures change, entirely from the new `` rendering plus the PLIST escaping above; normalising both leaves all 14 byte-identical. So that a resolved unassigned member still pickles, `_unregistered_member` installs a `__reduce_ex__` rebuilding an equivalent unregistered member ([#575](https://github.com/JarryShaw/PyPCAPKit/issues/575)). +- `httpv2._guess_version` tried to parse a stream as HTTP/2 and read failure as "not HTTP/2", rather than identifying the version first, so a genuine connection preface followed by a real `SETTINGS` frame raised `ProtocolError: unknown HTTP version`. It now identifies before it parses: a preface is recognised as HTTP/2 outright, skipped rather than fed to `httpv2`, and counted as header so `info` stays byte-identical to reading the following frame alone (otherwise the injected `packet=self.packet.payload`, sliced from octet 9 of a buffer whose frame starts at 24, reported preface remnants as payload). A preface with no frame behind it raises `ProtocolError: HTTP/2: connection preface with no frame`; a bare mid-stream `SETTINGS` frame, an `Upgrade: h2c` request and non-HTTP input are unchanged, the first two left undecidable on purpose (`Upgrade: h2c` needs per-connection state a single payload cannot carry, and a mid-stream frame has no safe heuristic, since a `type <= 9` guess misfires on binary HTTP/1 bodies). Protochain over the sample captures is unchanged ([#800](https://github.com/JarryShaw/PyPCAPKit/issues/800)). +- `TCP.__proto__` bound `httpv1.HTTP` directly for ports 80 and 8080, so a segment on either port was HTTP/1 by assertion of the port alone; both versions share those ports, so the payload has to decide. Repointed to the generic `pcapkit.protocols.application.http.HTTP` proxy, whose identification became positive only once [#800](https://github.com/JarryShaw/PyPCAPKit/issues/800) and [#814](https://github.com/JarryShaw/PyPCAPKit/pull/814) landed, which is why this waited. `udp.py` already bound the proxy for both ports; this removes the asymmetry its docstring and `docs/source/contributing/pep.rst` documented as an open request. Over the sample captures every HTTP/1.1 frame keeps its chain, and nine genuine HTTP/2 frames in `options-transport.pcap` change `Ethernet:IPv4:TCP:Raw` to `Ethernet:IPv4:TCP:HTTP/2`. Not labelled breaking, but not free: TCP:80/8080 traffic that is neither valid HTTP/1 nor preface-carrying is now exposed to the fall-through arm, where the direct `httpv1` binding left it `Raw` regardless; a caller depending on that is the one who would notice ([#682](https://github.com/JarryShaw/PyPCAPKit/issues/682)). - **a breaking change to** 8 more bespoke registries: step 2 of [#860](https://github.com/JarryShaw/PyPCAPKit/issues/860), PR 1 of 2 (`AppType` is PR 2, [#874](https://github.com/JarryShaw/PyPCAPKit/pull/874) below), bringing `StatusCode`, `ReturnCode`, `ResponseKind`, `GroupingInformation`, `OptionType`, `FEATCode`, `Command` and `Method` onto `EnumRegistry` and applying [#775](https://github.com/JarryShaw/PyPCAPKit/issues/775)'s mint/unmint ruling to their `get()` as well as `_missing_`. The first five convert 12 unambiguous placeholder branches (`Unassigned`, `Unknown`, `opt_unknown`) to `_unregistered_member`; the three with a custom `__new__` (`StatusCode`, `ReturnCode`, `OptionType`) get an override reconstructing the attributes the base's generic helper would leave unset, and `StatusCode`/`ReturnCode`'s hand-written `get()`, still on the retired `default == -1` convention, is replaced by the base's (no caller relied on the old form). `OptionType` keeps its own `get()` for its multi-namespace dispatch, but a round-2 review found it still minted on both its int/namespace and `str` paths, and the live pcapng parse path (`PCAPNG._make_pcapng_options`) calls it with wire bytes, so parsing an undeclared option code still registered a permanent member; both paths now build an unregistered member. `FEATCode`, `Command` and `Method` mint the literal wire value as its own name rather than any placeholder; per the owner's ruling (*"get will not have sufficient information to create new ones"*: `Command` needs `feat`/`desc`/`type`/`conf` and `Method` needs `safe`/`idempotent`, which a bare wire string lacks), both `_missing_` and each class's own `get()` (a second, independent mint site) now build an unregistered member. `FEATCode`'s crawler now declares all 15 real FEAT-code values from the live IANA table instead of minting 10 of them as a side effect of evaluating `Command`'s rows at import time, the import-time-mutation shape [#861](https://github.com/JarryShaw/PyPCAPKit/pull/861) removed from `FilterType`; pinned count-agnostically so a future IANA update cannot fail a correct regeneration. Untouched, per rulings: `CommandType` stays `IntFlag` (real `A|P` composites in the generated data), and `TransportProtocol`'s `auto()` renumbering and `AppType` are PR 2's. `pcapkit/protocols/schema/misc/pcapng.py`'s `OptionEnumField.post_process` docstring is corrected: it still bypasses `OptionType.get()` directly, now for `pickle` round-trip safety ([#860](https://github.com/JarryShaw/PyPCAPKit/issues/860) already stops the minting that used to be the reason), since neither of `OptionType`'s two paths gets both pickling and correct rendering right at once ([#869](https://github.com/JarryShaw/PyPCAPKit/pull/869)). - `register` on `ProtocolBase`, `Frame`, `PCAPNG`, `SCTP`, `Link`, `Internet` and the concrete `Transport` subclasses (`TCP`, `UDP`) now raises `RegistryError` for a non-class argument, where it raised a bare `TypeError` ("issubclass() arg 1 must be a class") from inside `abc`. Each guard gains an explicit `isinstance(protocol, type)` test ahead of its `issubclass`, after the `ModuleDescriptor` is unwrapped, so a descriptor naming a non-class attribute is rejected too. All seven `Raises:` clauses already promised `RegistryError` if `protocol` "is not a ... subclass", which a non-class satisfied in spirit but not in fact; they now read "is not a class, or not a ... subclass". No guard's target class changes, and a wrong class still raises `RegistryError` as before. `Transport.register` on `Transport` itself still raises `UnsupportedCall` first; only its subclasses reach the guard. Not breaking for exception handling: `RegistryError` subclasses `TypeError` through `BaseError`, so `except TypeError` still catches it, and the type and message text are the only things a handler can key on that change. Two observable side effects do come with it, because `BaseError` is loud by default: the non-class path now emits pcapkit's one `CRITICAL` log record, and outside devmode it installs `sys.excepthook` and `threading.excepthook`, neither of which the bare `TypeError` from `abc` did. That is the wrong-class path's existing behaviour, so the fix makes the two consistent rather than introducing something new. Follows the same fix for the foundation registrars ([#1021](https://github.com/JarryShaw/PyPCAPKit/issues/1021)) ([#1026](https://github.com/JarryShaw/PyPCAPKit/issues/1026)). - `PCAPNG`'s `ns_dnsname`, `ns_dnsIP4addr` and `ns_dnsIP6addr` scope guards, on both the read and the make side, now say the option "must be in Name Resolution Block", which is the block they check; they used to name the systemd(1) Journal Export Block, with a literal `:manpage:` role in the text. `register_record`'s `RegistryWarning` likewise now says "name resolution record already registered". The make-side `ns_dnsIP4addr`/`ns_dnsIP6addr` messages also drop the lowercase `ns_dnsip4addr`/`ns_dnsip6addr` spelling for the `OptionType` member name. Two more guards named the wrong option: `_make_option_if_rxspeed`'s scope and duplicate guards now say `[if_rxspeed]` rather than `[if_txspeed]`, and `_read_option_epb_queue`'s length guard says `[epb_queue]` rather than `[epb_packetid]`. Only message text changes; the exception and warning types are unchanged ([#1038](https://github.com/JarryShaw/PyPCAPKit/issues/1038)). @@ -227,9 +227,9 @@ This resolves [#548](https://github.com/JarryShaw/PyPCAPKit/issues/548), which r #### Fixed - thirteen `re.sub` sites under `pcapkit/vendor/` passed `re.MULTILINE` as the fourth *positional* argument, which is `count`, not `flags`: substitution capped at 8 matches, and a `DeprecationWarning` on Python 3.13+ (a future `TypeError`). Found by an AST sweep rather than grep, since 8 of the 13 put the flag on a continuation line. The flag moves to `flags=` at all 13 sites (`default.py` plus one each in `hip/eddsa_curve.py`, `http/frame.py`, `http/method.py`, `http/status_code.py`, `ipv4/router_alert.py`, `ipv6/option.py`, `ipv6/router_alert.py`, `ipv6/tagger_id.py`, `reg/ethertype.py`, `tcp/flags.py`, `tcp/mp_tcp_option.py` and `tcp/option.py`). Behaviour preserving: the shared pattern (`r'\r*\n'`) has no MULTILINE-sensitive anchor, so no `const/` regeneration was needed ([#796](https://github.com/JarryShaw/PyPCAPKit/issues/796)). -- `main` was red ([#866](https://github.com/JarryShaw/PyPCAPKit/issues/866)): 12 tests across three files, all `RecursionError` (`tests/const/test_const_enum_lookup.py`; two `test_const_enum_no_mint.py` methods, two registries each; 7 of the 12 in `tests/protocols/misc/test_pcapng_unit.py`). `pcapkit/vendor/pcapng/record_type.py` and `secrets_type.py` rendered a two-line `_missing_` body whose first line did not return: `cls._unregistered_member(value, 'Unassigned')` followed by `return cls(value)`. That was harmless while the first line was `extend_enum(...)`, which registers the member so `cls(value)` found it; [#861](https://github.com/JarryShaw/PyPCAPKit/pull/861) (above) replaced it with `_unregistered_member`, which deliberately does not register, so `cls(value)` missed and re-entered `_missing_`. [#861](https://github.com/JarryShaw/PyPCAPKit/pull/861) half-corrected the two files: it fixed the *generated* modules (leaving the old `return cls(value)` as dead code) and never touched the two *crawlers*, which is why [#861](https://github.com/JarryShaw/PyPCAPKit/pull/861)'s own CI stayed green until a live regeneration from the stale crawlers reintroduced the defect; [#861](https://github.com/JarryShaw/PyPCAPKit/pull/861)'s cross-review had verified byte-identical regeneration on a 5-of-82 sample that missed these two. Both crawlers now collapse to the single returning line the other 101 registries use, and the two const files are regenerated (`record_type.py` fetches the network for its own `LINK` constant, so that regeneration needs connectivity, though it came back byte-identical; `secrets_type.py` is offline). The new guard sits at the crawler-emission layer rather than the behavioural one `test_const_enum_lookup.py` already swept: a hard-coded `_missing_` body must end in a return, and no emitted line may re-enter `_missing_` for the same value -- deliberately narrower than "every emitted line must return", since 53 crawlers legitimately emit 109 non-returning lines in the shape `Vendor.process` uses. The guard sees only 35 of 96 crawlers; the other 61 are covered behaviourally. Closes [#866](https://github.com/JarryShaw/PyPCAPKit/issues/866) ([#867](https://github.com/JarryShaw/PyPCAPKit/pull/867)). -- `python -m pcapkit.vendor ` could not fail: `run()` swallowed every crawler exception as a filterable `VendorRuntimeWarning` and `main()` always returned `0`, so a broken crawler regenerating nothing looked like success to a human and to the cron job. Per the owner's ruling (*"only discard changes made by a non-zero sub-vendor... for cron jobs, at most warn about failed sub-vendors, but never fail the entire job"*), three pieces land. **Exit code** (`pcapkit/vendor/__main__.py`): `run()` returns whether its target succeeded; `main()` attempts every target and returns `1` if any raised, writing the failing target's qualified name and exception `repr` to stderr unconditionally via `print()` rather than the filterable warning. **Per-target snapshot and restore**: a new `_snapshot_and_restore` context manager backs up a target's const file with `tempfile.mkstemp` before it runs (resolving the destination through `Vendor._dest_path`, now a `classmethod` in `pcapkit/vendor/default.py` so it can be called before `Vendor.__init__` fetches, renders or writes), restores the backup via `os.replace` on any exception before re-raising, and discards it on success. A target whose destination cannot be resolved (a crawler outside the real tree) runs unprotected rather than failing the snapshot step, since `Vendor.__init__` hits the same resolution failure itself; a destination that does not exist yet is left alone. **Workflow** (`.github/workflows/cron-vendor.yml`): the "Update Vendor" step no longer aborts under `bash -e` when `pcapkit-vendor` exits non-zero. The exit code is captured around that one command, so targets that succeeded are still `isort`ed, diffed, committed and pushed, while a "Warning" block naming the failed target(s) is appended to `$GITHUB_STEP_SUMMARY`, a `::warning::` annotation is emitted per failure, and the step's own exit code is always `0`. A per-target `git checkout -- ` restore step was dropped: with snapshot/restore, a failed target's const file is never touched. `tests/vendor/test_vendor_exit_code_unit.py` (5 methods, through the real entrypoints with stub `Vendor` subclasses) and `tests/vendor/test_vendor_snapshot_restore_unit.py` (3: success replaces the previous content, failure leaves it byte-for-byte intact, a read-only destination is left as it was) make no network calls. Closes [#872](https://github.com/JarryShaw/PyPCAPKit/issues/872) ([#873](https://github.com/JarryShaw/PyPCAPKit/pull/873)). -- three follow-ups from [#873](https://github.com/JarryShaw/PyPCAPKit/pull/873)'s review, none changing merged behaviour: one unpinned invariant and two inaccurate pieces of prose/lint. **1.** `except BaseException:` in `_snapshot_and_restore` is pinned: `test_a_keyboard_interrupt_still_restores_and_propagates` asserts content restored byte-for-byte, no `.bak` surviving, and the `KeyboardInterrupt` still propagating rather than becoming a `False` return; mutating the clause to `except Exception:` fails it. **2.** A false docstring clause is corrected: it said a crawler whose `_dest_path` cannot be resolved "goes on to fail loudly anyway once `Vendor.__init__` calls the same `_dest_path` itself", which holds for one of the function's two triggers and not the other. An instance-method-only override raises `TypeError` on the *class* call `_snapshot_and_restore` makes (no `self` to bind), but the *instance* call inside `__init__` resolves fine, so that target runs unprotected and **succeeds silently**: `run()` returns `True`, the file is replaced, and nothing warns that no snapshot was taken. **3.** The `# pylint: disable=no-else-raise` pragma is **kept**. Removing it rested on the belief that R1720 never applies to `try`/`except`/`else`, a message-control artefact (`--disable=all --enable=no-else-raise` does not re-enable the check the way `--enable=R` does); under the `Makefile`'s flags, which CI runs, pylint 4.0.8 flags it. The `else:` is deliberate: it makes the backup cleanup unreachable from the `except`. Left out of scope: an untested `os.close(fd)` leak (harmless, the path still exists and `copy2` reopens it) and the semantically identical unindented form of the same `else:`. Closes [#875](https://github.com/JarryShaw/PyPCAPKit/issues/875) ([#876](https://github.com/JarryShaw/PyPCAPKit/pull/876)). +- `main` was red ([#866](https://github.com/JarryShaw/PyPCAPKit/issues/866)): 12 tests across three files, all `RecursionError` (`tests/const/test_const_enum_lookup.py`; two `test_const_enum_no_mint.py` methods, two registries each; 7 of the 12 in `tests/protocols/misc/test_pcapng_unit.py`). `pcapkit/vendor/pcapng/record_type.py` and `secrets_type.py` rendered a two-line `_missing_` body whose first line did not return: `cls._unregistered_member(value, 'Unassigned')` followed by `return cls(value)`. That was harmless while the first line was `extend_enum(...)`, which registers the member so `cls(value)` found it; [#861](https://github.com/JarryShaw/PyPCAPKit/pull/861) (above) replaced it with `_unregistered_member`, which deliberately does not register, so `cls(value)` missed and re-entered `_missing_`. [#861](https://github.com/JarryShaw/PyPCAPKit/pull/861) fixed the *generated* modules but not the two *crawlers*, so [#861](https://github.com/JarryShaw/PyPCAPKit/pull/861)'s CI stayed green until a live regeneration reintroduced the defect; [#861](https://github.com/JarryShaw/PyPCAPKit/pull/861)'s review had sampled 5 of 82 regenerations, missing these two. Both crawlers now collapse to the single returning line the other 101 registries use, and the two const files are regenerated. A new guard at the crawler-emission layer requires a hard-coded `_missing_` body to end in a return and no emitted line to re-enter `_missing_` for the same value -- deliberately narrower than "every emitted line must return", which `Vendor.process`'s shape legitimately breaks. The guard sees only 35 of 96 crawlers; the other 61 are covered behaviourally. Closes [#866](https://github.com/JarryShaw/PyPCAPKit/issues/866) ([#867](https://github.com/JarryShaw/PyPCAPKit/pull/867)). +- `python -m pcapkit.vendor ` could not fail: `run()` swallowed every crawler exception as a filterable `VendorRuntimeWarning` and `main()` always returned `0`, so a broken crawler regenerating nothing looked like success to a human and to the cron job. Per the owner's ruling (*"only discard changes made by a non-zero sub-vendor... for cron jobs, at most warn about failed sub-vendors, but never fail the entire job"*), three pieces land. **Exit code** (`pcapkit/vendor/__main__.py`): `run()` returns whether its target succeeded; `main()` attempts every target and returns `1` if any raised, writing the failing target's qualified name and exception `repr` to stderr unconditionally via `print()` rather than the filterable warning. **Per-target snapshot and restore**: a new `_snapshot_and_restore` context manager backs up a target's const file with `tempfile.mkstemp` before it runs (resolving the destination through `Vendor._dest_path`, now a `classmethod` in `pcapkit/vendor/default.py` so it can be called before `Vendor.__init__` fetches, renders or writes), restores the backup via `os.replace` on any exception before re-raising, and discards it on success. A target whose destination cannot be resolved (a crawler outside the real tree) runs unprotected rather than failing the snapshot step, since `Vendor.__init__` hits the same resolution failure itself; a destination that does not exist yet is left alone. **Workflow** (`.github/workflows/cron-vendor.yml`): the "Update Vendor" step no longer aborts under `bash -e` when `pcapkit-vendor` exits non-zero. The exit code is captured around that one command, so targets that succeeded are still `isort`ed, diffed, committed and pushed, while a "Warning" block naming the failed target(s) is appended to `$GITHUB_STEP_SUMMARY`, a `::warning::` annotation is emitted per failure, and the step's own exit code is always `0`. A per-target `git checkout -- ` restore step was dropped: with snapshot/restore, a failed target's const file is never touched. Closes [#872](https://github.com/JarryShaw/PyPCAPKit/issues/872) ([#873](https://github.com/JarryShaw/PyPCAPKit/pull/873)). +- three follow-ups from [#873](https://github.com/JarryShaw/PyPCAPKit/pull/873)'s review, none changing merged behaviour: one unpinned invariant and two inaccurate pieces of prose/lint. **1.** `except BaseException:` in `_snapshot_and_restore` is pinned: a test asserts that a `KeyboardInterrupt` still restores the file and propagates. **2.** A false docstring clause is corrected: a crawler whose `_dest_path` is an instance-method-only override fails the *class* call `_snapshot_and_restore` makes but not the *instance* call inside `__init__`, so that target runs unprotected and **succeeds silently**. **3.** The `# pylint: disable=no-else-raise` pragma is **kept**: the belief that R1720 never applies to `try`/`except`/`else` was a message-control artefact (`--disable=all --enable=no-else-raise` does not re-enable the check the way `--enable=R` does), and under the `Makefile`'s flags, which CI runs, pylint flags the `else:`, which is deliberate because it makes the backup cleanup unreachable from the `except`. Left out of scope: an untested `os.close(fd)` leak (harmless, since `copy2` reopens the path) and the semantically identical unindented form of the same `else:`. Closes [#875](https://github.com/JarryShaw/PyPCAPKit/issues/875) ([#876](https://github.com/JarryShaw/PyPCAPKit/pull/876)). - `TLSKeyLabel` tracked the Mozilla NSS key-log wiki, but `draft-ietf-opsawg-pcapng-06` S4.7 now points at `[I-D.ietf-tls-keylogfile]` (draft-05), published as [RFC 9850](https://datatracker.ietf.org/doc/html/rfc9850), "The SSLKEYLOGFILE Format for TLS", whose S4.2 creates the IANA "TLS SSLKEYLOGFILE Labels" registry. Against it, `TLSKeyLabel` was missing `ECH_SECRET` and `ECH_CONFIG` ([RFC 9850](https://datatracker.ietf.org/doc/html/rfc9850) SS2.3/4.2), so `TLSKeyLabel('ECH_SECRET')` raised `ValueError` on a legitimate Decryption Secrets Block; both are added, `ECH_SECRET` keeping the file's `# nosec B105` bandit convention for `*_SECRET` members. `RSA` has no row in [RFC 9850](https://datatracker.ietf.org/doc/html/rfc9850)'s registry (it is an NSS-only label the NSS wiki records as removed in NSS 3.34) and is kept, since removing a public member is breaking, with a docstring comment marking it the historical exception. `pcapkit/vendor/pcapng/secrets_type.py`'s stale `# NSS Key Log Format` comment is repointed at RFC 9850, and `TLSKeyLabel`'s docstring cites it. A new test pins the member set against RFC 9850 plus the `RSA` exception. Closes [#882](https://github.com/JarryShaw/PyPCAPKit/issues/882) ([#883](https://github.com/JarryShaw/PyPCAPKit/pull/883)). ### Project infrastructure @@ -244,10 +244,10 @@ This resolves [#548](https://github.com/JarryShaw/PyPCAPKit/issues/548), which r #### Changed - renames with no compatibility alias left behind: `examples/sample` and `examples/samples` -- one letter apart, holding different things -- are now `examples/captures` and `examples/generators`. -- the README is a landing page now, and Markdown rather than reStructuredText. `README.rst` (424 lines) became `README.md` (102), keeping what a reader arriving from PyPI or a search result needs (what the library is, why it exists rather than Scapy or DPKT, how to install it, a worked example, where the documentation lives). **Module Structure**, **Engine Comparison**, **Engine support by Python version**, **Test Environment**, **Test Results** and **Installation Notes** were already duplicated in fuller form in `docs/source/index.rst`, so they are linked rather than restated; the two blocks that existed nowhere else moved: **Testing** is now `docs/source/testing.rst`, in the index toctree, and the `pipenv` and `make setup` block joined the Installation section of `docs/source/index.rst`. Requested by the project owner, and a deliberate exception to the reStructuredText convention, for the README only, since its renderers are GitHub and PyPI rather than Sphinx. `setup.py` reads `README.md` and declares `text/markdown`, and `MANIFEST.in` gains `include README.md` because `global-include *.rst` no longer matches. That line is belt-and-braces rather than load-bearing, contrary to what this entry claimed when written: setuptools' `sdist` ships whichever of `README`, `README.rst`, `README.txt` and `README.md` exists before `MANIFEST.in` is read, so deleting the line leaves the sdist listing byte-identical, and `setup.py` reads the file from where it executes, which under `pip` is the unpacked sdist. Of the `include` lines only `CHANGELOG.md` is load-bearing, measured in the [#631](https://github.com/JarryShaw/PyPCAPKit/pull/631) entry below. References moved with the rename: `examples/benchmark/Dockerfile` copies `README.md` (a literal `COPY` of the old name would have failed the layer and taken `make bench` with it), and benchmark prose naming the root README as the destination of generated tables now names `docs/source/index.rst`. `examples/benchmark/benchmark.py` still says `README.rst` on purpose, meaning the benchmark suite's own README ([#619](https://github.com/JarryShaw/PyPCAPKit/pull/619)). +- the README is a landing page now, and Markdown rather than reStructuredText. `README.rst` (424 lines) became `README.md` (102), keeping what a reader arriving from PyPI or a search result needs (what the library is, why it exists rather than Scapy or DPKT, how to install it, a worked example, where the documentation lives). **Module Structure**, **Engine Comparison**, **Engine support by Python version**, **Test Environment**, **Test Results** and **Installation Notes** were already duplicated in fuller form in `docs/source/index.rst`, so they are linked rather than restated; the two blocks that existed nowhere else moved: **Testing** is now `docs/source/contributing/testing.rst`, and the `pipenv` and `make setup` block joined the Installation section of `docs/source/index.rst`. Requested by the project owner, and a deliberate exception to the reStructuredText convention, for the README only, since its renderers are GitHub and PyPI rather than Sphinx. `setup.py` reads `README.md` and declares `text/markdown`, and `MANIFEST.in` gains `include README.md`, which is belt-and-braces rather than load-bearing: setuptools' `sdist` ships a `README.md` regardless. Of the `include` lines only `CHANGELOG.md` is load-bearing, measured in the [#631](https://github.com/JarryShaw/PyPCAPKit/pull/631) entry below. `examples/benchmark/Dockerfile` copies `README.md`, and `examples/benchmark/benchmark.py` still says `README.rst` on purpose, meaning the benchmark suite's own README ([#619](https://github.com/JarryShaw/PyPCAPKit/pull/619)). - `CODE_OF_CONDUCT.md` moves from Contributor Covenant 1.4 to 3.0, at the maintainer's request. The text is the canonical 3.0 Markdown fetched from https://www.contributor-covenant.org/version/3/0/code_of_conduct/code_of_conduct.md rather than transcribed. Three things needed deciding. 3.0's two `[NOTE` placeholders are filled or removed: the reporting channel names `jarryshaw@icloud.com` (the contact 1.4 carried and `SECURITY.md` uses as email fallback) plus GitHub's report-abuse form for a report about the maintainer, which a single-maintainer project cannot otherwise cover; the enforcement placeholder is an instruction to the adopter and is removed. 3.0's plural "Community Moderators" become the singular maintainer (eight occurrences), this repository having no moderators. The four-rung ladder (Warning, Temporarily Limited Activities, Temporary Suspension, Permanent Ban) is a suggestion and is **kept**, each rung mapping to a lever one person holds on GitHub. 3.0 is CC BY-SA 4.0 where 1.4 carried no licence notice, so the attribution names version 3.0, links the permanent `version/3/0/` URL, carries the CC BY-SA 4.0 notice, indicates changes were made as BY requires, and says the share-alike term covers this document only (the code remains BSD-3-Clause, `LICENSE` untouched). The ladder renders as four list items each nesting three, checked against GitHub's Markdown API; [#613](https://github.com/JarryShaw/PyPCAPKit/pull/613) had to repair the 1.4 file when a stray list marker collapsed it into one nested item ([#624](https://github.com/JarryShaw/PyPCAPKit/pull/624)). -- reconciled which private (`_xxx`) attributes and methods the Sphinx build documents, replacing an ad hoc mix with one rule: document the contract, hide the recipe. A member every subclass must implement, or whose shape a caller depends on, stays documented despite its underscore; a private helper that only keeps one method short does not. Seven module-level directives for pure implementation detail (`esp._resolve`, `esp._CRYPTO`, `ngap._convert`, `ngap._revert`, `ngap._PYCRATE`, `ngap._PDU_LOCK`, `pypcapfile._NamedStream`) were dropped, while 39 `autoattribute` directives were added for class-private state that *is* contract, among them `Extractor._flag_f`, the `PyPCAP` and `PCAP_CT` engines' `_backend` (the only two of the six third-party engines with one), `TraceFlow`'s internal fields, `FieldBase`/`Field` internals, the built-in `PCAP` engine's `_gbhdr`, `_vinfo`, `_nnsec` and `_dlink` (shared with `PyPCAP`, `PCAP_CT` and `PyPCAPFile`) and `PCAPNG`'s `_ctx`/`_ctx_list`. The 128 runtime definitions of `_missing_` (121 under `pcapkit.const`, 7 inline in `pcapkit.protocols`) gained one write-up in place of a directive per class, a new "Unrecognised Values" section in `docs/source/pcapkit/const/index.rst` cross-referenced from `registry.rst` (`conf.py` already excludes `_missing_` from every `autoclass`). `CONTRIBUTING.md` gained the rule as a named section. `sphinx-build -b html` gives 53 warnings on `main` and 54 after, the addition being the pre-existing bare `Type` ambiguity newly rendered by `TraceFlow._foutio`'s new directive, tracked as [#709](https://github.com/JarryShaw/PyPCAPKit/issues/709). No line under `pcapkit/` changed ([#684](https://github.com/JarryShaw/PyPCAPKit/issues/684)). -- `examples/captures/out.json`, `out.plist`, `out.txt` and `pcapng.txt` are no longer tracked in git; they are build output, not fixtures, and a tracked rendering with no reader goes stale silently. `pcapng.txt` is exactly that: it still recorded `packet -> NIL` for the four Enhanced Packet Blocks of `dhcp.pcapng` long after [#683](https://github.com/JarryShaw/PyPCAPKit/pull/683) gave those blocks their captured octets back. Rather than regenerate it again and leave the drift free to recur, the four files are removed from the index and folded into `examples/captures/`'s blanket `.gitignore` rule; the genuine inputs `in.pcap` and `dhcp.pcapng` stay tracked. `examples/legacy_smoke/Makefile` and its `README.rst` now describe regenerating these reports via `make fixtures`, and `tests/project/test_capture_tracking.py` (5 tests, 8 subtests) pins the invariant; 2 of the 5 fail against the pre-change tree with the four reports restored, the other 3 passing on both by construction ([#685](https://github.com/JarryShaw/PyPCAPKit/issues/685)). +- reconciled which private (`_xxx`) attributes and methods the Sphinx build documents, replacing an ad hoc mix with one rule: document the contract, hide the recipe. A member every subclass must implement, or whose shape a caller depends on, stays documented despite its underscore; a private helper that only keeps one method short does not. Seven module-level directives for pure implementation detail (`esp._resolve`, `esp._CRYPTO`, `ngap._convert`, `ngap._revert`, `ngap._PYCRATE`, `ngap._PDU_LOCK`, `pypcapfile._NamedStream`) were dropped, while 39 `autoattribute` directives were added for class-private state that *is* contract, among them `Extractor._flag_f`, the `PyPCAP` and `PCAP_CT` engines' `_backend` (the only two of the six third-party engines with one), `TraceFlow`'s internal fields, `FieldBase`/`Field` internals, the built-in `PCAP` engine's `_gbhdr`, `_vinfo`, `_nnsec` and `_dlink` (shared with `PyPCAP`, `PCAP_CT` and `PyPCAPFile`) and `PCAPNG`'s `_ctx`/`_ctx_list`. The 128 runtime definitions of `_missing_` (121 under `pcapkit.const`, 7 inline in `pcapkit.protocols`) gained one write-up in place of a directive per class, a new "Unrecognised Values" section in `docs/source/pcapkit/const/index.rst` cross-referenced from `registry.rst` (`conf.py` already excludes `_missing_` from every `autoclass`). `CONTRIBUTING.md` gained the rule as a named section. The one new Sphinx warning, a bare `Type` newly rendered by `TraceFlow._foutio`'s directive, is [#709](https://github.com/JarryShaw/PyPCAPKit/issues/709). No line under `pcapkit/` changed ([#684](https://github.com/JarryShaw/PyPCAPKit/issues/684)). +- `examples/captures/out.json`, `out.plist`, `out.txt` and `pcapng.txt` are no longer tracked in git; they are build output, not fixtures, and a tracked rendering with no reader goes stale silently. `pcapng.txt` is exactly that: it still recorded `packet -> NIL` for the four Enhanced Packet Blocks of `dhcp.pcapng` long after [#683](https://github.com/JarryShaw/PyPCAPKit/pull/683) gave those blocks their captured octets back. Rather than regenerate it again and leave the drift free to recur, the four files are removed from the index and folded into `examples/captures/`'s blanket `.gitignore` rule; the genuine inputs `in.pcap` and `dhcp.pcapng` stay tracked. `examples/legacy_smoke/Makefile` and its `README.rst` now describe regenerating these reports via `make fixtures`, and `tests/project/test_capture_tracking.py` pins the invariant ([#685](https://github.com/JarryShaw/PyPCAPKit/issues/685)). - `tests/project/test_isort_clean.py` gated on `isort` with a `try`/`except ImportError` inside `setUpClass`, which `tests/_dependency_gates.py` cannot see (`_gates_of()` walks a decorator list, never a function body), and `isort` was in no `pyproject.toml` extra, so the file reported `OK (skipped=1)` on every CI leg and the dependency-gate guard could not tell. It now gates on a module-level `HAS_ISORT` and a class-level `@unittest.skipUnless` so `gated_scopes()` counts it, `MODULE_PROVIDERS` maps `isort`, and `isort` joins the `test` extra rather than being excluded (`lint.yml` installs none, so unlike `mypy` there is no existing check to defer to). No workflow changed, every pytest job already building on `.[test]`; `gated_scopes()` goes 250 to 251 and `dependency_gate_gaps()` holds at 18. The third instance of the hazard [#745](https://github.com/JarryShaw/PyPCAPKit/issues/745) exists to stop ([#766](https://github.com/JarryShaw/PyPCAPKit/issues/766)). - `tests/vendor/test_vendor_reg_apptype_generator_unit.py` gated on `mypy` with no `MODULE_PROVIDERS` entry, so `_top_level_providers()` died on a bare `KeyError: 'mypy'` once the gate became visible -- a further instance of the hazard [#745](https://github.com/JarryShaw/PyPCAPKit/issues/745) exists to stop, like [#766](https://github.com/JarryShaw/PyPCAPKit/issues/766) above. `MODULE_PROVIDERS['mypy'] = ('mypy',)` fixes the lookup, and the three bare `MODULE_PROVIDERS[...]` subscripts route through that function, which raises an `AssertionError` naming the module, key and table on any future gap. `mypy` is deliberately not added to a `pyproject.toml` extra: no extra carries it, it is a `Pipfile` dev-dependency that `lint.yml` installs and already runs whole-package, and `lint.yml` runs no pytest job for `pytest_jobs()` to see, so the fix is a `DEPENDENCY_GATE_EXCLUSIONS['HAS_MYPY']` entry on `test`, `engine-tests` and `gate`. `dependency_gate_gaps()` goes 15 to 18 `(flag, job)` pairs, all three new and expected; ([#779](https://github.com/JarryShaw/PyPCAPKit/issues/779)). - `engine='pyshark'` had no CI job driving it through a real `tshark`-parsed capture. `pypcap-parity` installs `pyshark` but its `apt-get install` named no `tshark` package, so `PyShark.unsupported_reason()` declined the engine, the extractor fell back to the built-in parser, and the one real-extraction assertion, `assertGreater(extractor.length, 0)`, passed against the fallback's output: a vacuous pass invisible in every skip count. `pypcap-parity` now installs `tshark` with the debconf pre-seed `engine-tests` carries, and the test proves which engine ran (no `EngineWarning`, `extractor._exnam == 'pyshark'`, a `PyShark` instance), as the PyPCAP parity test does. Only `tests/integration/test_engine_runtime.py` and `test_engine_parity.py` change; the length assertion is kept ([#846](https://github.com/JarryShaw/PyPCAPKit/pull/846)). @@ -265,7 +265,7 @@ This resolves [#548](https://github.com/JarryShaw/PyPCAPKit/issues/548), which r - two inaccurate claims in the [#630](https://github.com/JarryShaw/PyPCAPKit/pull/630) entry above, caught in review after [#630](https://github.com/JarryShaw/PyPCAPKit/pull/630) had merged; both concern the history of `LICENSE`, which is untouched here. 2017 was and remains the right answer, so this is a changelog-prose correction. The entry said the file "until then was stock MPL text carrying no author notice at all, so 2018 is the first start year the project ever asserted and it was already a year late when it was written". That was true of the single blob `bc836cfa2` replaced and wrong as a description of the era: the pre-BSD file was MIT, then GPL v3, then Apache 2.0, then MPL 2.0; the MIT and GPL texts both carried an author notice naming **2017**; and `2018` first appeared in the Apache notice of 2018-12-08 as a single year, current when written. The claim also contradicted the entry's own dating of the first commit to 2017-11-07. The history supports a stronger argument than the one written, which is why this is corrected rather than deleted: 2017 is the year asserted from the first commit, and [#630](https://github.com/JarryShaw/PyPCAPKit/pull/630) restores it. The `BSD 3-Clause License` title line is reattributed from opensource.org's template to choosealicense.com's, whose first line is that title and which carries the `Copyright (c) [year], [fullname]` comma this file uses. The error came from generalising the pre-BSD era off two blobs that were the same file (`bc836cfa2^:LICENSE` and `1c69341dc:LICENSE`); enumerating all 14 commits that touched `LICENSE` caught it ([#638](https://github.com/JarryShaw/PyPCAPKit/pull/638)). - the release workflow published to PyPI and to Anaconda with nothing a human had to approve. `create-release.yml`'s `pypi` job had `environment: release` commented out while the `id-token: write` from the same upstream snippet was re-added live below it, so the block read as disabled as a unit when only its gate still was; the `conda` job had no `environment:` at all. Neither needed a tag: the workflow also fires on completion of `Vendor Update`, so a scheduled registry crawl that bumped the version was enough to publish to two public indexes, a median three minutes from bump to upload. Four jobs reach outside the run and all four are now gated: `github` on `github-release` (the GitHub Release and the `v*` tag), `tag` on `conda-tag` (the commit pushed to `main`), `pypi` on `pypi`, `conda` on `anaconda`. One environment per credential rather than a shared `release`, because approval is granted to an environment and not a job, so one `release` approved once would release both indexes together. PyPI has to be answerable alone: an accepted version number cannot be reused, and `skip-existing: true` makes the re-upload *succeed* having published nothing, so an unintended publish permanently consumes the number the intended release wanted. The workflow half is only half the fix: an `environment:` naming one that does not exist is created implicitly with no protection rules and the job proceeds unapproved, so this is inert until each of the four exists in repository settings with a required reviewer, and each one's "Deployment branches and tags" must stay unrestricted or the `tags: v*` trigger fails outright instead of pausing. `github-pages` is the standing example, carrying no protection rule since 2021. The new test asserts the general rule, that no job running a publishing action or a `git push` may omit an `environment:`, so a publishing job added later without a gate fails rather than ships ([#641](https://github.com/JarryShaw/PyPCAPKit/issues/641)). - three test comments stated an exact count of committed captures under `examples/captures/`, a number that drifts whenever the tracked set changes and in one case was wrong when written: `tests/_tiers.py` said "six," `tests/test_tier_guard.py` "the moment a seventh capture is committed," and `tests/integration/_helpers.py` "four," which undercounted on the day it was written. All three now describe the invariant (`_tiers.py`: "the moment somebody commits another capture... or stops committing one"; `_helpers.py`: "...are fixtures -- some of them committed --..."; `test_tier_guard.py` drops its ordinal). Prose-only, verified with `coverage.parser.PythonParser` that no executable statement moved ([#700](https://github.com/JarryShaw/PyPCAPKit/issues/700)). -- `test_every_tracked_name_exists_and_matches_git` in `tests/test_tier_guard.py` checked neither the git index nor the filesystem despite its name: it asserted only that the tracked-capture list was non-empty and no name contained a `/`, so a hardcoded list passed whether or not it matched reality. Sibling `test_capture_suggestions_are_captures` had the same gap. Found while cross-reviewing [#703](https://github.com/JarryShaw/PyPCAPKit/pull/703)'s first commit (prose-only, so out of scope there); the fix rides in as that PR's second and third commits, and [#703](https://github.com/JarryShaw/PyPCAPKit/pull/703)'s description names both [#700](https://github.com/JarryShaw/PyPCAPKit/issues/700) and [#708](https://github.com/JarryShaw/PyPCAPKit/issues/708). The first test now re-derives the expected set independently, shelling out to `git ls-files -z` under `examples/captures/` rather than calling back into `_tiers.py`, asserts equality against it, and adds a per-name `Path.is_file()` check. Its sibling gets a narrower fix: equality against `committed_capture_names()`'s filter re-implemented inline over `_tiers.committed_captures()`, which catches a wrong filter but still trusts `committed_captures()` for the tracked set. A cross-review of the first version found it wrong twice: a stale `_tiers.py` docstring still claimed `test_tier_guard.py` "only stats `in.pcap`," falsified by the new per-name loop; and the new `assertEqual` was vacuous on an empty tracked set (`() == ()`), having dropped the non-emptiness guard the other test kept. Both are corrected in a follow-up commit. Even now a hardcoded literal that happens to match today's names passes: the test detects a set that is wrong *at the moment it runs*, not hardcoding as a practice, turning a permanent blind spot into a tripwire that fires on the next change to the tracked set ([#708](https://github.com/JarryShaw/PyPCAPKit/issues/708)). +- `test_every_tracked_name_exists_and_matches_git` in `tests/test_tier_guard.py` checked neither the git index nor the filesystem despite its name: it asserted only that the tracked-capture list was non-empty and no name contained a `/`, so a hardcoded list passed whether or not it matched reality. Sibling `test_capture_suggestions_are_captures` had the same gap. Fixed in [#703](https://github.com/JarryShaw/PyPCAPKit/pull/703); [#703](https://github.com/JarryShaw/PyPCAPKit/pull/703)'s description names both [#700](https://github.com/JarryShaw/PyPCAPKit/issues/700) and [#708](https://github.com/JarryShaw/PyPCAPKit/issues/708). The first test now re-derives the expected set from `git ls-files -z` under `examples/captures/`, asserts equality against it, and checks each name with `Path.is_file()`; its sibling re-implements `committed_capture_names()`'s filter inline, which catches a wrong filter but still trusts `committed_captures()` for the tracked set. A hardcoded literal that happens to match the tracked names still passes; the test catches a set that is wrong when it runs ([#708](https://github.com/JarryShaw/PyPCAPKit/issues/708)). --- diff --git a/docs/source/changelog/1.5.0.rst b/docs/source/changelog/1.5.0.rst index a5df9636d..d0cb449b2 100644 --- a/docs/source/changelog/1.5.0.rst +++ b/docs/source/changelog/1.5.0.rst @@ -4,10 +4,9 @@ The largest release since 1.0, and the first recorded here as it happened rather than reconstructed. Three more extraction engines, ESP with payload decryption, SCTP and NGAP over SCTP, a library logger that no longer hijacks the consumer's, -and a defect programme run through the issue tracker spanning 254 issues and pull -requests, from :pr:`121` to :pr:`883` as of this revision. The programme is still running, -so those figures are a snapshot rather than a total -- they are re-derived from -the entries below each time this file is regenerated. +and a defect programme run through the issue tracker, citing 266 issues and pull +requests from :pr:`121` to :issue:`1050` as of this revision. The programme is still +running, so those figures are a snapshot rather than a total. Preceded by ``1.5.0a1`` (2026-09-15), ``1.5.0b1`` and ``1.5.0b2`` (both 2026-09-18) and ``1.5.0b3`` (2026-09-19), all published as prereleases and so @@ -38,9 +37,8 @@ Changed ``/etc/services`` because IANA names no precedence, and ``get_all()`` reaches the aliases (:pr:`754`). * the six ``%``-formatted ``raise ValueError(...)`` calls in the generated - ``pcapkit.const.reg.apptype.apptype``, left by :issue:`759` so its diff over a - 12,391-member file stayed reviewable, are f-strings, matching the other 605 - tree-wide (:issue:`792`). The remainder followed: three ``_missing_`` guards + ``pcapkit.const.reg.apptype.apptype``, left by :issue:`759`, are f-strings + (:issue:`792`). The remainder followed: three ``_missing_`` guards (``pcapkit.vendor.tcp.flags``, ``.ftp.command``, ``.http.method``), ``AppType``'s ``__new__``/``__repr__``/``__str__``, and the ``extend_enum`` fallbacks in ``get()`` and in the span-handling tail. The ``consider-using-f-string`` disable drops from @@ -54,21 +52,16 @@ Changed templates. The other bespoke templates (``{const,vendor}/ftp/return_code.py``, ``{const,vendor}/http/status_code.py``, ``vendor/http/frame.py``) and ``vendor/default.py`` still carry it, so :issue:`804`'s claim that nothing in - ``pcapkit/{const,vendor}/{ftp,http}/`` needs it holds for this pair only. - ``test_the_disable_drops_only_where_nothing_else_needs_percent_formatting`` asserted - the opposite for this pair and is flipped; ``repr()`` is asserted member by member - against the old ``%`` output (:issue:`804`). + ``pcapkit/{const,vendor}/{ftp,http}/`` needs it holds for this pair only; ``repr()`` + output is unchanged (:issue:`804`). * ``test_const_enum_no_mint.py``'s ``is_manufactured()`` exemption for :pr:`878`'s (above) 53 preserved hex-suffixed names moves from a per-*file* to a per-*name-shape* one. The per-file ``HEX_SUFFIXED_NAME_EXEMPT_PATHS`` was shape-blind: rewriting a branch to ``'Xyplex_%d' % value`` -- exactly the value-derived shape the predicate exists to flag -- left the sweep green. The check - now matches an AST ``%``-``BinOp`` whose left side is a string ending in ``'_0x%s'`` - and whose right side equals a parsed ``hex(value)[2:].upper().zfill(4)`` (via - ``ast.dump()``, immune to quote-style or whitespace drift). The exact - ``exempt_hits == 53`` bound stays, so adding or removing a branch still forces a - human look. Only ``tests/const/test_const_enum_no_mint.py`` changed. Closes - :issue:`879` (:pr:`881`). + now exempts only a ``'..._0x%s' % hex(value)[2:].upper().zfill(4)`` expression, and + the exact ``exempt_hits == 53`` bound stays. Test-only. Closes :issue:`879` + (:pr:`881`). Fixed ~~~~~ @@ -96,15 +89,12 @@ Fixed ``default`` -- ``Hardware.get(99999, 0)`` raised ``ValueError`` instead of returning ``0``. The integer path now consults ``default`` before letting the error escape. ``-1``, the placeholder the generated signature already carried, still separates "no - default" from "a default"; it is to be replaced by an identity sentinel in :pr:`859` - (not yet landed) once the shared base :pr:`858` (below) puts ``get()`` behind one - call site. The sweep covers 110 of the 118 integer registries, not the three - :issue:`584` named; ``pcapng`` ``OptionType`` and ``reg`` ``AppType`` carry a - bespoke integer fallback and are deliberately left alone, since neither drops a - default by raising. Not reachable from wire data, so a contract fix rather than a - parse fix. Applied to the nine vendor templates and the 113 generated modules; a new - test renders the shared template and compares it with the generated module, so a - regeneration cannot undo it (:issue:`584`). + default" from "a default"; :pr:`859` replaced it with an identity sentinel once the + shared base :pr:`858` (below) put ``get()`` behind one call site. The sweep covers + 110 of the 118 integer registries, not the three :issue:`584` named; ``pcapng`` + ``OptionType`` and ``reg`` ``AppType`` carry a bespoke integer fallback and are + deliberately left alone, since neither drops a default by raising. Not reachable + from wire data, so a contract fix rather than a parse fix (:issue:`584`). * ``_missing_`` in the four Mobility Header flag enumerations (``BindingACKFlag``, ``BindingUpdateFlag``, ``HandoverACKFlag``, ``HandoverInitiateFlag``) ended in ``return cls(value)``, the constructor that had just failed, so every in-range value @@ -114,14 +104,12 @@ Fixed ``aenum`` ``Flag`` machinery that resolves exactly those values; ``pcapkit/const/tcp/flags.py`` had none and so never had the defect (it has one now, from :issue:`647` below, with the same tail). All four now end in - ``return super()._missing_(value)``, as ``pcapkit/vendor/default.py`` emits and 75 - of the 117 modules under ``pcapkit/const/`` already did (77 with :issue:`647`'s two - new modules). ``F(0)`` is an empty flag and ``BindingACKFlag(0x06)`` is ``S|D``. The - ``extend_enum`` mint used by ``pcapng/record_type.py`` and ``secrets_type.py`` was - rejected for a flag type: naming ``0x06`` ``Unassigned_0x06`` would hide the - composite and add a ``_member_map_`` entry per bit pattern, up to 65536 for - ``BindingUpdateFlag``. The range guard is untouched. Fixed in the four - ``pcapkit/vendor/mh/`` templates and regenerated (:issue:`623`). + ``return super()._missing_(value)``, as ``pcapkit/vendor/default.py`` emits and most + const modules (:issue:`647`'s two new ones included) already did, so ``F(0)`` is an + empty flag and ``BindingACKFlag(0x06)`` is ``S|D``. Minting through + ``extend_enum`` was rejected for a flag type: naming ``0x06`` ``Unassigned_0x06`` + would hide the composite and add a member per bit pattern. The range guard is + untouched (:issue:`623`). * six of the 123 constant registries rejected an invalid value differently from the built-in ``enum``, and three did not reject it at all. ``Flags``, ``ftp.command.CommandType`` and ``reg.apptype.TransportProtocol`` are ``IntFlag`` @@ -141,14 +129,7 @@ Fixed ``Method('FROBNICATE')``, ``ProtectionAuthority(1 << 70)`` and ``AppType.get(65000, proto=tcp)`` still register. The three flag guards end in ``return super()._missing_(value)``, so in-range composites still decompose and - :issue:`623` stays fixed. ``pcapkit/vendor/tcp/flags.py`` carried the value checker - all along as ``FLAG = 'isinstance(value, int) and 4 <= value <= 15'`` without - interpolating it; those are *bit offsets*, so emitting it unchanged would have - rejected every composite and ``Flags(0)``. It now reads ``0 <= value <= 0xFFFF``. - ``TransportProtocol`` reads its bound from ``cls.__members__``, because ``get`` - grows the registry at runtime and a literal would reject the member it had just - added. Fixed in the four bespoke templates under ``pcapkit/vendor/`` and regenerated - from live IANA data (42 insertions, no deletions, in 4 of 134 files). A new + :issue:`623` stays fixed; ``Flags`` accepts ``0 <= value <= 0xFFFF``. A new ``tests/const/test_const_enum_builtin_parity.py`` asserts that every one of the 123 registries raises the same exception type as a stdlib ``enum.IntEnum``; it keys on ``aenum.Enum`` because an ``IntFlag`` is not an ``IntEnum`` subclass, which is how @@ -173,14 +154,11 @@ Fixed caller checking equality, since ``AppType.__eq__`` compares on ``port`` alone. ``TransportProtocol`` is an ``aenum.IntFlag`` and a member carries the whole set IANA assigned the service, so ``tcp | udp`` is an ordinary value to pass back in. - Sweeping all 10,625 multi-transport members through ``get(m.port, proto=m.proto)`` - gave 46 answers naming another service at 20 ports; 44 differ only in not being the - port's canonical, which a single-bit lookup does too, leaving two real defects: port - 888 (``accessbuilder`` resolved to TCP's ``cddbp``) and port 999 (``puprouter`` - resolved to TCP's ``garcon``). It now raises ``ProtocolError``, a ``ValueError`` - subclass, so the documented contract holds. No caller is affected: the ten library - call sites each name one transport, and ``register_apptype``, the one reader of a - composite ``proto``, tests it with ``in`` (:issue:`759`). + Over every multi-transport member this misrouted two: port 888 (``accessbuilder`` + resolved to TCP's ``cddbp``) and port 999 (``puprouter`` resolved to TCP's + ``garcon``). It now raises ``ProtocolError``, a ``ValueError`` subclass, so the + documented contract holds. No library caller passes a composite + (:issue:`759`). * ``get()``'s string-key miss and every generated registry's ``_missing_`` bounded-range branch called ``aenum.extend_enum()`` on an unrecognised value, so e.g. ``Hardware(40)`` minted a permanent member. Fixed at the two shared sites in @@ -194,11 +172,10 @@ Fixed the ``KeyError`` into ``ProtocolError``. Riding along: ``pcapkit/toolkit/pyshark.py`` gains a two-entry ``FILTER_NAME_TO_LINKTYPE`` (``eth`` to ``ETHERNET``, ``tr`` to ``IEEE802_5``), consulted ahead of - ``LinkType.get`` since pyshark reports a filter name, not a DLT name. Out of scope - and deferred: 1026 minting call sites (1007 in ``_missing_``, 7 in ``get``, 12 - hand-written in ``pcapkit/protocols/internet/mh.py`` and - ``pcapkit/protocols/application/ngap.py``); the 106 ``register``/``register_alias`` - sites are the by-design caller-named path (:pr:`838`). + ``LinkType.get`` since pyshark reports a filter name, not a DLT name. The other + minting call sites in ``_missing_``, ``get`` and the hand-written ``mh.py`` and + ``ngap.py`` were deferred; the 106 ``register``/``register_alias`` sites are not + deferred but the by-design caller-named path (:pr:`838`). * **a breaking change to** ``pcapkit.const.ipx.socket.Socket``'s ``_missing_``: three of its five archived range branches were strict subsets of a range tested earlier -- ``(0x0020, 0x003F)`` after ``(0x0001, 0x0BB8)``, and ``(0x4000, 0x4FFF)`` and @@ -208,9 +185,7 @@ Fixed change name**: 32 from ``Registered by Xerox`` to ``Experimental``, 4095 from ``Dynamically Assigned`` to ``Dynamically Assigned Socket Numbers`` and 32764 from ``Dynamically Assigned`` to ``Statically Assigned Socket Numbers``. No defined - member changes and every value still resolves. The prior tests had pinned the - shadowing as intended (``# masks 'Experimental'``); they are corrected rather than - dropped, and a new test pins all three ranges as reachable (:pr:`847`). + member changes and every value still resolves (:pr:`847`). * ``LinkType(209).name`` returned ``'IPMB_LINUX'``, which tcpdump's own table marks "Legacy names (do not use)", because a 2024-05-04 regeneration emitted that row above the current ``I2C_LINUX = 209`` and ``aenum`` gives a value to @@ -218,9 +193,7 @@ Fixed whose note mentions "legacy" after every current row, so the current name is defined first; today that affects value 209 alone. ``IPMB_LINUX`` stays a reachable alias (``LinkType['IPMB_LINUX']`` still resolves to 209). Not - breaking: no name disappears, no value or membership changes, and - ``LinkType.IPMB_LINUX is LinkType.I2C_LINUX`` holds before and after; only the - canonical name for the shared value moves (:pr:`848`). + breaking: only the canonical name for the shared value moves (:pr:`848`). * **a breaking change to** 82 more const registries' ``_missing_``: applies the owner's mint/unmint ruling (settled on :pr:`847`, confirmed on :issue:`775` -- *"a final concrete assigned name -> mint; a notation for the readers -> unmint"*) across @@ -240,10 +213,11 @@ Fixed real ownership fact. 7 of the 89 are left alone: ``CGAType``'s ``Tag_`` mint is not an IANA-style range, and the other 6 files sit on 9 classes not yet on ``EnumRegistry`` (``AppType``, ``StatusCode``, ``ftp.return_code``, ``ftp.command``, - ``Method``, ``OptionType``), blocked on :pr:`859`/:issue:`860`. Each conversion + ``Method``, ``OptionType``), left to :pr:`859`/:issue:`860`. Each conversion changes both the vendor crawler and the generated file, per :pr:`838`'s and - :pr:`858`'s precedent. ``docs/source/conventions.rst``'s worked examples already - agreed with keeping ``Registered by Xerox`` under Mint (:pr:`861`). + :pr:`858`'s precedent. The worked examples in + ``docs/source/contributing/conventions/mint-criterion.rst`` keep + ``Registered by Xerox`` under Mint (:pr:`861`). * **a breaking change to** ``EtherType``'s ``_missing_`` ordering: values ``0x0101``-``0x01FF`` now resolve to their Old Xerox Experimental name instead of an ``IEEE802_3_Length_Field_0x0101``-style one. The generated method tested @@ -252,10 +226,8 @@ Fixed IPX fix, these bounds come from the live IANA CSV, so the fix is general: ``pcapkit/vendor/reg/ethertype.py``'s ``process()`` collects range rows and a new ``EtherType._insert_range`` places each ahead of the first already-placed range that - fully contains it, leaving non-overlapping pairs in CSV order. Of all 56 range - tests, this pair is the only containment. - ``tests/const/test_const_ethertype_862_unit.py`` pins the symptom, the root cause - and the ordering rule. Closes :issue:`862` (:pr:`865`). + fully contains it, leaving non-overlapping pairs in CSV order; this pair is the only + containment among the 56 ranges. Closes :issue:`862` (:pr:`865`). * **a breaking change to** ``Method``: ``__new__`` called ``str.__new__(cls)`` with no argument, so all 40 registered members' ``str`` payload was empty regardless of value -- ``str(Method.GET) == ''`` and ``Method.GET == 'GET'`` was ``False``; @@ -269,8 +241,8 @@ Fixed ``json.dumps(Method.GET)`` emits ``"GET"`` not ``""``; ``sorted()`` is lexicographic rather than input-order-stable; and ``{m: ... for m in Method}`` gains 40 entries where it had 1, because all 40 members previously hashed as ``''`` and compared - equal. Nothing under ``pcapkit/protocols`` relied on the old behaviour - (``httpv1.py``/``httpv2.py`` read only ``.value``). Closes :issue:`870` (:pr:`871`). + equal. Nothing under ``pcapkit/protocols`` relied on the old behaviour. Closes + :issue:`870` (:pr:`871`). * **a breaking change to** ``TransportProtocol`` and ``AppType``: step 2 of :issue:`860`, PR 2 of 2 (the 8 non-``AppType`` bespoke registries were :pr:`869`, above). ``AppType`` and its four transport subclasses move onto ``EnumRegistry`` and @@ -292,10 +264,9 @@ Fixed changes what specific integers *mean*: on ``AppType.get(80, proto=...)``, a bare ``4`` resolved to ``SCTP.http`` on ``main`` and now **silently** resolves to ``DCCP.unknown``, because a real member sits at ``4`` under both numberings; a bare - ``8`` used to mint ``DCCP.PORT_80_dccp`` and now raises ``ValueError``. The silent - reroute is disclosed here, and the composed-value case is deliberately untested, per - the owner's ruling that it is "an obsoleted path from a breaking change". - (:pr:`874`). + ``8`` used to mint ``DCCP.PORT_80_dccp`` and now raises ``ValueError``. The + composed-value case is deliberately untested, per the owner's ruling that it is "an + obsoleted path from a breaking change" (:pr:`874`). * closes the remaining scope of :issue:`775`: ``EtherType`` (52 branches) and ``Socket`` (1), the last registries whose ``_missing_`` minted a permanent member, now return a non-registering ``cls._unregistered_member(value, name)``. The issue's @@ -305,18 +276,12 @@ Fixed and return the identical object on a repeat; it now returns ```` with ``__members__`` unchanged, absent from ``_value2member_map_``, equal but not identical across lookups -- likewise - ``Socket(0x0010)``. Weaker verification, disclosed: ``vendor/ipx/socket.py`` - regenerates byte-identically offline, but ``vendor/reg/ethertype.py`` needs the live - IANA CSV, so it rests on a mechanical substitution matching exactly 52 lines plus a - run against the committed CSV fixture. Keeping each branch's hex-suffixed name + ``Socket(0x0010)``. The regenerated ``EtherType`` was not checked against the live + IANA CSV, only against the committed fixture. Keeping each branch's hex-suffixed name reintroduces the value-derived shape ``test_const_enum_no_mint.py``'s ``is_manufactured()`` sweep forbids; rather than weaken the predicate, a narrow - ``HEX_SUFFIXED_NAME_EXEMPT_PATHS`` covers exactly these two files, since the - collision the check guards against cannot occur once ``_unregistered_member`` never - registers. ``pcapkit/corekit/enum.py``'s docstring now names ``CGAType`` alone as - still minting. ``tests/vendor/test_ipx_socket_unit.py`` is unaffected - (``EXPECTED_MISSING_NAMES`` still carries ``0x0010: 'Registered by Xerox_0x0010'``) - (:pr:`878`). + exemption covers exactly these two files, since the collision the check guards + against cannot occur once ``_unregistered_member`` never registers (:pr:`878`). pcapkit.corekit --------------- @@ -326,18 +291,12 @@ Added * ``tests/corekit/test_fields_numbers_width_repair.py``, covering each octet boundary in its own method, since the defect is a pattern and a single case - would pass against a fix that special-cased the reported width. Each boundary - is asserted as a pair (the value below, which always packed, and the value - above, which did not), so a width shifted by one in the other direction fails - too. It asserts widths and octets rather than exception types, because the - exception depends on whether the mis-sized width has a native ``struct`` code - (:issue:`599`). -* ``FieldBaseShortReadPaddingSideTests``, ten cases over 133 subtests covering both - byte orders at 2, 4 and 8 octets and every truncation point, including the two - figures :issue:`604` reports as literals. Eight fail on the unfixed tree; the other - two are guard rails -- a full read at every width and order, which may not move, and - a read against an empty buffer, which pads to zeros either way and is the - :issue:`431` behaviour the option and list loops depend on (:issue:`604`). + would pass against a fix that special-cased the reported width (:issue:`599`). +* ``FieldBaseShortReadPaddingSideTests``, covering both byte orders at 2, 4 and 8 + octets and every truncation point, including the two figures :issue:`604` reports, + plus guard rails for a full read and for a read + against an empty buffer, the :issue:`431` behaviour the option and list loops depend + on (:issue:`604`). Changed ~~~~~~~ @@ -351,41 +310,26 @@ Changed ``ModuleDescriptor`` is resolved once and written back; a *miss* deliberately is not -- recording a miss in a class-level ``collections.defaultdict`` is the defect :issue:`421`/:pr:`426` fixed at this layer, :issue:`425`/:pr:`428` for the option, - chunk and block registries, and :issue:`555`/:pr:`560` at the schema layer. So every - unrecognised frame resolved the fallback descriptor again (48 of the 52 resolutions - in an extraction of ``many_interfaces.pcapng``); a resolution now costs ~117 ns - rather than ~436 ns. **This is not measurable in** ``extract()`` **wall clock**: 48 - avoided calls is ~17 us against a ~37 ms extraction, inside run-to-run variance, and - the "~40% of cumulative time" reading that prompted the work was an artifact of - ``_import_next_layer`` being a recursive dispatcher. The change is taken for its - shape: ``sys.modules`` stays the only module cache, so its invalidation is the - interpreter's, whereas a memo of the class would serve the pre-reload class after an - ``importlib.reload`` forever and fail ``isinstance`` against the live one. Proposed - by ``@Ts-Boom`` in :pr:`563`, whose profiling found the miss path; the - implementation differs because that one added a second, never-invalidated cache of - resolved classes (:issue:`574`). + chunk and block registries, and :issue:`555`/:pr:`560` at the schema layer. A + resolution now costs ~117 ns rather than ~436 ns, which is **not measurable in** + ``extract()`` **wall clock**. The change is taken for its shape: ``sys.modules`` + stays the only module cache, whereas a memo of the class would serve the pre-reload + class after an ``importlib.reload`` forever and fail ``isinstance`` against the live + one. Proposed by ``@Ts-Boom`` in :pr:`563`; the implementation differs because that + one added a second, never-invalidated cache of resolved classes (:issue:`574`). * ``SeekableReader.truncate()`` raises instead of returning a size, and the misspelled ``writeable()`` is now ``writable()``. Both break external callers; nothing inside the package calls either. ``io.IOBase`` documents one gate over two methods -- "If False, write() and truncate() will raise OSError" -- and this reader's ``writable()`` answers False, yet ``truncate`` - returned its new size. Every ordinary read-only file object in CPython raises - here (``open(path, 'rb').truncate()`` gives ``io.UnsupportedOperation``), so - this is not a pure-Python nicety that the C path skips. The refusal is - ``UnsupportedOperation('truncate')``, the same one ``write`` raises; - pcapkit's subclasses ``io.UnsupportedOperation``, so the house exception *is* - the one the contract names. The old resizing is not deleted, only made - private as ``_truncate_buffer``: it resizes a private lookback window, not - the stream (the issue's own counter-argument), and it is the only route to - the "position sits before the window" state that ``seek`` and the four - buffered read paths must refuse, for which :issue:`643` and :issue:`644` - landed tests. ``writeable()`` was never an override of anything -- - ``SeekableReader.writable is io.IOBase.writable`` was True -- so ``io``, - ``shutil`` and third parties never saw it; both answered False, which hid it, - and the value reported is unchanged. The new test asserts the property, that - a ``SeekableReader`` raises the same exception type as a read-only - ``io.BufferedReader``, capturing that type by running the call so it tracks - CPython across 3.10--3.15. (:issue:`645`). + returned its new size, where ``open(path, 'rb').truncate()`` raises. The refusal is + ``UnsupportedOperation('truncate')``, the same one ``write`` raises; pcapkit's + subclasses ``io.UnsupportedOperation``. The old resizing is kept as the private + ``_truncate_buffer``: it resizes a private lookback window, not the stream, and it is + the only route to the "position sits before the window" state that ``seek`` and the + four buffered read paths must refuse, for which :issue:`643` and :issue:`644` landed + tests. ``writeable()`` was never an override of anything, so ``io``, ``shutil`` and + third parties never saw it, and the value reported is unchanged (:issue:`645`). * **a breaking change to** ``@final``, now enforced at runtime on ``Info`` and ``Schema``. ``info_final`` and ``schema_final`` both end ``return final(cls)``, so every finalised class carried ``typing.final``'s ``__final__`` marker and nothing @@ -401,19 +345,10 @@ Changed subclass declaration will not see it. ``@info_final @final`` and ``@final @info_final`` both finalise silently; the re-entry check keys on ``__finalised__`` rather than ``__final__``, because only the former records *the - decorator* having run and decorators apply bottom-up, so a ``__final__`` test would - read a class marked an instant earlier as already finalised. ``@info_final`` twice - warns and returns the class unchanged. Every check reads its marker from the class's - own ``__dict__``, since the markers are ordinary inheriting attributes. + decorator* having run. ``@info_final`` twice warns and returns the class unchanged. ``pcapkit.utilities.compat`` takes ``final`` from ``typing_extensions`` below 3.11 - rather than 3.8, because ``typing.final`` only records ``__final__`` from 3.11 and - the guards were otherwise a silent no-op on the 3.10 leg. - ``EnumSchema.__init_subclass__`` calls the base hook first so a refused declaration - cannot leave ``__enum__`` pointing at a discarded class, and - ``pcapkit.protocols.schema.misc.pcapng``'s ``Option.__init_subclass__`` is reordered - to match. Additive in tree: no ``Info`` or ``Schema`` class is subclassed after - finalising or carries ``__final__`` without ``FinalisedState.FINAL``, so the guard - cannot fire on the library itself (:issue:`778`). + rather than 3.8, because ``typing.final`` only records ``__final__`` from 3.11. + The guard cannot fire on the library's own classes (:issue:`778`). Fixed ~~~~~ @@ -440,27 +375,15 @@ Fixed ``FieldBase.unpack`` now keeps a per-context ledger of octets supplied against octets synthesised, and raises ``FieldValueError`` once the total of shortfalls *past 65,536 octets* passes ``262144 + 16 * supplied``. The same 200 blocks retain 0.2 MiB (54.6x). - A shortfall of 65,536 octets or fewer is padded unconditionally and charged to - nothing, and that band is load-bearing. 65,536 is the span of a 16-bit wire length - (IP header, IPv6 payload, TCP/IPv4 option, PCAP-NG option), so no shortfall from a - snapshot-truncated capture is subject to the budget. Without the band, a running - budget alone made the *same* legitimate 54-octet frame declaring an IPv4 total length - of 65,535 parse one way on 37 of 40 identical calls and another on the rest, - depending on what had been parsed before; a guard whose answer moves with history is - worse than the amplification it bounds. **What this does not close**: a length - declared by a 16-bit field can still be repeated without limit. A crafted - 80,048-octet PCAP-NG file of 2,000 Enhanced Packet Blocks, each with one option - declaring 65,535 octets against four real ones, retains 125.00 MiB of zeros (1,637x - its size), before and after. That is not an oversight: a bare 40-octet IPv4 header - declaring total length 65,535, which is what a legitimate capture of offload-sized - segments truncated to its snapshot length looks like, amplifies by **the same - 1,637x**, and no budget at this layer separates them. Separating them needs the - frame's own ``incl_len``/``orig_len``, knowable at the protocol layer and not here. - Nor is the budget per file: nothing resets the ledger, so it bounds everything a - context has parsed, which is still proportionate to the octets that context was given; - tightening it is a one-line change at whichever layer owns a run. Padding tallies and - each parse's outcome are identical to before across every capture in - ``examples/captures/`` truncated at some 8,700 offsets (:issue:`573`). + A shortfall of 65,536 octets or fewer, the span of a 16-bit wire length, is padded + unconditionally and charged to nothing, so no snapshot-truncated capture is subject + to the budget; without that band the same legitimate frame parsed differently + depending on what had been parsed before. **What this does not close**: a length + declared by a 16-bit field can still be repeated without limit, and at this layer a + crafted file is indistinguishable from a legitimate snapshot-truncated capture of + offload-sized segments; separating them needs the frame's own + ``incl_len``/``orig_len``. Nothing resets the ledger, so it bounds everything a + context has parsed rather than one file (:issue:`573`). * which exception a malformed TCP SACK option raised depended on unrelated process state: a clean interpreter raised ``ProtocolError`` as documented, but a process that had popped ``pcapkit.corekit.fields.misc`` from ``sys.modules`` @@ -471,8 +394,7 @@ Fixed is a second, distinct class, so ``isinstance`` misclassified the field and billed each item by its declared length instead of what it consumed. **Any caller relying on the previously-observed** ``FieldValueError`` **now gets** - ``ProtocolError``, deterministically, matching the docstring. Fixed by - importing at module level. + ``ProtocolError``, deterministically, matching the docstring. * a ``NumberField`` whose ``length`` was a callable could not pack or parse at any width ``struct`` has a native integer code for. ``length`` is ``-1`` until the callable resolves, ``-1`` has no native code, and the template builder raised @@ -494,31 +416,21 @@ Fixed ``65536`` at two, ``16777216`` at three. **The reach is wider than "just past a boundary"**: every bit length that is not a multiple of eight is wrong, so ``1`` (bit length 1, floored to *zero* octets) and every value from 1 to 127 failed too. - It is now the ceiling it was meant to be, matching the ``math.ceil(n / 8)`` idiom - used elsewhere. The repair is reached only by packing a field the caller never - resolved (a schema resolves every field first), which is why it survived - :issue:`591`'s suite: its five repair-path values (``0xFF``, ``0xFFFF``, - ``0xFFFFFFFF``, ``0xFFFFFFFFFFFFFFFF``, ``0x800001``) have bit lengths of 8, 16, 32, - 64 and 24, where floor and ceiling agree. :issue:`591`'s fix neither caused nor - masked this, but it changed the failure: a mis-sized 1, 2 or 4 octets now surfaces - from ``struct.pack`` as ``'B' format requires 0 <= number <= 255`` rather than - ``OverflowError`` from ``int.to_bytes``. Deliberately left alone: a signed field is - sized without room for its sign bit, so an unresolved signed field still cannot pack - ``128``; and an unresolved field's bit mask is ``-1``, making the masking and sign - remap no-ops. The same ``math.ceil(x.bit_length() // 8)`` survives at the two ILNP - nonce option builders in ``hopopt.py`` and ``ipv6_opts.py``, a separate change - (:issue:`599`). + It is now the ceiling it was meant to be. The repair is reached only by packing a + field the caller never resolved (a schema resolves every field first), which is why + :issue:`591`'s suite, whose values all had byte-aligned bit lengths, missed it; + :issue:`591`'s fix neither caused nor masked it. + Deliberately left alone: a signed field is sized without room for its sign bit, so + an unresolved signed field still cannot pack ``128``. The same typo at the two ILNP + nonce option builders is a separate change (:issue:`599`). * ``FieldBase.unpack`` padded a short read on the wrong side. When a buffer falls short of the declared length -- the accommodation that lets a snapshot-truncated capture parse (:issue:`431`) -- the shortfall was zero-filled with ``rjust()``, placing the zeros at the *front*, as if the octets never read were the leading ones; a short read has lost the trailing ones. It is now ``ljust()``, correct for **both** - byte orders. The two directions failed differently and only one was loud: one octet - of a four-octet little-endian ``120`` read as ``2013265920`` (inflated by - ``2 ** 24``), while three octets of a four-octet big-endian ``0x01020304`` read as - ``0x10203`` (scaled *down* by 256). The second is the dangerous one, since a value - smaller than the truth passes a sanity check where an inflated one overruns, which - is why the big-endian half went unnoticed. A full read is untouched (:issue:`604`). + byte orders: one octet of a four-octet little-endian ``120`` read as ``2013265920``, + and three octets of a four-octet big-endian ``0x01020304`` as ``0x10203``. A full + read is untouched (:issue:`604`). * ``SeekableReader.truncate`` put its padding where the reader's bookkeeping says the content is, and ``read`` then returned one octet fewer than the buffer held. Same family as :issue:`604`, but two defects. The buffer keeps content at @@ -543,17 +455,14 @@ Fixed which ``seek`` reads as its licence to fetch more; clamping ``_buffer_cur`` alone made that sum under-report, and the next forward ``seek`` silently spliced in octets from the wrong offset (``read(8)``, ``truncate(3)``, ``seek(6)``, ``read(1)`` over - ``b'abcdefghijklmnop'`` returned ``b'l'`` where ``b'g'`` sits at offset 6). All - three came from fuzzing random operation sequences against the buffer's invariants: - of 3000 rounds, 2372 left the bookkeeping inconsistent and 683 raised an - undocumented exception, and none do now. Latent rather than live -- nothing here - calls ``truncate``, though it is public -- and invisible to existing tests, which - asserted the return value and never the content (:issue:`622`). + ``b'abcdefghijklmnop'`` returned ``b'l'`` where ``b'g'`` sits at offset 6). Latent + rather than live: nothing here calls ``truncate``, though it is public + (:issue:`622`). * the 16-bit band the :issue:`573` entry above records as deliberately left open: a length declared by a 16-bit wire field could still be repeated without limit. The - crafted 80,048-octet PCAP-NG of 2,000 Enhanced Packet Blocks that entry describes - synthesised 131,070,000 octets of zeros (1637.393x its size) and now synthesises - none, with all 2,000 frames and options still parsed. The bound is taken one layer + crafted PCAP-NG of 2,000 Enhanced Packet Blocks, each with an option declaring 65,535 + octets against four real ones, synthesised 1,637x its size in zeros and now + synthesises none, with every frame and option still parsed. The bound is taken one layer up, where the missing information exists: a PCAP-NG block's Block Total Length is authoritative and cross-checked against its trailing copy, so the option area is that length less the fixed fields, ``captured_len`` and its padding, and an option @@ -567,27 +476,18 @@ Fixed ledger already budgets. A second, ``bounded_area``, clamps a packet block's option area to the octets the block itself holds less the trailing Block Total Length: the area was otherwise sized from a declared length nothing checks against the file, so a - block declaring 1,000,000 octets while holding 36 had a 999,964-octet area and 65,535 - zeros were synthesised from 36 (1,820x) with no warning. The cross-review found that - one. Both are no-ops on a well-formed block. The five non-packet blocks' option areas - keep the framing assumption; the general fix for a declared length reaching a read is - :issue:`678`. Clamping rather than refusing keeps the :issue:`431` accommodation, - since a PCAP-NG block read has no catch point above ``FieldBase.unpack`` and one - refusal would abort a whole extraction rather than one block; and the clamp reads only - the block's own declared framing, never a running total, so identical input answers - identically whatever preceded it. The skip for a remainder already past zero is - load-bearing on the unpacking path (a ten-octet area leaves ``__length__`` at -2 - before a payload sizes itself, and clamping would give the field a ``'-2s'`` - template); it is *not* what protects packing, since ``BytesField`` and - ``StringField`` repair a negative width in ``pre_process``. No clamp fires across the - six PCAP-NG sample captures (338 options), and 501 truncation levels of - ``dhcp.pcapng`` give byte-identical outcomes, most being the uncaught ``ValueError`` - that :issue:`678` records. The bound is pinned as a property, not one - input: across every combination of one to three options and nine declared lengths the - options never report more octets than the area, and the ratio is flat across 1, 8, 64 - and 512 blocks, which separates a bound from a smaller constant. The worst ratio over - five adversarial shapes is now 0.862x, against 1,637x, 960x and 224x for the same - three before (:issue:`594`). + block declaring 1,000,000 octets while holding 36 synthesised 65,535 zeros with no + warning. Both are no-ops on a well-formed block. The five non-packet blocks' option + areas keep the framing assumption; the general fix for a declared length reaching a + read is :issue:`678`. Clamping rather than refusing keeps the :issue:`431` + accommodation, since a PCAP-NG block read has no catch point above + ``FieldBase.unpack`` and one refusal would abort a whole extraction rather than one + block; and the clamp reads only the block's own declared framing, never a running + total, so identical input answers identically whatever preceded it. The skip for a + remainder already past zero is load-bearing on the unpacking path, where clamping + would give the field a negative template such as ``'-2s'``. No clamp fires on the + sample captures, and :issue:`678`'s uncaught ``ValueError`` on truncated input is + unchanged (:issue:`594`). * **a breaking change to** ``FieldBase.length``: it called ``struct.calcsize`` on a template built from a negative resolved length (e.g. ``'-5s'``, from a ``length=`` callback whose running counter had been overdrawn), raising a bare @@ -613,13 +513,11 @@ Fixed generator's bare ``extend_enum(cls, name, value)``, so ``pcapkit.const`` ships two contradictory ``register`` contracts until :pr:`858` (below). The six also gain the generated template's ``KeyError`` catching on a missing name, so - ``Flags.get('NOPE', 0)`` returns ``0`` rather than raising (a widening, undisclosed - until this PR); and a key that is neither ``int`` nor ``str``, which raised - ``TypeError`` on the five ``IntFlag`` registries' hand-copied ``get`` - (``Flags.get(3.5)``, since ``cls[key]`` is a containment test on a ``Flag``), now - raises ``ValueError``, tried as a value. ``ExtensionHeader``, the only ``IntEnum`` - among them, already raised ``KeyError`` there, the shift :pr:`858`'s entry - attributes to the other 105 (:pr:`855`). + ``Flags.get('NOPE', 2048)`` returns ``Flags.ACK`` rather than raising; and a key that is neither + ``int`` nor ``str``, which raised ``TypeError`` on the five ``IntFlag`` registries + (``Flags.get(3.5)``), now raises ``ValueError``, tried as a value. On + ``ExtensionHeader`` it raised ``KeyError``, the shift :pr:`858`'s entry attributes + to the other 105 (:pr:`855`). * **a breaking change to** ``EnumRegistry.register()``, extended: the 105 const registries still on the generated template's bare ``extend_enum`` now inherit the guard :pr:`855` (above) added, so ``TransType.register(6, 'TOTALLY_NEW_NAME')`` @@ -627,14 +525,11 @@ Fixed ``register_alias()``, where it used to mint and raise nothing. ``pcapkit/vendor/default.py``'s template now emits ``class {NAME}(EnumRegistry, IntEnum)`` and drops its own - ``get``/``register``/``_unregistered_member``. Across 121 const modules holding 127 - enum classes: 6 already used ``EnumRegistry`` (:pr:`855`); of the other 115 modules, - 105 share the template byte-for-byte and are converted, and the remaining 10 keep - their bespoke ``__new__`` and are left alone: ``ftp/command`` (4 classes), - ``ftp/return_code`` (3), ``http/method``, ``http/status_code``, - ``pcapng/option_type``, ``reg/apptype/apptype.py`` (``AppType`` and - ``TransportProtocol``) and its four transport subclasses. That makes **111 of the - 127 classes** inherit ``EnumRegistry``, up from 6. ``get()``'s dispatch shifts only + ``get``/``register``/``_unregistered_member``. The 10 modules with a bespoke + ``__new__`` (``ftp/command``, ``ftp/return_code``, ``http/method``, + ``http/status_code``, ``pcapng/option_type`` and ``reg/apptype``) are left alone, so + **111 of the 127 classes** inherit ``EnumRegistry``, up from 6 (:pr:`855`). + ``get()``'s dispatch shifts only for a key that is neither ``int`` nor ``str`` (``KeyError``, tried as a name; now ``ValueError``, tried as a value), a key type nothing in the library passes. ``pcapkit/corekit/enums.py`` is renamed **``enum.py``**, the maintainer's ruling @@ -652,8 +547,7 @@ Fixed converts; a dict lookup never reaches ``_missing_``. A name still wins when a string is both a name and a different member's value. Step 1 of :issue:`860`; step 2, converting the bespoke registries, follows (:pr:`869`, :pr:`874`, - below). The ``IntEnum``/``IntFlag`` non-``str`` path is unchanged, measured on - ``TransType`` and ``HandoverACKFlag``. Widens rather than breaks: a call that + below). The non-``str`` path is unchanged. Widens rather than breaks: a call that raised ``KeyError`` now resolves, and no caller that got a correct result gets a different one (:pr:`863`). * **a breaking change to** ``EnumRegistry.get()``: its docstring claimed "It never @@ -667,17 +561,10 @@ Fixed exception permits. A ``default`` naming no registered member now falls through to the error ``key`` itself would have raised, rather than a fresh error about the default -- the accepted cost of the ruling. The docstring's "It never mints" is - qualified to ``default`` only. New ``GetDefaultNoMintTests`` pin the issue's repro - on the real ``EtherType`` registry, the preserved key-path mint, and default - resolution on an ``int`` and a ``str`` registry. Four assertions in - ``tests/const/test_const_enum_get.py`` and - ``tests/const/test_const_enum_builtin_parity.py`` that encoded the old contract are - retargeted rather than weakened; a fifth, - ``Hardware.get('Definitely-Not-A-Member', 40)`` (``40`` sits in ``Hardware``'s - declared-but-unassigned range and now raises ``KeyError`` naming the original key), - waited on :pr:`865` because its test file was contended, then landed once :pr:`865` - (above) merged, with ``Hardware``'s member count unchanged at 42. Closes - :issue:`864` (:pr:`868`). + qualified to ``default`` only. So ``Hardware.get('Definitely-Not-A-Member', 40)``, + with ``40`` in ``Hardware``'s declared-but-unassigned range, now raises ``KeyError`` + naming the original key; its test waited on :pr:`865` (above), then landed once + :pr:`865` merged. Closes :issue:`864` (:pr:`868`). pcapkit.dumpkit --------------- @@ -696,17 +583,10 @@ Fixed ``Flags::0 [0]`` and ``Flags(8)`` gives ``Flags::8 [8]``. That is the spelling the enumeration libraries use for an undeclared residue (``Flags(2057).name`` is ``ACK|9``), and it cannot be mistaken for a member name, since an identifier may not - begin with a digit, whereas ``NONE`` is a real declared name elsewhere. Three sites - carried the identical interpolation (the ``OrderedMultiDict`` key path, the - ``addon`` branch and the scalar return) and now share one ``render_enum`` helper. - The guard is on ``name is None`` and not on the value, because the defect was never - about zero (``Flags(1)``, ``Flags(8)``, ``Flags(9)`` and ``Flags(65536)`` are - equally nameless) and is not an ``aenum`` quirk: stdlib ``enum.IntFlag`` answers - ``name is None`` at the same values. Five of the seven flag registries are nameless - at zero (``Flags`` and the four Mobility Header flags; the other two declare - ``undefined = 0``). Pre-existing since 2023-04-28 and surfaced by :pr:`634`, which - made a flagless TCP segment reach this path. The committed example dumps do not move - (:issue:`648`). + begin with a digit, whereas ``NONE`` is a real declared name elsewhere. The guard is + on ``name is None`` and not on the value, because the defect was never about zero + (``Flags(1)`` and ``Flags(9)`` are equally nameless). Surfaced by :pr:`634`, which + made a flagless TCP segment reach this path (:issue:`648`). * ``tests/dumpkit/test_nameless_enum_rendering_unit.py`` (added by :pr:`670`) carried two probes assuming every flag-enum registry is 16 bits wide: ``test_scalar_return_renders_a_nameless_member_as_its_value`` used one fixed tuple @@ -716,14 +596,9 @@ Fixed runtime), the three ``mh`` ack/initiate flags 8, and ``mh.binding_update_flag.BindingUpdateFlag`` and ``tcp.flags.Flags`` 16. ``tcp.flags.Flags(65536)`` correctly raises (``0 <= value <= 0xFFFF``), so the sweep - was failing on correct behaviour. A new ``_field_mask`` derives each registry's own - bound from its declared members, ``_nameless_values`` returns the values that bound - admits but no member names (zero, each undeclared bit alone, all combined), and - ``test_a_value_past_the_field_is_refused_rather_than_rendered`` pins the boundary: - the widest in-field value is accepted, one past it is refused. A cross-review found - all seven guard ``raise`` lines it reaches (``tcp/flags.py``'s own included) were - already covered by a test predating the fix, so none was newly reached. Nothing - under ``pcapkit/`` changed (:issue:`702`). + was failing on correct behaviour. The probes now derive each registry's bound from + its declared members, and a new test pins that one past the widest in-field value is + refused. Test-only (:issue:`702`). * the plist writer emitted mapping keys unescaped, so a key carrying ``&``, ``<`` or ``>`` produced a report that is not well-formed XML. ``dictdumper``'s ``_append_dict`` interpolates the key into ``'{item}'`` and calls @@ -731,15 +606,10 @@ Fixed that build a mapping now escape their keys through one ``escape_key`` helper -- a ``MultiDict``, where :issue:`575` escaped an enum-derived key inline and nothing else, and a plain dict, where nothing was escaped. The json, tree and text writers - are untouched (``escape_key`` is a no-op for them and a plain dict is not rebuilt). - Across 6 captures and 5 formats, exactly 2 of 30 reports changed (``test.pcapng``'s - plist and xml, one writer under two names), by one line each; that capture is why a - non-``str`` key is handled at all, its decryption secrets block keying TLS key log - entries by a raw bytes client random whose repr carries all three characters. The - upstream half is ``JarryShaw/DictDumper#125`` (``_append_dict`` should call - ``_encode_value`` on a key); this does not wait on it. ``PcapngUnescapedKeyTests``' - docstrings, which claimed both the json and plist reports were unparseable, are - reworded: only the json writer's quoting defect stands, upstream as + are untouched. A non-``str`` key is escaped too: a PCAP-NG decryption secrets block + keys TLS key log entries by a raw bytes client random whose repr can carry all three + characters. The upstream half is ``JarryShaw/DictDumper#125``; this does not wait on + it. The json writer's quoting defect stands, upstream as ``JarryShaw/DictDumper#121`` (:issue:`772`). pcapkit.foundation @@ -819,15 +689,11 @@ Changed ``__protocol_name__`` still set the name a class reports, registered or not. Each metaclass gained a class-level ``registry`` property mirroring ``EnumSchema.registry``, and a ``Dumper`` subclass no longer touches the filesystem - while its ``class`` statement runs (inferring ``fmt`` from ``kind`` meant - instantiating it against a ``NamedTemporaryFile``). ``Engine``'s keyword is - ``engine=`` rather than the ``name=`` first shipped, because ``name`` cannot be a - class keyword on Python 3.10: ``mcls``, ``name``, ``bases`` and ``namespace`` - collide with ``abc.ABCMeta.__new__``'s parameters (positional-or-keyword before - 3.11, positional-only from 3.11), raising ``TypeError`` before the hook runs. Those - four are the whole collision surface (measured on 3.10.21, 3.11.15 and 3.14.7), and - ``engine=``, ``protocol=`` and ``fmt=`` are outside it. There is no ``name=`` alias: - a keyword that works on some interpreters and not others is the trap being removed. + while its ``class`` statement runs. ``Engine``'s keyword is ``engine=`` rather than + the ``name=`` first shipped, because ``mcls``, ``name``, ``bases`` and ``namespace`` + collide with ``abc.ABCMeta.__new__``'s parameters on Python 3.10 and raise + ``TypeError`` before the hook runs. There is no ``name=`` alias: a keyword that works + on some interpreters and not others is the trap being removed. * **a breaking change to** ``Extractor.register_engine``, ``register_reassembly`` and ``register_traceflow``, and the ``register_extractor_*`` wrappers over them: each now accepts only a subclass of the public ``Engine``, ``Reassembly`` or @@ -838,14 +704,10 @@ Changed carrying the registration hook. Third-party code that subclasses ``EngineBase``, ``ReassemblyBase`` or ``TraceFlowBase`` directly and registers through these functions now gets ``RegistryError`` naming the public class; subclass the public - class instead. The error messages are unchanged. The built-ins are unaffected - because they are declared directly in ``Extractor``'s class-body mappings as - ``ModuleDescriptor`` literals and never pass through a registrar at all; a new - internal path, ``Extractor._register_internal_engine`` and its two siblings, - checks the base for programmatic internal registration and is what the - :issue:`513` regression test now exercises. ``register_protocol`` is - untouched: every in-house protocol derives from ``ProtocolBase`` and none from - ``Protocol``. + class instead. The error messages are unchanged. The built-ins never pass through a + registrar; ``Extractor._register_internal_engine`` and its two siblings check the + base for internal registration, and are what the :issue:`513` regression test now + exercises. ``register_protocol`` is untouched. * extraction is around 46% faster on a 1,117-frame HTTP capture, with byte-identical output (:pr:`420`). A reassembled datagram's payload is analysed on first read rather than eagerly, cutting IP reassembly's own cost by 90.7% and TCP's by 23.7% @@ -856,12 +718,10 @@ Changed either (:pr:`427`). * **a breaking change to** ``Extractor.engine`` and ``Extractor.record_header``: both are now declared to return ``EngineBase[_P]`` rather than ``Engine``, and the private - ``_exeng`` attribute follows. Both built-in engines subclass ``EngineBase`` directly, - so neither was ever an ``Engine``; the declaration now names the class the object - has, and the ``cast`` calls that hid the gap now launder only the frame type - parameter. Nothing is lost: ``Engine`` adds only the ``__init_subclass__`` - registration hook, and the frame type parameter is preserved. Two things are - visible to a type checker. Code passing the result to an ``Engine``-typed + ``_exeng`` attribute follows. The built-in engines subclass ``EngineBase`` directly, + so none was ever an ``Engine``, which adds only the ``__init_subclass__`` + registration hook. Two things are visible to a type checker. Code passing the + result to an ``Engine``-typed parameter must now accept ``EngineBase``; and because the old declarations were *unparameterised*, ``engine.read_frame()`` and ``record_header().read_frame()`` used to be ``Any`` and are now the extractor's own frame type, so an assignment @@ -870,14 +730,11 @@ Changed ``module``, ``registry``, ``extractor``, ``unsupported_reason``, ``run``, ``read_frame``, ``close`` and the rest) moved from ``Engine`` to ``EngineBase``, where they are defined, and ``Engine`` now documents only the - ``__init_subclass__`` registration hook it adds. ``Extractor.engine`` declares - ``EngineBase[_P]`` as its return type, but that class was documented - ``:no-members:`` under *Internal Definitions*, so the type a reader followed - led to no API. Cross-references to ``Engine.run``, ``Engine.read_frame``, - ``Engine.close`` and ``Engine.unsupported_reason`` now point at ``EngineBase``, - and the ``PCAP`` engine's ``run`` docstring lost the verbless sentence - "will also be save the current ``PCAP`` engine instance". ``EngineBase`` is - still not exported in ``__all__``; that is left to :issue:`1016` (:issue:`1024`). + ``__init_subclass__`` registration hook it adds, so the ``EngineBase[_P]`` that + ``Extractor.engine`` returns leads to an API. Cross-references to ``Engine.run``, + ``Engine.read_frame``, ``Engine.close`` and ``Engine.unsupported_reason`` now point + at ``EngineBase``. Exporting ``EngineBase`` in ``__all__`` is left to :issue:`1016` + (:issue:`1024`). Fixed ~~~~~ @@ -895,19 +752,14 @@ Fixed ``not self._flag_s`` (the flag that gated the ``open()``), and the ``SeekableReader`` wrapping a non-seekable input asked for ``stream_closing=not self._flag_s`` in the same wrong direction. The leak was one - descriptor per extraction for the life of the process, and the production root cause - of the ``ResourceWarning`` that reddened :pr:`577`, :pr:`596` and :pr:`600`; - :issue:`606` made ``tests/utilities/test_stacklevel.py`` immune to foreign warnings, - which repaired the CI signal but not the leak. Closing the caller's stream was the - more dangerous half: silent data loss for anyone passing an open file they meant to - keep reading. Both conditions now follow ownership via a single - ``Extractor._owns_input``, and a finaliser releases the handle of an extraction - *abandoned* before end of file (``auto=False``, iterated part way and dropped), - which reaches ``_cleanup`` by no route and was the second warning. A path-given - extraction used to leave 1 descriptor open and a stream-given one came back closed - and unreadable; the two files :issue:`610` names emit 0 such warnings where they - emitted 2. ``__exit__`` deliberately still closes unconditionally, since a caller - who scopes an ``Extractor`` with ``with`` asked for that (:issue:`610`). + descriptor per extraction, the root cause of the ``ResourceWarning`` that reddened + :pr:`577`, :pr:`596` and :pr:`600` (:issue:`606` had only made the test immune). + Both conditions now follow ownership via a single ``Extractor._owns_input``, and a + finaliser releases the handle of an extraction *abandoned* before end of file + (``auto=False``, iterated part way and dropped); the two files :issue:`610` names + emit no such warning. ``__exit__`` deliberately still + closes unconditionally, since a caller who scopes an ``Extractor`` with ``with`` + asked for that (:issue:`610`). * ``extract(..., no_eof=True)`` never returned. End of stream was detected and ``ExtractionWarning: EOF reached`` fired, but all three loops that handle it (``record_frames``, ``__next__``, ``__call__``) read ``if self._flag_n: continue``, @@ -919,26 +771,19 @@ Fixed finished one: ``prepare`` raises end of stream when zero bytes remain and restores the position first, so two consecutive ends of stream at the same position mean nothing arrived between them (``Extractor._note_eof_progress``). A pipe is - unaffected, since a read there *blocks* while the writer is open but idle. Also - fixed, not in the issue: ``pcapkit -`` hung on **any** finished stdin; - ``cat in.pcap | python -m pcapkit -`` went from killed at 25s to exiting 0 with a - byte-identical dump. **One deliberate narrowing**: a *seekable* input does not - block, so its two probes fall microseconds apart and a file still being appended to - ends at the data present when the extraction reached it. The previous behaviour was - unbounded, which is the defect; a timed grace period would make the cut-off - intermittent rather than absent, so following a growing file is left to a policy of - its own and the decision is pinned by a test. Pre-existing and untouched: an append - landing *mid-record* raises ``ValueError: read length must be non-negative or -1``, - so ``no_eof`` over a growing file only ever worked for a writer flushing whole - records. Both ``no_eof`` docstrings say all of this. The end-to-end regression tests - are bounded by a **child process** rather than ``tests._support.time_limit``, - because the in-process deadline proved *intermittent* on this loop (once it escaped - entirely, past ten minutes and 13.2 GB RSS), and an intermittent guard against a - hang leaves a wedged suite rather than a red test. Two guesses at the cause are - ruled out: the parse path's two ``except Exception`` handlers are never entered - during the spin (``prepare`` raises before any next-layer decode), and the memory is - the retry loop emitting tens of thousands of ``EOF reached`` records a second, which - the runner retains (:issue:`620`). + unaffected, since a read there *blocks* while the writer is open but idle. This also + fixes ``pcapkit -``, which hung on **any** finished stdin. **One deliberate + narrowing**: a *seekable* input does not block, so a file still being appended to + ends at the data present when the extraction reached it; a timed grace period would + make the cut-off intermittent rather than absent, so following a growing file is + left to a policy of its own. Pre-existing and untouched: an append landing + *mid-record* raises ``ValueError: read length must be non-negative or -1``. The + end-to-end tests are bounded by a **child process** rather than + ``tests._support.time_limit``, because the in-process deadline proved intermittent on + this loop, and an intermittent guard against a hang leaves a wedged suite rather than + a red test. Two guesses at the cause are ruled out: the parse path's two + ``except Exception`` handlers are never entered during the spin, and the memory is + the retry loop's ``EOF reached`` records, which the runner retains (:issue:`620`). * ``register_protocol`` displaced one protocol class with another and said nothing. The registry behind ``pcapkit.protocols.__proto__`` is keyed on ``cls.__name__.upper()``, and three dispatchable classes are all named ``HTTP`` (the @@ -954,37 +799,25 @@ Fixed built-in. A plain ``import pcapkit`` emits zero new warnings: the three built-in ``HTTP`` classes derive from ``ProtocolBase``, so ``__init_subclass__`` never fires for them, and import-time seeding keys off the distinct names ``HTTP``, ``HTTPv1`` - and ``HTTPv2``. The issue called this the one registrar in - ``pcapkit/foundation/registry/`` with no warning; more accurately, none warned by - itself, the siblings only by delegating to a classmethod with an - ``if code in cls.__xxx__: warn(...)`` guard (:pr:`726` later gave all seven the - identity guard described elsewhere), and this is the only ``__name__.upper()``-keyed - registry in ``pcapkit/`` that mutates its target with no guard and no delegate able - to supply one. The new guard deliberately reads "key present and the incumbent is a - different class" where siblings warned on mere presence then: this key is derived - from the class, and this function is the funnel all nine wrapper registrars end in, - so registering one class under two codes (``register_tcp`` then ``register_udp``, - supported and documented) reaches it twice with nothing displaced, and warning there - would put noise on a documented path whose wholesale filter would then hide the real - collision. The collision is reported, not resolved: re-keying is a registry-format - change that none of the bare-name readers can absorb (each degrades *silently* to a - string or ``Raw`` on a miss), so it belongs to the registry redesign in - :issue:`514`, which this is sequenced ahead of (:issue:`675`). + and ``HTTPv2``. The guard deliberately reads "key present and the incumbent is a + different class", not mere presence (:pr:`726` later gave all seven siblings the + same identity guard): this function is the funnel all nine wrapper registrars end + in, so registering one class under two codes (``register_tcp`` then + ``register_udp``, a documented path) reaches it twice with nothing displaced. The + collision is reported, not resolved: re-keying is a registry-format change that none + of the bare-name readers can absorb (each degrades *silently* to a string or ``Raw`` + on a miss), so it belongs to the registry redesign in :issue:`514`. The issue called + this the one registrar with no warning; in fact none warned by itself, the siblings + only by delegating to a guarded classmethod (:issue:`675`). * two ``#:`` autodoc comments in ``pcapkit/foundation/traceflow/traceflow.py`` named a bare ``Type``, which Sphinx resolves against every class named ``Type`` in the project (five) rather than ``typing.Type``, and silently linked to - ``pcapkit.const.l2tp.type.Type``, an L2TP field-type enum. Line 424 - (``#: ~typing.Type[Dumper]: Dumper class.``, spelled out since :issue:`709` fixed - it) is the live case: once :issue:`684` rendered ``TraceFlow._foutio``, the built - docs pointed at the wrong class with no warning. Both sites now spell it - ``~typing.Type[Dumper]``, as eight other files already do. Line 146 (the first line - of ``__output__``'s ``#:`` block) is fixed for the same reason but is currently - inert, since the ``# type:`` comment at line 162 spells the same bare - ``Type[Dumper]``; it is insurance for whatever eventually renders that type. This - pair is part of :issue:`709`. Four more bare ``Type`` sites, the hand-written - ``:type:`` fields at ``docs/source/pcapkit/foundation/engines/engine.rst:40``, - ``.../reassembly/reassembly.rst:33`` and ``:43``, and - ``.../traceflow/traceflow.rst:40``, are fixed separately by :pr:`714` + ``pcapkit.const.l2tp.type.Type``, an L2TP field-type enum; once :issue:`684` + rendered ``TraceFlow._foutio``, the built docs pointed at the wrong class with no + warning. Both sites now spell it ``~typing.Type[Dumper]``, one pair of the sites + :issue:`709` lists (:issue:`709`). Four more + bare ``Type`` sites, in hand-written ``:type:`` fields under + ``docs/source/pcapkit/foundation/``, are fixed separately by :pr:`714` (:issue:`709`). * ``register_protocol``'s overwrite warning could claim a protocol was replaced with itself. The guard was correct (``incumbent is not protocol``, an identity check from @@ -993,11 +826,9 @@ Fixed ``tests/protocols/test_construction_keyword_check_unit.py``'s ``_protocol_class`` does) produces two distinct classes sharing one ``__module__`` and ``__qualname__``, so a real overwrite read "overwriting X with X." The message now compares the two - reprs and, only when they coincide, appends each object's ``id()``. - ``__module__``/``__qualname__`` was rejected as the disambiguator: for the reported - shape they are exactly what the coinciding repr already renders. The new - ``test_register_protocol_disambiguates_classes_sharing_a_repr`` fails against the - unfixed message (:issue:`710`). + reprs and, only when they coincide, appends each object's ``id()``; + ``__module__``/``__qualname__`` would not disambiguate, being what the repr already + renders (:issue:`710`). * five more registrars warned on mere key presence rather than an actual overwrite, outside the wording of :issue:`718`'s identity guard (``incumbent is not None and incumbent is not new``, landed by :pr:`726`), whose @@ -1005,10 +836,8 @@ Fixed and ``register_traceflow`` on ``Extractor``, and ``register_dumper`` on both ``Extractor`` and ``TraceFlow``. All five now compare the incumbent by identity before warning; both ``register_dumper`` sites compare only the stored dumper, so - re-registering with just a new file extension stays silent too, judged defensible - rather than comparing the full ``(dumper, ext)`` pair. Non-breaking: a correct - caller sees strictly fewer warnings and no change to return value or exception. Five - new tests pin the silent/warns-anyway split (:issue:`739`). + re-registering with just a new file extension stays silent too. Non-breaking: a + correct caller sees strictly fewer warnings (:issue:`739`). * **six registrars** now raise ``RegistryError`` for a non-class argument, where a bare ``TypeError`` ("issubclass() arg 1 must be a class") used to escape -- from the guard itself for the two ``register_dumper`` methods, and from inside ``abc`` for the other @@ -1022,24 +851,16 @@ Fixed descriptor. No guard's target class changes, and a wrong class still raises ``RegistryError`` as before. ``RegistryError`` subclasses ``TypeError``, so ``except TypeError`` still catches it; a caller matching the exact type, or the old - message, does not. **Not every site is covered**: six of the thirteen bare - ``issubclass`` guards in the package are fixed here, and the same guard remains in - the ``register`` classmethods of ``ProtocolBase``, ``Frame``, ``PCAPNG``, ``SCTP``, - ``Link``, ``Internet`` and ``Transport``, all of which still leak ``TypeError`` for a - non-class. ``Transport.register`` is reachable despite its ``UnsupportedCall``, which - is gated on ``cls is Transport`` and so fires only for the abstract class -- the guard - below it leaks through ``TCP.register`` and ``UDP.register``, which is the only way it - is ever called. :issue:`1026` tracks all seven (:issue:`1021`). + message, does not. The same guard in the seven protocol ``register`` classmethods is + fixed separately by :issue:`1026` (:issue:`1021`). * a missing extraction engine package now gives one ``EngineWarning`` rather than two (:issue:`1045`, :pr:`1046`). ``Extractor.import_test`` warned that the package was absent, then ``Extractor.run`` warned again when it fell back to the default engine; - ``run`` now stays silent there. The surviving message is the one ``run`` already - emitted, ``engine () is not installed; using default engine instead`` - (the module name is in backticks in the real message, which this changelog's - Markdown generator cannot show literally), so ``import_test`` now says the same. - Only code that matched ``import_test``'s old wording, - ``extraction engine '' not available; using default engine instead``, or - that expects two warnings, breaks. + ``run`` now stays silent there, and ``import_test`` says what ``run`` used to: + ``engine () is not installed; using default engine instead``, the + module name in backticks. Only + code that matched ``import_test``'s old wording, or that expects two warnings, + breaks. pcapkit.protocols ----------------- @@ -1104,60 +925,38 @@ Added matching the other four families. Backed by ``pcapkit.foundation.registry.protocols.register_protocol_code``, also callable directly for a class that declined at definition time. It was written up as the - mechanism :issue:`548` (``TransType.L2TP`` registered nowhere) needs, "now a - one-declaration change"; investigating :issue:`548` found otherwise -- 115 is an - :rfc:`3931` L2TPv3-over-IP header with no class to dispatch to, so the declaration - would have pointed the :rfc:`2661` parser at it (see the corresponding **Fixed** - entry below). The mechanism is unaffected, and its worked example now names - ``L2TPv3`` rather than ``L2TPv2`` (:pr:`570`). + mechanism :issue:`548` needs; :issue:`548` turned out to need a missing class + instead (115 is :rfc:`3931` L2TPv3 over IP, not :rfc:`2661`; see the corresponding + **Fixed** entry below), so its worked example names ``L2TPv3`` (:pr:`570`). * ``tests/protocols/test_dispatch_reachability_unit.py``, the coverage :issue:`548` asked for: every ``ProtocolBase`` descendant whose ``__index__`` returns an enum member must be reachable under that code in the registry its enum *type* designates, read from the same ``_CODE_DESTINATIONS`` table backing ``code=`` so - the two cannot drift. Where ``test_dispatch_registry_unit.py`` walks the 38 - entries that exist and checks each parses, this walks the classes and catches - one nothing registered, the shape in which ``OSPF`` once shipped reachable from - no table. 23 claims verified, no gaps; a companion case injects a gap and - confirms the audit reports it (:issue:`548`). + the two cannot drift. It walks the classes rather than the registry entries, so it + catches a class nothing registered, the shape in which ``OSPF`` once shipped + (:issue:`548`). * ``tests/protocols/transport/test_tcp_mptcp_join_flag_ordering_unit.py``, covering all three MP_JOIN layouts through the *public* constructor, the construct-pack-parse cycle for each, the stale-flags case that rules out a zero-valued default, the statement order, and controls for the parse path and the flag-independent options. - It closes the gap that let 100% statement and branch coverage of the two changed - modules coexist with a broken public path: the old cases reached - ``_make_mptcp_join`` by assigning a Python ``set`` to ``_flags`` on a bare - ``TCP.__new__(TCP)``, which executes every branch while bypassing both the ordering - and the accumulator's type. The stale ``tcp-mptcp/MP_JOIN`` entry is deleted from - ``EXPECTED_FAILURES`` and the MP_JOIN exclusion in - ``test_tcp_mptcp_subtype_unit.py`` is lifted (:issue:`587`). + The old cases bypassed the public path, so full coverage coexisted with a broken + one. The stale ``tcp-mptcp/MP_JOIN`` entry is deleted from ``EXPECTED_FAILURES`` + (:issue:`587`). * ``tests/protocols/test_option_generator_tcp_base_unit.py``, asserting that every ``TCP_BASE`` key is a parameter ``TCP.make`` declares (derived from - ``inspect.signature``, so it catches a fourth misspelling nobody has made yet) - and that the built segment carries the stated header in both data model and - packed octets. Two of the three wrong names were invisible to any assertion - about a *value*, the value asked for equalling the default used instead - (:issue:`602`). + ``inspect.signature``) and that the built segment carries the stated header in both + data model and packed octets (:issue:`602`). * ILNP nonce sizing coverage in ``tests/protocols/internet/test_ipv6_extension_unit.py``, one test per protocol, asserting the declared length, the exact packed octets and the construct-pack-parse - cycle over ten nonces. The existing suite missed the defect because its only ILNP - nonce was ``0xFFFFFF`` (bit length 24, a multiple of eight, where floor division and - the ceiling agree), the blind spot that hid the same typo in ``numbers.py`` behind - bit lengths 8, 16, 24, 32 and 64. Every new case bar two controls therefore has a - bit length that is *not* a multiple of eight, several below 256, and the test - asserts that at least six stay non-byte-aligned and one stays below 256, so rounding - them off later cannot disarm it (:issue:`601`). + cycle over ten nonces, most with a bit length that is *not* a multiple of eight, + where floor division and the ceiling disagree (:issue:`601`). * ``SOLUTION`` parameter width coverage in ``tests/protocols/internet/test_hip_unit.py``: eight widths asserting the declared length, the reader's acceptance and the construct-pack-parse cycle, plus a 57-bit case pinning the 20-octet length :rfc:`5201#section-5.2.5` requires of HIPv1. - Five widths (1, 9, 12, 17 and 25 bits) are deliberately *not* multiples of eight, - because at a multiple of eight the defective ``ceil(bits / 4)`` coincides with - the correct width, which is why every fixture reaching this builder passed; the - round-trip case in ``examples/generators/options.py`` supplies no ``random`` or - ``solution``, so it exercised the formula at zero bits. 57 bits discriminates on - the HIPv1 path where 64 does not: both formulas give 20 at a full-width value - (:issue:`608`). + Five widths are deliberately *not* multiples of eight, where the defective + ``ceil(bits / 4)`` coincides with the correct width (:issue:`608`). Changed ~~~~~~~ @@ -1196,65 +995,41 @@ Changed and ``layer='application'`` now does -- the same inversion applies to RARP -- but the extracted protochain is unchanged at every ``layer=`` value, including ``'internet'`` (``IPv4:OSPFIGP`` before and after, as IPv4 - ends the chain itself). Measured: ``IPv4:OSPFv2:Raw`` for a headerless ``IPV4`` capture, - ``Ethernet:RARP:Raw`` for a padded RARP frame. + ends the chain itself). * renames with no compatibility alias left behind: PCAP-NG ``Option`` subclasses spell the namespace class keyword ``ns=`` instead of ``namespace=`` (:issue:`439`). * ``tests/protocols/transport/test_tcp_udp_unit.py`` now reaches the MP_JOIN dispatchers through ``TCP()`` itself, instead of assigning a Python ``set`` to - ``_flags`` on a bare ``TCP.__new__(TCP)``. A ``set`` answers the membership tests - ``_make_mptcp_join`` and ``_read_mptcp_join`` use, so every flag branch ran and both - TCP modules read 100% coverage, while the attribute had neither the - ``aenum.IntFlag`` type production assigns nor the ordering that governs when it - exists -- how :issue:`587` stayed invisible behind that number, and the - ``cast('Enum_Flags', 0)`` no-op behind it. Reverting :issue:`587`'s hoist now fails - two of the file's 17 tests with - ``AttributeError: 'TCP' object has no attribute '_flags'``, and restoring the - ``cast`` fails two with - ``TypeError: argument of type 'int' is not a container or iterable``; all 17 passed - before. The library is unchanged and coverage does not move -- the point is what the - same numbers are now worth (:issue:`603`). + ``_flags`` on a bare ``TCP.__new__(TCP)``. A ``set`` answers the same membership + tests, so every branch ran while the attribute had neither the type production + assigns nor the ordering that governs when it exists -- how :issue:`587` stayed + invisible. Reverting :issue:`587`'s hoist, or restoring the + ``cast('Enum_Flags', 0)`` seed, now fails the file. Test-only (:issue:`603`). * building a protocol through its constructor with a keyword that names nothing now raises ``UnsupportedCall`` instead of discarding it. **This is a behaviour change to a public API**: every ``make`` in the tree ends its signature with ``**kwargs`` and reads nothing out of it, so a misspelled keyword was accepted, dropped, and the field - it named kept its default -- wrong octets, nothing said. That is what :issue:`602` - cost: ``examples/generators/options.py`` asked for ``seq=1`` where ``TCP.make`` spells - it ``seq_no``, and 25 generated fixture frames carried sequence number ``0``. - :issue:`541` and :issue:`556` were the same silence. The schema layer was never so - permissive (``Schema.__update__`` warns ``UnknownFieldWarning``), and that asymmetry - is what this closes. The check sits in ``ProtocolBase.__init__`` rather than ``make``, - because ``__post_init__`` passes one ``**kwargs`` to the construction *and* to the - parse of what it has just constructed, so a keyword declared only by ``read`` - legitimately travels through ``make`` (``HIP.read`` declares ``extension``, which - ``HIP.make`` does not, and the option generator depends on it). The accepted set is - the union of every keyword-taking parameter of ``make``, ``read``, ``pack``, - ``unpack``, ``__post_init__`` and ``__init__`` across the MRO, computed once per class - from ``inspect.signature``. Parsing is deliberately untouched: there the keywords are - whatever the engines and the four ``_import_next_layer`` implementations forward, a - protocol cannot know which its parent passed on, and a dropped parse keyword changes - how a packet is read, not what its octets say. Two opt-in escapes exist per class - through a new ``__keywords__``: a set, for a keyword read out of ``**kwargs`` by name - (``ESP.read`` with ``packet``); and ``None``, for a dispatcher whose real signature - belongs to a class chosen at call time (only ``HTTP.make`` forwarding to - ``HTTPv1``/``HTTPv2``). The message names the near neighbour (``seq`` reports *did you - mean 'seq_no'?*). ``from_data`` warns ``UnknownFieldWarning`` rather than raising, - because its keywords are whatever ``_make_data`` returned, so the defect is a key of - that mapping disagreeing with the signature it is spread into, and the person who - meets it cannot fix it. This surfaced four latent bugs in this repository, all - residue of :issue:`602` and fixed here: the ``_TCP_BASE`` of - ``examples/generators/dispatch.py`` and three stale copies under - ``tests/protocols/transport/``, each building segments with sequence number ``0``. - Three more are reported, not fixed, being a defect per protocol: ``Frame._make_data`` - returns ``ts_src`` where ``make`` declares ``ts_sec``, ``L2TPv2._make_data`` returns - ``prio`` where it declares ``priority``, and ``Header._make_data`` returns a - ``magic_number`` that ``Header.make`` does not take, so ``from_data`` has been - dropping a frame's timestamp, an L2TPv2 priority bit and a capture's byte order, and - now says so. One limitation: a *direct* ``SomeProtocol.make(...)`` call is not checked - and still discards silently, since the check sits where every producer's keywords - converge rather than inside each of the 30 ``make`` implementations; - ``object.__new__(cls).make(**kwargs)``, the idiom ``HTTP.make`` uses, reaches it - (:issue:`617`). + it named kept its default -- wrong octets, nothing said, as in :issue:`602`, + :issue:`541` and :issue:`556`. The check sits in ``ProtocolBase.__init__`` rather + than ``make``, because ``__post_init__`` passes one ``**kwargs`` to the construction + *and* to the parse of what it has just constructed, so a keyword declared only by + ``read`` legitimately travels through ``make`` (``HIP.read`` declares + ``extension``). The accepted set is the union of every keyword-taking parameter of + ``make``, ``read``, ``pack``, ``unpack``, ``__post_init__`` and ``__init__`` across + the MRO. Parsing is deliberately untouched: a protocol cannot know which keywords its + parent passed on. A new ``__keywords__`` opts a class out: a set, for a keyword read + out of ``**kwargs`` by name (``ESP.read`` with ``packet``); and ``None``, for a + dispatcher whose real signature belongs to a class chosen at call time (only + ``HTTP.make``). The message names the near neighbour (``seq`` reports *did you mean + 'seq_no'?*). ``from_data`` warns ``UnknownFieldWarning`` rather than raising, because + the defect is then a ``_make_data`` key the caller cannot fix. Four leftover copies + of :issue:`602`'s misspelt TCP base, in ``examples/generators/dispatch.py`` and + ``tests/protocols/transport/``, are fixed. Three such ``_make_data`` keys are + reported, not fixed: ``Frame._make_data`` returns ``ts_src`` where ``make`` declares + ``ts_sec``, ``L2TPv2._make_data`` returns ``prio`` where it declares ``priority``, and + ``Header._make_data`` returns a ``magic_number`` that ``Header.make`` does not take. + A *direct* ``SomeProtocol.make(...)`` call is not checked and still discards + silently (:issue:`617`). * **a breaking change to** ``Application``: it no longer forbids payload dispatch outright. It encodes "no further protocol layer above this one", which undissected trailer bytes are not, so ``_decode_next_layer`` and ``_import_next_layer`` accept @@ -1538,29 +1313,22 @@ Fixed the obstacle: ``MPTCPDSS.ssn``, ``dl_len`` and ``checksum`` have always been ``ConditionalField`` on the sibling ``M`` flag. The ``SwitchField`` form is kept for the narrower reason the note now gives: ``ConditionalField``'s ``length`` - forwards to the wrapped field without consulting the condition, so it is safe here - only because ``Schema.pack`` and ``Schema.unpack`` special-case that wrapper by - name, whereas a ``SwitchField`` always resolves to a concrete field. Replacing it - would be a behaviour change and is not made (:issue:`603`). + forwards to the wrapped field without consulting the condition, whereas a + ``SwitchField`` always resolves to a concrete field (:issue:`603`). * the one assertion :issue:`604` left pinning the old padding side, red on ``mainline`` since :pr:`621` merged. ``TCPUDPUnitTests.test_a_truncated_option_still_parses_its_declared_length`` expected a truncated TCP option's ``data`` as the synthesised zero octets *followed by* the real ones, as ``rjust()`` produced; :pr:`621` made the padding ``ljust()`` - everywhere but could not retarget this file, since :pr:`612` owned it and editing - concurrently risked discarding work that has since landed. The real octets now come - first for both parametrised widths, and the docstring, which said the short read was - *left*-padded, is corrected. Test-only; the sibling case in - ``tests/protocols/internet/test_ipv4_unit.py`` was retargeted in :pr:`621` (two - subtest failures before, none after; :issue:`604`). + everywhere but could not retarget this file, since :pr:`612` owned it. The real + octets now come first, and the docstring is corrected. Test-only; the sibling case in + ``tests/protocols/internet/test_ipv4_unit.py`` was retargeted in :pr:`621` + (:issue:`604`). * ``main`` went red the moment :issue:`604`'s ``ljust()`` landed, because that test still pinned the head-padded short read: ``Reserved_79`` declaring ``length=12`` over 6 real octets now reports ``aabbccddeeff00000000`` where the test expected - ``00000000aabbccddeeff``, failing both subtests (``declared_length=12`` and ``=32``) - while the 17 other cases stayed green. The inputs discriminate: ``trailing`` is - non-zero and the pad width is 4 and 24. :pr:`621` left the file alone because - :pr:`612` owned it at the time, and merged two minutes ahead of the cross-review - verdict that named it (:issue:`604`, :pr:`621`). + ``00000000aabbccddeeff``. :pr:`621` left the file alone because :pr:`612` owned it + at the time (:issue:`604`, :pr:`621`). * the HIP ``SOLUTION`` builder sized the parameter with ``4 + math.ceil(max(random.bit_length(), solution.bit_length()) / 4)``, an invalid shorthand for two fields. :rfc:`7401#section-5.2.5` gives ``Length`` as @@ -1591,23 +1359,12 @@ Fixed ``Flags`` declares no ``_missing_``, so ``Flags(0)`` is an ordinary ``aenum.IntFlag`` pseudo-member and does not meet the ``RecursionError`` the same construction hits on the ``pcapkit.const.mh`` flag enumerations (:issue:`623`). - Nothing a caller can reach changed: the three MP_JOIN layouts of :rfc:`8684` section - 3.2 are chosen from these membership tests, but ``mptcp_data_selector`` rejects a - flagless MP_JOIN with a ``FieldError`` first, so the ``TypeError`` was latent, and - only by virtue of a guard in another file. Called directly on a flagless parsed - segment the dispatcher now raises ``ProtocolError`` naming an invalid flags - combination. One difference *is* observable, a dump-format change rather than a - value change: a flagless segment's ``connection`` renders as the string - ``Flags::None [0]`` where it rendered as the number ``0``, in the JSON, tree and - PLIST outputs. The value and wire bytes are unchanged; ``connection`` no longer - switches JSON type with the flags (a number for a flagless segment, a string such as - ``Flags::ACK [2048]`` for every other). The literal ``None`` is a separate rendering - defect in the hook ``pcapkit.dumpkit.common.make_dumper`` installs, which - interpolates ``o.name`` without accounting for a nameless composite member; it is - left to its own change and pinned by a test. The committed example dumps are - unaffected (``examples/captures/in.pcap`` has no flagless segment). Coverage cannot - see this fix, since the changed line already executed; the added subtests are the - evidence (:issue:`616`). + The MP_JOIN path was latent: ``mptcp_data_selector`` rejects a flagless MP_JOIN + before :rfc:`8684` section 3.2's three layouts are chosen. One dump-format change is + observable: a flagless segment's ``connection`` renders as a string where it + rendered as the number ``0``, so it no longer switches JSON type with the flags. That + string read ``Flags::None [0]`` until the ``pcapkit.dumpkit`` fix above + (:issue:`616`). * **a breaking change to a public attribute.** ``Frame.len`` and ``Frame.cap_len`` were filled from *opposite* wire fields depending on the container format, so ``frame.len`` meant the captured length out of a ``.pcap`` and the on-wire length @@ -1615,27 +1372,18 @@ Fixed length (the record header's ``orig_len``) and ``cap_len`` the octets stored (``incl_len``). **Code reading ``frame.len`` or ``frame.cap_len`` from a ``.pcap`` gets the other field's value than before**, and for a truncated frame that is a - different number rather than a relabelling. Which reader to move was a decision, not - settled by seniority: the PCAP spelling is the *older* (``c43892af``, 2022-01-11, - with the data model's docstrings agreeing a day later), the opposite spelling in - ``pcapkit.toolkit.pcapng`` arrived 15 months later in ``25f216f4`` (2023-04-27), and - both were internally consistent. What settles it is that the *names* are - Wireshark's: ``epan/dissectors/packet-frame.c`` registers ``frame.len`` as "Frame - length on the wire" and ``frame.cap_len`` as "Frame length stored into the capture - file", and raises ``frame.len_lt_caplen`` expert info - (``PI_MALFORMED``/``PI_ERROR``) on ``frame_len < cap_len``, which could not be - malformed if ``len`` were the smaller, captured one. Both formats define the - underlying fields alike (``pcap-savefile(5)``'s ``incl_len``/``orig_len``, - ``draft-ietf-opsawg-pcapng``'s Captured/Original Packet Length). The internal - consumer moved with it: ``Frame.read`` hands ``_decode_next_layer`` the octets - *present*, now ``frame.cap_len``; handing over the on-wire length would be the - declared-length-exceeds-available-octets fault of :issue:`554`, :issue:`573` and - :issue:`594` on every snapped frame. The dumpers are untouched (``PCAPIO`` packs its - record header from ``frame_info.incl_len`` and ``orig_len``, already right in both - readers). It went unnoticed because the two lengths differ only for a snapshot-cut - frame and no such fixture existed until :pr:`614` added one (``big_endian.pcap``'s - third frame, 1200 octets on the wire against 96 captured), so every earlier - assertion compared a value against itself (:issue:`618`). + different number rather than a relabelling. Seniority did not settle it: the PCAP + spelling is the *older* (``c43892af``, 2022-01-11) and PCAP-NG's arrived in + ``25f216f4`` (2023-04-27), both internally consistent. The *names* settle it, being + Wireshark's: ``frame.len`` is "Frame length on the wire" and ``frame.cap_len`` + "Frame length stored into the capture file", and ``frame.len_lt_caplen`` expert info + flags ``frame_len < cap_len`` as malformed, which it could not be if ``len`` were + the smaller, captured one. ``Frame.read`` hands + ``_decode_next_layer`` the octets *present*, now ``frame.cap_len``; handing over the + on-wire length would be the declared-length-exceeds-available-octets fault of + :issue:`554`, :issue:`573` and :issue:`594` on every snapped frame. The dumpers are + untouched. The two lengths differ only for a snapshot-cut frame, and no such fixture + existed until :pr:`614` (:issue:`618`). * the HIP ``PUZZLE`` and ``SOLUTION`` builders derived three wire-format values from the payload value instead of the data model, and all three were wrong. **This changes two public data models and the octets both parameters emit.** The width of @@ -1791,35 +1539,18 @@ Fixed hard-coded. Teaching ``ProtocolBase.__init__`` to skip a protocol that had set ``packet`` was rejected: it heals ``frame.info.packet`` while ``frame.packet`` still reports the whole block as header. The override is a plain ``property``, not the - ``cached_property`` it replaces, which a cross-review on a second model settled: the - inherited one caches because it *reads the stream*, where this one only walks - buffers already filled, and caching it would let a second ``unpack`` on one instance - return the *first* call's octets; the unit test swaps the block on a live instance - and fails ``b'payload' != b'cached'`` under a cache. ``ProtocolBase.packet`` gains - the contract in its docstring and no executable line. This moves the parse output of - every PCAP-NG capture on the success path, so it ships labelled breaking: - ``frame.info.packet`` goes from ``b''`` to hundreds of octets, + ``cached_property`` it replaces: the inherited one caches because it *reads the + stream*, where this one only walks buffers already filled, and caching it would let + a second ``unpack`` on one instance return the *first* call's octets. This moves the + parse output of every PCAP-NG capture on the success path, so it ships labelled + breaking: ``frame.info.packet`` goes from ``b''`` to hundreds of octets, ``frame.packet.header`` from the whole block to the pre-payload prefix. It was also - a wire-format defect: ``PCAPIO`` writes ``value.packet`` after each 16-octet record - header, so dumping those four blocks produced a **104-octet** file whose record - headers declared hundreds of octets and delivered none, desynchronising any reader - that walks by ``incl_len`` (pcapkit refused its own output with - ``ValueError: read length must be non-negative or -1``, and scapy returned 1 packet - of 64 octets instead of 5). It round-trips now. - ``tests/protocols/test_pcapng_regression.py`` grows 4 tests to 11 and 3 subtests to - 24, with payload expectations derived twice (literals, and a ``struct``-only - re-parse independent of the code under test) and each synthesised payload a - different length modulo 4, so an offset off by a field or a payload that picked up - block padding cannot pass by coincidence; shapes no fixture exercised (an option - area *after* the captured data, a big-endian section, snapped Enhanced and Simple - Packet Blocks) are covered. ``pcapkit/protocols/protocol.py`` is identical in every - coverage column, confirming its change is docstring-only. The uncaught - ``ValueError`` on an EOF-truncated PCAP-NG (:issue:`678`) is untouched: over the 101 - whole-percent prefixes ``max(1, 1508 * i // 100)`` both trees give 96 of that - ``ValueError``, one ``FormatError``, one ``ProtocolError`` and three parses. - ``examples/captures/pcapng.txt``, a hand-regenerated legacy reference, still showed - the old ``packet -> NIL`` here; :issue:`685` (below) later removed it from the index - rather than regenerating it again (:issue:`646`). + a wire-format defect: ``PCAPIO`` writes ``value.packet`` after each record header, + so a dump declared hundreds of octets per record and delivered none, + desynchronising any reader that walks by ``incl_len``. It round-trips now. The + uncaught ``ValueError`` on an EOF-truncated PCAP-NG (:issue:`678`) is untouched, and + :issue:`685` (below) later removed the stale ``examples/captures/pcapng.txt`` from + the index (:issue:`646`). * an unpadded HTTP/2 ``DATA``, ``HEADERS`` or ``PUSH_PROMISE`` frame parsed with its whole payload silently discarded. Three payload length callbacks in ``pcapkit/protocols/schema/application/httpv2.py`` put the conditional in the wrong @@ -1834,48 +1565,20 @@ Fixed ``(A - B) if T else 0`` and ``A - (B if T else 0)`` are both ``A - B`` for truthy ``T``; the padded arm has no off-by-one (:issue:`668` asked that be checked), because ``Schema.unpack`` decrements ``packet['__length__']`` by each field's width, - so ``pad_len``'s own octet is already out by the time the payload field runs. Three - sibling sites that were never wrong (``ContinuationFrame.fragment``, - ``UnassignedFrame.data``, ``GoawayFrame.debug``) localise the defect to the grouping - and are pinned as controls. Nothing caught it: ``test_option_roundtrip_unit`` drives - every frame type through ``make`` -> parse -> ``make``, where both sides agreed on - an empty payload (its generator passes no payload for these frames), and - ``test_httpv2_frame_readers_cover_successful_frames`` uses hand-built schema stubs, - so the callback never ran. The same lambda also governs **packing**, which neither - the issue nor the first revision noticed and a cross-review on a second model did: - ``BytesField`` consults ``length`` on both paths, so an unpadded ``DATA`` frame - carrying twelve octets packed to a nine-octet header whose length field declares - **21**, body absent (unpadded ``HEADERS`` and ``PUSH_PROMISE`` declared 29 against 9 - and 33 against 13). pcapkit was *emitting* malformed HTTP/2 that desynchronises any - reader walking a stream by that field; a ``make`` -> parse -> ``make`` cycle of a - non-empty unpadded payload now closes byte-for-byte - (``HTTPv2ConstructedFrameDeclaresWhatItWritesUnitTests``). The new - ``tests/protocols/application/test_httpv2_payload_length_unit.py`` reads wire octets - (23 tests, 12 subtests over padded, unpadded and padding-only shapes of all three - frames plus both ``PRIORITY`` combinations) and asserts payload *octets*, never - length, because a length assertion survives the right width read from the wrong - offset; padded cases use non-zero padding ``b'\xde\xad\xbe\xef'`` rather than the - zeros :rfc:`9113#section-6.1` tells a sender to use, so failing to subtract it - yields a different byte string. Six wrong fixes written over the schema confirm the - tests discriminate, each rejected; the cross-review's own, ``max(computed, 1)`` on - the two ``fragment`` fields, **passed** the first revision's tests because only - ``DataFrame`` had a padding-only case (a padding-only ``HEADERS`` frame then leaked - one padding octet); the missing padding-only cases and - ``test_the_padded_arm_reaches_zero_and_is_not_clamped`` close it, ``0`` being a - legitimate answer. ``EXPECTED_FAILURES`` is unmoved (its one HTTP/2 entry - ``httpv2-frame/PRIORITY`` keeps its ``CONSTRUCT`` status, ``PriorityFrame`` having - no payload field or ``PADDED`` flag; its cited ``httpv2.py:572`` is stale, the - ``header.length != 9`` guard now being at ``:650``). The construct side needed no - change: ``_make_http_length`` always computed - ``len(frame.data) + (pad_len + 1 if pad_len else 0)``, and the shared length - callback disagreed on both paths. An AST sweep of all 496 package files found one - other site of the same shape, ``pcapkit/protocols/schema/internet/ipv4.py:336`` - (``TSOption.remainder``), and it is correct: its ``else 0`` is intentional because - ``ts_data`` consumes the whole option data area in that arm. Ships labelled - breaking: restoring a dropped payload moves the parse output of essentially every - HTTP/2 capture, so a golden file or regression baseline that recorded the empty - value sees it change, and constructed frames change byte-for-byte, from malformed to - correct (:issue:`668`). + so ``pad_len``'s own octet is already out by the time the payload field runs. The + same lambda also governs **packing**: an unpadded ``DATA`` frame carrying twelve + octets packed to a nine-octet header whose length field declared **21**, body + absent, so pcapkit was *emitting* malformed HTTP/2; a ``make`` -> parse -> ``make`` + cycle of a non-empty unpadded payload now closes byte-for-byte. The one other + conditional of this shape, ``TSOption.remainder`` in + ``pcapkit/protocols/schema/internet/ipv4.py``, is correct: its ``else 0`` is + intentional because ``ts_data`` consumes the whole option data area in that arm. The + new tests use + non-zero padding rather than the zeros :rfc:`9113#section-6.1` tells a sender to use, + so failing to subtract it yields a different byte string. Ships labelled breaking: + restoring a dropped payload moves the parse output of essentially every HTTP/2 + capture, and constructed frames change byte-for-byte, from malformed to correct + (:issue:`668`). * the schema half of every option-like registrar overwrote silently while the parser half of the same call warned. ``EnumSchema.register`` was a bare ``cls.__enum__[code] = schema``, and is the schema half of 14 public registrars, so @@ -1892,24 +1595,15 @@ Fixed overwritten. Their firing condition is harmonised onto the narrower guard :pr:`681` gave ``register_protocol`` (:issue:`718`, :pr:`726`) rather than presence alone: that key is derived from the value it stores, whereas these take a caller-supplied - ``code``. Five of the seven (``Link`` 7 entries, ``Internet`` 16, ``Frame`` 3, - ``PCAPNG`` 3, ``SCTP`` 2) ship pre-seeded with unresolved ``ModuleDescriptor`` - values, as do ``TCP`` (4) and ``UDP`` (3), each keeping its own dict rather than - filling the one ``ProtocolBase`` and abstract ``Transport`` share and leave at 0 - (``Transport.register`` raises ``UnsupportedCall``). Where a table is pre-seeded, - the incumbent may be a two-string descriptor while the replacement is the class it - names. Resolution happens earlier, on the incoming argument only - (``if isinstance(protocol, ModuleDescriptor): protocol = protocol.klass``), so the - guard's ``is`` comparison measures a resolved class against an unresolved descriptor - and warns on re-registering the same class under its own pre-seeded code - (``Internet.register(TransType.TCP, TCP)``), a false positive the harmonised guard - does not close. ``ContextRegistry.register`` already raises on a duplicate, and the - reassembly and ESP registrars append to lists with no key (duplicate - security-association SPIs are designed, resolved by scoring), so none of the three - takes a guard. ``import pcapkit`` holds at one warning, a third-party deprecation, - and no ``RegistryWarning``, over the 327 registry writes it performs (one being - ``R1CounterParameter``'s second code, from :issue:`690`), none landing on a key - already present (:issue:`692`). + ``code``. Where a table is pre-seeded with an unresolved ``ModuleDescriptor``, the + guard's ``is`` comparison measures the resolved incoming class against the + descriptor, so re-registering the same class under its own pre-seeded code + (``Internet.register(TransType.TCP, TCP)``) still warns, a false positive the + harmonised guard does not close. ``ContextRegistry.register`` already raises on a + duplicate, and the reassembly and ESP registrars append to lists with no key, so + none of the three takes a guard. ``import pcapkit`` emits no ``RegistryWarning``: + none of its 327 registry writes (one being ``R1CounterParameter``'s second code, from + :issue:`690`) lands on a key already present (:issue:`692`). * every EOF-truncated PCAP-NG file raised a bare ``ValueError`` out of ``Extractor``, losing the whole extraction rather than the one truncated block. Block Total Length is cross-checked against its trailing copy and never against the file, so a last @@ -1924,39 +1618,30 @@ Fixed ``bounded_area`` and applied to the eight unwrapped ``length - N`` spans and eight ``__option_padding__`` fields: ``_TextField.__call__`` builds ``f'{length}s'`` unconditionally, so a negative became ``'-8s'`` and ``struct.calcsize`` raised an - exception caught nowhere a frame loop looks. Across all 1,509 octet boundaries of - ``examples/captures/dhcp.pcapng``, 6 levels parsed before and 1,495 do now; the 14 - that still raise are twelve files too short to hold a block and two cuts into the - Interface Description Block's ``if_tsresol`` option, so none costs a frame that was - in the file. A mutation fuzz removes both of :issue:`678`'s families; two others - remain, unchanged: ``ValueError: N is not a valid BlockType`` (filed as - :issue:`701`; an unassigned block type raises from ``aenum`` inside - ``EnumField.post_process``, so the registry's ``UnknownBlock`` default is - unreachable) and ``MemoryError`` from :pr:`593`'s 32-bit band. Three bare exceptions + exception caught nowhere a frame loop looks. Both of :issue:`678`'s failure families + are gone; two others remain: + ``ValueError: N is not a valid BlockType`` (filed as :issue:`701`) and + ``MemoryError`` from :pr:`593`'s 32-bit band. Three bare exceptions in ``SystemdJournalExportBlock.post_process``, reachable from valid input, are fixed - with it: the body's NUL alignment padding was read as a field *name*, so every - journal entry of unaligned length raised ``struct.error`` (a NUL-only line now ends - the entry); a binary field's declared length was unbounded against its entry - (``OverflowError`` at ``2**63`` and above, silently wrong below; now clamped and + with it: the body's NUL alignment padding was read as a field *name* + (``struct.error``; a NUL-only line now ends the entry); a binary field's declared + length was unbounded against its entry (``OverflowError``; now clamped and reported); and a non-UTF-8 name, key or value raised ``UnicodeDecodeError`` (now - ``errors='replace'`` and reported, as ``StringField`` does). The silent loss of - every field after a binary one is filed as :issue:`704`. The five unclamped - non-packet option areas keep the per-block framing assumption :pr:`676`'s note - describes. Also here, the fifth and last registrar with no overwrite guard: - ``Option.register`` now reports a displaced option schema as a ``RegistryWarning`` - naming every namespace the ``ns='opt'`` fan-out displaced something in, once per - registration, tested by membership (subscripting the per-namespace ``defaultdict`` - would insert ``UnknownOption`` for a code nobody registered). The condition is - identity-based like the seven parser registrars and :pr:`681`'s guard for - ``register_protocol`` (:issue:`718`, :pr:`726`); a namespace the call itself creates - is exempt. Flooring the three decryption-secrets payloads that read ``__length__`` - *whole* was tried and reverted: ``Schema.pack`` leaves that key at ``-1`` for - "unknown", so it packed nothing and turned ``pcapng-secrets/TLS_Key_Log`` and - ``.../WireGuard_Key_Log`` from ``MISMATCH`` to ``OK`` (an empty payload equals an - empty payload). Ships labelled breaking: well-formed captures are byte-identical, - but any truncated PCAP-NG now yields the frames before the cut where it raised, so a - caller reading "extraction raised" as "this file is unusable" gets a partial result - whose last frame may carry zero-padded octets, and eight short-file depths move from + ``errors='replace'`` and reported). The silent loss of every field after a binary + one is filed as :issue:`704`. The five unclamped non-packet option areas keep the + per-block framing assumption :pr:`676`'s note describes. Also here, the fifth and + last registrar with no overwrite guard: ``Option.register`` now reports a displaced + option schema as a ``RegistryWarning`` naming every namespace the ``ns='opt'`` + fan-out displaced something in, tested by membership (subscripting the per-namespace + ``defaultdict`` would insert ``UnknownOption`` for a code nobody registered). The + condition is identity-based like the seven parser registrars and :pr:`681`'s guard + for ``register_protocol`` (:issue:`718`, :pr:`726`); a namespace the call itself + creates is exempt. Flooring the + three decryption-secrets payloads that read ``__length__`` *whole* was tried and + reverted, since ``Schema.pack`` leaves that key at ``-1`` for "unknown". Ships + labelled breaking: well-formed captures are byte-identical, but any truncated + PCAP-NG now yields the frames before the cut where it raised, the last of which may + carry zero-padded octets, and eight short-file depths move from ``ProtocolError: unknown byteorder magic`` to ``StreamEOFError``. The :pr:`683` drift :issue:`685` tracks is unchanged (:issue:`678`). * HIP's ``R1_COUNTER``/``R1_Counter`` parameter packed 12 octets where @@ -2063,8 +1748,7 @@ Fixed consult and mint only once the value is unresolvable; a genuine miss gets ``EnumField._unregistered_member``, a real member of the same registry built through its storage base's ``__new__``, so ``isinstance`` holds and the lookup table stops - growing. ``extend_enum`` calls drop 111 to 0 on ``http.pcap`` and 15 to 2 on - ``many_interfaces.pcapng`` (both from the untouched ``_missing_``). Riding along: + growing. Riding along: PCAP-NG option namespaces stop cross-contaminating: once a code was minted under ``opt``, ``OptionType.get`` merged ``opt`` over the requested namespace and 6 of the other 7 namespaces answered ``opt_unknown`` for it; each now gets its own @@ -2072,11 +1756,9 @@ Fixed across the sample captures, as :pr:`420` and :pr:`427` did, and this deliberately misses that bar: 14 of 23 captures change, entirely from the new ```` rendering plus the PLIST escaping above; normalising both leaves all 14 - byte-identical. A ``pickle`` round-trip of a resolved unassigned member worked before - (it was minted, so registered) and would have broken silently after; - ``_unregistered_member`` now installs a ``__reduce_ex__`` rebuilding an equivalent - unregistered member, checked on protocols 0-5 under both picklers and - ``copy``/``deepcopy`` (:issue:`575`). + byte-identical. So that a resolved unassigned member still pickles, + ``_unregistered_member`` installs a ``__reduce_ex__`` rebuilding an equivalent + unregistered member (:issue:`575`). * ``httpv2._guess_version`` tried to parse a stream as HTTP/2 and read failure as "not HTTP/2", rather than identifying the version first, so a genuine connection preface followed by a real ``SETTINGS`` frame raised ``ProtocolError: unknown HTTP version``. @@ -2089,23 +1771,18 @@ Fixed ``SETTINGS`` frame, an ``Upgrade: h2c`` request and non-HTTP input are unchanged, the first two left undecidable on purpose (``Upgrade: h2c`` needs per-connection state a single payload cannot carry, and a mid-stream frame has no safe heuristic, since a - ``type <= 9`` guess misfires on binary HTTP/1 bodies). Protochain over all 23 sample - captures (1,604 frames, 231 HTTP-bearing) is byte-identical to the unfixed tree - (:issue:`800`). + ``type <= 9`` guess misfires on binary HTTP/1 bodies). Protochain over the sample + captures is unchanged (:issue:`800`). * ``TCP.__proto__`` bound ``httpv1.HTTP`` directly for ports 80 and 8080, so a segment on either port was HTTP/1 by assertion of the port alone; both versions share those ports, so the payload has to decide. Repointed to the generic ``pcapkit.protocols.application.http.HTTP`` proxy, whose identification became positive only once :issue:`800` and :pr:`814` landed, which is why this waited. ``udp.py`` already bound the proxy for both ports; this removes the asymmetry its - docstring and ``docs/source/pep.rst`` documented as an open request (prose-only on - that side). Protochain over all 23 captures (1,604 frames) is *not* byte-identical, - and that is the fix: all 231 HTTP/1.1 frames keep their chain, and nine frames in - ``options-transport.pcap`` change ``Ethernet:IPv4:TCP:Raw`` to - ``Ethernet:IPv4:TCP:HTTP/2`` -- genuine HTTP/2 frames this library's own - ``httpv2.HTTP.make`` built, previously refused by the HTTP/1 parser. - ``_guess_version``'s entry count over the corpus goes 0 to 252 (231 HTTP/1.1, 9 - HTTP/2, 12 fall-throughs that stay ``Raw``). Not labelled breaking, but not free: + docstring and ``docs/source/contributing/pep.rst`` documented as an open request. + Over the sample captures every HTTP/1.1 frame keeps its chain, and nine genuine + HTTP/2 frames in ``options-transport.pcap`` change ``Ethernet:IPv4:TCP:Raw`` to + ``Ethernet:IPv4:TCP:HTTP/2``. Not labelled breaking, but not free: TCP:80/8080 traffic that is neither valid HTTP/1 nor preface-carrying is now exposed to the fall-through arm, where the direct ``httpv1`` binding left it ``Raw`` regardless; a caller depending on that is the one who would notice (:issue:`682`). @@ -2345,22 +2022,16 @@ Fixed That was harmless while the first line was ``extend_enum(...)``, which registers the member so ``cls(value)`` found it; :pr:`861` (above) replaced it with ``_unregistered_member``, which deliberately does not register, so ``cls(value)`` - missed and re-entered ``_missing_``. :pr:`861` half-corrected the two files: it - fixed the *generated* modules (leaving the old ``return cls(value)`` as dead code) - and never touched the two *crawlers*, which is why :pr:`861`'s own CI stayed green - until a live regeneration from the stale crawlers reintroduced the defect; - :pr:`861`'s cross-review had verified byte-identical regeneration on a 5-of-82 - sample that missed these two. Both crawlers now collapse to the single returning - line the other 101 registries use, and the two const files are regenerated - (``record_type.py`` fetches the network for its own ``LINK`` constant, so that - regeneration needs connectivity, though it came back byte-identical; - ``secrets_type.py`` is offline). The new guard sits at the crawler-emission layer - rather than the behavioural one ``test_const_enum_lookup.py`` already swept: a - hard-coded ``_missing_`` body must end in a return, and no emitted line may re-enter - ``_missing_`` for the same value -- deliberately narrower than "every emitted line - must return", since 53 crawlers legitimately emit 109 non-returning lines in the - shape ``Vendor.process`` uses. The guard sees only 35 of 96 crawlers; the other 61 - are covered behaviourally. Closes :issue:`866` (:pr:`867`). + missed and re-entered ``_missing_``. :pr:`861` fixed the *generated* modules but + not the two *crawlers*, so :pr:`861`'s CI stayed green until a live regeneration + reintroduced the defect; :pr:`861`'s review had sampled 5 of 82 regenerations, + missing these two. Both crawlers now collapse to the single returning line the other + 101 registries use, and the two const files are regenerated. A new guard at the + crawler-emission layer requires a hard-coded ``_missing_`` body to end in a return + and no emitted line to re-enter ``_missing_`` for the same value -- deliberately + narrower than "every emitted line must return", which ``Vendor.process``'s shape + legitimately breaks. The guard sees only 35 of 96 crawlers; the other 61 are covered + behaviourally. Closes :issue:`866` (:pr:`867`). * ``python -m pcapkit.vendor `` could not fail: ``run()`` swallowed every crawler exception as a filterable ``VendorRuntimeWarning`` and ``main()`` always returned ``0``, so a broken crawler regenerating nothing @@ -2388,35 +2059,24 @@ Fixed failed target(s) is appended to ``$GITHUB_STEP_SUMMARY``, a ``::warning::`` annotation is emitted per failure, and the step's own exit code is always ``0``. A per-target ``git checkout -- `` restore step was dropped: with - snapshot/restore, a failed target's const file is never touched. - ``tests/vendor/test_vendor_exit_code_unit.py`` (5 methods, through the real - entrypoints with stub ``Vendor`` subclasses) and - ``tests/vendor/test_vendor_snapshot_restore_unit.py`` (3: success replaces - the previous content, failure leaves it byte-for-byte intact, a read-only - destination is left as it was) make no network calls. Closes :issue:`872` - (:pr:`873`). + snapshot/restore, a failed target's const file is never touched. Closes + :issue:`872` (:pr:`873`). * three follow-ups from :pr:`873`'s review, none changing merged behaviour: one unpinned invariant and two inaccurate pieces of prose/lint. **1.** ``except BaseException:`` in ``_snapshot_and_restore`` is pinned: - ``test_a_keyboard_interrupt_still_restores_and_propagates`` asserts content restored - byte-for-byte, no ``.bak`` surviving, and the ``KeyboardInterrupt`` still - propagating rather than becoming a ``False`` return; mutating the clause to - ``except Exception:`` fails it. **2.** A false docstring clause is corrected: it - said a crawler whose ``_dest_path`` cannot be resolved "goes on to fail loudly - anyway once ``Vendor.__init__`` calls the same ``_dest_path`` itself", which holds - for one of the function's two triggers and not the other. An instance-method-only - override raises ``TypeError`` on the *class* call ``_snapshot_and_restore`` makes - (no ``self`` to bind), but the *instance* call inside ``__init__`` resolves fine, so - that target runs unprotected and **succeeds silently**: ``run()`` returns ``True``, - the file is replaced, and nothing warns that no snapshot was taken. **3.** The - ``# pylint: disable=no-else-raise`` pragma is **kept**. Removing it rested on the - belief that R1720 never applies to ``try``/``except``/``else``, a message-control - artefact (``--disable=all --enable=no-else-raise`` does not re-enable the check the - way ``--enable=R`` does); under the ``Makefile``'s flags, which CI runs, pylint - 4.0.8 flags it. The ``else:`` is deliberate: it makes the backup cleanup unreachable + a test asserts that a ``KeyboardInterrupt`` still restores the file and propagates. + **2.** A false docstring clause is corrected: a crawler whose ``_dest_path`` is an + instance-method-only override fails the *class* call ``_snapshot_and_restore`` + makes but not the *instance* call inside ``__init__``, so that target runs + unprotected and **succeeds silently**. **3.** The + ``# pylint: disable=no-else-raise`` pragma is **kept**: the belief that R1720 never + applies to ``try``/``except``/``else`` was a message-control artefact + (``--disable=all --enable=no-else-raise`` does not re-enable the check the way + ``--enable=R`` does), and under the ``Makefile``'s flags, which CI runs, pylint flags + the ``else:``, which is deliberate because it makes the backup cleanup unreachable from the ``except``. Left out of scope: an untested ``os.close(fd)`` leak (harmless, - the path still exists and ``copy2`` reopens it) and the semantically identical - unindented form of the same ``else:``. Closes :issue:`875` (:pr:`876`). + since ``copy2`` reopens the path) and the semantically identical unindented form of + the same ``else:``. Closes :issue:`875` (:pr:`876`). * ``TLSKeyLabel`` tracked the Mozilla NSS key-log wiki, but ``draft-ietf-opsawg-pcapng-06`` S4.7 now points at ``[I-D.ietf-tls-keylogfile]`` (draft-05), published as :rfc:`9850`, "The SSLKEYLOGFILE Format for TLS", whose S4.2 @@ -2494,23 +2154,17 @@ Changed **Test Environment**, **Test Results** and **Installation Notes** were already duplicated in fuller form in ``docs/source/index.rst``, so they are linked rather than restated; the two blocks that existed nowhere else moved: **Testing** is now - ``docs/source/testing.rst``, in the index toctree, and the ``pipenv`` and - ``make setup`` block joined the Installation section of ``docs/source/index.rst``. Requested - by the project owner, and a deliberate exception to the reStructuredText convention, - for the README only, since its renderers are GitHub and PyPI rather than Sphinx. + ``docs/source/contributing/testing.rst``, and the ``pipenv`` and ``make setup`` block + joined the Installation section of ``docs/source/index.rst``. Requested by the + project owner, and a deliberate exception to the reStructuredText convention, for the + README only, since its renderers are GitHub and PyPI rather than Sphinx. ``setup.py`` reads ``README.md`` and declares ``text/markdown``, and ``MANIFEST.in`` - gains ``include README.md`` because ``global-include *.rst`` no longer matches. That - line is belt-and-braces rather than load-bearing, contrary to what this entry claimed - when written: setuptools' ``sdist`` ships whichever of ``README``, ``README.rst``, - ``README.txt`` and ``README.md`` exists before ``MANIFEST.in`` is read, so deleting the - line leaves the sdist listing byte-identical, and ``setup.py`` reads the file from - where it executes, which under ``pip`` is the unpacked sdist. Of the ``include`` lines - only ``CHANGELOG.md`` is load-bearing, measured in the :pr:`631` entry below. References - moved with the rename: ``examples/benchmark/Dockerfile`` copies ``README.md`` (a - literal ``COPY`` of the old name would have failed the layer and taken ``make bench`` - with it), and benchmark prose naming the root README as the destination of generated - tables now names ``docs/source/index.rst``. ``examples/benchmark/benchmark.py`` still - says ``README.rst`` on purpose, meaning the benchmark suite's own README (:pr:`619`). + gains ``include README.md``, which is belt-and-braces rather than load-bearing: + setuptools' ``sdist`` ships a ``README.md`` regardless. Of the ``include`` lines + only ``CHANGELOG.md`` is load-bearing, measured in the :pr:`631` entry below. + ``examples/benchmark/Dockerfile`` copies ``README.md``, and + ``examples/benchmark/benchmark.py`` still says ``README.rst`` on purpose, meaning the + benchmark suite's own README (:pr:`619`). * ``CODE_OF_CONDUCT.md`` moves from Contributor Covenant 1.4 to 3.0, at the maintainer's request. The text is the canonical 3.0 Markdown fetched from https://www.contributor-covenant.org/version/3/0/code_of_conduct/code_of_conduct.md @@ -2548,9 +2202,8 @@ Changed "Unrecognised Values" section in ``docs/source/pcapkit/const/index.rst`` cross-referenced from ``registry.rst`` (``conf.py`` already excludes ``_missing_`` from every ``autoclass``). ``CONTRIBUTING.md`` gained the rule as a named section. - ``sphinx-build -b html`` gives 53 warnings on ``main`` and 54 after, the addition being - the pre-existing bare ``Type`` ambiguity newly rendered by ``TraceFlow._foutio``'s new - directive, tracked as :issue:`709`. No line under ``pcapkit/`` changed (:issue:`684`). + The one new Sphinx warning, a bare ``Type`` newly rendered by ``TraceFlow._foutio``'s + directive, is :issue:`709`. No line under ``pcapkit/`` changed (:issue:`684`). * ``examples/captures/out.json``, ``out.plist``, ``out.txt`` and ``pcapng.txt`` are no longer tracked in git; they are build output, not fixtures, and a tracked rendering with no reader goes stale silently. ``pcapng.txt`` is exactly that: it still recorded @@ -2560,9 +2213,7 @@ Changed into ``examples/captures/``'s blanket ``.gitignore`` rule; the genuine inputs ``in.pcap`` and ``dhcp.pcapng`` stay tracked. ``examples/legacy_smoke/Makefile`` and its ``README.rst`` now describe regenerating these reports via ``make fixtures``, and - ``tests/project/test_capture_tracking.py`` (5 tests, 8 subtests) pins the invariant; 2 - of the 5 fail against the pre-change tree with the four reports restored, the other 3 - passing on both by construction (:issue:`685`). + ``tests/project/test_capture_tracking.py`` pins the invariant (:issue:`685`). * ``tests/project/test_isort_clean.py`` gated on ``isort`` with a ``try``/``except ImportError`` inside ``setUpClass``, which ``tests/_dependency_gates.py`` cannot see (``_gates_of()`` walks a decorator list, @@ -2787,21 +2438,12 @@ Fixed checked neither the git index nor the filesystem despite its name: it asserted only that the tracked-capture list was non-empty and no name contained a ``/``, so a hardcoded list passed whether or not it matched reality. Sibling - ``test_capture_suggestions_are_captures`` had the same gap. Found while cross-reviewing - :pr:`703`'s first commit (prose-only, so out of scope there); the fix rides in as that - PR's second and third commits, and :pr:`703`'s description names both :issue:`700` and - :issue:`708`. The first test now re-derives the expected set independently, shelling - out to ``git ls-files -z`` under ``examples/captures/`` rather than calling back into - ``_tiers.py``, asserts equality against it, and adds a per-name ``Path.is_file()`` - check. Its sibling gets a narrower fix: equality against - ``committed_capture_names()``'s filter re-implemented inline over - ``_tiers.committed_captures()``, which catches a wrong filter but still trusts - ``committed_captures()`` for the tracked set. A cross-review of the first version found - it wrong twice: a stale ``_tiers.py`` docstring still claimed ``test_tier_guard.py`` - "only stats ``in.pcap``," falsified by the new per-name loop; and the new - ``assertEqual`` was vacuous on an empty tracked set (``() == ()``), having dropped the - non-emptiness guard the other test kept. Both are corrected in a follow-up commit. Even - now a hardcoded literal that happens to match today's names passes: the test detects a - set that is wrong *at the moment it runs*, not hardcoding as a practice, turning a - permanent blind spot into a tripwire that fires on the next change to the tracked set - (:issue:`708`). + ``test_capture_suggestions_are_captures`` had the same gap. Fixed in :pr:`703`; + :pr:`703`'s description names both :issue:`700` and + :issue:`708`. The first test now re-derives the expected set from + ``git ls-files -z`` under ``examples/captures/``, asserts equality against it, and + checks each name with ``Path.is_file()``; its sibling re-implements + ``committed_capture_names()``'s filter inline, which catches a wrong filter but still + trusts ``committed_captures()`` for the tracked set. A hardcoded literal that happens + to match the tracked names still passes; the test catches a set that is wrong when it + runs (:issue:`708`).