Repository navigation
fix(mh): stop the four RFC-inline helper enums from minting on unassigned bytes - #889
Conversation
|
Qualifying something I told you when recommending the raise — the safety argument is verified on a path real captures do not take. I said raising "costs one MH message's structured parse, not the capture", because the lookups sit under But Mobility Header is an IPv6 extension header; the IPv4 wrapping is a synthetic harness. And IPv6 does not merely hand the layer back — it keeps walking its own chain: Measured: What I have not established: I tried to reproduce the failure on unfixed The cross-review's first task is to settle it on the PR head and to test whether This does not change your ruling — the four enums should still raise, and the |
…gned bytes Per the owner's ruling on #877, FastBindingAcknowledgmentStatus, IPv6AddressPrefixCode, LocalizedRoutingStatus and LMAAddressCode stay immutable: none of their RFCs keep an IANA registry, so an unassigned byte is closed rather than pending assignment. - `_missing_` raises a new `EnumValueError` (`pcapkit.utilities.exceptions`, paired with `EnumError` the way `FieldValueError` pairs with `FieldError`) instead of minting an `Unassigned_N` member. Over IPv4 that costs one MH message, via the `@beholder`-decorated `_import_next_layer`. Over IPv6, where Mobility Header chains as one of IPv6's own extension headers, it costs the whole IPv6 packet: the walk's `proto = info.next` (`ipv6.py:338`) has nothing to read off the `Raw` fallback, and that second, unrelated `AttributeError` is what a further-out `@beholder` actually catches. That walk defect is pre-existing -- a truncated or malformed extension header hits it on unmodified `main` too -- but a well-formed packet naming merely an unassigned byte is a far likelier way to reach it. Neither case is a capture-level failure; sibling frames are unaffected either way. - `get()`'s `default=-1` mint path is removed for the two enums that keep `get()`. It now raises the same `EnumValueError` for an unknown key, closing the defect where a second unknown name aliased the first at the shared sentinel `-1` and bypassed `_missing_`'s own range guard. - `LocalizedRoutingStatus.get`/`LMAAddressCode.get` are deleted outright: zero callers repo-wide, tests included. - Two existing tests pinning the old minting/aliasing behaviour are re-pointed to the new contract. A new test proves the aliasing defect, the range invariant and the IPv4 degrade path, each shown to fail without this fix; it also pins the IPv6 whole-packet defect as it stands today, so whoever fixes ipv6.py:338 sees this test and knows to revisit it. tests/protocols/internet/test_mh_unit.py: 51 run, 0 failures, 0 errors. Refs #880 (mh.py half only; ngap.py's ProcedureCode/ProtocolIE are separate, per-key work tracked there)
93bb2e0 to
a47d231
Compare
|
GOOD TO GO on The code was never in question. What was wrong is what the code said, in four shipping docstrings. Verified on the amended head: Each of the four now cites Three things the author did beyond what I asked, all right calls:
That IPv6 subTest is the part worth keeping: it pins today's defective behaviour as an explicit assertion, so whoever fixes #891 (the The earlier verdict's other findings stand confirmed and needed no rework: the four enums are correctly closed with Still references #880 rather than closing it: |
…red -1 mint (#880) Completes #880's ngap.py half (the mh.py half landed in #889). Per the owner's ruling, ProcedureCode/ProtocolIE are genuinely open per 3GPP TS 38.413, so they want a per-key unique value rather than the shared -1 sentinel #880 measured aliasing on -- sourced from pycrate's own compiled NGAP specification through a vendor crawler, not hand-maintained or minted on demand. - Add pcapkit/vendor/ngap/{procedure_code,protocol_ie}.py: crawlers whose _request() imports pycrate_asn1dir.NGAP (no network fetch -- pycrate is already an installed optional dependency) and reads NGAP_Constants' (name, value) pairs, filtered by each value's own typeref. - Add pcapkit/const/ngap/{procedure_code,protocol_ie}.py, generated: EnumRegistry + IntEnum, matching the house pattern. get() no longer mints on an unresolvable string key (raises, like every other converted registry); _missing_ answers an in-range value it has not seen with a throwaway, non-registering member instead of a shared-value extend_enum mint, which is what stops two different unknown keys from aliasing. - pcapkit/protocols/application/ngap.py now re-exports both from pcapkit.const.ngap under their historical names, dropping ~600 lines of hand-rolled IntEnum + extend_enum logic. - Extend four tests/const/ generic sweeps' pinned counts (+2 registries); tests/const/test_const_registry_protocol.py's own census is untouched (out of scope) and now needs 122->124, 105->107. New tests in tests/vendor/ and tests/protocols/application/, each shown to fail without this fix. `pytest tests/const/ tests/vendor/ (excluding the reachability test) tests/protocols/application/test_ngap_unit.py`: 384 passed, 41026 subtests, 1 known pre-existing failure (the census above); coverage on the touched files 96-100%.
…red -1 mint (#880) Completes #880's ngap.py half (the mh.py half landed in #889). Per the owner's ruling, ProcedureCode/ProtocolIE are genuinely open per 3GPP TS 38.413, so they want a per-key unique value rather than the shared -1 sentinel #880 measured aliasing on -- sourced from pycrate's own compiled NGAP specification through a vendor crawler, not hand-maintained or minted on demand. - Add pcapkit/vendor/ngap/{procedure_code,protocol_ie}.py: crawlers whose _request() imports pycrate_asn1dir.NGAP (no network fetch -- pycrate is already an installed optional dependency) and reads NGAP_Constants' (name, value) pairs, filtered by each value's own typeref. - Add pcapkit/const/ngap/{procedure_code,protocol_ie}.py, generated: EnumRegistry + IntEnum, matching the house pattern. get() no longer mints on an unresolvable string key; _missing_ answers an in-range value it has not seen with a throwaway, non-registering member instead of a shared-value extend_enum mint, which is what stops two different unknown keys from aliasing. - pcapkit/protocols/application/ngap.py now re-exports both from pcapkit.const.ngap under their historical names, dropping ~600 lines of hand-rolled IntEnum + extend_enum logic. - Update tests/const/test_const_registry_protocol.py's own census (122 const files -> 124, 105 generated-shape -> 107) and its narrative paragraph, since the two new const files land on the generated side. Extend four other tests/const/ generic sweeps' pinned counts the same way (+2 registries each). - .github/workflows/cron-vendor.yml installs pycrate for the scheduled regeneration job: it runs pcapkit-vendor with no target, which now reaches both new crawlers, and pycrate is deliberately excluded from the `[all]` extra end users install. New tests in tests/vendor/ and tests/protocols/application/, each shown to fail without this fix. tests/const/test_const_registry_protocol.py in full via plain unittest: Ran 74 tests, OK. pytest tests/const/ tests/vendor/ (excluding the reachability test) tests/protocols/application/test_ngap_unit.py: 384 passed, 41026 subtests, 0 known failures; coverage on the touched files 96-100%.
…red -1 mint (#880) Completes #880's ngap.py half (the mh.py half landed in #889). Per the owner's ruling, ProcedureCode/ProtocolIE are genuinely open per 3GPP TS 38.413, so they want a per-key unique value rather than the shared -1 sentinel #880 measured aliasing on -- sourced from pycrate's own compiled NGAP specification through a vendor crawler, not hand-maintained or minted on demand. - Add pcapkit/vendor/ngap/{procedure_code,protocol_ie}.py: crawlers whose _request() imports pycrate_asn1dir.NGAP (no network fetch -- pycrate is already an installed optional dependency) and reads NGAP_Constants' (name, value) pairs, filtered by each value's own typeref. - Add pcapkit/const/ngap/{procedure_code,protocol_ie}.py, generated: EnumRegistry + IntEnum, matching the house pattern. get() no longer mints on an unresolvable string key; _missing_ answers an in-range value it has not seen with a throwaway, non-registering member instead of a shared-value extend_enum mint, which is what stops two different unknown keys from aliasing. - pcapkit/protocols/application/ngap.py now re-exports both from pcapkit.const.ngap under their historical names, dropping ~600 lines of hand-rolled IntEnum + extend_enum logic. - Update tests/const/test_const_registry_protocol.py's own census (122 const files -> 124, 105 generated-shape -> 107) and its narrative paragraph, since the two new const files land on the generated side. Extend four other tests/const/ generic sweeps' pinned counts the same way (+2 registries each). - Per the owner's ruling, declare pycrate in pyproject.toml's `vendor` extra (developer-facing, alongside requests/beautifulsoup4) rather than installing it ad hoc in a workflow; `all` still excludes it, unchanged. cron-vendor.yml installs `.[all,vendor]` so the scheduled regeneration job -- which runs pcapkit-vendor with no target and so now reaches both new crawlers -- has pycrate_asn1dir.NGAP importable. Extended one dependency-gate test (tests/test_tier_guard.py) that pinned `vendor`'s exact requirement-name set. New tests in tests/vendor/ and tests/protocols/application/, each shown to fail without this fix. tests/const/test_const_registry_protocol.py in full via plain unittest: Ran 74 tests, OK. tests/test_tier_guard.py: Ran 107 tests, OK. pytest tests/const/ tests/vendor/ (excluding the reachability test) tests/protocols/application/test_ngap_unit.py: 384 passed, 41026 subtests, 0 known failures; coverage on the touched files 96-100%.
- Added IPv6_GenericExt, a generic RFC 6564 §4 parser for IPv6 extension headers: it reads the guaranteed next-header/Hdr-Ext-Len octets and derives next/length/protocol from the actual parse, applying AH's and IPv6-Frag's own length rules where they diverge. Gated to version==6, since it is registered into the Internet-wide dispatch table for Shim6 and would otherwise also activate for an IPv4 payload carrying protocol byte 140. - IPv6._decode_next_layer now stops its walk structurally -- on any layer whose parsed info carries no `next` field -- instead of reading `info.next` unconditionally and crashing the whole packet when it hits ESP, BIT-EMU, 253, 254, or any other IANA code with no dedicated parser (the actual #891 defect). IPv6._import_next_layer substitutes IPv6_GenericExt for a recognised header whose own parser raises. - IPv6_GenericExt.alias is hyphenated ('IPv6-GenericExt'), matching its siblings, so IPv6's `alias.lstrip('IPv6-')` keys the packet dict as 'genericext' rather than '_genericext'. - An overrun warns in the house SchemaWarning convention, then stops the walk rather than clipping: the length also decides where the next header starts, so a clipped skip distance would fabricate a layer. - Exported IPv6_GenericExt from the pcapkit, pcapkit.protocols, pcapkit.protocols.data and pcapkit.protocols.schema aggregators. - Re-pointed the #889 test pinning the pre-fix whole-packet collapse, and added coverage for both dispatch paths, the AH/Frag rules, the overrun stop, the version gate, and an unimplemented terminal code (BIT-EMU). Build/test: tests/protocols/internet/ (260 passed, 868 subtests) and tests/project/{test_public_api,test_isort_clean,test_module_isolation}.py via the worktree's own pcapkit; coverage on the touched files rose from 81% to 94% with the new tests added. `make isort` clean; the two remaining mypy findings and the pylint findings pre-exist on main.
- Added IPv6_GenericExt, a generic RFC 6564 §4 parser for IPv6 extension headers: it reads the guaranteed next-header/Hdr-Ext-Len octets and derives next/length/protocol from the actual parse, applying AH's and IPv6-Frag's own length rules where they diverge. Gated to version==6, since it is registered into the Internet-wide dispatch table for Shim6 and would otherwise also activate for an IPv4 payload carrying protocol byte 140. - IPv6._decode_next_layer now stops its walk structurally -- on any layer whose parsed info carries no `next` field -- instead of reading `info.next` unconditionally and crashing the whole packet when it hits ESP, BIT-EMU, 253, 254, or any other IANA code with no dedicated parser (the actual #891 defect). IPv6._import_next_layer substitutes IPv6_GenericExt for a recognised header whose own parser raises. - IPv6_GenericExt.alias is hyphenated ('IPv6-GenericExt'), matching its siblings, so IPv6's `alias.lstrip('IPv6-')` keys the packet dict as 'genericext' rather than '_genericext'. - An overrun warns in the house SchemaWarning convention, then stops the walk rather than clipping: the length also decides where the next header starts, so a clipped skip distance would fabricate a layer. - Exported IPv6_GenericExt from the pcapkit, pcapkit.protocols, pcapkit.protocols.data and pcapkit.protocols.schema aggregators. - Re-pointed the #889 test pinning the pre-fix whole-packet collapse, and added coverage for both dispatch paths, the AH/Frag rules, the overrun stop, the version gate, and an unimplemented terminal code (BIT-EMU). Build/test: tests/protocols/internet/ (260 passed, 868 subtests) and tests/project/{test_public_api,test_isort_clean,test_module_isolation}.py via the worktree's own pcapkit; coverage on the touched files rose from 81% to 94% with the new tests added. `make isort` clean; the two remaining mypy findings and the pylint findings pre-exist on main.
Please follow the guide below
make pylint,make mypy,make isort)make testpasses, and a test case covers the changedocs/source/changelog/and regeneratedCHANGELOG.md-- N/A -- changelog centralised in docs(changelog): shared 1.5.0 changelog — long-lived, merges last (#610, #616, #617, #618, #620) #657What is the purpose of your pull request?
fix— corrects a defectDescription of your pull request and other information
This is the
mh.pyhalf of #880 only.ngap.py'sProcedureCode/ProtocolIEare the other two of the six sites #880 names, ruled to be sourced from pycrate through a vendor crawler instead (separate work), so #880 stays open.Per the owner's ruling on #877,
FastBindingAcknowledgmentStatus,IPv6AddressPrefixCode,LocalizedRoutingStatusandLMAAddressCodeno longer mint anUnassigned_Nmember for a value their RFC leaves unassigned._missing_raises a newEnumValueError(pcapkit.utilities.exceptions, pairing withEnumErrorthe wayFieldValueErrorpairs withFieldError) instead -- no existing exception there fit "no member for this value in a closed enum".get()'sdefault=-1mint path -- the shared-sentinel aliasing defect #880 measured -- is removed for the two enums that keepget(); it now raises the same way_missing_does.LocalizedRoutingStatus.get/LMAAddressCode.getare deleted outright, verified zero callers repo-wide, tests included.The consequence of the raise differs by wrapping protocol, and this matters for anyone reading the diff. Over IPv4, which dispatches Mobility Header through the base, non-chain-walking
_import_next_layer, the@beholderdecorator there degrades just the one MH message toRaw-- not a capture-level failure. Over IPv6, Mobility Header is also one of IPv6's own chained extension headers, so the fallback lands inside IPv6's own extension-header walk, which then doesproto = info.next(ipv6.py:338) on it;Raw's info has no.next, so that second, unrelatedAttributeErroris what a further-out@beholderactually catches -- degrading the whole IPv6 packet, header fields included. That walk defect is pre-existing (a truncated or malformed extension header hits the identicalAttributeErroron unmodifiedmain), but this change adds a far likelier route to it: a well-formed packet naming merely an unassigned byte, where before only a malformed one got there. Sibling frames are unaffected either way.Two tests pinning the old minting/round-trip behaviour are re-pointed; a new test proves the aliasing defect is gone and the
0 <= value <= 255range invariant holds, demonstrates the IPv4 degrade path, and pins the IPv6 whole-packet-to-Rawbehaviour as it stands today -- deliberately, so whoever fixesipv6.py:338sees that assertion and knows to revisit it. Each new/changed assertion is shown to fail without this fix.tests/protocols/internet/test_mh_unit.py: 51 run, 0 failures, 0 errors.