feat(protocols): opt-in code= registration with enum-type inference for Protocol/ProtocolBase

CHANGELOG.md

@@ -20,6 +20,7 @@ The largest release since 1.0, and the first recorded here as it happened rather
 - **Changed** -- `Probe`, `CipherSuite` and `IntegritySuite` are `Info` subclasses rather than `typing.NamedTuple`, and no `NamedTuple` remains in the package. They are Mappings now, so `len()` and iteration yield field names rather than values.
 - **Changed** -- renames with no compatibility alias left behind: `HoleDiscriptor` is spelled `HoleDescriptor` and its package alias `TCP_HoleDiscriptor` is `TCP_HoleDescriptor` (#350); PCAP-NG `Option` subclasses spell the namespace class keyword `ns=` instead of `namespace=` (#439); and `examples/sample` and `examples/samples` -- one letter apart, holding different things -- are now `examples/captures` and `examples/generators`.
 - **Changed** -- subclass registration is **opt-in** for `Engine`, `Reassembly`, `TraceFlow` and `dumpkit`'s `Dumper` (#514). Each registers if and only if its registry keyword is given -- `engine=` for `Engine`, `protocol=` for `Reassembly` and `TraceFlow`, `fmt=` for `Dumper`. Previously an absent keyword fell back to the class' own name, so *every* subclass of the public class was registered, and declining meant subclassing the parallel `*Base` class under an alias -- which is what every built-in does, and why the public classes had **0** subclasses between them against the `*Base` classes' 9, 5, 2 and 3. **This breaks out-of-tree code that subclasses one of the four and relies on the derived key**; pass the keyword, or call the matching `register_*` function. Nothing the library ships is affected, and the `*Base` classes remain importable. Two things that were silent are now loud: an unrecognised class keyword raises `UnsupportedCall` instead of being swallowed by `**kwargs` -- which used to register the class under its own name, so passing `name=` to a `Reassembly` subclass silently ignored the key it was given, `protocol=` being the real one -- and `Dumper`'s `ext=` without `fmt=` likewise. A class attribute is not an opt-in: `__engine_name__` and `__protocol_name__` still set the name a class reports, registered or not. Each metaclass also gained a class-level `registry` property mirroring `EnumSchema.registry`. As a side effect a `Dumper` subclass no longer touches the filesystem while its `class` statement runs: inferring `fmt` from the `kind` property meant instantiating the class against a `NamedTemporaryFile` mid-definition. `Engine`'s keyword is `engine=` rather than the `name=` this first shipped with, because `name` cannot be passed as a class keyword at all on Python 3.10: `mcls`, `name`, `bases` and `namespace` collide with `abc.ABCMeta.__new__`'s own parameters, which are positional-or-keyword before 3.11 and positional-only from 3.11, so a class statement naming any of the four raises `TypeError` from the metaclass before the hook is reached. Those four are the whole of the `ABCMeta.__new__` collision surface, measured on 3.10.21, 3.11.15 and 3.14.7; `engine=`, `protocol=` and `fmt=` are all outside it, so the documented registration path works on every supported version. There is no `name=` alias -- a keyword that worked on some interpreters and not others is the trap being removed, not a compatibility measure.
+- **Added** -- `Protocol`/`ProtocolBase` gain a `code=` class keyword for the next-layer dispatch registries -- `Link.__proto__`, `Internet.__proto__`, `TCP.__proto__`, `UDP.__proto__`, `SCTP.__proto__`, `Frame.__proto__` and `PCAPNG.__proto__` -- the same opt-in treatment #514 gave `Engine`, `Reassembly`, `TraceFlow` and `Dumper` above, extended to the one family that needed a registration key invented rather than merely un-guarded. Omitting `code` leaves a subclass unregistered, exactly as before this keyword existed: the built-in dispatch tables are still populated by literal assignment in each layer module, not by `__init_subclass__`, so nothing the library ships moves. `code` accepts a bare enum member, whose *type* infers the destination -- `EtherType` means `Link`, `TransType` means `Internet`, `PayloadProtocolIdentifier` means `SCTP`, and `LinkType` means *both* `Frame` **and** `PCAPNG`, deterministically, mirroring what `register_linktype` already does by hand -- or a `{destination: key}` mapping, required for a raw `int` such as a TCP/UDP port number, which cannot say by itself which transport it belongs to. Either form may appear in an iterable, so one declaration can register a class into several registries at once, e.g. a `L2TP` subclass reachable both by IP protocol number and by a UDP port. The explicit mapping form is accepted even for a key whose type could be inferred -- being more explicit than required is never an error. Inference refuses rather than guesses: an enum member whose type names no known destination raises `RegistryError` instead of silently doing nothing or picking an arbitrary registry, and an unrecognised class keyword raises `UnsupportedCall`, matching the other four families. Backed by the new `pcapkit.foundation.registry.protocols.register_protocol_code`, which can also be called directly to register a class that declined at class-definition time. This is the mechanism #548 (`TransType.L2TP` registered nowhere) needs and does not yet use -- fixing it is now a one-declaration change, left for its own issue rather than folded in here.
 - **Changed** -- extraction is around 46% faster on a 1,117-frame HTTP capture, with byte-identical output (#420). A reassembled datagram's payload is now analysed on first read rather than eagerly, which cuts IP reassembly's own cost by 90.7% and TCP's by 23.7% -- IP reassembly submits a datagram for every frame, fragmented or not (#424). Flow tracing over the same capture went from 1416.6 ms to 744.0 ms, because the flow dumper had been handing each record to a `Frame` constructor that re-dissected the whole protocol stack to return bytes it had just been given; options are no longer parsed twice either (#427). All output compared byte-for-byte across the sample captures in each case.
 - **Fixed** -- next-layer, option, chunk, block and parameter dispatch all read `defaultdict` registries, so a lookup miss inserted the key into class-level state shared by every later instance, after which a legitimate `register_*` call warned that the code was already registered. Every read now goes through a lookup that does not grow the table, and `IPv4.__option__` and `HIP.__parameter__` became inspectable class attributes rather than names assembled at call time (#426, #428, #429, #434). One break comes with it: a tuple-registered handler pair written to the documented `OptionParser`/`OptionConstructor` signature now works where it could previously never be called at all, and a pair written with an explicit leading `self` -- the only shape that used to work -- now does not.
 - **Fixed** -- the identical defect one layer up, in the schema layer's own `EnumSchema.registry`: `Option.registry[code]` for an unregistered `code` inserted the default schema under that code, so a single lookup made an unassigned TCP option number, e.g. `156`, read back as registered for the rest of the process. `EnumSchema.__enum__` is now built (or, when a subclass seeds it manually in its own class body -- `PCAPNG.Option`'s namespaced mapping, `TCP.MPTCP`'s plain one) as a retention-safe mapping that still returns the registered default on a miss, it just stops recording it; `.registry` keeps returning the same object it always did, so nothing that held a reference to it is affected (#555).

docs/source/changelog/1.5.0.rst

@@ -171,6 +171,36 @@ pull requests between #326 and #509.
   so the documented registration path works on every supported version. There is
   no ``name=`` alias -- a keyword that worked on some interpreters and not others
   is the trap being removed, not a compatibility measure.
+* **Added** -- ``Protocol``/``ProtocolBase`` gain a ``code=`` class keyword for
+  the next-layer dispatch registries -- ``Link.__proto__``,
+  ``Internet.__proto__``, ``TCP.__proto__``, ``UDP.__proto__``,
+  ``SCTP.__proto__``, ``Frame.__proto__`` and ``PCAPNG.__proto__`` -- the same
+  opt-in treatment #514 gave ``Engine``, ``Reassembly``, ``TraceFlow`` and
+  ``Dumper`` above, extended to the one family that needed a registration key
+  invented rather than merely un-guarded. Omitting ``code`` leaves a subclass
+  unregistered, exactly as before this keyword existed: the built-in dispatch
+  tables are still populated by literal assignment in each layer module, not by
+  ``__init_subclass__``, so nothing the library ships moves. ``code`` accepts a
+  bare enum member, whose *type* infers the destination -- ``EtherType`` means
+  ``Link``, ``TransType`` means ``Internet``, ``PayloadProtocolIdentifier``
+  means ``SCTP``, and ``LinkType`` means *both* ``Frame`` **and** ``PCAPNG``,
+  deterministically, mirroring what ``register_linktype`` already does by
+  hand -- or a ``{destination: key}`` mapping, required for a raw ``int`` such
+  as a TCP/UDP port number, which cannot say by itself which transport it
+  belongs to. Either form may appear in an iterable, so one declaration can
+  register a class into several registries at once, e.g. a ``L2TP`` subclass
+  reachable both by IP protocol number and by a UDP port. The explicit
+  mapping form is accepted even for a key whose type could be inferred --
+  being more explicit than required is never an error. Inference refuses
+  rather than guesses: an enum member whose type names no known destination
+  raises ``RegistryError`` instead of silently doing nothing or picking an
+  arbitrary registry, and an unrecognised class keyword raises
+  ``UnsupportedCall``, matching the other four families. Backed by the new
+  ``pcapkit.foundation.registry.protocols.register_protocol_code``, which can
+  also be called directly to register a class that declined at class-definition
+  time. This is the mechanism #548 (``TransType.L2TP`` registered nowhere)
+  needs and does not yet use -- fixing it is now a one-declaration change, left
+  for its own issue rather than folded in here.
 * **Changed** -- extraction is around 46% faster on a 1,117-frame HTTP capture,
   with byte-identical output (#420). A reassembled datagram's payload is now
   analysed on first read rather than eagerly, which cuts IP reassembly's own

docs/source/ext.rst

@@ -227,6 +227,35 @@ The following code snippet shows how to create a new protocol class:
    # register protocol class
    register_ethertype(EtherType.Internet_Protocol_version_4, MyIPv4)
 
+.. note::
+
+   Registering after the fact, as above, is one option. The other is passing
+   ``code=`` at class definition, which is **opt-in** -- a class that omits it
+   stays unregistered, exactly as if no ``register_*`` call had been made
+   either:
+
+   .. code-block:: python
+
+      class MyIPv4(Internet[IPv4Data, IPv4Schema],
+                   schema=IPv4Schema, data=IPv4Data,
+                   code=EtherType.Internet_Protocol_version_4):
+          ...
+
+   The key's own type decides which registry it goes to -- an
+   :class:`~pcapkit.const.reg.ethertype.EtherType` member always means
+   :class:`~pcapkit.protocols.link.link.Link`, a
+   :class:`~pcapkit.const.reg.transtype.TransType` member always means
+   :class:`~pcapkit.protocols.internet.internet.Internet`, and so on -- so a
+   raw :class:`int`, such as a TCP or UDP port, cannot say which registry it
+   belongs to and must be given explicitly: ``code={TCP: 21}``. Either form
+   may appear in an iterable to register one class into several registries
+   at once. See
+   :meth:`ProtocolBase.__init_subclass__
+   <pcapkit.protocols.protocol.ProtocolBase.__init_subclass__>` for the full
+   rule, and
+   :func:`~pcapkit.foundation.registry.protocols.register_protocol_code` for
+   what runs underneath it.
+
 Extending Existing Protocol
 ---------------------------
 

pcapkit/all.py

@@ -80,6 +80,7 @@
 
     # pcapkit.foundation.registry
     'register_protocol',
+    'register_protocol_code',
     'register_linktype',
     'register_pcap', 'register_pcapng',
     'register_ethertype',

pcapkit/foundation/__init__.py

@@ -27,6 +27,7 @@
     'TraceFlowManager',
 
     'register_protocol',
+    'register_protocol_code',
     'register_linktype',
     'register_pcap', 'register_pcapng',
     'register_ethertype',

pcapkit/foundation/registry/__init__.py

@@ -25,6 +25,7 @@
     'register_extractor_reassembly', 'register_extractor_traceflow',
 
     'register_protocol',
+    'register_protocol_code',
 
     'register_linktype',
     'register_pcap', 'register_pcapng',

pcapkit/foundation/registry/protocols.py

@@ -7,10 +7,19 @@
 This module provides the protocol registries for :mod:`pcapkit`.
 
 """
+import collections.abc
+import enum
 from typing import TYPE_CHECKING, cast, overload
 
+import aenum
+
 from pcapkit.const.reg.apptype import AppType as Enum_AppType
 from pcapkit.const.reg.apptype import TransportProtocol
+from pcapkit.const.reg.ethertype import EtherType as Enum_EtherType
+from pcapkit.const.reg.linktype import LinkType as Enum_LinkType
+from pcapkit.const.reg.transtype import TransType as Enum_TransType
+from pcapkit.const.sctp.payload_protocol_identifier import \
+    PayloadProtocolIdentifier as Enum_PayloadProtocolIdentifier
 from pcapkit.corekit.module import ModuleDescriptor
 from pcapkit.protocols import __proto__ as protocol_registry
 from pcapkit.protocols.application.httpv2 import HTTP as HTTPv2
@@ -49,7 +58,7 @@
 from pcapkit.utilities.logging import get_logger
 
 if TYPE_CHECKING:
-    from typing import Optional, Type
+    from typing import Any, Iterator, Optional, Type
 
     from pcapkit.const.hip.parameter import Parameter as HIP_Parameter
     from pcapkit.const.http.frame import Frame as HTTP_Frame
@@ -104,6 +113,7 @@
 
 __all__ = [
     'register_protocol',
+    'register_protocol_code',
 
     'register_linktype',
     'register_pcap', 'register_pcapng',
@@ -151,6 +161,127 @@ def register_protocol(protocol: 'Type[Protocol]') -> 'None':
     logger.debug('registered protocol: %s', protocol.__name__)
 
 
+#: Enum type -> the class(es) owning the :attr:`ProtocolBase.__proto__
+#: <pcapkit.protocols.protocol.ProtocolBase.__proto__>` dispatch registry
+#: keyed by that enum type -- the "registry-of-registries" that lets
+#: ``code=`` infer a destination from a key's own type, per GH-514. This is
+#: not an invention: it is exactly the targeting
+#: :func:`register_ethertype`, :func:`register_transtype`,
+#: :func:`register_linktype` and :func:`register_sctp` already hard-code by
+#: hand, moved into one table so :func:`register_protocol_code` can consult
+#: it. ``LinkType`` naming two classes is deliberate, not ambiguous --
+#: :func:`register_linktype` already fans out to both.
+_CODE_DESTINATIONS: 'dict[type, tuple[Type[Protocol], ...]]' = {
+    Enum_EtherType: (Link,),
+    Enum_TransType: (Internet,),
+    Enum_PayloadProtocolIdentifier: (SCTP,),
+    Enum_LinkType: (Frame, PCAPNG),
+}
+
+
+def _iter_code_targets(code: 'Any') -> 'Iterator[tuple[Type[Protocol], Any]]':
+    """Flatten a ``code=`` argument into ``(destination, key)`` pairs.
+
+    Args:
+        code: See :func:`register_protocol_code`.
+
+    Yields:
+        One ``(destination, key)`` pair per registration the caller asked
+        for -- possibly several, e.g. a single :class:`~pcapkit.const.reg.\
+linktype.LinkType` member yields both :class:`Frame` and :class:`PCAPNG`.
+
+    Raises:
+        RegistryError: If a value has no destination it can name or infer;
+            see :func:`register_protocol_code`.
+
+    """
+    if isinstance(code, dict):
+        yield from code.items()
+        return
+
+    if isinstance(code, (enum.Enum, aenum.Enum)):
+        destinations = _CODE_DESTINATIONS.get(type(code))
+        if destinations is None:
+            raise RegistryError(
+                f'no destination registry is known for enum type {type(code).__name__!r} '
+                f'(key {code!r}); pass an explicit destination, e.g. code={{SomeClass: {code!r}}}')
+        for destination in destinations:
+            yield destination, code
+        return
+
+    if isinstance(code, (str, bytes)):
+        raise RegistryError(f'code must be an enum member, a {{destination: key}} mapping, or an '
+                            f'iterable thereof, not {code!r}')
+
+    if isinstance(code, collections.abc.Iterable):
+        for item in code:
+            yield from _iter_code_targets(item)
+        return
+
+    raise RegistryError(f'raw key {code!r} has no destination registry it can infer; pass an '
+                        f'explicit destination, e.g. code={{TCP: {code!r}}}')
+
+
+def register_protocol_code(protocol: 'Type[Protocol]', code: 'Any') -> 'None':
+    r"""Register ``protocol`` into the next-layer dispatch registry (or
+    registries) named by ``code``.
+
+    This is what backs the ``code`` keyword of
+    :meth:`ProtocolBase.__init_subclass__
+    <pcapkit.protocols.protocol.ProtocolBase.__init_subclass__>`; it can also
+    be called directly to register a class that declined at class-definition
+    time.
+
+    ``code`` is normalised into a flat sequence of targets, where each item
+    is either:
+
+    * an enum member, whose *type* is looked up in a small table mapping
+      enum type to the class(es) owning the matching ``__proto__`` --
+      :class:`~pcapkit.const.reg.ethertype.EtherType` to
+      :class:`~pcapkit.protocols.link.link.Link`,
+      :class:`~pcapkit.const.reg.transtype.TransType` to
+      :class:`~pcapkit.protocols.internet.internet.Internet`,
+      :class:`~pcapkit.const.sctp.payload_protocol_identifier.PayloadProtocolIdentifier`
+      to :class:`~pcapkit.protocols.transport.sctp.SCTP`, and
+      :class:`~pcapkit.const.reg.linktype.LinkType` to *both*
+      :class:`~pcapkit.protocols.misc.pcap.frame.Frame` **and**
+      :class:`~pcapkit.protocols.misc.pcapng.PCAPNG`; or
+    * a :class:`dict` mapping an explicit destination class to a key -- the
+      only form accepted for a bare :class:`int`, since e.g. a port number
+      does not say by itself whether it means
+      :class:`~pcapkit.protocols.transport.tcp.TCP` or
+      :class:`~pcapkit.protocols.transport.udp.UDP`;
+
+    or an iterable of either, to register ``protocol`` into several
+    registries from one call -- e.g. a
+    :class:`~pcapkit.protocols.link.l2tp.L2TP` subclass reachable both by its
+    IP protocol number and by a UDP port:
+
+    .. code-block:: python
+
+       register_protocol_code(L2TPv2, [TransType.L2TP, {UDP: 1701}])
+
+    The explicit mapping form is accepted for any key, even one whose type
+    could be inferred -- being more explicit than required is never an
+    error.
+
+    Args:
+        protocol: Protocol class to register.
+        code: Registration key(s); see above.
+
+    Raises:
+        RegistryError: If a bare :class:`int` (or any other non-enum, non-
+            mapping value) is given without an explicit destination, or if
+            an enum member's type names no known destination registry --
+            inference refuses rather than guesses.
+
+    """
+    for destination, key in _iter_code_targets(code):
+        destination.register(key, protocol)
+        logger.debug('registered %s into %s.__proto__: %s', protocol.__name__,
+                     destination.__name__, key)
+
+
 ###############################################################################
 # Top-Level Registries
 ###############################################################################