refactor(corekit)!: move decorators into corekit (#1735)

docs/source/contributing/conventions/index.rst

@@ -13,6 +13,8 @@ House Conventions
    arisen. :ref:`sentinel-convention` governs :mod:`pcapkit.corekit`,
    :ref:`extension-header-subclassing` a protocol class hierarchy,
    :ref:`protocol-layer-placement` which subpackage a dissector belongs in,
+   :ref:`utilities-or-corekit` which of :mod:`pcapkit.utilities` and
+   :mod:`pcapkit.corekit` a support module belongs in,
    :ref:`process` the repository rather than any of its code,
    :ref:`documentation` the prose itself -- on these pages and in the API
    reference -- rather than any code at all, and :ref:`test-coverage` what the
@@ -37,3 +39,4 @@ House Conventions
    process
    documentation
    test-coverage
+   utilities-or-corekit

docs/source/contributing/conventions/utilities-or-corekit.rst

@@ -0,0 +1,27 @@
+.. _utilities-or-corekit:
+
+Choosing Between ``utilities`` and ``corekit``
+----------------------------------------------
+
+Which of the two support packages a module belongs in. Settled by the owner on
+:issue:`1712`, over two alternatives that sorted modules by packet knowledge or by
+who uses them. This rule can be checked from the import graph alone, and it needed
+the fewest moves: one.
+
+:mod:`pcapkit.utilities` **is for cross-cutting services**: exceptions, warnings,
+logging, the compatibility layer, configuration, and wrappers around third-party
+libraries such as :mod:`~pcapkit.utilities.chardet`. The question to ask of a module
+is *could any code use this service, and does it import nothing from* :mod:`pcapkit`
+*outside* ``utilities``? Both halves have to hold, and *nothing* covers a top-level
+import, one inside a function and one under ``TYPE_CHECKING`` alike.
+:file:`tests/project/test_utilities_import_boundary_1735_unit.py` checks every module
+under :file:`pcapkit/utilities/` for it.
+
+:mod:`pcapkit.corekit` **is for the types the packet model is built from**: what
+:mod:`pcapkit.protocols`, :mod:`pcapkit.const` and :mod:`pcapkit.foundation`
+subclass, instantiate or decorate with. The question is *is this part of what a
+protocol or its schema is made of?* A support module that fails the ``utilities``
+question belongs here instead. :mod:`pcapkit.corekit.decorators` is the worked example: it acts on a
+protocol's file and falls back to protocol classes, so it moved out of ``utilities``
+on :issue:`1735`. ``corekit`` may import ``utilities``; the reverse is what the rule
+forbids.

docs/source/pcapkit/corekit/decorators.rst

@@ -0,0 +1,39 @@
+Decorator Functions
+===================
+
+.. module:: pcapkit.corekit.decorators
+
+:mod:`pcapkit.corekit.decorators` contains several useful
+decorators, including :func:`~pcapkit.corekit.decorators.seekset`,
+:func:`~pcapkit.corekit.decorators.beholder` and
+:func:`~pcapkit.corekit.decorators.prepare`.
+
+.. note::
+
+   This module was ``pcapkit.utilities.decorators`` until :issue:`1735` moved it
+   here, and :mod:`pcapkit.utilities` no longer exports the three decorators.
+   Nothing is left at the old path, so import them from this module.
+
+.. autodecorator:: pcapkit.corekit.decorators.seekset
+
+.. autodecorator:: pcapkit.corekit.decorators.beholder
+
+.. autodecorator:: pcapkit.corekit.decorators.prepare
+
+.. important::
+
+   All three decorators above are designed for decorating *class methods*.
+   For more information, please refer to the documentation of each
+   decorator function.
+
+Type Variables
+--------------
+
+.. data:: pcapkit.corekit.decorators.R_seekset
+   :type: typing.Any
+
+.. data:: pcapkit.corekit.decorators.R_beholder
+   :type: pcapkit.protocols.protocol.ProtocolBase
+
+.. data:: pcapkit.corekit.decorators.R_prepare
+   :type: pcapkit.protocols.schema.schema.Schema

docs/source/pcapkit/corekit/index.rst

@@ -10,7 +10,8 @@ collection :class:`~pcapkit.corekit.protochain.ProtoChain`, the
 :class:`~pcapkit.corekit.multidict.MultiDict` family for multi-entry
 mappings, the :class:`~pcapkit.corekit.fields.field.Field` family for data
 parsing, the :class:`~pcapkit.corekit.context.ContextRegistry` channel for
-caller-supplied information a protocol needs but the wire does not carry, and
+caller-supplied information a protocol needs but the wire does not carry, the
+:mod:`~pcapkit.corekit.decorators` that protocols and schemas parse through, and
 the :class:`~pcapkit.corekit.enum.EnumLookup`/
 :class:`~pcapkit.corekit.enum.EnumRegistry` bases every constant enumeration
 inherits from.
@@ -20,6 +21,7 @@ inherits from.
 
    fields/index
    context
+   decorators
    enum
    infoclass
    io

docs/source/pcapkit/protocols/application/ngap.rst

@@ -32,7 +32,7 @@ it; an NGAP payload simply degrades to the opaque payload path, exactly as an
 unregistered PPID would, because
 :meth:`~pcapkit.protocols.protocol.Protocol._import_next_layer`, which
 :class:`~pcapkit.protocols.transport.sctp.SCTP` inherits, is wrapped in
-:func:`~pcapkit.utilities.decorators.beholder` and falls back to
+:func:`~pcapkit.corekit.decorators.beholder` and falls back to
 :class:`~pcapkit.protocols.misc.raw.Raw`.
 
 Decoder Dependency

docs/source/pcapkit/utilities/index.rst

@@ -4,16 +4,14 @@ Module Utilities
 
 .. module:: pcapkit.utilities
 
-:mod:`pcapkit.utilities` contains several useful functions
-and classes which are foundations of :mod:`pcapkit`, including
-decorator function :func:`~pcapkit.utilities.decorators.seekset`
-and :func:`~pcapkit.utilities.decorators.beholder`, and
-several user-refined exceptions and warnings.
+:mod:`pcapkit.utilities` holds the cross-cutting services :mod:`pcapkit` is
+built with: user-refined exceptions and warnings, logging, a compatibility
+layer, and wrappers around third-party libraries. It imports nothing from
+:mod:`pcapkit` outside itself; see :ref:`utilities-or-corekit`.
 
 .. toctree::
    :maxdepth: 2
 
-   decorators
    exceptions
    warnings
    logging

examples/generators/dispatch.py

@@ -786,7 +786,7 @@ class Outcome(NamedTuple):
     detail: 'str'
     #: The first ``'error'`` value found anywhere in the parsed frame's nested
     #: info mapping, or ``''`` if none is present. This is what
-    #: :func:`pcapkit.utilities.decorators.beholder` records when a dissector
+    #: :func:`pcapkit.corekit.decorators.beholder` records when a dissector
     #: raises and gets replaced with :class:`~pcapkit.protocols.misc.raw.Raw`,
     #: so it is the fragment :data:`KNOWN_DEGRADED` pins against.
     error: 'str'

pcapkit/all.py

@@ -145,5 +145,4 @@
     # pcapkit.utilities -- unlisted: its names stay reachable as attributes, but
     # ``import *`` leaves out generic helpers such as ``warn``, ``reset``,
     # ``configure`` and ``detect``
-    #'beholder_ng', 'seekset_ng',                            # Decorators
 ]

pcapkit/utilities/decorators.py → pcapkit/corekit/decorators.py

@@ -3,12 +3,17 @@
 """Decorator Functions
 =========================
 
-.. module:: pcapkit.utilities.decorators
+.. module:: pcapkit.corekit.decorators
 
-:mod:`pcapkit.utilities.decorators` contains several useful
-decorators, including :func:`~pcapkit.utilities.decorators.seekset`,
-:func:`~pcapkit.utilities.decorators.beholder` and
-:func:`~pcapkit.utilities.decorators.prepare`.
+:mod:`pcapkit.corekit.decorators` contains several useful
+decorators, including :func:`~pcapkit.corekit.decorators.seekset`,
+:func:`~pcapkit.corekit.decorators.beholder` and
+:func:`~pcapkit.corekit.decorators.prepare`.
+
+They act on a protocol's file and on the schema it unpacks, and
+:func:`~pcapkit.corekit.decorators.beholder` falls back to protocol classes,
+so they live with the packet model rather than in :mod:`pcapkit.utilities`
+(:issue:`1735`).
 
 """
 import functools

pcapkit/foundation/extraction.py

@@ -1571,7 +1571,7 @@ def _note_eof_progress(self) -> 'bool':
         :issue:`620` reported.
 
         The signal is the input's own position. End of stream is raised by
-        :func:`~pcapkit.utilities.decorators.prepare` when the bytes remaining in
+        :func:`~pcapkit.corekit.decorators.prepare` when the bytes remaining in
         the stream measure zero, and it restores the position before raising, so
         the position at end of stream is stable. Two consecutive ends of stream at
         the *same* position therefore mean nothing arrived between them.

pcapkit/protocols/application/ngap.py

@@ -30,7 +30,7 @@
 unregistered PPID would, because
 :meth:`~pcapkit.protocols.protocol.Protocol._import_next_layer`, which
 :class:`~pcapkit.protocols.transport.sctp.SCTP` inherits, is wrapped in
-:func:`~pcapkit.utilities.decorators.beholder` and falls back to
+:func:`~pcapkit.corekit.decorators.beholder` and falls back to
 :class:`~pcapkit.protocols.misc.raw.Raw`.
 
 Decoder Dependency

pcapkit/protocols/data/internet/ipv6_ext.py

@@ -48,6 +48,6 @@ class IPv6_Ext(Protocol):
     #: Header-specific content after the two fixed octets, as consumed.
     data: 'bytes'
     #: Original parsing error, if this instance was reached as a
-    #: :func:`~pcapkit.utilities.decorators.beholder` fallback rather than by
+    #: :func:`~pcapkit.corekit.decorators.beholder` fallback rather than by
     #: direct dispatch.
     error: 'Optional[Exception]'

pcapkit/protocols/internet/internet.py

@@ -18,11 +18,11 @@
 
 from pcapkit.const.ipv6.extension_header import ExtensionHeader as Enum_ExtensionHeader
 from pcapkit.const.reg.transtype import TransType as Enum_TransType
+from pcapkit.corekit.decorators import beholder
 from pcapkit.corekit.module import ModuleDescriptor
 from pcapkit.corekit.protochain import ProtoChain
 from pcapkit.protocols.protocol import _PT, _ST, ProtocolBase, _rebuilding
 from pcapkit.protocols.schema.schema import keep_short_read, replay_short_read
-from pcapkit.utilities.decorators import beholder
 from pcapkit.utilities.exceptions import stacklevel
 from pcapkit.utilities.warnings import ProtocolWarning, RegistryWarning, warn
 

pcapkit/protocols/internet/ipv6.py

@@ -31,6 +31,7 @@
 
 from pcapkit.const.ipv6.extension_header import ExtensionHeader as Enum_ExtensionHeader
 from pcapkit.const.reg.transtype import TransType as Enum_TransType
+from pcapkit.corekit.decorators import beholder
 from pcapkit.corekit.infoclass import Info
 from pcapkit.corekit.multidict import MultiDict, OrderedMultiDict
 from pcapkit.corekit.protochain import ProtoChain
@@ -40,7 +41,6 @@
 from pcapkit.protocols.protocol import FRAME_LAYER_LIMIT, _frame_depth
 from pcapkit.protocols.schema.internet.ipv6 import IPv6 as Schema_IPv6
 from pcapkit.protocols.schema.internet.ipv6 import jumbo_payload_length, jumbogram_error
-from pcapkit.utilities.decorators import beholder
 from pcapkit.utilities.exceptions import ProtocolError
 
 if TYPE_CHECKING:
@@ -72,7 +72,7 @@ class IPv6(IP[Data_IPv6, Schema_IPv6],
     #: When that dedicated parser raises, :meth:`_import_next_layer`
     #: substitutes :class:`~pcapkit.protocols.internet.ipv6_ext.IPv6_Ext`
     #: instead of letting the generic
-    #: :func:`~pcapkit.utilities.decorators.beholder` fall back to plain
+    #: :func:`~pcapkit.corekit.decorators.beholder` fall back to plain
     #: :class:`~pcapkit.protocols.misc.raw.Raw`, which has no ``next`` field
     #: and so cannot continue the walk in :meth:`_decode_next_layer`.
     #:
@@ -606,7 +606,7 @@ def _decode_next_layer(self, ipv6: 'Data_IPv6', proto: 'Optional[int]' = None,
             # :class:`~pcapkit.protocols.misc.raw.Raw`, which does not (see
             # :attr:`__generic_ext_codes__`). Reading ``info.next`` on that
             # would raise ``AttributeError`` and let a further-out
-            # :func:`~pcapkit.utilities.decorators.beholder` degrade the
+            # :func:`~pcapkit.corekit.decorators.beholder` degrade the
             # *whole* packet.
             #
             # Such a layer is not recorded as an extension header: it consumed
@@ -725,7 +725,7 @@ def _import_next_layer(self, proto: 'int', length: 'Optional[int]' = None, *,  #
             raises, this substitutes
             :class:`~pcapkit.protocols.internet.ipv6_ext.IPv6_Ext` for it
             rather than letting the exception reach the
-            :func:`~pcapkit.utilities.decorators.beholder` decorating this
+            :func:`~pcapkit.corekit.decorators.beholder` decorating this
             method, which would substitute plain
             :class:`~pcapkit.protocols.misc.raw.Raw` -- and ``Raw`` has no
             ``next`` field, so it could not continue

pcapkit/protocols/internet/ipv6_ext.py

@@ -107,7 +107,7 @@
 2. **A recognised header whose own parser raises.** ``HOPOPT``,
    ``IPv6-Route``, ``IPv6-Opts``, ``MH``, ``HIP``, ``IPv6-Frag`` and ``AH`` all
    have dedicated classes. When one of *those* raises,
-   :func:`~pcapkit.utilities.decorators.beholder`
+   :func:`~pcapkit.corekit.decorators.beholder`
    (:meth:`Protocol._import_next_layer <pcapkit.protocols.protocol.ProtocolBase._import_next_layer>`)
    would substitute plain :class:`~pcapkit.protocols.misc.raw.Raw`, which loses
    ``next`` and crashes the walk as in path 1.
@@ -381,7 +381,7 @@ def read(self, length: 'Optional[int]' = None, *,  # pylint: disable=arguments-d
                   stands in for, e.g. ``0`` for ``HOPOPT`` or ``140`` for
                   ``Shim6``.
                 * ``error`` -- the parsing error, when this instance was
-                  reached as a :func:`~pcapkit.utilities.decorators.beholder`
+                  reached as a :func:`~pcapkit.corekit.decorators.beholder`
                   fallback rather than by direct dispatch.
 
         Returns:
@@ -578,7 +578,7 @@ def __post_init__(self, file: 'Optional[IO[bytes] | bytes]' = None, length: 'Opt
             reading ``IPv4:IPv6-Ext:...`` instead of the ``IPv4:Shim6`` that a
             plain, non-continuing ``Raw`` gives. Rejecting here sends
             construction back through
-            :func:`~pcapkit.utilities.decorators.beholder` at the *caller's*
+            :func:`~pcapkit.corekit.decorators.beholder` at the *caller's*
             layer, which substitutes that same ``Raw``: the version-agnostic
             behaviour, with no IPv4-specific code.
 

pcapkit/protocols/misc/pcapng.py

@@ -1316,7 +1316,7 @@ def read(self, length: 'Optional[int]' = None, *, _read: 'bool' = True,
             # copy and never against the file, so on a capture cut short it runs
             # past the real end -- and seeking past the end is legal and silent.
             # The *next* block read would then measure a negative remainder,
-            # since ``pcapkit.utilities.decorators.prepare`` derives it as the
+            # since ``pcapkit.corekit.decorators.prepare`` derives it as the
             # end of the stream less the current position, and a negative
             # ``__length__`` is what ``pcapng_block_selector`` hands to
             # :class:`~pcapkit.corekit.fields.misc.SchemaField`, which fails with
@@ -1530,7 +1530,7 @@ def _check_block_floor(self, length: 'Optional[int]') -> 'None':
         :meth:`FieldBase.unpack <pcapkit.corekit.fields.field.FieldBase.unpack>`,
         which has no catch point above it and would abort the whole extraction --
         this costs no frame that was actually in the file. It is the same signal
-        :func:`~pcapkit.utilities.decorators.prepare` raises for a *derived*
+        :func:`~pcapkit.corekit.decorators.prepare` raises for a *derived*
         remainder of exactly zero; a remainder of one, two or three octets
         passes that check and is caught here instead.
 

pcapkit/protocols/misc/raw.py

@@ -139,7 +139,7 @@ def __post_init__(self, file: 'Optional[IO[bytes] | bytes]' = None,
         self._file = io.BytesIO(self._data)
 
         # NOTE: The buffer above is all there is to parse, so its size is
-        # declared rather than left for :func:`~pcapkit.utilities.decorators.prepare`
+        # declared rather than left for :func:`~pcapkit.corekit.decorators.prepare`
         # to derive: a derived zero is read as the end of the stream, whereas
         # an empty raw payload is a payload all the same (:issue:`1285`).
         if length is None:

pcapkit/protocols/protocol.py

@@ -35,6 +35,7 @@
 import aenum
 
 from pcapkit.corekit.context import ContextRegistry
+from pcapkit.corekit.decorators import beholder, seekset
 from pcapkit.corekit.fields.misc import ConditionalField, SchemaField, SwitchField
 from pcapkit.corekit.module import ModuleDescriptor
 from pcapkit.corekit.protochain import ProtoChain
@@ -48,7 +49,6 @@
 from pcapkit.protocols.schema.schema import Schema
 from pcapkit.utilities.chardet import detect
 from pcapkit.utilities.compat import cached_property, get_annotations
-from pcapkit.utilities.decorators import beholder, seekset
 from pcapkit.utilities.exceptions import (BaseError, ProtocolError, ProtocolNotFound,
                                           ProtocolNotImplemented, RegistryError, StructError,
                                           UnsupportedCall, stacklevel)

pcapkit/protocols/schema/schema.py

@@ -8,14 +8,14 @@
 import itertools
 from typing import TYPE_CHECKING, Any, Generic, TypeVar, cast
 
+from pcapkit.corekit.decorators import prepare
 from pcapkit.corekit.fields.collections import ListField, OptionField
 from pcapkit.corekit.fields.field import NO_VALUE, FieldBase
 from pcapkit.corekit.fields.misc import (ConditionalField, ForwardMatchField, PayloadField,
                                          SchemaField)
 from pcapkit.corekit.fields.strings import PaddingField
 from pcapkit.corekit.infoclass import FinalisedState
 from pcapkit.utilities.compat import Mapping, final
-from pcapkit.utilities.decorators import prepare
 from pcapkit.utilities.exceptions import NoDefaultValue, ProtocolUnbound, SchemaError, stacklevel
 from pcapkit.utilities.warnings import RegistryWarning, SchemaWarning, UnknownFieldWarning, warn
 

pcapkit/utilities/__init__.py

@@ -5,19 +5,17 @@
 
 .. module:: pcapkit.utilities
 
-:mod:`pcapkit.utilities` contains several useful functions
-and classes which are foundations of :mod:`pcapkit`, including
-decorator function :func:`~pcapkit.utilities.decorators.seekset`
-and :func:`~pcapkit.utilities.decorators.beholder`, etc., and
-several user-refined exceptions and warnings.
+:mod:`pcapkit.utilities` holds the cross-cutting services
+:mod:`pcapkit` is built with: user-refined exceptions and
+warnings, logging, a compatibility layer, and wrappers around
+third-party libraries. It imports nothing from :mod:`pcapkit`
+outside itself (:issue:`1712`).
 
 """
 from pcapkit.utilities.chardet import detect
-from pcapkit.utilities.decorators import beholder, prepare, seekset
 from pcapkit.utilities.exceptions import stacklevel
 from pcapkit.utilities.logging import configure, ensure_output, get_logger, logger, reset
 from pcapkit.utilities.warnings import warn
 
 __all__ = ['logger', 'get_logger', 'configure', 'reset', 'ensure_output',
-           'warn', 'stacklevel', 'detect',
-           'beholder', 'prepare', 'seekset']
+           'warn', 'stacklevel', 'detect']