docs: correct documentation that stated untrue things (#546)

.github/workflows/deploy-pages.yml

@@ -57,7 +57,14 @@ jobs:
         with:
           python-version: '3.14'
 
-      - name: Install and Build 🔧 # This example project is built using npm and outputs the result to the 'build' folder. Replace with the commands required to build your project, or remove this step entirely if your site is pre-built.
+      # Builds the Sphinx documentation into `docs/build/html`, which the deploy
+      # step below publishes. `PCAPKIT_SPHINX` is what makes the
+      # `if SPHINX_TYPE_CHECKING:` TypedDicts visible to autodoc; `PCAPKIT_DEVMODE`
+      # puts the library logger at DEBUG on stderr and leaves tracebacks untrimmed,
+      # so an autodoc import failure is legible in the run log. Both are read at
+      # import time, so they are exported before anything imports `pcapkit` --
+      # `docs/source/conf.py` sets `PCAPKIT_SPHINX` itself for the same reason.
+      - name: Install and Build 🔧
         run: |
           set -x
 

CHANGELOG.md

@@ -7,7 +7,7 @@ The largest release since 1.0, and the first recorded here as it happened rather
 - **Added** -- ESP parsing and construction [[RFC 4303](https://datatracker.ietf.org/doc/html/rfc4303)], with optional payload decryption and ICV verification through `cryptography` (`pip install pypcapkit[crypto]`) (#378). Keys reach the dissector through a new caller-state channel, `pcapkit.corekit.context`, surfaced as the `context=` keyword on `extract()` and `Extractor`, carrying an `esp.SecurityAssociation`. Nothing here raises: with no association the SPI and sequence number are still reported and the ciphertext is left opaque, with `status=NO_SA`, and a failed decryption or ICV check is recorded on the parsed result the same way. Extended Sequence Numbers, TFC padding and anti-replay are not implemented, and an unsupported cipher or MAC is refused with a clear error rather than half-processed.
 - **Added** -- SCTP as a transport protocol [[RFC 9260](https://datatracker.ietf.org/doc/html/rfc9260)]: all 13 chunk types, 8 chunk parameters and 13 error causes, with the CRC32c both recorded and verifiable -- it covers the SCTP packet alone, with no IP pseudo-header, so it can be checked from the SCTP bytes. Upper layers register on the DATA chunk's Payload Protocol Identifier through `register_sctp`, not on a port (#379).
 - **Added** -- NGAP over SCTP (3GPP TS 38.413), decoding aligned PER through `pycrate` (`pip install pypcapkit[NGAP]`) (#251, #417). Decoding is generic by ASN.1 shape rather than per-procedure, so all 81 elementary procedures and 438 protocol IEs work and a new 3GPP release needs no code change. Registered as a default on PPID 60 and 66, but only 60 decodes: PPID 66 is an NGAP PDU inside a DTLS record, and there is no DTLS dissector. `pycrate` is deliberately excluded from the `all` extra -- it is LGPL-2.1+ where this package is BSD-3-Clause, and lands some 238 MB to obtain one module.
-- **Added** -- the Mobility Header registry, completed (#383, #437). The [RFC 5568](https://datatracker.ietf.org/doc/html/rfc5568) fast-handover messages and options first, then all 24 registered message data types, 70 of the 71 registered options -- with nested sub-option registries for the flow identification, access network identifier, quality-of-service and LMA-controlled MAG parameter families -- and all 4 CGA extensions. Only the CGA Parameters option remains on the generic handler.
+- **Added** -- the Mobility Header registry, completed (#383, #437). The [RFC 5568](https://datatracker.ietf.org/doc/html/rfc5568) fast-handover messages and options first, then all 24 registered message data types, all 71 registered options -- with nested sub-option registries for the flow identification, access network identifier, quality-of-service and LMA-controlled MAG parameter families -- and all 4 CGA extensions. The CGA Parameters option was the last to come off the generic handler, which is what let the four CGA extensions round-trip end to end, since it is the only option that can carry one on the wire.
 - **Added** -- dispatch entries for dissectors that existed but were reachable from no registry (#436): FTP-DATA on TCP 20, HTTP/1 on TCP 8080, HTTP on UDP 8080, `L2TPv2` on UDP 1701 and OSPF at `TransType` 89. `VLAN` became an abstract base with `C_Tag` (802.1Q) and `S_Tag` (802.1ad) as concrete subclasses, so a Q-in-Q frame no longer collapses into one opaque `Raw`; `L2TP` likewise became a base, with `L2TPv2` carrying the [RFC 2661](https://datatracker.ietf.org/doc/html/rfc2661) implementation.
 - **Added** -- `pcapkit.utilities.logging` as a real interface: `get_logger()` for per-module children, `configure()` to set level, handler, stream, format or propagation at runtime, `reset()` to return to library-neutral, and `ensure_output()`. Seventeen modules now log under their own `__name__`, so a consumer can silence `pcapkit.foundation.registry` while keeping `pcapkit.foundation.extraction` (#384).
 - **Added** -- `conflict` on the reassembly data models: absolute, inclusive ranges where two fragments claimed the same span with different bytes, which was previously lost silently on both the IP (#482) and TCP (#443, #478) paths.

SECURITY.md

@@ -2,18 +2,81 @@
 
 ## Supported Versions
 
-Use this section to tell people about which versions of your project are
-currently being supported with security updates.
+Security fixes land on the current stable line and on the development line. Older
+lines are not patched -- upgrading is the fix.
 
-| Version | Supported          |
-| ------- | ------------------ |
-| 1.0.x   | :white_check_mark: |
-| 0.x.x   | :x:                |
+| Version         | Supported          | Notes                                   |
+| --------------- | ------------------ | --------------------------------------- |
+| 1.5.x           | :white_check_mark: | development line, currently pre-release |
+| 1.4.x           | :white_check_mark: | current stable                          |
+| 1.3.x and older | :x:                | no longer patched                       |
+| 0.x             | :x:                | no longer patched                       |
+
+Supported interpreters are those the test matrix actually covers, currently
+CPython 3.10 through 3.14. `pyproject.toml` declares `requires-python = ">=3.6, <4"`
+through the `bpc-*` source conversion, but 3.8 and 3.9 are best-effort and below
+3.8 is intent rather than something that is exercised -- see the note above
+`requires-python` in `pyproject.toml`. A report that only reproduces on an
+interpreter outside the tested range is still welcome; it may be fixed by raising
+the floor rather than by patching.
 
 ## Reporting a Vulnerability
 
-Use this section to tell people how to report a vulnerability.
+**Please do not open a public issue for a security problem.** Public issues are
+the right place for ordinary bugs, and the wrong place for anything exploitable.
+
+Use GitHub's private vulnerability reporting, which is enabled on this repository:
+
+- <https://github.com/JarryShaw/PyPCAPKit/security/advisories/new>
+
+That opens a private advisory visible only to the maintainer, lets patches be
+prepared before anything is public, and issues a CVE if one is warranted. If you
+cannot use it, email the project contact listed in
+[`CODE_OF_CONDUCT.md`](CODE_OF_CONDUCT.md) instead and say that the report is a
+security issue.
+
+Please include whatever you have:
+
+- the version of `pypcapkit` and the Python interpreter,
+- what an attacker gains, and what they need in order to get it,
+- a capture file or a short script that reproduces it -- **strip anything
+  sensitive from a capture first**, since a pcap taken from a real network tends
+  to carry more than the bug.
+
+## What to expect
+
+This is a single-maintainer project, so response is best-effort rather than
+contractual:
+
+- an acknowledgement that the report arrived, normally within a week,
+- an assessment of whether it is accepted, and if so a rough severity, once it has
+  been reproduced,
+- a fix on the supported lines above, released as a patch version, with credit in
+  the changelog unless you would rather not be named.
+
+If a report is declined, you will be told why -- most often that it describes
+expected behaviour for a packet parser. Note the threat model below before
+deciding whether you have a finding.
+
+## Threat model
+
+`pcapkit` parses attacker-controlled bytes by design: a capture file, or a live
+stream off the wire, is untrusted input. Vulnerabilities in that parsing are in
+scope. In particular:
+
+- memory exhaustion, unbounded allocation, or a hang on a malformed packet,
+- an uncaught exception escaping the public API where a `pcapkit` exception was
+  promised -- parse errors are supposed to arrive as
+  `pcapkit.utilities.exceptions` types, not as an arbitrary traceback,
+- anything that executes code, writes outside the requested output path, or reads
+  an unrelated file as a result of the *contents* of a capture.
+
+Out of scope:
 
-Tell them where to go, how often they can expect to get an update on a
-reported vulnerability, what to expect if the vulnerability is accepted or
-declined, etc.
+- the optional third-party extraction engines' own defects (`dpkt`, `scapy`,
+  `pyshark`, `pypcap`, `pcap-ct`, `pypcapfile`) -- report those upstream, though
+  do say so here if `pcapkit` passes them something it should not,
+- a capture that parses to the wrong values without any security consequence;
+  that is an ordinary bug and belongs in a public issue,
+- needing elevated privileges to capture live traffic, which is the operating
+  system's requirement rather than this project's.

docs/source/changelog/1.5.0.rst

@@ -52,10 +52,12 @@ pull requests between #326 and #509.
   package is BSD-3-Clause, and lands some 238 MB to obtain one module.
 * **Added** -- the Mobility Header registry, completed (#383, #437). The
   :rfc:`5568` fast-handover messages and options first, then all 24 registered
-  message data types, 70 of the 71 registered options -- with nested sub-option
+  message data types, all 71 registered options -- with nested sub-option
   registries for the flow identification, access network identifier,
   quality-of-service and LMA-controlled MAG parameter families -- and all 4 CGA
-  extensions. Only the CGA Parameters option remains on the generic handler.
+  extensions. The CGA Parameters option was the last to come off the generic
+  handler, which is what let the four CGA extensions round-trip end to end,
+  since it is the only option that can carry one on the wire.
 * **Added** -- dispatch entries for dissectors that existed but were reachable
   from no registry (#436): FTP-DATA on TCP 20, HTTP/1 on TCP 8080, HTTP on UDP
   8080, ``L2TPv2`` on UDP 1701 and OSPF at ``TransType`` 89. ``VLAN`` became an

docs/source/conf.py

@@ -9,8 +9,6 @@
 import logging
 import os
 import pkgutil
-import sys
-import typing
 from typing import TYPE_CHECKING
 
 # NB: a private module of ``sphinx-autodoc-typehints``, used deliberately -- see
@@ -30,7 +28,7 @@
 import pcapkit
 
 if TYPE_CHECKING:
-    from typing import Any, Dict, List, Optional
+    from typing import Any, Dict, Optional
     from sphinx.application import Sphinx
 
 logger = logging.getLogger('pcapkit-sphinx')
@@ -335,56 +333,32 @@ def claim_attribute_signature(app: 'Sphinx', what: str, name: str,  # pylint: di
     return None
 
 
-def remove_module_docstring(app: 'Sphinx', what: str, name: str,  # pylint: disable=unused-argument
-                            obj: 'Any', options: 'Dict[str, Any]', lines: 'List[str]') -> None:  # pylint: disable=unused-argument
-    if what == "module" and "pcapkit" in name:
-        module = sys.modules.get(name)
-        if module is not None:
-            logger.info('reloading module: %s', name)
-            typing.TYPE_CHECKING = True
-            importlib.reload(module)
-            logger.info('reloaded module: %s', name)
-        #lines.clear()
-
-
-def process_docstring(app: 'Sphinx', what: str, name: str,  # pylint: disable=unused-argument
-                      obj: 'Any', options: 'Dict[str, Any]', lines: 'List[str]') -> None:  # pylint: disable=unused-argument
-    if what == "module" and "pcapkit" in name:
-        module = importlib.import_module(name)
-        typing.TYPE_CHECKING = True
-        importlib.reload(module)
-
-
-def process_fields(app: 'Sphinx', what: str, name: str, obj: 'Any', options: 'Dict[str, Any]', lines: 'List[str]') -> 'None':
-    if what == 'attribute' \
-        and name.startswith('pcapkit.protocols.schema') \
-        and type(obj).__module__.startswith('pcapkit.corekit.fields'):
-
-        print(name, obj)
-        #lines.append(':param packet: Packet data.',)
-
-
-def source_read(app: 'Sphinx', docname: str, source_text: str) -> 'None':  # pylint: disable=unused-argument
-    print(docname, source_text)
+# NB: four handlers used to sit here -- ``remove_module_docstring``,
+# ``process_docstring``, ``process_fields`` and ``source_read`` -- each with its
+# ``app.connect`` call commented out in ``setup`` below, so none of them had run
+# for as long as they had been in the file. They are deleted rather than restored,
+# because restoring them would either duplicate work now done properly or make the
+# build unusable:
+#
+#   ``remove_module_docstring`` and ``process_docstring`` were two attempts at the
+#   same thing -- flipping :data:`typing.TYPE_CHECKING` and reloading each module
+#   so autodoc could see the ``if TYPE_CHECKING:`` names. ``bind_type_checking_names``
+#   above now solves that, per module and without mutating a global mid-build.
+#
+#   ``process_fields`` only ever printed; the ``lines.append`` that was its point
+#   was itself commented out.
+#
+#   ``source_read`` printed every document's entire source text, which on this
+#   project is several thousand pages of build log.
+#
+# If the ``TYPE_CHECKING`` behaviour is ever wanted again, ``bind_type_checking_names``
+# is where it belongs, not a reload hook.
 
 
 def setup(app: 'Sphinx') -> None:
-    #app.connect('autodoc-process-docstring', process_docstring, 0)
-    #app.connect("autodoc-process-docstring", remove_module_docstring)
     app.connect('builder-inited', bind_type_checking_names)
     app.connect('autodoc-skip-member', maybe_skip_member)
     # NB: below ``sphinx_autodoc_typehints``, which connects at the default 500 and
     # would otherwise win the tie on registration order -- conf.py's ``setup`` runs
     # after the extensions in ``extensions`` have been set up.
     app.connect('autodoc-process-signature', claim_attribute_signature, priority=400)
-    #app.connect('source-read', source_read)
-    #app.connect('autodoc-process-docstring', process_fields)
-
-    # typing.TYPE_CHECKING = True
-    # for name, module in sys.modules.copy().items():
-    #     if 'pcapkit' not in name:
-    #         continue
-
-    #     logger.info('reloading module: %s', name)
-    #     importlib.reload(module)
-    #     logger.info('reloaded module: %s', name)

docs/source/ext.rst

@@ -83,6 +83,8 @@ The following table shows all available protocol classes in :mod:`pcapkit`:
 |                                                                  | HTTP Family    | :class:`pcapkit.protocols.application.httpv1.HTTP`                                  |
 +                                                                  +                +-----------------------+-------------------------------------------------------------+
 |                                                                  |                | :class:`pcapkit.protocols.application.httpv2.HTTP`                                  |
++                                                                  +----------------+-----------------------+-------------------------------------------------------------+
+|                                                                  | :class:`pcapkit.protocols.application.ngap.NGAP`                                                     |
 +------------------------------------------------------------------+----------------+-----------------------+-------------------------------------------------------------+
 |                                                                  |                | :class:`pcapkit.protocols.misc.pcap.header.Header`                                  |
 +                                                                  + PCAP Format    +-----------------------+-------------------------------------------------------------+

docs/source/pcapkit/corekit/context.rst

@@ -20,13 +20,13 @@ Rather than adding protocol specific keyword arguments to
 :class:`~pcapkit.foundation.extraction.Extractor`, such information is passed
 as a :class:`~pcapkit.corekit.context.ContextRegistry` -- a mapping of
 protocol index ID (c.f. :meth:`Protocol.id
-<pcapkit.protocols.protocol.ProtocolBase.id>`) to a
+<pcapkit.protocols.protocol.Protocol.id>`) to a
 :class:`~pcapkit.corekit.context.ProtocolContext` instance. The registry is
 handed to :class:`~pcapkit.foundation.extraction.Extractor` once, and is then
 propagated down the protocol stack by
-:meth:`Protocol._import_next_layer <pcapkit.protocols.protocol.ProtocolBase._import_next_layer>`,
+:meth:`Protocol._import_next_layer <pcapkit.protocols.protocol.Protocol._import_next_layer>`,
 so that a protocol nested arbitrarily deep can reach it through
-:meth:`Protocol._get_context <pcapkit.protocols.protocol.ProtocolBase._get_context>`.
+:meth:`Protocol._get_context <pcapkit.protocols.protocol.Protocol._get_context>`.
 
 Decoding an ESP tunnel end to end:
 

docs/source/pcapkit/foundation/extraction.rst

@@ -62,6 +62,7 @@ extracts parametres from a PCAP file.
    .. autoattribute:: _flag_d
    .. autoattribute:: _flag_e
    .. autoattribute:: _flag_q
+   .. autoattribute:: _flag_r
    .. autoattribute:: _flag_t
    .. autoattribute:: _flag_v
    .. autoattribute:: _flag_n
@@ -79,6 +80,7 @@ extracts parametres from a PCAP file.
 
    .. autoattribute:: _exlyr
    .. autoattribute:: _exptl
+   .. autoattribute:: _exctx
 
    .. automethod:: __iter__
    .. automethod:: __next__

docs/source/pcapkit/foundation/reassembly/ip/ipv4.rst

@@ -24,7 +24,7 @@ Terminology
        Data structure for **IPv4 datagram reassembly**
        (:meth:`IPv4.reassembly <pcapkit.foundation.reassembly.reassembly.Reassembly.reassembly>`)
        is as following, with ``ipv4`` the protocol instance
-       (``frame['IPv4']``) and ``ipv4_info`` its :attr:`~pcapkit.protocols.protocol.ProtocolBase.info`
+       (``frame['IPv4']``) and ``ipv4_info`` its :attr:`~pcapkit.protocols.protocol.Protocol.info`
        -- the header fields come off the latter, the raw octets off the former:
 
        .. code-block:: python

docs/source/pcapkit/foundation/reassembly/ip/ipv6.rst

@@ -24,7 +24,7 @@ Terminology
        Data structure for **IPv6 datagram reassembly**
        (:meth:`IPv6.reassembly <pcapkit.foundation.reassembly.reassembly.Reassembly.reassembly>`)
        is as following, with ``ipv6_info`` the IPv6
-       :attr:`~pcapkit.protocols.protocol.ProtocolBase.info` and
+       :attr:`~pcapkit.protocols.protocol.Protocol.info` and
        ``ipv6_frag_info`` the Fragment header's:
 
        .. code-block:: python

docs/source/pcapkit/protocols/application/application.rst

@@ -6,8 +6,8 @@ Base Protocol
 :mod:`pcapkit.protocols.application.application` contains only
 :class:`~pcapkit.protocols.application.application.Application`,
 which is a base class for application layer protocols, eg.
-:class:`HTTP/1.* <pcapkit.protocols.application.application.httpv1>`,
-:class:`HTTP/2 <pcapkit.protocols.application.application.httpv2>`
+:class:`HTTP/1.* <pcapkit.protocols.application.httpv1.HTTP>`,
+:class:`HTTP/2 <pcapkit.protocols.application.httpv2.HTTP>`
 and etc.
 
 .. autoclass:: pcapkit.protocols.application.application.Application

docs/source/pcapkit/protocols/application/http.rst

@@ -7,8 +7,8 @@ HTTP - Hypertext Transfer Protocol
 :class:`~pcapkit.protocols.application.http.HTTP`
 only, which is a base class for Hypertext Transfer
 Protocol (HTTP) [*]_ family, eg.
-:class:`HTTP/1.* <pcapkit.protocols.application.application.httpv1>`
-and :class:`HTTP/2 <pcapkit.protocols.application.application.httpv2>`.
+:class:`HTTP/1.* <pcapkit.protocols.application.httpv1.HTTP>`
+and :class:`HTTP/2 <pcapkit.protocols.application.httpv2.HTTP>`.
 
 .. autoclass:: pcapkit.protocols.application.http.HTTP
    :no-members: