diff --git a/docs/source/conf.py b/docs/source/conf.py index 9b8978cd3b..a2c2be33b4 100644 --- a/docs/source/conf.py +++ b/docs/source/conf.py @@ -88,8 +88,7 @@ # # These are deliberately opt-in per site rather than an automatic ``#NNN`` rule, # because no digit-keyed pattern can tell a citation from a packet-diagram label: -# ``#\d{3}`` will miss four-digit numbers once the tracker reaches them -- there are -# none yet -- and already misses 17 two-digit ones, while ``#\d+`` catches the 45 +# ``#\d{3}`` misses every two- and four-digit number, while ``#\d+`` catches the 45 # one-digit RFC diagram labels across ``pcapkit/protocols/internet/hip.py`` (36), # ``pcapkit/protocols/transport/sctp.py`` (7) and # ``pcapkit/protocols/schema/internet/hip.py`` (2) -- ``DH GROUP ID #1``, @@ -98,8 +97,8 @@ # # The caption is ``#%s`` for both ``:issue:`` and ``:pr:`` so that converting a bare # citation or an explicit link changes the markup and not the rendered text. A -# citation written as an inline literal is the one exception: it rendered as monospace -# and now renders as a link in body font. The two roles exist separately because the +# citation written as an inline literal is the one exception: converting it turns +# monospace into a link in body font. The two roles exist separately because the # issue-versus-pull-request distinction is itself a documented convention (see # ``contributing/conventions/documentation.rst``), and a single role would flatten it # in the source even though GitHub redirects between ``/issues/NNN`` and ``/pull/NNN`` @@ -322,12 +321,11 @@ def bind_type_checking_names(app: 'Sphinx') -> None: annotations resolvable also makes a malformed one **fatal**. A quoted annotation with stray whitespace inside the quotes -- ``rank: ' int'``, the space that belongs after the colon typed one character late -- is meaningless - to a type checker and was previously harmless here, because these annotations - failed earlier with :exc:`NameError`, which - :func:`sphinx.util.typing.get_type_hints` catches. Now they resolve, and Python - 3.14's :mod:`annotationlib` rejects the leading space with a - :exc:`SyntaxError` that :func:`~sphinx.util.typing.get_type_hints` does *not* - catch, so it aborts the whole build. Two such annotations existed and both are + to a type checker, and is harmless while the annotation fails earlier with + :exc:`NameError`, which :func:`sphinx.util.typing.get_type_hints` catches. + Once it resolves, Python 3.14's :mod:`annotationlib` rejects the leading space + with a :exc:`SyntaxError` that :func:`~sphinx.util.typing.get_type_hints` does + *not* catch, so it aborts the whole build. Two such annotations existed and both are fixed at source; a third would stop the build rather than be worked around here, which is the right way round for a typo. diff --git a/docs/source/contributing/conventions/index.rst b/docs/source/contributing/conventions/index.rst index 20b33b1804..a6fd0cbe26 100644 --- a/docs/source/contributing/conventions/index.rst +++ b/docs/source/contributing/conventions/index.rst @@ -8,17 +8,17 @@ House Conventions automated contributor would otherwise have to rediscover by reading a closed issue thread. Each ruling names where it was settled. - :ref:`mint-criterion` and :ref:`registry-protocol` govern - :mod:`pcapkit.const`, which is where the settled questions have mostly - arisen. :ref:`sentinel-convention` governs :mod:`pcapkit.corekit`, - :ref:`extension-header-subclassing` a protocol class hierarchy, - :ref:`protocol-layer-placement` which subpackage a dissector belongs in, - :ref:`utilities-or-corekit` which of :mod:`pcapkit.utilities` and - :mod:`pcapkit.corekit` a support module belongs in, - :ref:`process` the repository rather than any of its code, + :ref:`mint-criterion` governs :mod:`pcapkit.const`, which is where the + settled questions have mostly arisen, :ref:`sentinel-convention` + :mod:`pcapkit.corekit`, and :ref:`registry-protocol` :mod:`pcapkit.const` + again. :ref:`extension-header-subclassing` governs a protocol class + hierarchy, :ref:`protocol-layer-placement` which subpackage a dissector + belongs in, :ref:`process` the repository rather than any of its code, :ref:`documentation` the prose itself -- on these pages and in the API - reference -- rather than any code at all, and :ref:`test-coverage` what the - test suite has to prove. + reference -- rather than any code at all, :ref:`test-coverage` what the + test suite has to prove, and :ref:`utilities-or-corekit` which of + :mod:`pcapkit.utilities` and :mod:`pcapkit.corekit` a support module belongs + in. **A ruling that stays in its thread is a ruling that gets rediscovered.** So when a question is answered in a way the code cannot express on its own -- a diff --git a/docs/source/contributing/conventions/process.rst b/docs/source/contributing/conventions/process.rst index f4c0902ad1..e3e2b1f8f8 100644 --- a/docs/source/contributing/conventions/process.rst +++ b/docs/source/contributing/conventions/process.rst @@ -42,7 +42,7 @@ once: they are on demand -- one extra at a time, ``pip install pypcapkit[Scapy]`` and so on. This holds however easy they are to install, and it is what the ruling changed: four of them used to be in ``all``. -* **``PyPCAP`` and ``PCAP_CT`` were already excluded for installability**, which the +* ``PyPCAP`` **and** ``PCAP_CT`` **were already excluded for installability**, which the ruling leaves untouched. ``pypcap`` is an sdist-only C extension needing a compiler and the libpcap development files; ``pcap-ct`` and ``libpcap`` are published only as pre-releases, and ``all`` should not be how somebody acquires a @@ -317,7 +317,7 @@ carrying it are that kind: of the numbering, with nothing labelled at all between them and that change. Three of those seven are distribution rollups, each also carrying ``release``; the other four are early ``refactor``/``feat`` work from before the project stabilised. On - issues it is sparser still, appearing only from :issue:`775` up. So ``breaking``'s + issues it is sparser still, appearing only from :issue:`719` up. So ``breaking``'s absence on an old pull request is weak evidence at best. The current figures, rather than these: diff --git a/docs/source/contributing/conventions/protocol-layer-placement.rst b/docs/source/contributing/conventions/protocol-layer-placement.rst index 5963fd27af..167c0dea08 100644 --- a/docs/source/contributing/conventions/protocol-layer-placement.rst +++ b/docs/source/contributing/conventions/protocol-layer-placement.rst @@ -184,14 +184,17 @@ protocol number is refused. **Swapping** :class:`~pcapkit.protocols.link.link.Link` **for** :class:`~pcapkit.protocols.application.application.Application` **is not name-for-name.** -Counting the names each class body sets, whether new or overriding -:class:`~pcapkit.protocols.protocol.ProtocolBase`'s, ``Link`` has -seven -- ``__data__``, ``__layer__``, ``__proto__``, ``__schema__``, -``_read_protos``, ``register`` and ``layer`` -- and ``Application`` eight: the four -they share (``__data__``, ``__layer__``, ``__schema__``, ``layer``) plus -``__index__``, ``__post_init__``, ``_decode_next_layer`` and ``_import_next_layer``. -``Link`` overrides three of those that ``Application`` does not -- ``__proto__``, -``register`` and ``_read_protos`` -- but all three also exist on ``ProtocolBase``, so +Counting the names in each class's own namespace, whether new or overriding +:class:`~pcapkit.protocols.protocol.ProtocolBase`'s, and including those +``ProtocolBase.__init_subclass__`` sets there, ``Link`` has +ten -- ``__data__``, ``__layer__``, ``__post_init__``, ``__proto__``, +``__schema__``, ``_read_protos``, ``_shipped_proto``, ``from_data``, ``register`` and +``layer`` -- and ``Application`` nine: the six they share (``__data__``, ``__layer__``, +``__post_init__``, ``__schema__``, ``from_data``, ``layer``) plus ``__index__``, +``_decode_next_layer`` and ``_import_next_layer``. +``Link`` overrides four of those that ``Application`` does not -- ``__proto__``, +``register``, ``_read_protos`` and ``_shipped_proto``, the last set because ``Link`` +defines its own ``__proto__`` -- but all four also exist on ``ProtocolBase``, so the strict difference ``Link`` minus ``Application`` minus ``ProtocolBase`` is **empty** and every one of them still resolves on ``OSPF``. What changes is which registry it resolves *to*: ``OSPF.__proto__`` is ``ProtocolBase.__proto__`` rather than diff --git a/docs/source/contributing/conventions/registry-protocol.rst b/docs/source/contributing/conventions/registry-protocol.rst index ba56f4735b..e35639a173 100644 --- a/docs/source/contributing/conventions/registry-protocol.rst +++ b/docs/source/contributing/conventions/registry-protocol.rst @@ -199,8 +199,8 @@ instance nor the type:: TypeError: super(type, obj): obj must be an instance or subtype of type Both forms share ``instance or subtype of type``, which is the only part of the -message anything here relies on -- pinning either full sentence would pass on two of -the five supported versions and fail on the other three. +message anything here relies on -- pinning either full sentence would fail on two or +three of the five supported versions. Two corollaries worth stating, because the obvious summary of this is wrong in both directions. ``RuntimeError: super(): no arguments`` is a *different* failure, raised diff --git a/docs/source/contributing/conventions/test-coverage.rst b/docs/source/contributing/conventions/test-coverage.rst index 49085b5596..ee4d2e55d3 100644 --- a/docs/source/contributing/conventions/test-coverage.rst +++ b/docs/source/contributing/conventions/test-coverage.rst @@ -7,8 +7,10 @@ The pages before this one rule on the library, the repository and the prose. Thi rules on what the test suite has to prove, and what a change has to bring to it. The owner set the direction on :issue:`1203`, paraphrased here as on the other pages: every round trip the library offers must pass and match exactly, the tests must cover all of -them, and the rules belong on a page rather than in review threads. :issue:`1202` tracks -extending round-trip coverage to the paths that do not have it yet. +them, and the rules belong on a page rather than in review threads. :issue:`1202` +extended round-trip coverage past the registries, to the :mod:`pcapkit.corekit` fields, +schemas and ``Info``, the :mod:`pcapkit.foundation` reassembly, flow tracing, engines and +registries, and the output formats. Round-Trip Invariants ~~~~~~~~~~~~~~~~~~~~~ @@ -42,18 +44,16 @@ The rules those tables carry: * **Every registered code has a case.** ``test_every_registered_code_has_a_case`` fails otherwise. The option suite's ``SKIP`` table is the only exit, and each of its - five entries states why a case would assert nothing: TCP's ``End_of_Option_List`` - and ``No_Operation`` are padding that ``_make_tcp_options`` drops, ``Multipath_TCP`` - is the envelope of the ``tcp-mptcp`` family and would duplicate one of its subtypes, - and ``PadN`` under HOPOPT and IPv6-Opts emits the same option-less header as - ``Pad1``. A code that fails is never skipped. + three entries states why a case would assert nothing: ``Multipath_TCP`` is the + envelope of the ``tcp-mptcp`` family and would duplicate one of its subtypes, and + ``PadN`` under HOPOPT and IPv6-Opts emits the same option-less header as ``Pad1``. + A code that fails is never skipped. * **A failing case is an entry, never a changed argument.** An ``EXPECTED_FAILURES`` entry records the status, a failure-detail fragment narrow enough to match one site, and the defect: the ``file:line`` that causes it and the issue that tracks it. Routing a case around a defect hides it: the generator's ``HIP_COPIES`` once put two parameters in every HIP packet, so the unrepresentable length of a lone one - (:issue:`651`) was never constructed. Entries older than this page name the - ``file:line`` alone, and :issue:`1202` is where they gain their issue. + (:issue:`651`) was never constructed. ``KNOWN_DEGRADED`` entries have a different shape -- a fragment and a reason -- and its two record why the probe's placeholder payload cannot reach the target, not a defect to fix, so the ``file:line``-and-issue rule does not apply to them. @@ -180,5 +180,5 @@ above: - Unit tests under :file:`tests/dumpkit/` asserting the output exactly, and a case in :file:`tests/integration/test_output_formats.py` for the file that lands on disk. That a format :mod:`pcapkit` also reads, such as PCAP, reproduces its - input byte for byte is the target :issue:`1202` tracks; nothing asserts it - yet. + input byte for byte is asserted over every sample capture by + :file:`tests/dumpkit/test_capture_dump_runtime.py` (:issue:`1202`). diff --git a/docs/source/contributing/pep.rst b/docs/source/contributing/pep.rst index a0787ee64c..a3c0f5af28 100644 --- a/docs/source/contributing/pep.rst +++ b/docs/source/contributing/pep.rst @@ -225,7 +225,7 @@ DTLS **Not started**, and it has no stub in the tree -- of the TLS/SSL and DTLS pair, only TLS/SSL does. It earns its own entry because the registries -have moved ahead of it, and a growing part of SCTP's surface now names a protocol +have moved ahead of it, and a growing part of SCTP's surface names a protocol that does not exist: * the DTLS chunk, type 65 of :class:`~pcapkit.const.sctp.chunk.Chunk`, and the @@ -262,10 +262,11 @@ small part of each is bound to a dissector; everything else resolves to :class:`~pcapkit.protocols.misc.raw.Raw`, so the capture parses without complaint and yields nothing useful. -* **3 of the 219** :class:`~pcapkit.const.reg.linktype.LinkType` values -- - ``ETHERNET``, ``IPV4`` and ``IPV6``, declared identically in - :class:`~pcapkit.protocols.misc.pcap.frame.Frame` and - :class:`~pcapkit.protocols.misc.pcapng.PCAPNG`. +* **5 of the 219** :class:`~pcapkit.const.reg.linktype.LinkType` values -- + ``NULL``, ``ETHERNET``, ``LOOP``, ``IPV4`` and ``IPV6``, declared identically + in :class:`~pcapkit.protocols.misc.pcap.frame.Frame` and + :class:`~pcapkit.protocols.misc.pcapng.PCAPNG`, which also read ``RAW`` as + ``IPV4`` or ``IPV6`` by its version nibble. * **17 of the 151** :class:`~pcapkit.const.reg.transtype.TransType` values, in :attr:`Internet.__proto__ `. @@ -318,9 +319,11 @@ A handful wanted only a table entry, because the dissector was already there: :class:`~pcapkit.protocols.link.vlan.VLAN`, which carries the shared tag layout; the split exists so that a Q-in-Q frame's two tags stay distinct in the parsed output. -* ``LinkType`` ``NULL``, ``LOOP`` and ``RAW`` carry bare IPv4 or IPv6, both of - which pcapkit dissects. ``NULL`` and ``LOOP`` need their four-octet address - family word skipped first, and ``RAW`` needs a version sniff. +* **Done.** ``LinkType`` ``NULL`` and ``LOOP`` are bound to + :class:`~pcapkit.protocols.link.loopback.Loopback`, which reads the four-octet + address family word and hands 2 to IPv4 and 24, 28 or 30 to IPv6 + (:issue:`1574`). ``RAW`` has no binding of its own: its bare IPv4 or IPv6 is + dispatched by the version nibble of the first octet (:issue:`1502`). Three follow-ups the above deliberately left alone: @@ -634,8 +637,7 @@ Anyone wanting to run the tests wants the repository, which is where they are. exact value stays in the sibling field every timestamp already has, such as ``time_epoch``. ``PlistRoundTripTests.test_plist_report_round_trips_through_plistlib``, in - ``tests/integration/test_output_formats.py``, now runs rather than being - skipped. + ``tests/integration/test_output_formats.py``, checks the round trip. Reassembly Beyond IP and TCP ---------------------------- @@ -734,8 +736,9 @@ Two smaller items in the same subsystem: the tracer hands each traced segment straight to it. :attr:`Index.packet ` then - holds one reassembled datagram per direction, and postpones twice: reading it is - what flushes the flow's reassembler, and each datagram's own + holds the reassembled datagrams, one per direction and acknowledgement number + (:class:`~pcapkit.foundation.traceflow.tcp.TCP` says why), and postpones twice: + reading it is what flushes the flow's reassembler, and each datagram's own :attr:`~pcapkit.foundation.reassembly.data.tcp.Datagram.packet` is parsed later still, through the same :class:`~pcapkit.foundation.reassembly.data.data.Deferred` arrangement the @@ -786,7 +789,7 @@ Two smaller items in the same subsystem: the offset-zero fragment has been received; a parser sends nothing, so the condition is only reported in the log record. And the memory motive is narrowed rather than removed -- an in-flight IP datagram identifier still - holds a fixed 72 KiB of preallocated space, but now for at most the timeout's + holds a fixed 72 KiB of preallocated space, but for at most the timeout's worth of capture time rather than for the life of the :class:`~pcapkit.foundation.extraction.Extractor`. @@ -888,13 +891,13 @@ Release Plan — 1.5.0 in Two Steps :data:`pcapkit.__version__` is the current version, and the release is sequenced against the waves above in two deliberate steps: -#. **A beta — ``1.5.0b1`` — when wave 1's remaining issues are closed.** +#. **A beta —** ``1.5.0b1`` **— when wave 1's remaining issues are closed.** *Done.* Wave 1's feature work had landed already; what remained was the defect tail in `the issue tracker `__, and closing it earned a beta rather than a final release, because the consistency sweep below had not run yet and was expected to find things. -#. **The official ``1.5.0`` when the post-wave-1 consistency sweep is done.** The +#. **The official** ``1.5.0`` **when the post-wave-1 consistency sweep is done.** The sweep is described under `Delivery Sequence`_ below — prose against code, missing tests, unaligned changes, and packet formats against the specifications. Its whole point is to find what one-at-a-time defect work @@ -973,7 +976,7 @@ the next push then tags and publishes. So: - marked prerelease * - ``1.5.0`` (step 2) - **no** - - **``main``** + - ``main`` - full release Both steps publish to PyPI — the ``pypi`` job carries no prerelease gate — but a @@ -1014,21 +1017,17 @@ pair alone would answer it twice. ``LINUX_SLL`` matters out of proportion to its size: every ``tcpdump -i any`` capture uses it, and neither variant has even a stub. -**Wave 2 or 3 — handlers for the** ``DLT_NULL`` **and** ``DLT_RAW`` **link -types.** Both are genuine DLTs that a capture can legitimately be rooted in — -``DLT_NULL`` (0) is BSD's loopback encapsulation, ``DLT_RAW`` (101) an IP packet -with no link header at all — and neither has a handler protocol class. The frame -dispatcher registers exactly three link types -(:file:`pcapkit/protocols/misc/pcap/frame.py`, ``__proto__``): -:attr:`~pcapkit.const.reg.linktype.LinkType.ETHERNET`, -:attr:`~pcapkit.const.reg.linktype.LinkType.IPV4` and -:attr:`~pcapkit.const.reg.linktype.LinkType.IPV6`, so a ``DLT_NULL`` or -``DLT_RAW`` capture reaches no dissector. This is a sibling of the ``LINUX_SLL`` -work above rather than a separate concern — all four are link types the library -enumerates but cannot parse. It surfaced while deciding what an unresolvable -link-layer name should do: neither value is an honest stand-in for "unknown link -type", which is why the toolkit raises instead of defaulting to -``LinkType.NULL``. +**Done ahead of wave 2 — handlers for the** ``DLT_NULL`` **and** ``DLT_RAW`` +**link types.** Both are genuine DLTs that a capture can legitimately be rooted +in — ``DLT_NULL`` (0) is BSD's loopback encapsulation, ``DLT_RAW`` (101) an IP +packet with no link header at all. +:class:`~pcapkit.protocols.link.loopback.Loopback` dissects ``DLT_NULL`` and +``DLT_LOOP`` (:issue:`1574`), and the frame dispatcher reads ``DLT_RAW`` as IPv4 +or IPv6 by its version nibble (:issue:`1502`), so of the link types this +section names only ``LINUX_SLL`` and ``LINUX_SLL2`` remain enumerated but +unparsed. Neither ``NULL`` nor ``RAW`` is an honest stand-in for "unknown link +type", which is why the toolkit raises for an unresolvable link-layer name +instead of defaulting to ``LinkType.NULL``. **Wave 3 — the remaining protocols** from the same list, taken three or four at a time. QUIC, DSL, FDDI and ISDN come last: QUIC because it is really HTTP/3 over a @@ -1080,14 +1079,14 @@ written down instead of a pair of figures that will be stale by the next merge: spec.loader.exec_module(m); \ print(collections.Counter(k.split('/')[0] for k in m.EXPECTED_FAILURES))" -Two things to be precise about before anyone starts. First, **"does not -round-trip" is not the same as "unimplemented"** — the entries are a mix, and -some are an implemented case carrying a tracked defect rather than a missing -implementation. Each entry names the defect behind it, so the list can be split -into *missing*, *defective* and *deliberately deferred*, which is the first task, -because the three want different work. Second, ``pcapng-option`` is the largest -family by a wide margin, so it is the item that decides whether -this is one wave's work or several; it deserves sizing before the rest. +Two things to be precise about before anyone starts. First, the round-trip half +is closed: ``EXPECTED_FAILURES`` is empty, so every case the harness enumerates +round-trips, and a new entry there would record a defect rather than a +deferral. Second, **round-tripping is not the same as being implemented** — the +harness walks the codes each protocol registers a handler for, not every code +its enumeration names, so what remains is the codes with no per-entry handler, +such as the 17 SCTP chunk types under `SCTP`_ above, which fall through to the +generic handlers and parse as opaque. This is deliberately **not** the same request as the ``__proto__`` dispatch registries. Those name whole protocols, so closing a gap there means diff --git a/docs/source/contributing/releasing.rst b/docs/source/contributing/releasing.rst index d6d0bb42bb..86505ba8ad 100644 --- a/docs/source/contributing/releasing.rst +++ b/docs/source/contributing/releasing.rst @@ -176,7 +176,7 @@ the default branch, and ``tag`` pulled ``main`` before tagging. again (``--clobber``). ``conda``'s upload is not gated on ``check_leg``, so a leg whose Anaconda upload landed but whose GitHub upload failed still gets its asset on a re-run. -* Two failures are new, and deliberate. With no release for the tag, the +* Two failures differ from the action, deliberately. With no release for the tag, the upload fails with ``gh``'s "release not found", where the action made one, with no notes and ``target_commitish`` unset (``src/github.ts:617-628,1026``), which Create a release defaults to "the repository's default branch"; the @@ -333,7 +333,7 @@ checks *is* the one it is about to create: exists. * ``pypi`` compares PyPI's file count for the version against the 7 expected -- the 6 supported legs' wheels plus the sdist kept only on the 3.15 leg; the - KTLO legs' wheels (#1633) are not counted. This job is + KTLO legs' wheels (`#1633 `__) are not counted. This job is already idempotent (``skip-existing: true`` on both upload steps), so the check exists only to save the cost of the build matrix, not for correctness. * ``conda`` compares Anaconda's aggregate count against the 10 expected (2 OS @@ -357,7 +357,7 @@ supported version added since (`#1660 release, and counting it would read a finished release as incomplete, run the gate and ``pypi``, and fail `One Commit per Release`_'s tag check on every run until the next release. A tag older than the file, as ``v1.5.0b12`` is, counts -the tiers the file was added with, which #1677 copied unchanged from the +the tiers the file was added with, which the change for `#1660 `__ copied unchanged from the workflows of the time. On the tag-push path ``startsWith(github.ref_name, 'v')`` bypasses the @@ -449,7 +449,7 @@ graph: ``tag`` and ``pypi`` both depend on ``github`` (``.github/workflows/create-release.yml:806,972``), and ``conda`` depends on ``tag`` and ``github`` (``:1208``) -- so nothing downstream of ``github`` can start before it is approved, and rejecting it leaves nothing tagged and -nothing published. Before this change ``tag`` depended on ``version_check`` +nothing published. Before that removal ``tag`` depended on ``version_check`` alone, so removing its reviewer without moving this dependency would have let it push a commit to ``main`` and cut a ``conda-*`` tag *ahead of* the approval it was supposed to wait for. @@ -459,7 +459,7 @@ Precautions .. warning:: - **Do not gate a release job on whether the ``v*`` tag exists.** ``github`` + **Do not gate a release job on whether the** ``v*`` **tag exists.** ``github`` creates the tag before ``pypi`` and ``conda`` upload, so a tag proves nothing about whether the upload finished. A job keyed on ``PCAPKIT_TAG_EXISTS`` skips itself on a retry after a partial release, and the run finishes green @@ -492,7 +492,7 @@ Neither is worth the risk of a double upload to an index that cannot take one back, and neither is needed, since a plain re-run self-heals; see `Recovery`_ below. -**``environment: pypi`` stays even though its reviewer is gone.** ``pypi`` +``environment: pypi`` **stays even though its reviewer is gone.** ``pypi`` publishes through PyPI's OIDC trusted publishing (``environment: pypi`` at ``:967``, ``id-token: write`` at ``:971``), and PyPI's trusted-publisher configuration can be scoped to a GitHub Actions environment name. If this @@ -502,7 +502,7 @@ this is standard OIDC trusted-publishing behaviour, not something the publisher's actual PyPI-side configuration was read to confirm here. Keep the name regardless, since there is no upside to removing it. -**``conda-tag`` writes to ``main``, so a release is not read-only on the +``conda-tag`` **writes to** ``main``\ **, so a release is not read-only on the branch.** After cutting ``conda-+0`` at the run's commit, the ``tag`` job pushes a commit setting :file:`conda/build` to ``0`` on top of ``main`` (``.github/workflows/create-release.yml:893-946``) -- a plain @@ -512,7 +512,7 @@ when ``main``'s tip already says ``0`` (the usual case, since :file:`util/bump_version.py` writes ``0``) or carries another ``__version__``, nor when origin already has a ``conda-+N`` tag other than ``+0``, since rewinding the counter would then reuse a tagged build number; the first -and last warn. #1634's own shape -- three Conda Update bumps that `#1632 +and last warn. `#1634 `__'s own shape -- three Conda Update bumps that `#1632 `__ left untagged -- gets the reset: ``main``'s 3 becomes 0. A rejected push is retried from a fresh fetch, three times at most. Approving ``github-release`` therefore also @@ -535,10 +535,10 @@ existence. ``pypi`` is safe to re-run (``skip-existing: true``), and so is ``conda``, because each matrix leg checks Anaconda for its own platform/Python distribution before uploading and skips only that leg's upload if it is already there. Neither replaces a file the GitHub Release -already has (#1640). +already has (`#1640 `__). **A re-run does not need to be verified by hand against the PyPI and Anaconda -listings, because ``release_status`` does it.** That job runs on every +listings, because** ``release_status`` **does it.** That job runs on every ``Create Release`` attempt regardless of what else skipped, and reconciles each of ``github``/``tag``/``pypi``/``conda`` *against its own evidence* rather than demanding the same outcome from all four -- a target counts as diff --git a/docs/source/contributing/testing.rst b/docs/source/contributing/testing.rst index 4f82319933..145f3085fb 100644 --- a/docs/source/contributing/testing.rst +++ b/docs/source/contributing/testing.rst @@ -29,8 +29,10 @@ the whole suite: The same fixtures back the demonstration scripts in :file:`examples/legacy_smoke/`, which read them as ``../captures/…``. -Continuous integration runs the ``make test`` selection, since the fixtures are -not in the repository. Wireshark's ``tshark`` is only required to exercise the +Continuous integration runs both: :file:`.github/workflows/unit-tests.yml` runs +the ``make test`` selection in its ``test`` job, and the fixture-dependent tests +in its ``integration`` job, which runs :file:`examples/generators/make_samples.py` +first. Wireshark's ``tshark`` is only required to exercise the :class:`PyShark ` engine, and is not needed by the test suite. @@ -47,5 +49,5 @@ they record what this version of :mod:`pcapkit` builds rather than what a third-party tool builds. :file:`tests/protocols/test_option_roundtrip_unit.py` runs the same cases without -needing a fixture at all, and carries a table of the ones that do not close the -cycle, each named against the defect that stops it. +needing a fixture at all, and fails on any case that does not close the cycle +unless its ``EXPECTED_FAILURES`` table names the defect that stops it. diff --git a/docs/source/contributing/workflows.rst b/docs/source/contributing/workflows.rst index 42b60deb2a..90d90ec3c9 100644 --- a/docs/source/contributing/workflows.rst +++ b/docs/source/contributing/workflows.rst @@ -412,9 +412,9 @@ its own ``github.run_id`` rather than on the ref, so it cannot cancel a run waiting there with release work. Keyed on the ref, run 38499295798 joined at 02:06:32Z on 2026-10-14, and the pending 38499150441, whose Vendor Update had succeeded, was cancelled a second later. A run whose Vendor Update concluded -``success`` with ``vendor-update`` skipped (#1748) is still keyed on the ref: +``success`` with ``vendor-update`` skipped (`#1748 `__) is still keyed on the ref: the group reads only the event, which does not say which jobs ran. It can still -displace a run pending there, and now releases nothing in its place, so that +displace a run pending there, and releases nothing in its place, so that release waits for the next Vendor Update on ``main``. ``pull_request_target`` @@ -470,7 +470,7 @@ CI Timing Review (`#1538 `__ **CI Timing** (:file:`ci-timing.yml`) runs :file:`util/ci_timing.py` every Monday, or on dispatch with a ``days`` window and an optional ``concurrency_hours`` runner-pool measurement. Through the API it lists every completed run in the window of Unit Tests, -Python Compatibility, Lint, CodeQL, GitHub Pages and Experimental Tests (#1732, so +Python Compatibility, Lint, CodeQL, GitHub Pages and Experimental Tests (`#1732 `__, so its two-hour legs stay measured), a day per query because a filtered listing stops at 1,000 runs (Pages completed 1,154 in the week to 2026-10-09). It then reads the jobs of an evenly spaced sample -- 150 push and 150 pull-request runs of Unit Tests, @@ -560,21 +560,20 @@ The cap line sits at 50% because at the fastest growth measured, +1.27 min/day o fixing pull request. Time to ``Required checks passed`` is HARD only as a regression because its p90 measures runner-queue load as much as test time: it was 19.6 min when the line was set and 36.7 min in the two days to 2026-10-10 with the pool saturated, -so an absolute HARD line would email about the queue every week. The 25% rise is an -initial value, to tune once weekly reports exist. ``W9`` is therefore expected to fire -while the pool stays saturated, and the first runs with JUnit will show whether +so an absolute HARD line would email about the queue every week. The 25% rise is a +starting value, to be tuned against the weekly reports. ``W9`` is therefore expected to +fire while the pool stays saturated, and the JUnit timings are what show whether ``W7``'s 10-second test line needs raising. What to do when one fires is on :doc:`conventions/process`. ``H1``, ``H2`` and ``H4`` on a non-blocking job are warnings, not HARD, so they do -not fail the run (#1732): on every job of :file:`experimental.yml`, which no +not fail the run (`#1732 `__): on every job of :file:`experimental.yml`, which no required context names, on every KTLO leg, a job whose name ends in ``(KTLO)`` -(#1633), and on a job whose name ends in ``(experimental)`` in any workflow. That +(`#1633 `__), and on a job whose name ends in ``(experimental)`` in any workflow. That last covers the experimental legs that ran in :file:`unit-tests.yml` under -``test``'s 45-minute cap before the split, whose runs stay in the 7-day window for a -week after it. The finding keeps its ``H`` id and says it is non-blocking. A +``test``'s 45-minute cap before the split. The finding keeps its ``H`` id and says it is non-blocking. A KTLO leg past its job's ``timeout-minutes`` is still cancelled, and its check run -reads red, though since #1738 nothing gating waits on it, so that warning is worth +reads red, though since `#1738 `__ nothing gating waits on it, so that warning is worth acting on before the cap. Work Kept Off the Pull-Request Queue (`#1538 `__) @@ -599,7 +598,7 @@ permission to the reusable workflow. A third cut, running the then-prerelease 3.15 engine cell on ``main`` pushes only, ended with `#1633 `__: 3.15 is -supported now, so its cells block on every run. +in the supported tier, so its cells block on every run. Required Status Checks ---------------------- @@ -665,12 +664,12 @@ triggers; the ``gate-only: true`` reusable calls documented above skip it via ``if: ${{ always() && inputs.gate-only != true }}`` (``unit-tests.yml:1939``), so it is never produced -- and never expected -- on those paths. -Python versions come in two tiers (#1633), and an experimental one (#1660, +Python versions come in two tiers (`#1633 `__), and an experimental one (`#1660 `__, below), declared in :file:`.github/python-versions.toml` (see `Python Versions`_). The **supported** tier, 3.11-3.15, blocks: a failed leg fails ``Required checks passed`` or its own required ``Compat`` context. The **KTLO** tier -- end-of-life 3.7-3.10, still built and tested -- never blocks, is never a required context, and never shows a commit or a -pull request red (#1672). In ``unit-tests.yml`` the KTLO legs are jobs of their own +pull request red (`#1672 `__). In ``unit-tests.yml`` the KTLO legs are jobs of their own since `#1738 `__: ``test-ktlo``, ``integration-ktlo`` and ``pypcap-parity-ktlo``, which neither ``required-checks`` nor ``unittest-ordering`` ``needs:``, so not even a failure no step can contain, such @@ -704,7 +703,7 @@ with supported legs. Every job holding KTLO legs works the same way: Two errors remain on a failed KTLO step, neither changing a conclusion: the runner's ``Process completed with exit code N.`` for a ``run:`` step, and an -action's own. And #1634's tag check still blocks the ``pypi`` KTLO legs: it fails +action's own. And `#1634 `__'s tag check still blocks the ``pypi`` KTLO legs: it fails every leg alike, and masked, the leg would go on to publish. 3.7 can neither install the ``test`` extra nor import the package, so each 3.7 leg @@ -713,8 +712,8 @@ install step, the ``Compat`` leg at ``Verify package`` -- and concludes ``success``. They run on ``ubuntu-22.04``, which retires on 2027-04-17 after brownouts from 2027-03-23 (actions/runner-images#14254). A brownout fails a ``test-ktlo`` or ``integration-ktlo`` leg before any step runs, where no step-level -``continue-on-error`` reaches; since #1738 that fails nothing gating, but the leg's -check run reads red, so they must still go before then (#1660). 3.6 is +``continue-on-error`` reaches; since `#1738 `__ that fails nothing gating, but the leg's +check run reads red, so they must still go before then (`#1660 `__). 3.6 is in no matrix: setup-python builds it only for retired Ubuntu images. Nor are 3.7-3.9 in ``engine-tests``, which installs every interpreter on ``ubuntu-latest``, where setup-python has no Ubuntu 26.04 build of any of them. Nor @@ -843,7 +842,7 @@ Python Versions Since `#1660 `__, :file:`.github/python-versions.toml` declares every Python version CI runs: one line per version, with its tier -- ``supported``, ``ktlo``, ``experimental``, or -``pre-release``, which none is today -- and, where ``ubuntu-latest`` cannot +``pre-release``, which no version carries -- and, where ``ubuntu-latest`` cannot provide it, the ``runner`` it needs. ``latest`` names the newest supported CPython, which every job on one interpreter runs. :file:`util/python_versions.py` reads the file with the runner's own ``python3`` -- its ``tomllib`` needs 3.11, and every image that @@ -853,7 +852,7 @@ job in ``unit-tests.yml`` and ``python-compatibility.yml``, ``version_check`` in through ``fromJSON``, and each single-interpreter job reads ``latest``. **A tier change is a one-line edit to that file, plus the matching edit to ruleset -23497679's ``Compat`` contexts.** The file's ``exemptions`` list each pin kept +23497679's** ``Compat`` **contexts.** The file's ``exemptions`` list each pin kept off ``latest``, with its reason in the comment above it, such as the conda jobs' ``!= '3.15'``. ``tests/project/test_python_versions_1660.py`` fails on a pin the file does not list, on an exemption with no comment, and on a diff --git a/docs/source/ext.rst b/docs/source/ext.rst index 4e2c4a3480..c0dafcaa5a 100644 --- a/docs/source/ext.rst +++ b/docs/source/ext.rst @@ -265,11 +265,13 @@ The following code snippet shows how to create a new protocol class: subclass of a dispatcher is checked normally. Prefer a declared parameter to either: it is also what documents the keyword to your callers. - Parsing is unaffected, and so is :meth:`ProtocolBase.from_data - `, which warns - :exc:`~pcapkit.utilities.warnings.UnknownFieldWarning` instead -- its keywords - come from your ``_make_data``, so a mismatch there is a disagreement between - two of your own mappings rather than a caller's typo. + Parsing is unaffected. :meth:`ProtocolBase.from_data + ` checks the keywords its + caller passes just as strictly, raising + :exc:`~pcapkit.utilities.exceptions.UnsupportedCall`, but only warns + :exc:`~pcapkit.utilities.warnings.UnknownFieldWarning` for a key that comes + from your ``_make_data`` -- a mismatch there is a disagreement between two of + your own mappings rather than a caller's typo. .. warning:: diff --git a/docs/source/index.rst b/docs/source/index.rst index f4e102765c..5cce1e487e 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -167,7 +167,7 @@ missing :program:`tshark`, a missing :manpage:`libpcap(3)`, the wrong ``pcap`` distribution) and a clean fall back, rather than an error from inside the third-party package. -Engine support by Python version +Engine Support by Python Version -------------------------------- Which engines can run at all, by interpreter. Verified by installing each engine @@ -350,6 +350,13 @@ plug-in functions, you may want to install the optional ones: explicitly: ``pip install pypcapkit[PyPCAP]`` or ``pip install pypcapkit[PCAP_CT]``. +.. versionchanged:: 1.5.0 + + ``all`` installs the core addons only: ``emoji``, ``cryptography`` and + ``pycrate``. The engines and vendor-crawler dependencies it carried in 1.4 + -- ``dpkt``, ``scapy``, ``pyshark``, ``requests`` and ``beautifulsoup4`` -- + moved to the ``dev`` extra, alongside ``pypcapfile`` (:issue:`910`). + .. warning:: **Install only one of** ``pypcap`` **and** ``pcap-ct``. Both own the top-level @@ -361,7 +368,7 @@ plug-in functions, you may want to install the optional ones: and which one won, but it cannot undo it. -------------------- -Engine prerequisites +Engine Prerequisites -------------------- Four of the engines need something beyond a ``pip install``. Each constraint @@ -465,7 +472,7 @@ For CLI usage, you will need to install the optional packages: pip install pypcapkit emoji ================== -Indices and tables +Indices and Tables ================== * :ref:`genindex` diff --git a/docs/source/pcapkit/const/index.rst b/docs/source/pcapkit/const/index.rst index 0624d97dfc..262411126d 100644 --- a/docs/source/pcapkit/const/index.rst +++ b/docs/source/pcapkit/const/index.rst @@ -62,14 +62,6 @@ Protocol Numbers reg -Miscellaneous -------------- - -.. toctree:: - :maxdepth: 2 - - pcapng - Link Layer ---------- @@ -78,7 +70,6 @@ Link Layer arp l2tp - ospf vlan Internet Layer @@ -112,3 +103,12 @@ Application Layer ftp http ngap + ospf + +Miscellaneous +------------- + +.. toctree:: + :maxdepth: 2 + + pcapng diff --git a/docs/source/pcapkit/corekit/index.rst b/docs/source/pcapkit/corekit/index.rst index efb2a6f53d..56fcf1645d 100644 --- a/docs/source/pcapkit/corekit/index.rst +++ b/docs/source/pcapkit/corekit/index.rst @@ -4,17 +4,22 @@ Core Utilities .. module:: pcapkit.corekit :mod:`pcapkit.corekit` holds the core utilities :mod:`pcapkit` is built on: -the :obj:`dict`-like :class:`~pcapkit.corekit.infoclass.Info`, the -:obj:`tuple`-like :class:`~pcapkit.corekit.version.VersionInfo`, the protocol -collection :class:`~pcapkit.corekit.protochain.ProtoChain`, the -:class:`~pcapkit.corekit.multidict.MultiDict` family for multi-entry -mappings, the :class:`~pcapkit.corekit.fields.field.Field` family for data -parsing, the :class:`~pcapkit.corekit.context.ContextRegistry` channel for -caller-supplied information a protocol needs but the wire does not carry, the -:mod:`~pcapkit.corekit.decorators` that protocols and schemas parse through, and +the :class:`~pcapkit.corekit.fields.field.Field` family for data parsing, the +:class:`~pcapkit.corekit.context.ContextRegistry` channel for caller-supplied +information a protocol needs but the wire does not carry, the +:mod:`~pcapkit.corekit.decorators` that protocols and schemas parse through, the :class:`~pcapkit.corekit.enum.EnumLookup`/ :class:`~pcapkit.corekit.enum.EnumRegistry` bases every constant enumeration -inherits from. +inherits from, the :obj:`dict`-like :class:`~pcapkit.corekit.infoclass.Info`, +the :class:`~pcapkit.corekit.io.SeekableReader` and stream proxies of +:mod:`~pcapkit.corekit.io`, the +:class:`~pcapkit.corekit.module.ModuleDescriptor` naming a class by module, the +:class:`~pcapkit.corekit.multidict.MultiDict` family for multi-entry mappings, +the :class:`~pcapkit.corekit.packet.DeferredPacket` mixin for a ``packet`` +parsed on first read, the protocol collection +:class:`~pcapkit.corekit.protochain.ProtoChain`, the shared +:mod:`~pcapkit.corekit.sentinels`, and the :obj:`tuple`-like +:class:`~pcapkit.corekit.version.VersionInfo`. .. toctree:: :maxdepth: 2 diff --git a/docs/source/pcapkit/foundation/index.rst b/docs/source/pcapkit/foundation/index.rst index 9a67cd571f..fc1a32762e 100644 --- a/docs/source/pcapkit/foundation/index.rst +++ b/docs/source/pcapkit/foundation/index.rst @@ -5,9 +5,11 @@ Library Foundation :mod:`pcapkit.foundation` is a collection of foundations for :mod:`pcapkit`, including PCAP file extraction tool -:class:`~pcapkit.foundation.extraction.Extractor`, TCP flow tracer -:class:`~pcapkit.foundation.traceflow.traceflow.TraceFlow`, registry management -APIs for :mod:`pcapkit`, and TCP/IP reassembly implementations. +:class:`~pcapkit.foundation.extraction.Extractor` and the +:mod:`~pcapkit.foundation.engines` it reads through, TCP/IP reassembly +implementations, TCP flow tracer +:class:`~pcapkit.foundation.traceflow.traceflow.TraceFlow`, and registry +management APIs for :mod:`pcapkit`. .. toctree:: :maxdepth: 3 diff --git a/docs/source/pcapkit/index.rst b/docs/source/pcapkit/index.rst index 63291a133c..acfd295566 100644 --- a/docs/source/pcapkit/index.rst +++ b/docs/source/pcapkit/index.rst @@ -27,10 +27,10 @@ schema definitions as well as various customisable interfaces. interface/index foundation/index protocols/index + utilities/index corekit/index toolkit/index dumpkit/index - utilities/index const/index vendor/index diff --git a/docs/source/pcapkit/protocols/internet/index.rst b/docs/source/pcapkit/protocols/internet/index.rst index 017f87e814..1467239a82 100644 --- a/docs/source/pcapkit/protocols/internet/index.rst +++ b/docs/source/pcapkit/protocols/internet/index.rst @@ -21,11 +21,11 @@ internet layer, with detailed implementation and methods. ipv6_route hopopt ipv6_option + mh ipsec ah esp hip - mh ipx .. todo:: diff --git a/docs/source/pcapkit/protocols/misc/index.rst b/docs/source/pcapkit/protocols/misc/index.rst index 2ae424fe85..3620256414 100644 --- a/docs/source/pcapkit/protocols/misc/index.rst +++ b/docs/source/pcapkit/protocols/misc/index.rst @@ -6,10 +6,11 @@ Auxiliary Protocols .. module:: pcapkit.protocols.schema.misc :mod:`pcapkit.protocols.misc` contains the auxiliary protocol implementations. -Such includes the :class:`~pcapkit.protocols.misc.raw.Raw` class for not-supported -protocols, the :class:`~pcapkit.protocols.misc.null.NoPayload` class for -indication of empty payload, the PCAP header classes, and the PCAP-NG -:class:`~pcapkit.protocols.misc.pcapng.PCAPNG` class. +Such includes the PCAP header classes, the PCAP-NG +:class:`~pcapkit.protocols.misc.pcapng.PCAPNG` class, the +:class:`~pcapkit.protocols.misc.raw.Raw` class for not-supported protocols, and +the :class:`~pcapkit.protocols.misc.null.NoPayload` class for indication of +empty payload. .. toctree:: :maxdepth: 1 diff --git a/docs/source/pcapkit/utilities/index.rst b/docs/source/pcapkit/utilities/index.rst index 26290bd47d..6a3f059372 100644 --- a/docs/source/pcapkit/utilities/index.rst +++ b/docs/source/pcapkit/utilities/index.rst @@ -5,8 +5,9 @@ Module Utilities .. module:: pcapkit.utilities :mod:`pcapkit.utilities` holds the cross-cutting services :mod:`pcapkit` is -built with: user-refined exceptions and warnings, logging, a compatibility -layer, and wrappers around third-party libraries. It imports nothing from +built with: user-refined exceptions and warnings, logging, the tunable limits +of :mod:`~pcapkit.utilities.config`, a compatibility layer, and wrappers around +third-party libraries. It imports nothing from :mod:`pcapkit` outside itself; see :ref:`utilities-or-corekit`. .. toctree:: diff --git a/docs/source/pcapkit/vendor/index.rst b/docs/source/pcapkit/vendor/index.rst index c3dcdba44a..6d897a2292 100644 --- a/docs/source/pcapkit/vendor/index.rst +++ b/docs/source/pcapkit/vendor/index.rst @@ -24,14 +24,6 @@ Protocol Numbers reg -Miscellaneous -------------- - -.. toctree:: - :maxdepth: 2 - - pcapng - Link Layer ---------- @@ -40,7 +32,6 @@ Link Layer arp l2tp - ospf vlan Internet Layer @@ -74,6 +65,15 @@ Application Layer ftp http ngap + ospf + +Miscellaneous +------------- + +.. toctree:: + :maxdepth: 2 + + pcapng Command Line Tool =================