From 30317895a4844f1df8dfee834c43a7506fe489ba Mon Sep 17 00:00:00 2001 From: Jarry Shaw Date: Sun, 4 Oct 2026 18:06:46 -0400 Subject: [PATCH] docs: cut the root markdown documents to what a reader needs MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - `README.md`, `CONTRIBUTING.md` and `SECURITY.md` lose restatement, hedging and background that repeats the sentence above it. `CODE_OF_CONDUCT.md` is adapted Contributor Covenant 3.0 text and is left byte-identical. - `CONTRIBUTING.md` said `_make_data` is overridden in 28 protocol modules. The real count is 29. The figure is untested prose that rots silently, so it is dropped rather than replaced. - `CONTRIBUTING.md` called the `_missing_` population "the 121 enumerations". 121 is the registry count, not the enumeration count: 130 enumerations exist under `pcapkit.const`, 122 define `_missing_`, and 121 of the 127 `EnumRegistry` subclasses do. Reworded to "121 registries". - The same sentence said that fallback "resolves an unregistered value and registers it". It does not register, for all but one registry: `docs/source/contributing/conventions/mint-criterion.rst` records that exactly one `_missing_` in the tree mints, `CGAType`'s, while 114 unmint and six only range-check. The claim is dropped. - `README.md` said the module-structure page covers "the eight subpackages". There are nine with an `__init__.py`: const, corekit, dumpkit, foundation, interface, protocols, toolkit, utilities, vendor. - Drops timed context per #719 — "currently", "today", and the narrative about older commit prefixes. Version-bounded support notes and the version table stay. Issue and pull-request citations are untouched: the `#587`/`#597` multiset in `CONTRIBUTING.md` is byte-identical, and the other three files carry none. `tests/project`: 268 passed, 1 skipped, 864 subtests passed. Part of #719. --- CONTRIBUTING.md | 174 +++++++++++++++++++++++------------------------- README.md | 48 ++++++------- SECURITY.md | 66 +++++++++--------- 3 files changed, 134 insertions(+), 154 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index aac0279fbd..7572c4a5dc 100755 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,31 +1,29 @@ # How to contribute -This document outlines the conventions on development workflow, commit message formatting and the -few repository-specific rules that are easy to trip over — the test tiers and the generated -changelog especially. The README is the authority on installing and building; this file covers what -happens after that. +This document covers the development workflow, commit message format and the repository-specific +rules that are easy to trip over — the test tiers and the generated changelog especially. The README +is the authority on installing and building; this file covers what happens after that. ## Getting started - Fork the repository on GitHub. -- Read the README for installation and build instructions, and the *Testing* page it links from its - *Documentation* table — `docs/source/contributing/testing.rst` — for the test commands. -- Set up a development environment. `make setup` runs `pipenv install --skip-lock --dev`, and the - `Makefile` exports `PIPENV_VENV_IN_PROJECT=1`, so the environment lands in `.venv/` inside the - checkout. **Only that environment has the dependencies** — the `make` targets below all run - through `pipenv run`, and invoking `pytest` or `sphinx` from a system interpreter will fail on - missing imports rather than on anything you changed. -- Looking for something to pick up? `docs/source/contributing/pep.rst` — rendered as the *Help - Wanted* page — is the maintained list of open proposals, kept in step with the code. +- Read the README for installation and build instructions, and the *Testing* page it links + (`docs/source/contributing/testing.rst`) for the test commands. +- Set up a development environment with `make setup`, which runs `pipenv install --skip-lock --dev`. + The `Makefile` exports `PIPENV_VENV_IN_PROJECT=1`, so the environment lands in `.venv/` inside the + checkout. **Only that environment has the dependencies**: the `make` targets below run through + `pipenv run`, and `pytest` or `sphinx` from a system interpreter fails on missing imports rather + than on anything you changed. +- Looking for something to pick up? `docs/source/contributing/pep.rst`, rendered as the *Help Wanted* + page, is the maintained list of open proposals. - Play with the project, submit bugs, submit patches! ## Contribution flow -This is a rough outline of what a contributor's workflow looks like: +A rough outline: -- Create a topic branch from where you want to base your work. This is usually `main`. -- Make commits of logical units, and add a test case if the change fixes a bug or adds new - functionality. +- Create a topic branch, usually from `main`. +- Make commits of logical units, and add a test case if the change fixes a bug or adds functionality. - Run the tests and make sure they pass (see below). - Add a changelog entry if the change is user-visible (see below). - Make sure your commit messages are in the proper format (see below). @@ -36,47 +34,45 @@ Thanks for your contributions! ## Running the tests -The suite runs in two tiers, and the tier is a property of a module's *path* rather than of the -command that collects it. `tests/_tiers.py` is the rule, and it documents itself at length. +The suite runs in two tiers. The tier is a property of a module's *path*, not of the command that +collects it; `tests/_tiers.py` is the rule. ```shell make test # the unit tier -- the selection CI runs make test-all # everything, regenerating the sample captures first ``` -`make test` is the selection in `.github/workflows/unit-tests.yml`: everything under `tests/` except -`tests/integration/` and the `*_runtime.py` / `*_regression.py` modules. It has to pass on a fresh -clone with nothing but the package and its test extra installed. +`make test` is the selection `.github/workflows/unit-tests.yml` runs: everything under `tests/` except +`tests/integration/` and the `*_runtime.py` / `*_regression.py` modules. It must pass on a fresh +clone with only the package and its test extra installed. The fixture-dependent tier reads sample captures under `examples/captures/`, most of which are -**not** tracked (see `.gitignore`); only a handful are committed. `make samples` rebuilds the rest, -and `make test-all` does that for you. This is the trap worth knowing -about: a unit-tier module that reads a *generated* capture passes on your machine, because you have -run `make samples`, and then fails on a fresh CI checkout with a missing-file error that blames the +**not** tracked (see `.gitignore`). `make samples` rebuilds them, and `make test-all` does that first. +The trap: a unit-tier module that reads a *generated* capture passes on your machine, where you have +run `make samples`, then fails on a fresh CI checkout with a missing-file error that blames the fixture rather than the tier rule. `tests/conftest.py` and `tests/test_tier_guard.py` catch most -shapes of it at collection time and tell you what to do; read `tests/_tiers.py` if the message is -not enough. +shapes of it at collection time and say what to do; `tests/_tiers.py` has the rest. ## Changelog entries Entries live in `docs/source/changelog/.rst`, one file per version, listed newest-first in the toctree of `docs/source/changelog.rst`. That tree is the single source of the project's history. -**`CHANGELOG.md` is generated — never edit it by hand.** It is a derivative of the newest entry -alone, produced by `util/changelog_md.py`, because its two consumers (the `Create Release` -workflow's release body and the source distribution) read Markdown rather than reStructuredText. -Add your bullet to the `.rst` entry, then regenerate: +**`CHANGELOG.md` is generated — never edit it by hand.** `util/changelog_md.py` derives it from the +newest entry alone, because its two consumers (the `Create Release` workflow's release body and the +source distribution) read Markdown rather than reStructuredText. Add your bullet to the `.rst` +entry, then regenerate: ```shell python util/changelog_md.py # rewrite CHANGELOG.md from the newest entry python util/changelog_md.py --check # exits 0 when they agree, prints a diff when they do not ``` -The generator needs only the standard library, so it runs against a bare interpreter. The -`Changelog drift` job in `.github/workflows/unit-tests.yml` runs `--check` on every push to `main` -and every pull request targeting it, and again whenever another workflow calls that one as a gate — -the vendor and conda updates, the pages deploy and the release all do, and the job carries no -`gate-only` guard to opt out. A hand-edited `CHANGELOG.md` will fail it. +The generator needs only the standard library. The `Changelog drift` job in +`.github/workflows/unit-tests.yml` runs `--check` on every push to `main`, every pull request +targeting it, and whenever another workflow calls that one as a gate (the vendor and conda updates, +the pages deploy and the release), with no `gate-only` guard to opt out. A hand-edited +`CHANGELOG.md` fails it. ## Documentation @@ -84,16 +80,15 @@ Documentation is reStructuredText under `docs/source/`, built with `make docs`. `pcapkit/` are reStructuredText too, and they are what the API reference renders from. The Markdown files at the repository root — this one, `CODE_OF_CONDUCT.md`, `SECURITY.md`, -`CHANGELOG.md` and the README — are a deliberate exception to that rule, because their consumers are -GitHub's own rendering and the release body rather than Sphinx. The issue and pull-request templates -under `.github/` are Markdown for the same reason. The exception ends there: anything added under -`docs/source/` is `.rst`. +`CHANGELOG.md` and the README — are a deliberate exception, because their consumers are GitHub's +rendering and the release body rather than Sphinx. The issue and pull-request templates under +`.github/` are Markdown for the same reason. Anything added under `docs/source/` is `.rst`. ### What belongs in the API reference: document the contract, hide the recipe -The tenet is to **keep the usage and extensibility clear and straightforward, while hiding the -recipe**. A member earns an autodoc directive because a reader needs it, not because of its spelling — -so a leading underscore is not by itself a reason to leave something out, and being public is not by +The tenet: **keep the usage and extensibility clear and straightforward, while hiding the recipe**. +A member earns an autodoc directive because a reader needs it, not because of its spelling — a +leading underscore is not by itself a reason to leave something out, and being public is not by itself a reason to put it in. **Contract — document it.** Anything a *caller* needs in order to use a class, and anything an @@ -101,45 +96,43 @@ itself a reason to put it in. - **Per-option and per-parameter `_read_*` / `_make_*` pairs.** These publish the data format: the keyword arguments a caller passes to construct that option, and the fields they get back when - parsing it. There is nowhere else to look it up, so they stay documented even though the dispatch + parsing it. There is nowhere else to look them up, so they stay documented even though dispatch reaches them through `getattr` rather than by name. - **Most class private attributes.** A private attribute that carries a subclass's state, or that a - subclass sets or reads, is contract. Keep it unless there is a specific reason it is not. + subclass sets or reads, is contract. Keep it unless there is a specific reason not to. - **Anything abstract, or implemented across subclasses.** An `@abstractmethod`, or an overridable hook a subclass is expected to provide — `_make_data` has a concrete base implementation on - `ProtocolBase` and is overridden in 28 protocol modules, which is exactly the case this covers. + `ProtocolBase` and is overridden across the protocol modules, which is exactly this case. - **Members whose observable behaviour is documented**, such as a method whose docstring records the warning it emits or a guarantee it makes. A reader who hits that warning looks it up here. -**Recipe — leave it out.** The implementation detail that is on nobody's usage or extensibility -surface. In practice this is chiefly **module-level privates**: a private helper function, a lazily -imported backend flag, an internal lock, a private wrapper class nobody constructs or subclasses. -Removing one of these takes its members with it, which is correct — a member of a private class is -reachable only through that class. +**Recipe — leave it out.** Implementation detail on nobody's usage or extensibility surface, chiefly +**module-level privates**: a private helper function, a lazily imported backend flag, an internal +lock, a private wrapper class nobody constructs or subclasses. Removing one takes its members with +it, which is correct — they are reachable only through that class. -The sweep runs **in both directions**. A must-implement member with no directive is the same defect as -a recipe body with one, only quieter: add the missing directive rather than aiming for a small diff. +The sweep runs **in both directions**: a must-implement member with no directive is the same defect as +a recipe body with one, only quieter. Add the missing directive rather than aiming for a small diff. Two mechanical points that decide real cases: - **A private base class must stay documented when a documented subclass carries `:show-inheritance:`.** Sphinx renders that subclass's `Bases:` line as a link into the private - class's page, so dropping the directive breaks a link a public page really does render. This is what - keeps `pcapkit.protocols.schema.misc.pcapng._OPT_Option` and its five siblings, plus `_IPField`, + class's page, so dropping the directive breaks a link a public page renders. This keeps + `pcapkit.protocols.schema.misc.pcapng._OPT_Option` and its five siblings, plus `_IPField`, `_IPInterfaceField` and `_TextField`. -- **Dunders keep their directives.** A `__dunder__` is reached through public syntax rather than by - name — `__len__` is what `len()` calls, `__getitem__` is what `obj[key]` does — so overriding one - changes behaviour a caller observes without ever writing the name. PyPCAPKit's own `__proto__`, - `__option__`, `__schema__` and `__protocol_name__` family is the documented extension contract that - subclass authors and the `register_*` functions write to: public API in everything but spelling. - -One shape to know about because it is invisible: every const enum carries a `_missing_` fallback that -resolves an unregistered value and registers it, rather than raising as a plain `enum` would. That is -deliberate and it is the extensibility behaviour of the whole `pcapkit.const` package, so it is -documented once on the package's landing page rather than restated on each of the 121 enumerations -under `pcapkit/const/` that implement it. Note `docs/source/conf.py` already names `_missing_` in -`autodoc_default_options['exclude-members']`, so a per-class directive would be arguing with the -project's own configuration. +- **Dunders keep their directives.** A `__dunder__` is reached through public syntax, not by name + (`len()` calls `__len__`, `obj[key]` calls `__getitem__`), so overriding one changes behaviour a + caller observes without writing the name. PyPCAPKit's own `__proto__`, `__option__`, `__schema__` + and `__protocol_name__` family is the documented extension contract that subclass authors and the + `register_*` functions write to: public API in everything but spelling. + +One invisible shape: the `pcapkit/const/` enums carry a `_missing_` fallback that resolves an +unregistered value rather than raising as a plain `enum` would. This is deliberate +and is the extensibility behaviour of the whole `pcapkit.const` package, so it is documented once on +the package's landing page rather than on each of the 121 registries that implement it. +`docs/source/conf.py` already names `_missing_` in `autodoc_default_options['exclude-members']`, so a +per-class directive would argue with the project's own configuration. ## Coding style @@ -156,31 +149,29 @@ make bandit # security lint make vermin # minimum-Python-version check ``` -Four of them — `pylint`, `mypy`, `bandit` and `vermin` — also run in CI, as the `Lint` job in +Four of them — `pylint`, `mypy`, `bandit` and `vermin` — also run in CI as the `Lint` job in `.github/workflows/lint.yml`: on every pull request against `main`, on a weekly Saturday schedule and on demand through `workflow_dispatch`, on Python 3.14 alone rather than across the test matrix. -The job drives them through the `Makefile` rather than restating their flags — `pylint`, `mypy` and -`bandit` by the targets above, `vermin` by its `vermin-ci` variant — with `RUN=` emptying the -`pipenv run` prefix. Both paths read the same flag variables, so a check cannot come out clean -locally and red in CI because the two definitions drifted. +The job drives them through the `Makefile` rather than restating their flags (`vermin` by its +`vermin-ci` variant), with `RUN=` emptying the `pipenv run` prefix. Both paths read the same flag +variables, so a check cannot come out clean locally and red in CI because the two drifted. **Those steps are advisory, not a gate.** Each carries `continue-on-error: true`, so a finding lands -as a non-blocking annotation and a red linter will not fail your pull request. That is a consequence -of none of the four being clean today; the workflow's header records the current counts and what -each tool would need before its `continue-on-error` line could be deleted. Read the job's run -summary — each step writes its own verdict there — and try not to add to the numbers. - -`isort` is the exception and is still local-only. It does appear in `cron-vendor.yml`, but as a -formatter that rewrites the regenerated constants rather than as a check, so nothing verifies import -ordering on a pull request. - -One trap in the target list above: `make vermin` redirects its report into `temp/vermin.txt` -rather than to your terminal, and vermin exits 1 today because `vermin.ini` sets `targets = 3.6` -against a real floor of 3.11. Make gives up at that redirect, so the run fails with -`make: *** [vermin] Error 1`, the report left in the file, and the line that would have opened it -never reached. `make vermin-ci` runs the same flags straight to stdout, which is why CI uses it -and why it is the easier of the two to read at a desk. Running the lot before you push still saves -a review round. +as a non-blocking annotation and a red linter will not fail your pull request; none of the four is +clean. The workflow's header records the counts and what each tool would need before +its `continue-on-error` line could be deleted. Read the job's run summary, where each step writes +its verdict, and try not to add to the numbers. + +`isort` is the exception and is local-only. It appears in `cron-vendor.yml` as a formatter that +rewrites the regenerated constants, not as a check, so nothing verifies import ordering on a pull +request. + +One trap: `make vermin` redirects its report into `temp/vermin.txt` rather than to your terminal, +and vermin exits 1 because `vermin.ini` sets `targets = 3.6` against a real floor of 3.11. Make +gives up at the redirect with `make: *** [vermin] Error 1`, the report left in the file and the line +that would have opened it never reached. `make vermin-ci` runs the same flags to stdout, which is why +CI uses it and why it is easier to read at a desk. Running the lot before you push still saves a +review round. ### Format of the Commit Message @@ -214,8 +205,7 @@ ones in use. `scope` names the part of the package affected — `tcp`, `corekit` or `test(utilities,foundation):`. The scope may be omitted where nothing narrower than the whole project applies, as in `docs:`. -Older history is mixed, and `git log` will show you a bare `protocols:` or `corekit:` subsystem -prefix from before this settled. Follow the form above for new work rather than the older one. +Older commits may carry a bare `protocols:` or `corekit:` subsystem prefix; follow the form above. Reference issues and pull requests as `#nnn`, in the subject where they fit and in the body otherwise. Keep the subject to one line and as short as clarity allows; there is no hard column diff --git a/README.md b/README.md index 1200b916a1..7b7bc0c1af 100644 --- a/README.md +++ b/README.md @@ -1,20 +1,17 @@ # PyPCAPKit -- Comprehensive Network Packet Analysis Library -> For any technical and/or maintenance information, please kindly refer to the +> For technical and maintenance information, see the > **[Official Documentation](https://jarryshaw.github.io/PyPCAPKit/)**. -The PyPCAPKit project is an open source Python program focused on network packet -parsing and analysis, which works as a comprehensive -[PCAP](https://en.wikipedia.org/wiki/Pcap) file extraction, construction and -analysis library, with [DictDumper](https://github.com/JarryShaw/DictDumper) as -its formatted output dumper. +PyPCAPKit is an open-source Python library for parsing, constructing and analysing +network packets and [PCAP](https://en.wikipedia.org/wiki/Pcap) files, with +[DictDumper](https://github.com/JarryShaw/DictDumper) as its formatted output dumper. -Unlike popular PCAP file extractors such as [Scapy](https://scapy.net), +Unlike popular PCAP extractors such as [Scapy](https://scapy.net), [DPKT](https://dpkt.readthedocs.io) and [PyShark](https://kiminewt.github.io/pyshark), -`pcapkit` is designed to be much more comprehensive: it reports more detailed -information about each packet, and offers a more *Pythonic* interface to work -with it. When that depth is not what you need, the same interface will also drive -six third-party extraction engines instead. +`pcapkit` reports more detail about each packet through a more *Pythonic* interface. +Where that depth is not needed, the same interface also drives six third-party +extraction engines. The whole project supports **Python 3.6** or later. @@ -41,14 +38,14 @@ pip install pypcapkit[cli] # command line interface pip install pypcapkit[all] # core addons only: cli + crypto + NGAP (pycrate) ``` -Every engine above is on demand; `all` bundles only the core addons the library -needs for full functionality. Four of the engines also need something beyond a -`pip install` -- a `tshark` binary, a C compiler, `libpcap` headers, or an older -interpreter -- and `pypcap`/`pcap-ct` must never be installed together. -The [installation guide](https://jarryshaw.github.io/PyPCAPKit/#installation) -covers every constraint and the reason for it, and `pcapkit` enforces each one in -code: asking for an engine that cannot run in the current environment warns with -the actual cause and falls back to `pcapkit`'s own parser. +Engines are on demand; `all` bundles only the core addons the library needs for +full functionality. Four engines also need something beyond `pip install` -- a +`tshark` binary, a C compiler, `libpcap` headers, or an older interpreter -- and +`pypcap`/`pcap-ct` must never be installed together. The +[installation guide](https://jarryshaw.github.io/PyPCAPKit/#installation) gives each +constraint and its reason. `pcapkit` enforces them in code: asking for an engine +that cannot run in the current environment warns with the cause and falls back to +its own parser. ## Usage @@ -66,11 +63,10 @@ datetime.datetime(2017, 11, 19, 15, 49, 5, 471719, tzinfo=datetime.timezone.utc) IPv6Address('fe80::a6:87f9:2793:16ee') ``` -The output above is from `examples/captures/in.pcap`, which is committed, so it -is reproducible from a clone. +The output is from the committed `examples/captures/in.pcap`, so it is reproducible +from a clone. -Reassembly, TCP flow tracing and a different engine are all keyword arguments on -the same call: +Reassembly, TCP flow tracing and the engine are keyword arguments to the same call: ```python >>> scapy = pcapkit.extract('in.pcap', nofile=True, engine='scapy') @@ -80,18 +76,18 @@ the same call: 3 ``` -More worked examples, including the command line interface, are in +More examples, including the command line interface, are in [How to ...](https://jarryshaw.github.io/PyPCAPKit/demo.html). ## Documentation The [official documentation](https://jarryshaw.github.io/PyPCAPKit/) is the -reference for everything below. The pages worth knowing by name: +reference. Pages worth knowing by name: | Page | What is in it | |---|---| | [API reference](https://jarryshaw.github.io/PyPCAPKit/pcapkit/index.html) | Every module, protocol and constant | -| [Module structure](https://jarryshaw.github.io/PyPCAPKit/#module-structure) | What each of the eight subpackages is for | +| [Module structure](https://jarryshaw.github.io/PyPCAPKit/#module-structure) | What each of the nine subpackages is for | | [Engine comparison](https://jarryshaw.github.io/PyPCAPKit/#engine-comparison) | Which engines exist, which Python versions they run on, and measured speed per packet | | [Engine support](https://jarryshaw.github.io/PyPCAPKit/pcapkit/foundation/engines/index.html) | What each engine does *not* support, and how the gap is surfaced | | [Installation](https://jarryshaw.github.io/PyPCAPKit/#installation) | Extras, engine prerequisites and the local development setup | diff --git a/SECURITY.md b/SECURITY.md index a3275610d7..9d64d3227d 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -7,41 +7,39 @@ lines are not patched -- upgrading is the fix. | Version | Supported | Notes | | --------------- | ------------------ | --------------------------------------- | -| 1.5.x | :white_check_mark: | development line, currently pre-release | +| 1.5.x | :white_check_mark: | development line, pre-release | | 1.4.x | :white_check_mark: | current stable | | 1.3.x and older | :x: | no longer patched | | 0.x | :x: | no longer patched | -Supported interpreters are those the test matrix actually covers, currently -CPython 3.10 through 3.14. `pyproject.toml` declares `requires-python = ">=3.6, <4"` -through the `bpc-*` source conversion, but 3.8 and 3.9 are best-effort and below -3.8 is intent rather than something that is exercised -- see the note above -`requires-python` in `pyproject.toml`. A report that only reproduces on an -interpreter outside the tested range is still welcome; it may be fixed by raising -the floor rather than by patching. +Supported interpreters are those the test matrix covers: CPython 3.10 +through 3.14. `pyproject.toml` declares `requires-python = ">=3.6, <4"` through the +`bpc-*` source conversion, but 3.8 and 3.9 are best-effort and below 3.8 is intent +rather than something exercised -- see the note above `requires-python` in +`pyproject.toml`. A report that only reproduces outside the tested range is still +welcome; it may be fixed by raising the floor rather than by patching. ## Reporting a Vulnerability **Please do not open a public issue for a security problem.** Public issues are -the right place for ordinary bugs, and the wrong place for anything exploitable. +for ordinary bugs, not anything exploitable. Use GitHub's private vulnerability reporting, which is enabled on this repository: - -That opens a private advisory visible only to the maintainer, lets patches be +It opens a private advisory visible only to the maintainer, lets patches be prepared before anything is public, and issues a CVE if one is warranted. If you cannot use it, email the project contact listed in -[`CODE_OF_CONDUCT.md`](CODE_OF_CONDUCT.md) instead and say that the report is a -security issue. +[`CODE_OF_CONDUCT.md`](CODE_OF_CONDUCT.md) and say the report is a security issue. Please include whatever you have: - the version of `pypcapkit` and the Python interpreter, - what an attacker gains, and what they need in order to get it, - a capture file or a short script that reproduces it -- **strip anything - sensitive from a capture first**, since a pcap taken from a real network tends - to carry more than the bug. + sensitive from a capture first**, since a pcap from a real network tends to carry + more than the bug. ## What to expect @@ -49,27 +47,25 @@ This is a single-maintainer project, so response is best-effort rather than contractual: - an acknowledgement that the report arrived, normally within a week, -- an assessment of whether it is accepted, and if so a rough severity, once it has - been reproduced, +- once reproduced, an assessment of whether it is accepted and a rough severity, - a fix on the supported lines above, released as a patch version, with credit in the changelog unless you would rather not be named. If a report is declined, you will be told why -- most often that it describes -expected behaviour for a packet parser. Note the threat model below before +expected behaviour for a packet parser. Check the threat model below before deciding whether you have a finding. ## Threat model -`pcapkit` parses attacker-controlled bytes by design: a capture file, or a live -stream off the wire, is untrusted input. Vulnerabilities in that parsing are in -scope. In particular: +`pcapkit` parses attacker-controlled bytes by design: a capture file or a live +stream off the wire is untrusted input. Vulnerabilities in that parsing are in +scope, in particular: - memory exhaustion, unbounded allocation, or a hang on a malformed packet, - an uncaught exception escaping the public API where a `pcapkit` exception was promised -- parse errors are meant to arrive as `pcapkit.utilities.exceptions` - types rather than as an arbitrary traceback. That is the intent rather than - something the parser manages everywhere today, so it is worth reporting even - though some cases are already known, + types rather than an arbitrary traceback. That is the intent, not something the + parser manages everywhere, so report it even though some cases are known, - anything that executes code, writes outside the requested output path, or reads an unrelated file as a result of the *contents* of a capture. @@ -85,26 +81,24 @@ Out of scope: ## Handling untrusted captures -The scope above is about what to report. This is what a caller can do in the -meantime, because hardening a parser against hostile input is ongoing work rather -than a finished state, and some of it is limited by what a given layer can even -tell apart -- a capture legitimately truncated in transit and one crafted to look -that way are not always separable from inside the field that declares the length. +Hardening a parser against hostile input is ongoing work, and some of it is limited +by what a layer can tell apart -- a capture legitimately truncated in transit and +one crafted to look that way are not always separable from inside the field that +declares the length. So if the captures you hand `pcapkit` are not ones you +produced yourself: -So if the captures you hand `pcapkit` are not ones you produced yourself: - -- **Bound the memory.** A malformed or hostile capture can expand to a great deal - more than its size on disk. Parse in a subprocess under an address-space limit +- **Bound the memory.** A malformed or hostile capture can expand to far more than + its size on disk. Parse in a subprocess under an address-space limit (`resource.RLIMIT_AS`) or a container memory limit, rather than relying on the parser to bound itself. - **Bound the time**, by the same mechanism and for the same reason. - **Expect failure, including failure the hierarchy does not cover.** Catch `pcapkit.utilities.exceptions.BaseError` for parse errors, and be ready for - built-in types such as `MemoryError` reaching you as well. + built-in types such as `MemoryError` as well. - **Do not trust a field more than the capture it came from.** A truncated or malformed field may parse to a value rather than raise, so a value that carries a security decision deserves its own validation. -None of this is peculiar to `pcapkit` -- it is the ordinary posture for giving -attacker-controlled bytes to any parser. It is written down because parsing them -is what `pcapkit` is for. +None of this is peculiar to `pcapkit`; it is the ordinary posture for giving +attacker-controlled bytes to any parser, written down here because parsing them is +what `pcapkit` is for.