diff --git a/CHANGELOG.md b/CHANGELOG.md index 7ff15100d..bd1fc8786 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -72,6 +72,7 @@ This is the resolution of #548, which reported `TransType.L2TP` (115) as registe - **Added** -- `examples/generators/endian.py`, and the byte-order tests that read what it writes. There was no big-endian `.pcap` in the repository at all, which is why #605 survived its own code review: the one-character fix leaves the corrected path exactly as untested as the broken one. The generator writes three captures -- `big_endian.pcap` (magic `a1 b2 c3 d4`), `big_endian_nanosecond.pcap` (`a1 b2 3c 4d`, the first fixture to take that branch of the magic-number table) and `little_endian.pcap` (`d4 c3 b2 a1`) -- carrying the *same three records* in each container, so the tests can assert that the byte order makes no difference to what is read out rather than only that the big-endian file matches numbers written down in a test. Frame 3 is captured short, 1200 octets on the wire cut to a 96-octet `snaplen`, so `incl_len` and `orig_len` differ and cannot both be satisfied by one byte-swapped value. `test_frame_endian_runtime.py` drives all three through `extract()` and walks each file's record chain with `struct` to derive its own expectations; a unit-tier case in `test_header_frame_unit.py` builds a two-record big-endian capture in memory instead, so the regression is also caught by the fixture-free selection CI runs on every push. All four fail on the unfixed tree -- the three fixture-backed ones by that `ValueError`, the in-memory one by `AssertionError: 3106905 != 1500000000` -- while the little-endian twin passes on both trees, which is what shows the records themselves are not the variable (#605). - **Changed** -- `tests/protocols/transport/test_tcp_udp_unit.py` now reaches the MP_JOIN dispatchers through `TCP()` itself, instead of assigning a Python `set` to `_flags` on a bare `TCP.__new__(TCP)`. A `set` answers the membership tests `_make_mptcp_join` and `_read_mptcp_join` use, so every flag branch ran and both TCP modules read 100% statement and branch coverage -- while the attribute had neither the `aenum.IntFlag` type production assigns nor the ordering that governs when it exists at all, which is how #587 stayed invisible behind that number and how the `cast('Enum_Flags', 0)` no-op behind it went unnoticed too. Measured on the rewrite, against the 17 tests of that file: revert #587's hoist and two of them fail with `AttributeError: 'TCP' object has no attribute '_flags'` where all 17 passed before; restore the `cast` and two fail with `TypeError: argument of type 'int' is not a container or iterable`, again where all 17 passed. The library is unchanged and the file's tests still pass, so the coverage numbers do not move -- the point is what the same numbers are now worth (#603). - **Fixed** -- documentation. `mptcp_dss_ack_selector`'s note said a corrected field-width lambda "would not have worked" and that fixing it belonged to `pcapkit.corekit.fields.numbers`, which is exactly where #598 then fixed it; the same paragraph sat in `test_tcp_mptcp_length_arithmetic_unit.py`'s module docstring, whose other stale claim was that MP_JOIN "cannot be built through the public `TCP()` constructor at all", true only until #587. A callable-length `NumberField` packs and unpacks both DSS widths now, and wire *absence* was never the obstacle either: `MPTCPDSS.ssn`, `dl_len` and `checksum` have always been `ConditionalField` on the sibling `M` flag, so the class already relied on that wrapper to keep a field off the wire. The `SwitchField` form is kept for the narrower reason the note now gives -- `ConditionalField`'s `length` forwards to the wrapped field without consulting the condition, so it is safe here only because `Schema.pack` and `Schema.unpack` special-case that wrapper by name, whereas a `SwitchField` always resolves to a concrete field. Replacing it would be a behaviour change and is not made (#603). +- **Changed** -- the README is a landing page now, and Markdown rather than reStructuredText. `README.rst` (424 lines) became `README.md` (103), keeping what a reader arriving from PyPI or a search result actually needs -- what the library is, why it exists rather than Scapy or DPKT, how to install it, a worked example, and where the documentation lives -- and dropping the technical detail the documentation already carried. **Module Structure**, **Engine Comparison**, **Engine support by Python version**, **Test Environment**, **Test Results** and **Installation Notes** were each already duplicated in `docs/source/index.rst`, in a fuller form, so they are linked rather than restated. Two blocks existed nowhere else and moved rather than going: **Testing** is now `docs/source/testing.rst`, registered in the index toctree, and the `pipenv` and `make setup` local development block joined the Installation section of `docs/source/index.rst`. Requested by the project owner, and a deliberate exception to the convention that documentation here is reStructuredText -- for the README only, since it is the one documentation file whose renderers are GitHub and PyPI rather than Sphinx. Accordingly `setup.py` reads `README.md` and declares its content type as `text/markdown`, and the `include README.md` line in `MANIFEST.in` is now the only thing that puts the README in a source distribution, because `global-include *.rst` no longer matches it -- which matters, since `setup.py` reads the file unguarded and an sdist without it cannot be installed. Verified with `twine check --strict` against a built sdist and wheel, both of which pass. The rename's own references moved with it, since a change that renames a file owns the references to it: `examples/benchmark/Dockerfile` copies `README.md` -- a literal `COPY` of the old name would have failed the layer outright and taken `make bench`, `make bench-quick` and `run.sh` with it -- and the benchmark harness prose that named the root README as the destination of its generated tables now names `docs/source/index.rst`, which is where those tables went. That covers the `Makefile` comment, `report.py`, `test_harness.py`, `run.sh` and the suite's own README, including the one place whose stated reason had inverted: the emitted markup is kept parseable by plain docutils, which is now a conservative choice rather than a hard requirement, because the page it lands on is rendered by Sphinx. `examples/benchmark/benchmark.py` still says `README.rst` and is left alone, because it means the benchmark suite's own README in the same directory, not the project's. - **Changed** -- `CODE_OF_CONDUCT.md` moves from Contributor Covenant 1.4 to Contributor Covenant 3.0, at the maintainer's request. The text is the canonical 3.0 Markdown fetched from https://www.contributor-covenant.org/version/3/0/code_of_conduct/code_of_conduct.md rather than a transcription, so the pledge, the encouraged and restricted behaviours and the scope are unaltered. Three things needed deciding rather than copying. 3.0 ships two `[NOTE` placeholders an adopter must fill: the reporting channel, which now names `jarryshaw@icloud.com` -- the same contact 1.4 carried and the one `SECURITY.md` already points at as its email fallback -- plus GitHub's report-abuse form for the case a single-maintainer project cannot otherwise cover, a report about the maintainer; and the enforcement section, whose placeholder is an instruction to the adopter and is removed. 3.0 then assigns enforcement throughout to plural "Community Moderators" (and once, inconsistently, to "Community Managers"), which this repository does not have, so all eight occurrences become the singular maintainer. The four-rung ladder -- Warning, Temporarily Limited Activities, Temporary Suspension, Permanent Ban -- is offered as a suggestion and is **kept**, because each rung maps onto a lever one person actually holds on GitHub: a private message, a locked thread, an interaction limit or block, a permanent block. Finally, 3.0 is licensed CC BY-SA 4.0 where 1.4's attribution paragraph carried no licence notice at all, so the attribution now names version 3.0, links the permanent `version/3/0/` URL, carries the CC BY-SA 4.0 notice and link, indicates that changes were made as BY requires, and says explicitly that the share-alike term covers this document only -- the code remains BSD-3-Clause and `LICENSE` is untouched. Rendering was checked against GitHub's own Markdown API rather than assumed: the ladder comes back as four list items each nesting three, which is what #613 had to repair in the 1.4 file when a stray list marker collapsed the whole document into one nested item (#624). Preceded by `1.5.0a1` (2026-09-15), `1.5.0b1` and `1.5.0b2` (both 2026-09-18) and `1.5.0b3` (2026-09-19), all published as prereleases and so resolved only by `pip install --pre`. `1.5.0b1` half-shipped: the tag, the GitHub release and the Conda deployments landed, but PyPI rejected the wheel because `twine check` found a Sphinx-only `:mod:` role in `README.rst`, which `pyproject.toml` declares as the dynamic long description. `1.5.0b2` is what reshipped it -- the release workflow is version-driven, so an existing version cannot republish -- and `1.5.0b3` followed the CI change that stops a TestPyPI outage from costing a release its wheels (#497, #498). diff --git a/MANIFEST.in b/MANIFEST.in index 89194f4fc..e17471ab6 100644 --- a/MANIFEST.in +++ b/MANIFEST.in @@ -11,7 +11,11 @@ prune examples/captures prune temp prune test -include README.rst +# Load-bearing, not belt-and-braces. ``global-include *.rst`` above matches only +# ``*.rst``, so since the README became Markdown this line is the only thing that +# puts it in an sdist -- and ``setup.py`` reads it unguarded for the +# ``long_description``, so an sdist without it cannot be installed at all. +include README.md include LICENSE # The changelog needs two lines of its own, and neither is redundant. @@ -24,7 +28,7 @@ include CHANGELOG.md # ``CHANGELOG.md`` carries only the version being released, so on its own the # distribution would describe one release and no history. ``prune docs`` above # drops the rest, which is where the history lives, so re-add just that one -# directory afterwards -- the same shape as ``include README.rst``, and for the +# directory afterwards -- the same shape as ``include README.md``, and for the # same reason. Each of those files is a self-contained reStructuredText entry # that renders standalone; the Sphinx index beside them is deliberately *not* # shipped, because it is a ``toctree`` that only Sphinx can read. diff --git a/Makefile b/Makefile index 70bde8b0c..3a71d8996 100644 --- a/Makefile +++ b/Makefile @@ -87,11 +87,11 @@ coverage: samples pipenv run coverage run -m pytest -q pipenv run coverage report -# The engine speed table in README.rst -- every supported Python version, one -# image each, measured in containers so the host does not affect the result. Needs -# docker, and nothing else: deliberately not run through pipenv, since the whole -# point is that the measuring environments are the pinned ones inside the images -# rather than whatever is installed here. +# The engine speed table in docs/source/index.rst -- every supported Python +# version, one image each, measured in containers so the host does not affect the +# result. Needs docker, and nothing else: deliberately not run through pipenv, +# since the whole point is that the measuring environments are the pinned ones +# inside the images rather than whatever is installed here. # # Expect around two hours at the defaults: five interpreters, seven environments, # and almost all of the measuring time is pyshark, which spawns a tshark process diff --git a/README.md b/README.md new file mode 100644 index 000000000..2f5b6bd19 --- /dev/null +++ b/README.md @@ -0,0 +1,102 @@ +# PyPCAPKit -- Comprehensive Network Packet Analysis Library + +> For any technical and/or maintenance information, please kindly refer to the +> **[Official Documentation](https://jarryshaw.github.io/PyPCAPKit/)**. + +The PyPCAPKit project is an open source Python program focused on network packet +parsing and analysis, which works as a comprehensive +[PCAP](https://en.wikipedia.org/wiki/Pcap) file extraction, construction and +analysis library, with [DictDumper](https://github.com/JarryShaw/DictDumper) as +its formatted output dumper. + +Unlike popular PCAP file extractors such as [Scapy](https://scapy.net), +[DPKT](https://dpkt.readthedocs.io) and [PyShark](https://kiminewt.github.io/pyshark), +`pcapkit` is designed to be much more comprehensive: it reports more detailed +information about each packet, and offers a more *Pythonic* interface to work +with it. When that depth is not what you need, the same interface will also drive +six third-party extraction engines instead. + +The whole project supports **Python 3.6** or later. + +## Installation + +```shell +pip install pypcapkit +``` + +Or from a clone, for the latest version and for development: + +```shell +git clone https://github.com/JarryShaw/PyPCAPKit.git +cd PyPCAPKit +pip install -e . +``` + +The extraction engines and plug-ins are optional extras: + +```shell +pip install pypcapkit[DPKT] # or Scapy, PyShark, PyPCAPFile, PyPCAP, PCAP_CT +pip install pypcapkit[crypto] # ESP payload decryption +pip install pypcapkit[cli] # command line interface +pip install pypcapkit[all] # every pure-Python extra +``` + +Four of the engines need something beyond a `pip install` -- a `tshark` binary, a +C compiler, `libpcap` headers, or an older interpreter -- and `all` deliberately +excludes both `pypcap` and `pcap-ct`, which must never be installed together. +The [installation guide](https://jarryshaw.github.io/PyPCAPKit/#installation) +covers every constraint and the reason for it, and `pcapkit` enforces each one in +code: asking for an engine that cannot run in the current environment warns with +the actual cause and falls back to `pcapkit`'s own parser. + +## Usage + +```python +>>> import pcapkit +>>> extraction = pcapkit.extract('in.pcap', nofile=True) +>>> len(extraction.frame) +6 +>>> frame = extraction.frame[0] +>>> str(frame.protochain) +'Ethernet:IPv6:IPv6_ICMP' +>>> frame.info.time +datetime.datetime(2017, 11, 19, 15, 49, 5, 471719, tzinfo=datetime.timezone.utc) +>>> frame.payload.payload.src +IPv6Address('fe80::a6:87f9:2793:16ee') +``` + +The output above is from `examples/captures/in.pcap`, which is committed, so it +is reproducible from a clone. + +Reassembly, TCP flow tracing and a different engine are all keyword arguments on +the same call: + +```python +>>> scapy = pcapkit.extract('in.pcap', nofile=True, engine='scapy') +>>> reasm = pcapkit.extract('in.pcap', nofile=True, reassembly=True, ipv6=True) +>>> flows = pcapkit.extract('in.pcap', nofile=True, trace=True, tcp=True) +>>> len(flows.trace) +3 +``` + +More worked examples, including the command line interface, are in +[How to ...](https://jarryshaw.github.io/PyPCAPKit/demo.html). + +## Documentation + +The [official documentation](https://jarryshaw.github.io/PyPCAPKit/) is the +reference for everything below. The pages worth knowing by name: + +| Page | What is in it | +|---|---| +| [API reference](https://jarryshaw.github.io/PyPCAPKit/pcapkit/index.html) | Every module, protocol and constant | +| [Module structure](https://jarryshaw.github.io/PyPCAPKit/#module-structure) | What each of the eight subpackages is for | +| [Engine comparison](https://jarryshaw.github.io/PyPCAPKit/#engine-comparison) | Which engines exist, which Python versions they run on, and measured speed per packet | +| [Engine support](https://jarryshaw.github.io/PyPCAPKit/pcapkit/foundation/engines/index.html) | What each engine does *not* support, and how the gap is surfaced | +| [Installation](https://jarryshaw.github.io/PyPCAPKit/#installation) | Extras, engine prerequisites and the local development setup | +| [Testing](https://jarryshaw.github.io/PyPCAPKit/testing.html) | Running the suite, and the sample captures it needs | +| [How to ...](https://jarryshaw.github.io/PyPCAPKit/demo.html) | Worked examples, library and CLI | +| [Extensions](https://jarryshaw.github.io/PyPCAPKit/ext.html) | Registering your own protocols, engines and dumpers | + +Release history is in [CHANGELOG.md](CHANGELOG.md), and contribution guidelines +are in [CONTRIBUTING.md](CONTRIBUTING.md). diff --git a/README.rst b/README.rst deleted file mode 100644 index 880c5f927..000000000 --- a/README.rst +++ /dev/null @@ -1,424 +0,0 @@ -PyPCAPKit - Comprehensive Network Packet Analysis Library -========================================================= - - For any technical and/or maintenance information, - please kindly refer to the |docs|_. - -.. |docs| replace:: **Official Documentation** -.. _docs: https://jarryshaw.github.io/PyPCAPKit/ - -The PyPCAPKit project is an open source Python program focus on network packet -parsing and analysis, which works as a comprehensive `PCAP`_ file extraction, -construction and analysis library. - - The whole project supports **Python 3.6** or later; CI covers 3.10 to 3.14, - and 3.15 as an allowed-to-fail leg. - ------ -About ------ - -PyPCAPKit is a comprehensive Python-native network packet analysis library, -with `DictDumper`_ as its formatted output dumper. - -Unlike popular PCAP file extractors, such as `Scapy`_, `DPKT`_, `PyShark`_, -and etc, ``pcapkit`` is designed to be much more comprehensive, which means -it is able to provide more detailed information about the packet, as well as -a more *Pythonic* interface for users to interact with. - -Module Structure ----------------- - -In ``pcapkit``, all files can be described as following eight parts. - -- Interface (``pcapkit.interface``) - - User interface for the ``pcapkit`` library, which - standardises and simplifies the usage of this library. - -- Foundation (``pcapkit.foundation``) - - Synthesises file I/O and protocol analysis, coordinates - information exchange in all network layers, as well as - provides the foundamental functions for ``pcapkit``. - -- Protocols (``pcapkit.protocols``) - - Collection of all protocol family, with detailed - implementation and methods. - -- Utilities (``pcapkit.utilities``) - - Auxiliary functions and tools for ``pcapkit``. - -- CoreKit (``pcapkit.corekit``) - - Core utilities for ``pcapkit`` implementation, mainly - for internal data structure and processing. - -- ToolKit (``pcapkit.toolkit``) - - Auxiliary tools for ``pcapkit`` to support the multiple - extraction engines with a unified interface. - -- DumpKit (``pcapkit.dumpkit``) - - File output formatters for ``pcapkit``. - -- Constants (``pcapkit.const``) - - Constant enumerations used in ``pcapkit`` for protocol - family extraction and representation. - -Engine Comparison ------------------ - -Due to the general overhead of ``pcapkit``, its extraction procedure takes -around *0.2* milliseconds per packet, which is already impressive but not enough -comparing to other popular extraction engines available on the market, given the -fact that ``pcapkit`` is a **comprehensive** packet processing module. - -Additionally, ``pcapkit`` introduced alternative extraction engines to accelerate -this procedure. By now ``pcapkit`` supports `Scapy`_, `DPKT`_, `PyShark`_, -`PyPCAP`_, `pcap-ct`_ and `PyPCAPFile`_, selected through ``engine='scapy'``, -``'dpkt'``, ``'pyshark'``, ``'pypcap'``, ``'pcap_ct'`` and ``'pypcapfile'`` -respectively; ``engine='default'`` (also spelled ``'pcapkit'``) is ``pcapkit``'s -own parser and the only one with no third-party requirement. - -`PyPCAP`_ and `pcap-ct`_ are two independent distributions of the same -``libpcap(3)`` interface, and both install a top-level ``pcap`` module, so -they are two engines rather than one. Upstream `PyPCAP`_ stops at Python 3.11; -`pcap-ct`_ covers 3.10 and newer. **Install exactly one of them** -- with both -present, ``pcap-ct`` wins the import and the other becomes unselectable, which -each engine detects and reports. - -Speed is not free. Every third-party engine supports **less** than the -``default`` one, and the newest ones support markedly less: - -- `PyPCAP`_ performs no protocol dissection at all, so it offers neither - reassembly nor flow tracing, and reads PCAP savefiles from disk only. -- `pcap-ct`_ reads the same interface, so it has exactly the same gaps. -- `PyPCAPFile`_ has no IPv6 decoder, so IPv6 reassembly is unavailable; IPv4 - and TCP reassembly still work, and it too is PCAP-only. -- `PyShark`_ performs no reassembly. - -Each gap is announced with a warning or an exception rather than silently -returning nothing. The `engine support documentation`_ tabulates them. - -Every engine also answers a preflight check before it is used -- -``unsupported_reason()`` -- so asking for one that cannot run in the current -environment produces a single warning naming the actual cause (a Python version, a -missing ``tshark``, a missing ``libpcap``, the wrong ``pcap`` distribution) and a -clean fall back to ``pcapkit``'s own parser, rather than an error from inside the -third-party package. - -Engine support by Python version -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Which engines can run at all, by interpreter. Verified by installing each engine -and extracting a capture on 3.10 through 3.14; 3.15 remains inferred. - -============== ======== ======== ======== ======== ======== ======== -Engine 3.10 3.11 3.12 3.13 3.14 3.15 -============== ======== ======== ======== ======== ======== ======== -``pcapkit`` yes yes yes yes yes yes* -``dpkt`` yes yes yes yes yes yes* -``scapy`` yes yes yes yes yes yes* -``pcap_ct`` yes yes yes yes yes yes* -``pypcap`` yes yes no no no no -``pypcapfile`` yes yes no no no no -``pyshark`` yes† yes† yes† yes† no no -============== ======== ======== ======== ======== ======== ======== - - ``*`` inferred, not measured -- no 3.15 interpreter was available. - - ``†`` also needs Wireshark's ``tshark``, which was absent, so only the - interpreter half was verified for ``pyshark``. - -``pypcap`` and ``pypcapfile`` stop at 3.11, and ``pyshark`` at 3.13, for the -reasons under `Engine prerequisites`_. **Python 3.11 is the last version on which -every engine can run** -- and even there ``pypcap`` and ``pcap_ct`` are mutually -exclusive, since both provide the ``pcap`` module, so no single environment ever -has all seven at once. - -Test Environment -~~~~~~~~~~~~~~~~ - -.. list-table:: - - * - Operating System - - macOS 26.6.2 - * - Chip - - Apple M2 Pro - * - Memory - - 16 GB - -Test Results -~~~~~~~~~~~~ - -Measured with ``examples/legacy_smoke/test_time.py``: 1,000 timed -extractions of ``examples/captures/in.pcap`` per engine and Python version. The -first extraction is discarded as a warm-up. Values are milliseconds per packet. - -=================== ======= ======= ======= ======= ======= -Engine 3.10 3.11 3.12 3.13 3.14 -=================== ======= ======= ======= ======= ======= -``pypcapfile`` [2]_ 0.0133 0.0093 -- -- -- -``dpkt`` 0.0163 0.0113 0.0118 0.0117 0.0125 -``pypcap`` [3]_ 0.0289 0.0212 -- -- -- -``pcap_ct`` [4]_ 0.0364 0.0320 0.0366 0.0322 0.0396 -``scapy`` 0.1082 0.0822 0.0877 0.0832 0.0967 -``pcapkit`` 0.2342 0.1906 0.1985 0.1920 0.2392 -``pyshark`` [1]_ 18.8158 18.7852 19.2694 20.3751 -- -=================== ======= ======= ======= ======= ======= - -The unavailable cells were attempted. They are not zeroes and must not be -compared with a measured row. - ------------- -Installation ------------- - -Simply run the following to install the current version from PyPI: - -.. code-block:: shell - - pip install pypcapkit - -Or install the latest version from the gi repository: - -.. code-block:: shell - - git clone https://github.com/JarryShaw/PyPCAPKit.git - cd pypcapkit - pip install -e . - # and to update at any time - git pull - -For local development with ``pipenv``, the repository already includes a -``Pipfile`` and ``Makefile`` targets that keep both the virtualenv and the -package caches inside the project directory: - -.. code-block:: shell - - make setup - -This resolves two common local setup issues on macOS/Homebrew installations: -``pipenv`` cache permission errors under ``~/Library/Caches`` and ``lxml`` -builds failing to locate Homebrew's ``libxml2``/``libxslt`` headers. - -If you prefer to run ``pipenv`` directly, use the same local cache layout and -skip any stale, user-local ``Pipfile.lock``: - -.. code-block:: shell - - PIPENV_VENV_IN_PROJECT=1 \ - PIPENV_CACHE_DIR=$PWD/.pipenv-cache \ - PIP_CACHE_DIR=$PWD/.pip-cache \ - pipenv install --skip-lock --dev - -And since ``pcapkit`` supports various extraction engines, and extensive -plug-in functions, you may want to install the optional ones: - -.. code-block:: shell - - # for DPKT only - pip install pypcapkit[DPKT] - # for Scapy only - pip install pypcapkit[Scapy] - # for PyShark only - pip install pypcapkit[PyShark] - # for PyPCAPFile only - pip install pypcapkit[PyPCAPFile] - # for PyPCAP only -- see the note below, this one builds from source - pip install pypcapkit[PyPCAP] - # for pcap-ct only -- the pure-Python alternative to PyPCAP, and the one that - # works on Python 3.12+; do not install it alongside PyPCAP - pip install pypcapkit[PCAP_CT] - # for ESP payload decryption - pip install pypcapkit[crypto] - # and to install the optional packages -- note this excludes PyPCAP and pcap-ct - pip install pypcapkit[all] - # or to do this explicitly - pip install pypcapkit dpkt scapy pyshark pypcapfile - -Installation Notes ------------------- - -The ``all`` extra deliberately excludes both ``pypcap`` and -``pcap-ct``, for different reasons. Everything -else in ``all`` is a pure-Python wheel, whereas ``pypcap`` compiles a C -extension; pulling it into ``all`` would demand a working compiler and the -`libpcap`_ development files from everyone installing ``pypcapkit[all]``. -``pcap-ct`` needs no compiler, but it and its ``libpcap`` dependency are -published only as **pre-releases** (1.3.0b3 and 1.11.0b29), and ``all`` should -not be how somebody ends up with a beta they did not ask for. Install either -explicitly: ``pip install pypcapkit[PyPCAP]`` or -``pip install pypcapkit[PCAP_CT]``. - -**Install only one of them.** Both distributions own the top-level ``pcap`` -module, and ``pip`` will install both without complaint. With both present the -``pcap-ct`` package wins the import and ``pypcap``'s extension module is -shadowed and unreachable, so ``engine='pypcap'`` stops working. ``pcapkit`` -detects that state and warns, naming both distributions and which one won, but -it cannot undo it. - -Engine prerequisites --------------------- - -Four of the engines need something beyond a ``pip install``. Each constraint is -also enforced in code -- the engine's ``unsupported_reason()`` is consulted before -anything is imported -- so hitting one produces a warning naming the cause and a -fall back to ``pcapkit``'s own parser, not an error from inside the third-party -package. - -``pyshark`` - Two requirements, and neither is visible to an import: the package imports - cleanly and then fails when used. - - - Drives Wireshark's ``tshark`` binary. It need not be on ``PATH``: - ``pyshark`` looks at ``tshark_path`` in its ``config.ini`` first, then - ``PATH`` on POSIX, both Program Files directories on Windows, and - ``/Applications/Wireshark.app`` on macOS. Install Wireshark (or just - ``tshark``) from your platform's package manager. - - Requires Python **3.13 or older**. ``pyshark`` 0.6 builds its event loop - with ``asyncio.get_event_loop_policy().get_event_loop()``, and from Python - **3.14** ``asyncio.get_event_loop()`` raises ``RuntimeError`` when there - is no current event loop instead of quietly creating one. Measured: a loop - is returned silently on 3.10 and 3.11, returned with a - ``DeprecationWarning`` on 3.12, and refused on 3.14. (3.13 was not available - to test and is expected to work, being on the deprecated-but-functional side - of that change.) - -``pypcap`` - Ships **no wheels** -- only an sdist -- so ``pip`` compiles it, and the build - needs both `libpcap`_'s headers (``pcap.h``) and its shared or static library: - - .. code-block:: shell - - # Debian/Ubuntu - sudo apt-get install libpcap-dev - # RHEL/Fedora/Amazon Linux - sudo dnf install libpcap-devel - # macOS - brew install libpcap - - Two caveats, both upstream problems rather than ``pcapkit`` ones: - - - ``pypcap`` 1.3.0 ships a **pre-generated** ``pcap.c`` produced by Cython - 0.29.x, which does not compile against the Python **3.12+** C API. Having - ``libpcap`` installed is therefore necessary but *not* sufficient: on 3.12 - or newer the build fails whatever else is present. Use Python **3.11 or - older** for this engine, or regenerate ``pcap.c`` with Cython 3 yourself. - - Its ``setup.py`` does not consult ``CFLAGS``/``LDFLAGS`` or - ``pkg-config``. It searches a fixed list of prefixes -- ``/usr``, - ``sys.prefix``, ``/opt/libpcap*``, ``../libpcap*``, ``../wpdpack*`` and the - macOS SDKs -- so a `libpcap`_ installed anywhere else, notably Homebrew's - keg-only prefix on Apple Silicon (``/opt/homebrew/opt/libpcap``), is not - found even though it is installed. Installing into ``sys.prefix``, or into - ``/opt/libpcap``, is what that search will pick up. - -``pcap_ct`` - The way to drive the same `libpcap`_ interface on Python **3.12 and newer**, - where ``pypcap`` cannot be built. Nothing to compile and no ``pcap.h`` needed: - `pcap-ct`_ is a ``ctypes`` reimplementation, and both it and its ``libpcap`` - dependency ship ``py3-none-any`` wheels. Verified reading a capture on Python - 3.10 and 3.14. - - Two caveats: - - - **A system ``libpcap`` is still required at run time.** The ``libpcap`` - distribution ships a vendored ``libpcap.so`` and, as published, does not use - it: its ``libpcap.cfg`` says ``LIBPCAP = None``, which sends its loader to - ``ctypes.util.find_library('pcap')``. So the library actually loaded is the - host's ``libpcap.so.1``, and with none present ``import pcap`` raises - ``OSError`` rather than ``ImportError``. Set ``LIBPCAP = tcpdump`` in - ``libpcap.cfg`` to use the vendored copy instead. - - Both distributions are **pre-releases**, and ``pcap-ct`` documents itself as - tracking the ``pypcap`` *1.2.3* interface. Every attribute the engine uses - was measured behaving identically to ``pypcap`` 1.3.0, but that is a - statement about the versions tested. - -``pypcapfile`` - Version 0.12.0 imports the ``imp`` module, which was **removed in Python - 3.12**, so ``pcapfile.savefile`` -- the module needed to read a capture -- - cannot be imported at all on 3.12 or newer. Upstream ``master`` has fixed - this but no release carries the fix yet, so this engine also requires Python - **3.11 or older** until 0.12.1 is published. - - **Note** -- ``pcapkit`` itself, and its ``default``, ``dpkt`` and ``scapy`` - engines, work - fine on current Python versions -- ``dpkt`` 1.9.8 and ``scapy`` 2.7.0 were both - measured reading a capture on Python 3.14. Only the four engines above carry - extra constraints, and asking for an engine that cannot run in the current - environment emits a warning naming the reason and falls back to ``pcapkit``'s - own parser rather than failing outright. - -For CLI usage, you will need to install the optional packages: - -.. code-block:: shell - - pip install pypcapkit[cli] - # or explicitly... - pip install pypcapkit emoji - -------- -Testing -------- - -The unit tests need nothing beyond the package itself and the sample captures -tracked in the repository: - -.. code-block:: shell - - make test - -The runtime, regression and integration tests additionally read sample captures -that are **not** tracked (see ``.gitignore``); -``examples/generators/make_samples.py`` reconstructs them into ``examples/captures/``, -and ``make test-all`` regenerates them before running the whole suite: - -.. code-block:: shell - - make samples # write examples/captures/*.pcap and *.pcapng - make test-all # regenerate the fixtures, then run every test - -The same fixtures back the demonstration scripts in -``examples/legacy_smoke/``, which read them as ``../captures/…``. - -One of the generators works differently from the rest and is worth knowing about. -``examples/generators/options.py`` does not describe packets it wants; it asks the -library which option, chunk, parameter, message, frame and block codes it -registers, and then constructs one of each through the public construction API, -parses it back, and constructs it again from what was parsed. The -``options-*.pcap`` captures are the cases that survive that round trip, so they -record what this version of ``pcapkit`` builds rather than what a third-party -tool builds. ``tests/protocols/test_option_roundtrip_unit.py`` runs the same -cases without needing a fixture at all, and carries a table of the ones that do -not yet close the cycle, each named against the defect that stops it. - -Continuous integration runs the ``make test`` selection, since the fixtures are -not in the repository. ``tshark`` is only required to exercise the PyShark -engine, and is not needed by the test suite. - -.. _PCAP: https://en.wikipedia.org/wiki/Pcap -.. _Scapy: https://scapy.net -.. _DPKT: https://dpkt.readthedocs.io -.. _PyShark: https://kiminewt.github.io/pyshark -.. _PyPCAP: https://github.com/pynetwork/pypcap -.. _pcap-ct: https://pypi.org/project/pcap-ct/ -.. _PyPCAPFile: https://github.com/kisom/pypcapfile -.. _libpcap: https://www.tcpdump.org -.. _DictDumper: https://github.com/JarryShaw/DictDumper -.. _engine support documentation: https://jarryshaw.github.io/PyPCAPKit/pcapkit/foundation/engines/index.html - -.. [1] `PyShark`_ 0.6 cannot create the implicit event loop it expects on Python - 3.14; it ran on 3.10--3.13. - -.. [2] `PyPCAPFile`_ 0.12.0 imports ``imp``, which Python removed in 3.12. - -.. [3] `PyPCAP`_ 1.3.0 could not be built here: its build does not search - Homebrew's libpcap library prefix. It is also unsupported on Python 3.12+. - -.. [4] The prerelease `pcap-ct`_ / `libpcap`_ wheels currently load Linux - ``libc.so.6`` on this macOS host, so their engine could not be preflighted. diff --git a/docs/source/changelog/1.5.0.rst b/docs/source/changelog/1.5.0.rst index aac296e78..d827348f0 100644 --- a/docs/source/changelog/1.5.0.rst +++ b/docs/source/changelog/1.5.0.rst @@ -941,6 +941,42 @@ pull requests between #326 and #509. is safe here only because ``Schema.pack`` and ``Schema.unpack`` special-case that wrapper by name, whereas a ``SwitchField`` always resolves to a concrete field. Replacing it would be a behaviour change and is not made (#603). +* **Changed** -- the README is a landing page now, and Markdown rather than + reStructuredText. ``README.rst`` (424 lines) became ``README.md`` (103), + keeping what a reader arriving from PyPI or a search result actually needs -- + what the library is, why it exists rather than Scapy or DPKT, how to install + it, a worked example, and where the documentation lives -- and dropping the + technical detail the documentation already carried. **Module Structure**, + **Engine Comparison**, **Engine support by Python version**, **Test + Environment**, **Test Results** and **Installation Notes** were each already + duplicated in ``docs/source/index.rst``, in a fuller form, so they are linked + rather than restated. Two blocks existed nowhere else and moved rather than + going: **Testing** is now ``docs/source/testing.rst``, registered in the index + toctree, and the ``pipenv`` and ``make setup`` local development block joined + the Installation section of ``docs/source/index.rst``. Requested by the project + owner, and a deliberate exception to the convention that documentation here is + reStructuredText -- for the README only, since it is the one documentation file + whose renderers are GitHub and PyPI rather than Sphinx. Accordingly + ``setup.py`` reads ``README.md`` and declares its content type as + ``text/markdown``, and the ``include README.md`` line in ``MANIFEST.in`` is now + the only thing that puts the README in a source distribution, because + ``global-include *.rst`` no longer matches it -- which matters, since + ``setup.py`` reads the file unguarded and an sdist without it cannot be + installed. Verified with ``twine check --strict`` against a built sdist and + wheel, both of which pass. The rename's own references moved with it, since a + change that renames a file owns the references to it: + ``examples/benchmark/Dockerfile`` copies ``README.md`` -- a literal ``COPY`` of + the old name would have failed the layer outright and taken ``make bench``, + ``make bench-quick`` and ``run.sh`` with it -- and the benchmark harness prose + that named the root README as the destination of its generated tables now names + ``docs/source/index.rst``, which is where those tables went. That covers the + ``Makefile`` comment, ``report.py``, ``test_harness.py``, ``run.sh`` and the + suite's own README, including the one place whose stated reason had inverted: the + emitted markup is kept parseable by plain docutils, which is now a conservative + choice rather than a hard requirement, because the page it lands on is rendered + by Sphinx. ``examples/benchmark/benchmark.py`` still says ``README.rst`` and is + left alone, because it means the benchmark suite's own README in the same + directory, not the project's. * **Changed** -- ``CODE_OF_CONDUCT.md`` moves from Contributor Covenant 1.4 to Contributor Covenant 3.0, at the maintainer's request. The text is the canonical 3.0 Markdown fetched from diff --git a/docs/source/index.rst b/docs/source/index.rst index 950f4f844..c7d9df2f0 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -29,6 +29,7 @@ construction and analysis library. ext demo + testing pep changelog @@ -263,6 +264,28 @@ Or install the latest version from the gi repository: # and to update at any time git pull +For local development with ``pipenv``, the repository already includes a +:file:`Pipfile` and :file:`Makefile` targets that keep both the virtualenv and +the package caches inside the project directory: + +.. code-block:: shell + + make setup + +This resolves two common local setup issues on macOS/Homebrew installations: +``pipenv`` cache permission errors under :file:`~/Library/Caches`, and ``lxml`` +builds failing to locate Homebrew's ``libxml2``/``libxslt`` headers. + +If you prefer to run ``pipenv`` directly, use the same local cache layout and +skip any stale, user-local :file:`Pipfile.lock`: + +.. code-block:: shell + + PIPENV_VENV_IN_PROJECT=1 \ + PIPENV_CACHE_DIR=$PWD/.pipenv-cache \ + PIP_CACHE_DIR=$PWD/.pip-cache \ + pipenv install --skip-lock --dev + And since :mod:`pcapkit` supports various extraction engines, and extensive plug-in functions, you may want to install the optional ones: diff --git a/docs/source/testing.rst b/docs/source/testing.rst new file mode 100644 index 000000000..95d953a60 --- /dev/null +++ b/docs/source/testing.rst @@ -0,0 +1,53 @@ +Testing +======= + +.. note:: + + This page came out of the project README when that was trimmed down to a + landing page. It is the only prose copy of the fixture story, so it lives here + rather than nowhere. + +Running the tests +----------------- + +The unit tests need nothing beyond the package itself and the sample captures +tracked in the repository: + +.. code-block:: shell + + make test + +The runtime, regression and integration tests additionally read sample captures +that are **not** tracked (see :file:`.gitignore`); +:file:`examples/generators/make_samples.py` reconstructs them into +:file:`examples/captures/`, and ``make test-all`` regenerates them before running +the whole suite: + +.. code-block:: shell + + make samples # write examples/captures/*.pcap and *.pcapng + make test-all # regenerate the fixtures, then run every test + +The same fixtures back the demonstration scripts in +:file:`examples/legacy_smoke/`, which read them as ``../captures/…``. + +Continuous integration runs the ``make test`` selection, since the fixtures are +not in the repository. Wireshark's ``tshark`` is only required to exercise the +:class:`PyShark ` engine, and is not +needed by the test suite. + +The option round-trip generator +------------------------------- + +One of the generators works differently from the rest and is worth knowing about. +:file:`examples/generators/options.py` does not describe packets it wants; it asks +the library which option, chunk, parameter, message, frame and block codes it +registers, and then constructs one of each through the public construction API, +parses it back, and constructs it again from what was parsed. The +:file:`options-*.pcap` captures are the cases that survive that round trip, so +they record what this version of :mod:`pcapkit` builds rather than what a +third-party tool builds. + +:file:`tests/protocols/test_option_roundtrip_unit.py` runs the same cases without +needing a fixture at all, and carries a table of the ones that do not yet close +the cycle, each named against the defect that stops it. diff --git a/examples/benchmark/Dockerfile b/examples/benchmark/Dockerfile index fb8ca327a..de71bdaf5 100644 --- a/examples/benchmark/Dockerfile +++ b/examples/benchmark/Dockerfile @@ -153,7 +153,7 @@ RUN set -eux; \ rm -f /tmp/pypcap-install.log # `pcapkit` last, because it is the thing under test and changes on every commit. -COPY pyproject.toml setup.py MANIFEST.in README.rst ./ +COPY pyproject.toml setup.py MANIFEST.in README.md ./ COPY pcapkit ./pcapkit # --no-deps so the pinned requirements above stay authoritative: `pcapkit` diff --git a/examples/benchmark/README.rst b/examples/benchmark/README.rst index 70e41516c..c5c2d4a7b 100644 --- a/examples/benchmark/README.rst +++ b/examples/benchmark/README.rst @@ -2,9 +2,9 @@ Engine Benchmark Suite ====================== -A containerised benchmark that produces the engine speed table in the project -``README.rst`` -- the whole table, every supported Python version, in one run. One -command, on any machine with Docker, and the output is two paste-ready +A containerised benchmark that produces the engine speed table in the project docs, +``docs/source/index.rst`` -- the whole table, every supported Python version, in one +run. One command, on any machine with Docker, and the output is two paste-ready reStructuredText snippets: .. code-block:: shell @@ -97,7 +97,7 @@ precisely the condition under which absolute times are comparable -- with each o A ratio there would be worse than redundant: ratios are normalised *within* one environment, so dividing across two interpreters yields a number that is neither engine's speed nor the interpreter's. The snippet says so in its own prose, because -the person reading the table in ``README.rst`` later is not the person who read this +the person reading the table in the docs later is not the person who read this file. **The ratio table is normalised to the default engine measured in the same @@ -115,7 +115,7 @@ engine's own milliseconds per packet. That is what lets a reader convert the rat back into times on the machine the run happened on. Both snippets carry their own ``Test Results`` heading -- the per-version one takes -the name, since it is what ``README.rst``'s **Test Results** section is, and the ratio +the name, since it is what the docs' **Test Results** section is, and the ratio one is ``Test Results (Relative)`` so that pasting both does not produce two sections with one name. The ``Test Environment`` block, which serves both, is in ``table.rst``. @@ -377,7 +377,7 @@ untracked): * - File - What it is * - ``table-versions.rst`` - - absolute ms/packet per engine and Python version -- ``README.rst``'s **Test + - absolute ms/packet per engine and Python version -- the docs' **Test Results** section, verbatim * - ``table.rst`` - ``Test Environment`` plus the machine-independent ratio table @@ -491,9 +491,12 @@ baseline, that machine drift cancels, that repeats are paired one to one rather averaged first, that the environments stitch together correctly, that an unmeasured engine keeps its reason, that the overlap marking neither over- nor under-claims, and that the emitted reStructuredText parses under **plain docutils** with no warnings and -uses no Sphinx-only roles. That last one matters because GitHub renders -``README.rst`` with docutils, where a ``:mod:`` role comes out as a visible error -block while looking perfectly fine in the project's own Sphinx docs. +uses no Sphinx-only roles. That last one is now a conservative choice rather than a +hard requirement: the snippets land in ``docs/source/index.rst``, which Sphinx +renders, so a ``:mod:`` role would survive there. Keeping them docutils-clean is what +lets a table be pasted into any reStructuredText a plain docutils reader will see -- +where a ``:mod:`` role comes out as a visible error block while looking perfectly fine +in the project's own Sphinx docs. The matrix adds its own statements to check, since most of the ways a version column can be wrong are silent: that a document lands in the column its *interpreter* names diff --git a/examples/benchmark/report.py b/examples/benchmark/report.py index f5b3e2cec..a098460c8 100644 --- a/examples/benchmark/report.py +++ b/examples/benchmark/report.py @@ -39,10 +39,15 @@ whose ranges overlap are marked: this run does not establish which of them is faster, and reporting their medians as though it did would be inventing precision. -The emitted markup is plain reStructuredText for **docutils**, which is what -renders the project's README on GitHub. No Sphinx-only roles (``:mod:``, -``:func:``, ``:manpage:``) appear in it -- only double-backtick literals -- because -docutils does not know them and renders them as errors. +The emitted markup is plain reStructuredText for **docutils**. No Sphinx-only +roles (``:mod:``, ``:func:``, ``:manpage:``) appear in it -- only double-backtick +literals -- because docutils does not know them and renders them as errors. + +The snippets land in :file:`docs/source/index.rst`, which Sphinx renders, so a +Sphinx-only role would in fact survive there. Staying docutils-clean is kept +deliberately anyway: it is what lets a table be pasted into any reStructuredText +a plain docutils reader will see, the project README having been one such +destination until it became Markdown. """ @@ -61,7 +66,7 @@ 'unmeasured_by_version', 'version_columns'] #: Engine name reported for ``pcapkit``'s own parser. ``pcapkit.extract`` accepts -#: ``'default'``; the README and the docs call it ``pcapkit``, and that is the +#: ``'default'``; the documentation calls it ``pcapkit``, and that is the #: spelling the table uses. BASELINE = 'default' @@ -70,7 +75,8 @@ #: Marks a row whose observed range overlaps another row's, i.e. a gap this run #: does not resolve. A double dagger rather than a footnote reference, so the -#: emitted snippet cannot collide with footnote numbers already in the README. +#: emitted snippet cannot collide with footnote numbers already on the page it is +#: pasted into. OVERLAP_MARK = '‡' #: Marks a row that was not measured, and points at the note carrying the reason. @@ -109,7 +115,7 @@ def _escape(text: 'str') -> 'str': * anything ending ``::`` -- *literal block expected; none found*. Each of those turns the snippet this module promises can be pasted into - :file:`README.rst` verbatim into a visible error block on GitHub, and the failure + :file:`docs/source/index.rst` verbatim into a visible error block, and the failure is invisible here because the harness's own fixtures are all well-behaved English. Args: @@ -285,8 +291,8 @@ def _fixed(value: 'float', decimals: 'int' = 4) -> 'str': number of decimal places -- ``0.01700`` above ``14.74`` -- which is precisely the layout that makes a column impossible to scan. Fixed places line the decimal points up, and they are also what the table this replaces in - :file:`README.rst` has always used, so a regenerated table is a diff of the - numbers rather than a diff of the formatting. + :file:`docs/source/index.rst` has always used, so a regenerated table is a diff + of the numbers rather than a diff of the formatting. Args: value: Number to format. @@ -729,7 +735,7 @@ def _provenance(documents: 'Sequence[dict[str, Any]]', image: 'Optional[str]', Deliberately excludes anything identifying the host it ran on -- no hostname, no kernel, no CPU model. Containerising the benchmark is what makes the host irrelevant, and printing host details would undo that while also making the - output awkward to paste into a public README. The architecture *is* included, + output awkward to paste into public documentation. The architecture *is* included, because it changes the numbers and identifies nothing. Args: @@ -879,7 +885,7 @@ def render_versions_rst(rows: 'Sequence[Row]', documents: 'Sequence[dict[str, An emulated: 'Optional[str]' = None) -> 'str': """Render the per-version table of absolute milliseconds per packet. - This is the snippet that replaces :file:`README.rst`'s **Test Results** section + This is the snippet that replaces :file:`docs/source/index.rst`'s **Test Results** section outright: engines down the side, Python versions across the top, ``--`` for a cell that could not be measured. It is self-contained -- heading, prose, table, notes -- because it is pasted as a unit, and a note explaining a gap is no use in a @@ -1055,7 +1061,7 @@ def render_rst(rows: 'Sequence[Row]', documents: 'Sequence[dict[str, Any]]', lines.append('') # A heading of its own rather than "Test Results", which is the per-version - # absolute table's -- both snippets go into the same README, and two sections + # absolute table's -- both snippets go onto the same page, and two sections # with one name is a document where a reader cannot say which table a sentence is # about. heading = 'Test Results (Relative)' @@ -1358,7 +1364,7 @@ def main(argv: 'Optional[list[str]]' = None) -> 'int': snippet = render_rst(rows, documents, args.image, args.emulated, missing) print('-' * 78) - print('reStructuredText below, ready to paste into README.rst') + print('reStructuredText below, ready to paste into docs/source/index.rst') print('-' * 78) print() print(versions_snippet) diff --git a/examples/benchmark/run.sh b/examples/benchmark/run.sh index 9184595b6..92bb019c2 100755 --- a/examples/benchmark/run.sh +++ b/examples/benchmark/run.sh @@ -54,7 +54,7 @@ usage() { usage: run.sh [options] Builds one benchmark image per Python version and runs them all, printing a -summary and two paste-ready reStructuredText tables for README.rst: absolute +summary and two paste-ready reStructuredText tables for the docs: absolute milliseconds per (engine, Python version), and engine ratios pooled across the whole matrix. @@ -456,7 +456,7 @@ docker cp "${REPORTER}:/out/." "${OUT_DIR}/" >/dev/null 2>&1 || status=1 echo >&2 if [ -f "${OUT_DIR}/table-versions.rst" ] && [ -f "${OUT_DIR}/table.rst" ]; then echo "==> wrote ${OUT_DIR}/table-versions.rst -- absolute milliseconds per engine" >&2 - echo " and Python version, ready to paste into README.rst's Test Results," >&2 + echo " and Python version, ready to paste into the docs' Test Results," >&2 echo " and ${OUT_DIR}/table.rst -- the machine-independent ratio view," >&2 echo ' alongside the raw JSON and the per-environment locks' >&2 diff --git a/examples/benchmark/test_harness.py b/examples/benchmark/test_harness.py index f0ae156b7..4949a63a2 100644 --- a/examples/benchmark/test_harness.py +++ b/examples/benchmark/test_harness.py @@ -17,8 +17,9 @@ * the overlap marking, which is what stops the report claiming a gap it did not measure; * that the emitted reStructuredText parses under **plain docutils** with no - errors and uses no Sphinx-only roles -- the README is rendered by docutils on - GitHub, where ``:mod:`` and friends come out as visible errors. + errors and uses no Sphinx-only roles, so that a table stays pasteable into any + reStructuredText a plain docutils reader will see, where ``:mod:`` and friends + come out as visible errors. Run with ``python -m pytest examples/benchmark/test_harness.py``. Nothing here needs docker, and only the driver-assertion tests need ``pcapkit`` importable. @@ -361,7 +362,7 @@ def test_an_identical_range_shows_one_value(self): class TestRenderedMarkup: - """What actually gets pasted into the README.""" + """What actually gets pasted into the documentation.""" def _snippet(self, emulated=None): """Render a full snippet from a two-environment run.""" @@ -474,7 +475,7 @@ def test_provenance_is_present_and_host_free(self): 'Passes', 'CPython 3.11.14', 'in.pcap', '1000', 'abc1234'): assert expected in snippet # The containerised design is what makes the host irrelevant; printing host - # details would undo it, and would leak machine identity into a public README. + # details would undo it, and would leak machine identity into public docs. import platform as host_platform # pylint: disable=import-outside-toplevel for leak in (host_platform.node(), host_platform.release(), host_platform.version()): if leak: @@ -897,7 +898,7 @@ def test_environments_disagreeing_about_one_cell_report_both(self): class TestVersionsMarkup: - """The per-version table as it will be pasted into README.rst.""" + """The per-version table as it will be pasted into docs/source/index.rst.""" def _snippet(self, missing=(), emulated=None, **kwargs): """Render the per-version snippet from a whole matrix run.""" @@ -1181,7 +1182,7 @@ def test_the_sample_column_is_not_called_passes(self): assert 'Passes' in snippet.split('Test Results (Relative)')[0] def test_the_two_tables_do_not_share_a_heading(self): - """Two sections named "Test Results" in one README is one too many. + """Two sections named "Test Results" on one page is one too many. Both snippets are pasted into the same document, and a reader then has no way to say which table a sentence underneath is about. @@ -1309,9 +1310,9 @@ def _documents(self, reason): def test_both_snippets_still_parse(self, reason): """A gcc diagnostic in a reason must not break the pasted table. - The promise is that these snippets go into README.rst verbatim, and GitHub - renders that with docutils -- where each of these strings produces a visible - error block instead of the table. + The promise is that these snippets go into docs/source/index.rst verbatim, + and that they stay parseable by plain docutils -- where each of these strings + produces a visible error block instead of the table. """ docutils_core = pytest.importorskip('docutils.core') @@ -1411,7 +1412,7 @@ def test_four_decimal_places_whatever_the_magnitude(self, value, expected): Significant figures -- which every other number in the report uses -- would give ``0.01700`` and ``14.74`` different numbers of decimal places in the same - column. Four places is also what the hand-maintained table in README.rst has + column. Four places is also what the hand-maintained table in the docs has always used, so a regenerated table diffs its numbers rather than its layout. """ diff --git a/setup.py b/setup.py index e60c56fa9..9b99e1a57 100644 --- a/setup.py +++ b/setup.py @@ -29,8 +29,8 @@ def get_long_description() -> "str": - """Extract description from README.rst, for PyPI's usage.""" - readme = os.path.join(os.path.dirname(os.path.abspath(__file__)), "README.rst") + """Extract description from README.md, for PyPI's usage.""" + readme = os.path.join(os.path.dirname(os.path.abspath(__file__)), "README.md") with open(readme, encoding="utf-8") as file: long_description = file.read() return long_description @@ -178,5 +178,5 @@ def run(self) -> "None": setup( cmdclass=cmdclass, long_description=get_long_description(), - long_description_content_type="text/x-rst", + long_description_content_type="text/markdown", )