Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 0 additions & 10 deletions .claude/settings.json

This file was deleted.

8 changes: 4 additions & 4 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,13 +14,13 @@ jobs:
id-token: write # mandatory for trusted publishing

steps:
- uses: actions/checkout@v6
- uses: actions/setup-python@v6
- uses: actions/checkout@v7
- uses: actions/setup-python@v7
with:
python-version: "3.14"

- uses: astral-sh/setup-uv@v7
- run: uvx --with tox-uv tox -e py,style
- uses: astral-sh/setup-uv@v10.2.0
- run: uvx --with tox-uv tox -e py,style,ty,pyrefly,pyright,mypy,zuban
- run: uv build
- name: Publish to PyPI
uses: pypa/gh-action-pypi-publish@release/v1
14 changes: 7 additions & 7 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,12 +21,12 @@ jobs:
python-version: ["3.10", "3.11", "3.12", "3.13", "3.14"]

steps:
- uses: actions/checkout@v6
- uses: actions/setup-python@v6
- uses: actions/checkout@v7
- uses: actions/setup-python@v7
with:
python-version: ${{ matrix.python-version }}

- uses: astral-sh/setup-uv@v7
- uses: astral-sh/setup-uv@v10.2.0
- run: uvx --with tox-uv tox -e py

- uses: coverallsapp/github-action@v2
Expand All @@ -50,10 +50,10 @@ jobs:
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v6
- uses: actions/setup-python@v6
- uses: actions/checkout@v7
- uses: actions/setup-python@v7
with:
python-version: "3.14"

- uses: astral-sh/setup-uv@v7
- run: uvx --with tox-uv tox -e style
- uses: astral-sh/setup-uv@v10.2.0
- run: uvx --with tox-uv tox -e style,ty,pyrefly,pyright,mypy,zuban
24 changes: 24 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
# Changelog

## v2.1.0

- portable-python's code is now checked by multiple type checkers on every change
- Bumped SQLite to 3.53.4 and gettext-tiny to 0.3.3; re-checked every other component against upstream (all already current, or deliberately held back — see the `version` property comments).
- Bug fixes:
- With `cpython-use-github: true`, a failed GitHub API call (e.g. rate limiting) crashed with a stack trace
- A build where `bin/python` could not be found crashed (instead of reporting it) when `cpython-validate-script` or `cpython-additional-packages` was configured
- Clearer error messages when the latest python version can't be determined, or a python family isn't supported
- A `{module}-configure` setting with several options was passed to `./configure` as one single argument
- Dry-run (`-n`) could modify files of an existing build (source patches, shebangs, permissions), a configured patch on a missing file is now reported
- Library use: `BuildSetup()` now takes a `PythonSpec`, get one via `required_python_spec("cpython:3.13.2")` (which validates it, `None` means latest)
- Tests cover more edge cases

## v2.0.2

- Corrected `readline` static linking

## v2.0.1

- Bumped the bundled components (OpenSSL, SQLite, Tcl/Tk, …) to current versions.
- Refreshed dependencies and modernized CI (now on uv / tox-uv, with updated GitHub Actions).
- Added an OKF documentation bundle under `docs/` (the old top-level docs folded into it).
2 changes: 1 addition & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,4 +22,4 @@ The authoritative, self-contained documentation is the OKF knowledge bundle unde

- Check **runez** before reimplementing file / system / CLI / logging / version helpers — see [`docs/guides/local-development.md`](docs/guides/local-development.md).
- The docs use the OKF format; keep the bundle conformant (`scripts/check_okf.py docs`) when you edit it.
- As changes land, keep a top section in [`docs/changelog.md`](docs/changelog.md) for the upcoming release — `## Unreleased`, or the next version directly once it's known. Maintain it as PRs come in; it's double-checked at release. Don't fuss over marking released-vs-unreleased (small audience, `git tag` says what's out).
- As changes land, keep a top section in [`CHANGELOG.md`](CHANGELOG.md) for the upcoming release — `## Unreleased`, or the next version directly once it's known. Maintain it as PRs come in; it's double-checked at release. Don't fuss over marking released-vs-unreleased (small audience, `git tag` says what's out).
1 change: 1 addition & 0 deletions MANIFEST.in
Original file line number Diff line number Diff line change
@@ -1,2 +1,3 @@
include LICENSE
include README.rst
include CHANGELOG.md
4 changes: 2 additions & 2 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -193,9 +193,9 @@ Portable Python can be used as a python library to invoke builds, or inspect an

Invoke a build from python code::

from portable_python import BuildSetup
from portable_python import BuildSetup, required_python_spec

setup = BuildSetup("cpython:3.9.7")
setup = BuildSetup(required_python_spec("cpython:3.9.7"))
setup.compile()


Expand Down
11 changes: 0 additions & 11 deletions SECURITY.md

This file was deleted.

6 changes: 3 additions & 3 deletions docs/architecture/build-setup.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,9 @@ timestamp: 2026-06-23T00:00:00Z
The build orchestrator (`__init__.py`). The [`build`](/cli/build.md) command — and library users — construct one from a python spec and call `compile()`:

```python
from portable_python import BuildSetup
from portable_python import BuildSetup, required_python_spec

BuildSetup("cpython:3.13.2").compile()
BuildSetup(required_python_spec("cpython:3.13.2")).compile()
```

## Mental model
Expand All @@ -26,4 +26,4 @@ It owns the [`Folders`](/architecture/config.md) (resolved paths) and the python

- **Start reading at `compile()`** — it calls everything else in order.
- A `--prefix` makes the build **non-portable**; without it the result is relocatable (see [ppp-marker](/concepts/ppp-marker.md) and [portability](/concepts/portability.md)).
- It requires a full `X.Y.Z` spec — a bare `3.13` is rejected; `"latest"` (or empty) resolves to the newest known version.
- It takes a `PythonSpec` with a full `X.Y.Z` version: `required_python_spec()` validates that (a bare `3.13` is rejected) and resolves `"latest"` (or empty) to the newest known version.
21 changes: 0 additions & 21 deletions docs/changelog.md

This file was deleted.

6 changes: 3 additions & 3 deletions docs/concepts/telltale-detection.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,15 +12,15 @@ Before compiling, the tool must decide, for each candidate [external module](/mo

## The `m_telltale` attribute

Each [`ModuleBuilder`](/architecture/module-builder.md) subclass declares `m_telltale`, a path (or list of paths) using an `{include}` placeholder that is expanded against the target's system include directories:
Each [`ModuleBuilder`](/architecture/module-builder.md) subclass declares `m_telltale`, a path (or several, space-separated) using an `{include}` placeholder that is expanded against the target's system include directories:

```python
class Openssl(ModuleBuilder):
m_telltale = "{include}/openssl/ssl.h"


class Sqlite(ModuleBuilder):
m_telltale = ["{include}/sqlite3.h"]
class LibFFI(ModuleBuilder):
m_telltale = "{include}/ffi.h {include}/ffi/ffi.h"
```

`PPG.find_telltale()` expands the placeholder against each `target.sys_include` directory and returns the first existing path, or `None`.
Expand Down
2 changes: 1 addition & 1 deletion docs/configuration/portable-python-yml.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ Replace `{module}` with the lowercased module name (e.g. `openssl`):
| `{module}-version` | Version to build. |
| `{module}-url` | Source URL (`$version` is substituted; may carry a `#sha256=…` fragment, verified on download). |
| `{module}-src-suffix` | Archive extension when the URL lacks one. |
| `{module}-configure` | **Replace** the default configure args (`$deps_lib` substituted). |
| `{module}-configure` | **Replace** the default configure args (space-separated, shell-style quoting; `$lib_dir` and `$lib64_dir` substituted). |
| `{module}-http-headers` | Headers for the download (env vars expanded). |
| `{module}-patches` | Source patches to apply before compiling. |
| `{module}-debian` | Debian package name for dependency detection. |
Expand Down
4 changes: 2 additions & 2 deletions docs/guides/ci-cd.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,13 +18,13 @@ Runs on every push to `main` and every pull request targeting `main`, with `canc
|-----|--------------|
| `test` | Matrix over Python `3.10`–`3.14` on `ubuntu-latest` (`fail-fast: false`). Runs `uvx --with tox-uv tox -e py`, then uploads coverage to Coveralls as a **parallel** build, flagged per version. |
| `coveralls-finish` | After all `test` jobs finish, signals Coveralls that the parallel build is complete. |
| `linters` | On Python `3.14`, runs `uvx --with tox-uv tox -e style` (ruff check + format diff). |
| `linters` | On Python `3.14`, runs `uvx --with tox-uv tox -e style,<typecheckers>` (ruff check + format diff, then the type checkers, see [type checker findings](/typecheck-findings.md)). |

## Release (`release.yml`)

Triggers on pushing a tag matching `v[0-9]*`. The single `publish` job (Python `3.14`, GitHub environment `release`):

1. `uvx --with tox-uv tox -e py,style` — full test + lint gate.
1. `uvx --with tox-uv tox -e py,style,<typecheckers>` — full test + lint + type check gate.
2. `uv build` — build the sdist + wheel.
3. `pypa/gh-action-pypi-publish` — publish to PyPI via **trusted publishing** (OIDC, `id-token: write` — no API token is stored).

Expand Down
9 changes: 5 additions & 4 deletions docs/guides/local-development.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,10 +21,11 @@ uv sync
The suite mocks `runez.run()` and builds in `--dryrun` mode, so it asserts the *planned* commands without actually compiling. Target is 100% coverage.

```shell
tox # all envs (py310-314, coverage, style)
tox -e py313 # one python version
tox -e style # lint check (ruff)
tox -e reformat # auto-fix formatting
tox # all envs (py310-314, coverage, style)
tox -e py313 # one python version
tox -e style # lint check (ruff)
tox -e reformat # auto-fix formatting
tox -e <typechecker> # one tox env per type checker, see tox.ini

.venv/bin/pytest tests/ # directly
.venv/bin/pytest tests/test_build.py::test_build_rc -vv # a single test
Expand Down
4 changes: 2 additions & 2 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ Start with the [Overview](/overview.md), then drill into the area you need.
* [Configuration](/configuration/index.md) - The `portable-python.yml` file, its sections and overrides.
* [Guides](/guides/index.md) - Playbooks for common tasks: building, extending, local development.
* [Release process](/release.md) - How to cut a release: changelog highlights, then a version tag.
* [Changelog](/changelog.md) - High-level release highlights, newest first.
* [Type checker findings](/typecheck-findings.md) - What introducing the type checkers found and fixed.

## Conventions

Expand All @@ -29,4 +29,4 @@ Start with the [Overview](/overview.md), then drill into the area you need.
## Authority

This bundle is the **authoritative** documentation for portable-python's internals, configuration, CLI, and development workflows.
The repository's `README.rst` remains the project's landing page (install, quick start, motivation).
The repository's `README.rst` remains the project's landing page (install, quick start, motivation), and its `CHANGELOG.md` has the high-level release highlights, newest first.
14 changes: 13 additions & 1 deletion docs/log.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,24 @@
# Knowledge Bundle Update Log

## 2026-10-09

* **Update**: [Configuration](/configuration/portable-python-yml.md): `{module}-configure` substitutes `$lib_dir` and `$lib64_dir` (not `$deps_lib`), and its value is split into separate args.
* **Creation**: Added the [type checker findings](/typecheck-findings.md) report.
* **Update**: The changelog moved out of the bundle, to `CHANGELOG.md` at the repo root (shipped in the sdist, like `README.rst`); [release process](/release.md) and the index point there.
* **Update**: [CI/CD](/guides/ci-cd.md): the `linters` job and the release gate run the type checkers.
* **Update**: `BuildSetup` takes a `PythonSpec` (via `required_python_spec()`) in [BuildSetup](/architecture/build-setup.md), and `m_telltale` is a space-separated `str` in [telltale detection](/concepts/telltale-detection.md).

## 2026-10-08

* **Update**: Listed the type checker tox envs (`tox -e <typechecker>`) in [local development](/guides/local-development.md).

## 2026-08-20

* **Update**: Bumped SQLite to 3.53.4 and gettext-tiny to 0.3.3, and recorded the stay-put rationale for LibFFI, OpenSSL and Readline in their `version` property comments (per the [bump components playbook](/guides/bump-components.md)).

## 2026-06-23

* **Creation**: Added the [release process](/release.md) playbook and the [changelog](/changelog.md).
* **Creation**: Added the [release process](/release.md) playbook and the changelog.
* **Creation**: Added the [bump component versions](/guides/bump-components.md) guide.
* **Update**: Pulled two verified facts in from the project wiki — the relativization tools (`patchelf` / `install_name_tool`) and `--prefix` / `--enable-shared` support ([PythonInspector](/architecture/python-inspector.md)), and the `#sha256=` download-checksum URL fragment ([configuration](/configuration/portable-python-yml.md)). Skipped the wiki's `sources:` download-url design — it's an unimplemented draft.
* **Update**: Trimmed the CLI pages — dropped the per-command option tables (which mirror `--help`) for purpose + non-obvious gotchas + examples, pointing at `--help` for the full flag list.
Expand Down
2 changes: 1 addition & 1 deletion docs/release.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ Review what changed since the last release:
git diff <last-tag>..main # e.g. git diff v2.0.0..main
```

Summarize the **highlights** in [`changelog.md`](/changelog.md) — short and high-level. It should
Summarize the **highlights** in `CHANGELOG.md` (at the repo root) — short and high-level. It should
give a reader a rough idea of the main changes since the last release, not a blow-by-blow; skip the
details.

Expand Down
111 changes: 111 additions & 0 deletions docs/typecheck-findings.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
---
type: Report
title: Type checker findings
description: What introducing multiple type checkers found in v2.0.2, and what v2.1 changed in response (bugs, design, minor).
tags: [report, type-checking, ci, quality]
timestamp: 2026-10-09T00:00:00Z
---

# Type checking: v2.0.2 → v2.1

What v2.1 changed in response to type checker findings on v2.0.2.

| Checker | Version | v2.0.2 | v2.1 | Actionable | Bugs | Design | Minor |
| --- | --- | ---: | ---: | ---: | ---: | ---: | ---: |
| ty | 0.0.85 | 51 | 0 | 51 | 0 | 5 | 6 |
| pyrefly | 1.3.2 | 96 | 0 | 96 | 1 | 6 | 7 |
| pyright | 1.1.414 | 93 | 0 | 93 | 1 | 6 | 8 |
| mypy | 2.4.0 | 32 | 0 | 32 | 0 | 3 | 4 |
| basedpyright | 1.40.2 | 2,979 | 2,576 | 646 | 1 | 6 | 8 |
| zuban | 0.10.0 | 38 | 0 | 38 | 2 | 4 | 5 |

- **v2.0.2**, **v2.1**: findings on that release's `src/` and `tests/`, both checked with v2.1's
settings (`tox.ini`, `pyproject.toml`). "v2.0.2" is commit `c298694`: v2.0.2 plus the component
bumps that preceded this work.
- **Actionable**: v2.0.2 findings resolved by a change in v2.1.
- **Bugs**, **Design**, **Minor**: how many of the changes listed under [Bug fixes](#bug-fixes),
[Design](#design) and [Minor](#minor) the checker spotted.
- basedpyright counts include its warnings: 110 errors + 2,869 warnings on v2.0.2, 5 + 2,571 on
v2.1. Its 5 remaining errors are import cycles (`__init__` ↔ `versions` ↔ `cpython` ↔
`inspector`). basedpyright is available locally (`tox -e basedpyright`), see [CI/CD](/guides/ci-cd.md)
for which checkers gate CI.
- v2.1's zeros need runez 5.10.1, which fixed an annotation portable-python tripped on (see
[Fixed in runez](#fixed-in-runez)).
- v2.0.2 had 3 `# type:` comments, v2.1 has none.

## Bug fixes

A checker pointed at code that misbehaved.

- With `cpython-use-github: true`, a failed GitHub API call (rate limiting, network error) crashed
with a `TypeError`: `RestClient.get()` returns `None` on failure, and the response was iterated
as-is [zuban, once `get_available_versions()` had a return annotation: mypy-style checkers skip
the body of unannotated functions]
- When `bin/python` couldn't be found after `make install`, a configured `cpython-validate-script`
or `cpython-additional-packages` ran `None` as the python executable before the "Can't find
bin/python" check happened: `bin_python` is now always a `Path`, and aborts with that message on
first use [pyrefly, pyright, basedpyright, zuban]

## Design

portable-python's invariants are now stated in its types, and enforced where they used to be
assumed.

- `BuildSetup` takes a `PythonSpec`, the new `required_python_spec()` turns CLI text (or
`None`/`"latest"`) into one, with a clear error when it isn't a full `X.Y.Z`
version [ty, pyrefly, pyright, basedpyright]
- `VersionFamily` is an ABC (`get_available_versions()` and `get_builder()` are abstract): `latest`
is always a `Version`, aborting with "Could not determine latest cpython version" instead of the
puzzling "Invalid python spec: None", and `available_versions` is always a dict [all six]
- `Folders`: `build`, `destdir`, `sources` and `ppp-marker` are required (clear error when not
configured), only `dist` and `logs` are optional, and folders are `Path`s [pyrefly, pyright,
basedpyright, zuban]
- `PPG.family()` always aborts for an unknown family (its `fatal=` parameter is gone) [all six]
- `m_telltale` is a plain `str` (several files are space-separated), instead of a mix of `str` and
`ClassVar[list]` [all six]
- Library tracking: `Trackable` is an ABC declaring `relative_path` and `tracked_category`,
`LibType` moved to `tracking.py` (the only enum `Tracker` was ever used with), and
`Tracker.report()` no longer overrides `represented()` with a different return type [ty, pyrefly,
pyright, basedpyright]

## Fixed in runez

portable-python tripped on this runez annotation, fixed in runez 5.10.1 rather than worked around.

- `to_path(None)` returns `None`: it always did at runtime, but runez 5.10.0's annotation only
allowed `str | Path` [all six]

## Removed

A checker flagged code that turned out not to be needed.

- The `__main__` block of `cli.py` re-imported `main` from its own module [ty, pyrefly, zuban]

## Minor

The checker was right, the fix changes little in practice.

- Attributes say what they hold, including `None` (`ModuleBuilder.m_*`, `parent_module`,
`ModuleCollection`, `_bin_python`, `BuildSetup.build_context`, ...) [all six]
- Base class methods declare their return type, so overrides agree (`url`, `auto_select_reason()`,
`candidate_modules()`, `get_problem()`, `xenv_LDFLAGS()`) [all six]
- `config.py`: `ConfigSource.get_value()` walks the key in a loop, `SingleFileMatch` sets its
attributes in `__init__`, and `is_match()` returns a `bool` [all six]
- `# type:` comments became annotations [pyrefly, pyright, basedpyright, zuban]
- Inspector: report lists hold strings, `relative_path()` returns a `str`, no `x and x.y`
chains [pyrefly, mypy, zuban]
- Version listing checks `v.patch` truthiness instead of `v.patch > 0` [ty, pyrefly, pyright,
basedpyright]
- Tests: no poking at `Config._sources`, the mock's patcher is created in `__init__` [ty, pyrefly,
pyright, basedpyright]
- `runez.ActivateColors` instead of `runez.colors.ActivateColors` [pyright, basedpyright]
- `recompress` takes family and version from a spec only when the path names one [pyright,
basedpyright]
- `run_make(cpu_count=...)` is annotated [ty]

## Not from a type checker

- Folder paths containing a space are refused by portable-python itself, with the reason spelled
out ("escapes too complex with makefiles, shell scripts etc"), instead of relying on a niche
runez helper
- New tests: missing `bin/python`, python.org listing no versions, folder path with a space
Loading
Loading