Skip to content

Gate CI on lint, types, and a coverage floor - #290

Merged
seanwevans merged 1 commit into
mainfrom
claude/ci-lint-gate
Aug 15, 2026
Merged

Gate CI on lint, types, and a coverage floor#290
seanwevans merged 1 commit into
mainfrom
claude/ci-lint-gate

Conversation

@seanwevans

Copy link
Copy Markdown
Owner

The problem

CI ran only tests. The repo has a complete .pre-commit-config.yaml — isort, black, pylint, flake8, mypy — but nothing enforced it outside a contributor's own machine. And the coverage job ran the suite, generated coverage.xml, uploaded it as an artifact, and never looked at the number. Both could regress silently, and one already had.

The change

A lint job that runs the pre-commit hooks at their pinned versions. Using pre-commit rather than re-listing the tools means CI and pre-commit install cannot drift apart about what passes. pytest is skipped in that job via SKIP=pytest — the matrix jobs already run the suite across six Python/OS combinations rather than once.

A coverage floor. Two parts:

  • Scope measurement to the package ([tool.coverage.run] source = ["pyisolate"]). Previously the report also counted the test files, which are ~100% by construction and inflate the total into a number that cannot meaningfully regress.
  • fail_under = 70, and the CI job now runs coverage report so the threshold is actually checked.

70 is deliberately below the current 73%: kernel-gated tests (Landlock, BPF, cgroup) skip on hosts lacking those features, so a floor set at the observed value would flake depending on the runner. It's a floor, not a target. The XML upload moved to if: always() so a regression is still inspectable when the check fails.

The gate immediately found drift on main

Which is rather the point:

  • tests/test_landlock.py — an import block isort wants collapsed to one line
  • tests/test_ebpf_contract.py — three statements over the line limit that black reformats

Both are fixed in this PR so the new job starts green. Neither is a behavior change.

Verified locally

pre-commit run --all-files (SKIP=pytest)  → isort/black/pylint/flake8/mypy all Passed
coverage report                            → TOTAL 73%, exit 0
coverage report --fail-under=95            → exit 2   (floor is wired, not decorative)
pytest -m "not soak"                       → 505 passed, 6 skipped

mypy is clean across 46 source files and pylint is 9.17 (above the existing fail-under = 8.0), so this gate reflects where the repo already is rather than imposing new debt.

Follow-up worth considering separately

Coverage is inverted: enforcement code is the least tested (child.py 27%, confine.py 45%, landlock.py 57%, thread.py 55%) while telemetry sits at 100%. A repo-wide floor doesn't capture that — per-module floors on the security-critical modules would, but that's a bigger conversation than this PR.


Generated by Claude Code

CI ran only tests. The repo has a full pre-commit config -- isort, black,
pylint, flake8, mypy -- but nothing enforced it outside a contributor's
own machine, and the coverage job measured coverage, uploaded the XML, and
never checked it. Both could regress silently.

Add a lint job that runs the pre-commit hooks at their pinned versions, so
CI and `pre-commit install` cannot disagree about what passes. pytest is
skipped in that job because the matrix jobs already run the suite.

Give coverage a floor. Scope measurement to the package (previously the
report also counted the test files, which are ~100% by construction and
inflate the total into something that cannot regress meaningfully) and set
fail_under to 70, below the current 73% so kernel-gated tests that skip on
hosts without Landlock/BPF/cgroup do not turn the floor into a flake. The
XML upload now runs even when the check fails, so a regression is still
inspectable.

The gate found existing drift on main, which is the argument for having it:
tests/test_landlock.py had an import block isort wanted to collapse and
tests/test_ebpf_contract.py had three statements over the line limit. Both
are fixed here so the new job starts green.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ebvMQ3vLxdK3joymz6Feg
@seanwevans
seanwevans merged commit b8a065d into main Aug 15, 2026
10 of 19 checks passed
@seanwevans
seanwevans deleted the claude/ci-lint-gate branch August 15, 2026 21:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants