Skip to content

ci(python): run GraalPy and free-threaded CPython as experimental, non-blocking test legs (#1660) - #1725

Merged
JarryShaw merged 1 commit into
mainfrom
ci/1660-experimental-tier
Oct 14, 2026
Merged

JarryShaw merged 1 commit into
mainfrom
ci/1660-experimental-tier

Conversation

@JarryShaw

Copy link
Copy Markdown
Owner

Please follow the guide below

What is the purpose of your pull request?

Tick the commit type your subject line carries.

  • fix — corrects a defect
  • feat — adds a feature
  • perf — changes performance, not behaviour
  • refactor — changes neither behaviour nor performance
  • test — tests only
  • docs — documentation only
  • ci — workflows or build tooling
  • release — bumps the version or rolls up a distribution
  • chore — anything else

Description of your pull request and other information

Part of #1660, the owner's option 1: GraalPy and free-threaded CPython run as experimental, non-blocking test legs, to be promoted only once green.

  • Tier: .github/python-versions.toml gains an experimental tier with 3.14t and graalpy-25.4, spelt as setup-python v7 reads them. util/python_versions.py gives each of them one test leg and nothing more: no integration, Compat, wheel or conda leg, and no required context.
  • Non-blocking:
    • The legs are named Python <version> (experimental) and run on the KTLO mechanics: the step guard, warning annotations and an Experimental summary step.
    • On these legs only, setup-python, install and tests are capped at 5, 10 and 25 minutes. A hung step therefore fails under continue-on-error, instead of the 45-minute job cancel that Required checks passed refuses (actions/runner StepsRunner.cs:322-353).
    • Every other leg's caps read 360, so its job cap binds as before.
  • Updater: experimental lines are hand-maintained, and update_python_versions.py keeps them as written.
  • util/ci_timing.py: it now reads a step cap written as an expression as no cap, where it used to crash on int().
  • Docs: the tiers section and step caps in workflows.rst. The line citations in workflows.rst and releasing.rst follow the moved lines.
  • Tests:
    • The tier, KTLO-leg, updater, unit-tests-queue, CI-timing and release-gate tests pass. The new ones fail on main.
    • Mutations of the caps are each caught.
    • actionlint reports no new finding.

…n-blocking test legs (#1660)

- python-versions.toml: an `experimental` tier, listed last in its table:
  free-threaded "3.14t" and a GraalPy table with "graalpy-25.4", spelt as
  setup-python's python-version reads them.
- python_versions.py: load() takes the tier for those two builds only, and
  refuses them any other tier until they have a wheel, a conda build and a
  Compat leg; `test` gains one leg each with `experimental: true`, and no
  other output (integration, Compat, parity, engines, wheels, conda) moves.
- unit-tests.yml: the legs are named `Python <version> (experimental)` and
  never block, by the KTLO mechanics: the step guard holds on both tiers,
  annotations are warnings, and an `Experimental summary` step reports each
  failed step. No Compat leg is experimental, so none is a required context.
  Their setup-python, install and test steps are capped at 5, 10 and 25
  minutes, so a slow leg fails a step instead of being cancelled at the
  job's 45; every other leg's caps are 360, so its job cap binds as before.
- ci_timing.py: a step cap written as an expression reads as none.
- update_python_versions.py: experimental entries are hand-maintained; every
  recompute keeps their lines as written, after the computed versions, and
  the proposal fingerprint leaves them out.
- workflows.rst: the tiers section, the step caps and the `test` leg count;
  unit-tests.yml line citations there and in releasing.rst follow the lines.

Tests: tests/project/test_python_versions_1660, _matrix_tiers_1633,
_ktlo_legs_stay_green_1672, _python_versions_update_1660, _unit_tests_queue,
_ci_timing and _release_gates pass; the new ones fail on origin/main.

Part of #1660
@JarryShaw JarryShaw added ci Pull requests that change CI or workflow configuration (ci: subject prefix) review: good-to-go Cross-review at the current head says ready; CI state is separate labels Oct 14, 2026
@JarryShaw

Copy link
Copy Markdown
Owner Author

Verdict: GOOD TO GO at 296ad6b71. Sonnet cross-reviews (author: Opus) checked two rounds.

  • Round 1 blocker, now fixed: a hung experimental leg could hit the 45-minute job cap and block every PR. The steps' caps now act first.
  • Timeouts don't block: read from actions/runner itself (StepsRunner.cs:310-353, ExecutionContext.cs:1422-1440). A step that times out under continue-on-error ends with outcome failure and result success, so the job concludes success.
  • Other legs unchanged: every job but test is identical to main. In test, the non-experimental caps evaluate to 360, above the job cap.
  • Spellings: 3.14t and graalpy-25.4 are what setup-python v7 accepts, checked in its source and the python-versions manifest.
  • Matrices: wheel, conda, Compat and integration are identical to main. Only test gains the two legs.
  • Mutations: six cap mutations are each caught. The seventh, raising the job cap, rightly still passes.
  • Non-blocking: the 25-minute test cap is a guess, since GraalPy's runtime is unmeasured. Running over shows as a warning.

@JarryShaw JarryShaw moved this to In review in PyPCAPKit Oct 14, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Coverage: 89.35% (unit tier, Python 3.15, 296ad6b71, Unit Tests run success)

Package Statements Missed Branches Partial Cover
pcapkit (top level) 104 3 20 3 95.16%
pcapkit/const 18860 954 2368 868 91.06%
pcapkit/corekit 2372 67 762 35 96.49%
pcapkit/dumpkit 275 3 100 3 98.40%
pcapkit/foundation 2885 144 1010 34 94.20%
pcapkit/interface 138 7 52 5 93.68%
pcapkit/protocols 17181 226 4590 190 98.07%
pcapkit/toolkit 1148 290 450 10 71.21%
pcapkit/utilities 500 4 150 4 98.77%
pcapkit/vendor 4417 2342 1010 157 43.38%

Per-file detail: the coverage-html artifact of this run.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci Pull requests that change CI or workflow configuration (ci: subject prefix)

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

1 participant