Repository navigation
ci(python): the GraalPy experimental leg reaches 26% of the suite before its 25-minute cap #1732
Description
Describe the bug
On the GraalPy 25.4 experimental leg added by #1725, the unit-test step reached about 26% of the suite before its 25-minute cap (job 115915976024). By that rate a full run takes about 95 minutes, past the job's 45-minute cap. The leg never finishes, so it can never turn green and be promoted (#1660, option 1). By design, the timeout only warns.
Expected behavior
The GraalPy leg finishes, so its result means something. Options:
- Longer caps for GraalPy alone, about 120 minutes for its job and test step. This costs runner time on every PR.
- A subset for GraalPy, for example the unit tier without the slow round-trip sweeps, as the KTLO legs do with
-m. This needs choosing and documenting. - Weekly only: run GraalPy on a schedule rather than on every PR. Results arrive later, and PR runs pay nothing.
- Keep as is. It keeps warning without ever finishing.
Additional context
The suite runs with -n auto --dist load, the same as other legs, so the slowness is GraalPy's own interpreter speed here. Recommended: option 3, a weekly run with long caps, since experimental results need not gate PRs.
Activity
JarryShaw commentedon Oct 14, 2026
i say we just do a subset then. roundtrip sweep should probably only run on the latest version.
JarryShaw commentedon Oct 14, 2026
Recorded: a subset. The round-trip sweeps run only on the
latestCPython leg, and every other leg, GraalPy included, skips them. A worker is on it. It will measure how long GraalPy takes without the sweeps, and set GraalPy's step cap from that measurement.I am reading "only on the latest version" as applying to all legs, not just GraalPy. That saves time on every PR. If you meant GraalPy alone, say so and I will narrow it.
JarryShaw commentedon Oct 14, 2026
The sweep subset is implemented: every
testleg exceptlatestskips the 41*roundtrip*modules. On its own, though, it does not make GraalPy fit:Required checks passed(unit-tests.yml:1885) andunittest-ordering(:1471) bothneeds: [test], so they wait for everytestleg, GraalPy included.Which way should GraalPy go?
Recommendation: 2, with the uncapped GraalPy run reported as a warning. The sweep subset ships either way.
JarryShaw commentedon Oct 14, 2026
take 2
JarryShaw commentedon Oct 14, 2026
Recorded: option 2.
testinto a job of their own.Required checks passednorunittest-orderingwaits on that job, so its caps can be large enough for GraalPy to finish without holding any PR.latesttestleg skips the 41*roundtrip*modules.A worker is updating the branch.
One question remains. The
integrationjob's three*roundtrip*_runtime.pymodules still run on every integration leg, where they take 19% of the 3.14 leg's time. Should the ruling cover them too? Until you say otherwise, the branch leavesintegrationunchanged.JarryShaw commentedon Oct 14, 2026
yes, the ruling cover them too.
4 remaining items
JarryShaw commentedon Oct 14, 2026
A
JarryShaw commentedon Oct 14, 2026
Recorded: A. In the weekly CI Timing report, cap-share findings on non-blocking legs (experimental and KTLO) become warnings, so the run stays green and the timings stay visible. Blocking legs still go HARD. Round 4 on this branch implements it.
ci(unit-tests): assign every test module a tier and run each tier whe…
test(project): keep the step stubs from measuring a second coverage c…
ci(unit-tests): cap the size of files tests write on every pytest step (
test(tiers): opt the remaining interpreter-sensitive modules into ALL (…
ci(unit-tests): run the whole suite on latest CPython (#1958) (#1976)
test(tiers): make ALL a curated opt-in tier (#1958) (#1978)