Skip to content

ci(python): a failed KTLO leg exits 0 and reports a warning, so main stays green (#1672) - #1679

Merged
JarryShaw merged 2 commits into
mainfrom
ci/1672-ktlo-legs-exit-green
Oct 13, 2026
Merged

JarryShaw merged 2 commits into
mainfrom
ci/1672-ktlo-legs-exit-green

Conversation

@JarryShaw

Copy link
Copy Markdown
Owner

Please follow the guide below

  • You will be asked some questions, please read them carefully and answer honestly

  • Put an x into all the boxes [ ] relevant to your pull request (like that [x])

  • Use Preview tab to see how your pull request will actually look like

  • Searched for similar pull requests

  • Followed the coding style (make pylint, make mypy, make isort)

  • make test passes, and a test case covers the change — tests cover it; make test (the whole suite) was not run, only the modules named under Tests

  • Added a changelog entry under docs/source/changelog/ and regenerated CHANGELOG.md, if the change is user-visible — N/A — centralised in docs(changelog): shared 1.5.0 changelog — long-lived, merges last (#610, #616, #617, #618, #620) #657

What is the purpose of your pull request?

Tick the commit type your subject line carries.

  • fix — corrects a defect
  • feat — adds a feature
  • perf — changes performance, not behaviour
  • refactor — changes neither behaviour nor performance
  • test — tests only
  • docs — documentation only
  • ci — workflows or build tooling
  • release — bumps the version or rolls up a distribution
  • chore — anything else

Description of your pull request and other information

Why. Job-level continue-on-error "prevents a workflow run from failing when a job fails"; the job itself still fails. On 4673cff, Compat's run 38453562652 succeeded, but check run 115490362417 (Compat Python 3.7 (KTLO)) concluded failure. It was the commit's only failing check, and statusCheckRollup read FAILURE.

How. Every job with a KTLO leg now uses the step-level form, which "prevents a job from failing when a step fails". Each step that can fail, checkout included, gets an id and continue-on-error: ${{ matrix.ktlo == true }}. Its "outcome is failure, but the final conclusion is success". A new last step, KTLO summary, turns each failed outcome in toJSON(steps) into one ::warning and one job-summary line. It runs jq, which is on every Ubuntu runner image. A step's own annotations take ANNOTATION_LEVEL, which is warning on a KTLO leg; engine-tests' 3.10 cell does the same. Compat and pypcap-parity now wait on the install's outcome as test does, and each pypi step waits on the one before it. On a supported leg these gates add nothing to the default success().

Unchanged. pypi's checkout and #1634's tag check still block every leg. Release status's KTLO notice still fires. A failed KTLO run: step still gets the runner's own Process completed with exit code N. error (ScriptHandler.cs:354 at 5f0b3ddcc). The runner records it before ExecutionContext.cs:1442-1446 applies continue-on-error. Check run 115490411891 shows it, and that check concluded success.

Tests. test_ktlo_legs_stay_green_1672.py fails each step of each job, on every leg, and checks three things. (a) No KTLO job fails. (b) KTLO legs write only warnings; the summary and cell scripts are executed. (c) Supported legs still fail with ::error. All 10 tests fail on origin/main and pass here, under plain unittest, and each of 10 single-line mutations is caught. The #1633 module now pins the warning text and retires the "red" phrasings. TestDocCitationsResolve now checks create-release.yml citations too, bare :N ones included; it fails on round 1's docs, which left nine citations, four of them :997, on lines that had moved. actionlint 1.7.12 adds no finding, and the touched tests/project modules and tests/test_tier_guard.py pass.

Closes #1672

…stays green (#1672)

Compat's KTLO legs carried job-level `continue-on-error`, which passes the run
but concludes the leg's check run `failure`: `Compat Python 3.7 (KTLO)` turned
every commit red. Every job holding a KTLO leg now works one way.

* Each step that can fail on a KTLO leg has an `id` and step-level
  `continue-on-error: ${{ matrix.ktlo == true }}`, checkout included; Compat
  drops its job-level form. The job concludes `success`.
* A step that needs an earlier one's work waits on its `outcome`: Compat and
  `pypcap-parity` wait on the install as `test` and `integration` do, and each
  `pypi` step on the one before it, so no KTLO leg builds or tests with the
  image's `python` after setup-python failed.
* A new last step, `KTLO summary`, turns each failed `outcome` into a
  `::warning` and a job-summary line. A step's own annotations take the job's
  `ANNOTATION_LEVEL`, `warning` on a KTLO leg; engine-tests' 3.10 cell too.
* `pypi`'s checkout and #1634's tag check still block every leg; Release
  status's KTLO notice stays.
* Prose and pins follow; tests/project/test_ktlo_legs_stay_green_1672.py steps
  through each job leg by leg and runs the scripts.

actionlint adds no finding; the touched tests/project modules and
tests/test_tier_guard.py pass.
…teps moved (#1672)

The first commit grew pypi by 29 lines, so nine citations below it named the
wrong line: conda's needs (four times as `:997`), both upload steps and their
release tags, and release_status's App-bot notice.

* releasing.rst and workflows.rst cite the lines the prose describes:
  `:1026`, `:933`/`:1283`, `:926-963`/`:1278-1313`, `:1385-1387`.
* TestDocCitationsResolve checks create-release.yml citations as it checks
  unit-tests.yml's, with each range's job, comma lists, and the bare `:N`
  form these two pages use; BARE_ELSEWHERE names the five bare citations of
  other files.

The new test fails on the first commit's docs (11 failures) and passes here,
as do the 1672, 1633, 1634, release-gates and unit-tests-queue modules.
@JarryShaw JarryShaw added ci Pull requests that change CI or workflow configuration (ci: subject prefix) review: good-to-go Cross-review at the current head says ready; CI state is separate labels Oct 13, 2026
@JarryShaw

Copy link
Copy Markdown
Owner Author

Verdict: GOOD TO GO at 88d6bbc9d. This passed a Sonnet cross-review on round 2. Round 1 had come back NEEDS CHANGES only for nine stale create-release.yml line citations in the docs. I re-ran the 1672 test, the queue test and the release-gate test: 10, 37 and 120 passed.

The change. Every KTLO leg (test, integration, pypcap-parity, Compat, pypi) now masks each step that can fail with step-level continue-on-error, and gates later steps on the earlier step's outcome. A final KTLO summary step turns each failed outcome into a ::warning and a job-summary line. Compat drops its job-level continue-on-error, which was what made the check run conclude failure on every run.

What the reviewer checked. It ran its own simulation over 339 scenarios:

Round 2 renumbers the citations and extends TestDocCitationsResolve to create-release.yml. It failed on round 1's docs with 11 failures.

Still shown: the runner's own "Process completed with exit code N." annotation stays on a masked step, though the check concludes success (ScriptHandler.cs:354). Removing it would need every step to exit 0, with the gating moved to outputs.

This conflicts with #1677 in 6 files. Whichever merges second gets rebased.

@github-actions

Copy link
Copy Markdown
Contributor

Coverage: 89.46% (unit tier, Python 3.14, 88d6bbc9d, Unit Tests run success)

Package Statements Missed Branches Partial Cover
pcapkit (top level) 104 3 20 3 95.16%
pcapkit/const 18858 954 2366 868 91.06%
pcapkit/corekit 2366 67 760 35 96.48%
pcapkit/dumpkit 275 3 100 3 98.40%
pcapkit/foundation 3031 152 1074 35 94.23%
pcapkit/interface 138 7 52 5 93.68%
pcapkit/protocols 17166 226 4580 190 98.07%
pcapkit/toolkit 1350 275 558 6 77.20%
pcapkit/utilities 494 4 146 4 98.75%
pcapkit/vendor 4409 2342 1006 157 43.25%

Per-file detail: the coverage-html artifact of this run.

@JarryShaw
JarryShaw merged commit 6402aeb into main Oct 13, 2026
50 checks passed
@github-project-automation github-project-automation Bot moved this from In review to Done in PyPCAPKit Oct 13, 2026
@JarryShaw
JarryShaw deleted the ci/1672-ktlo-legs-exit-green branch October 13, 2026 21:00
@JarryShaw JarryShaw removed the review: good-to-go Cross-review at the current head says ready; CI state is separate label Oct 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci Pull requests that change CI or workflow configuration (ci: subject prefix)

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

ci: a failed KTLO leg turns main and pull requests red; report it as a warning and exit 0

1 participant