aggregate-results: contract-only ci-status fails closed when the jobs API lags step data beyond the 15 s retry #883

Problem

A contract-only ci-status run (a pull_request event such as edited, labeled or unlabeled) fails closed even though a full run on the same head recorded ci-lanes=success:

##[warning]could not identify this run's gate job and running step in repos/melodic-software/claude-code-plugins/actions/jobs/<job id>: job 'ci-status' has no in-progress step named 'Aggregate lane results' after 5 reads. Cannot verify the step that wrote the carried status, so this run fails closed.
##[error]no successful ci-lanes status on <sha>; ...

The passing full run then re-runs the contract-only run, which fails the same way, so the newest ci-status on the head stays red. On 2026-10-24 this happened on every re-run attempt of the same run. This is the same message as closed #743, whose fix (#814) added a retry of at most 15 s. Today the jobs API's step data lagged by minutes, so that retry window does not cover it.

Seen in claude-code-plugins at the v0.58.0 pin (5e18f3d6). aggregate-results has not changed between 5e18f3d6 and origin/main (v0.65.7): git diff --stat 5e18f3d6 origin/main -- .github/actions/pr-require-checks/ prints nothing. A re-pin will not fix it.

Evidence

All times are 2026-10-24, UTC. All runs are in melodic-software/claude-code-plugins.

• PR #9683, head e7e77a92b7651274493751d5bfc7fed24e14fccb: full run https://github.com/melodic-software/claude-code-plugins/actions/runs/40112424087 succeeded. Contract-only run https://github.com/melodic-software/claude-code-plugins/actions/runs/40114806734 failed on all four attempts:

  • https://github.com/melodic-software/claude-code-plugins/actions/runs/40114806734/job/120933784217
  • https://github.com/melodic-software/claude-code-plugins/actions/runs/40114806734/job/120935337859
  • https://github.com/melodic-software/claude-code-plugins/actions/runs/40114806734/job/120935904644
  • https://github.com/melodic-software/claude-code-plugins/actions/runs/40114806734/job/120937305912 (the step ran 17:25:13 to 17:25:31 with STEP_NAME: Aggregate lane results and JOB_CHECK_RUN_ID: 120937305912, and logged the warning above after 5 reads)

• PR #9565, head 130abf3cd15255e82b1e128c7cc3692dcfbbb9f1: after full run https://github.com/melodic-software/claude-code-plugins/actions/runs/40113930347, contract-only run https://github.com/melodic-software/claude-code-plugins/actions/runs/40114804216 failed on two re-run attempts:

  • https://github.com/melodic-software/claude-code-plugins/actions/runs/40114804216/job/120935551814
  • https://github.com/melodic-software/claude-code-plugins/actions/runs/40114804216/job/120936549574

• PR #9634: https://github.com/melodic-software/claude-code-plugins/actions/runs/40113127210/job/120928817485 failed on attempt 1, so this is not limited to re-runs. Its reads ran from 16:47:25 to 16:47:43. Its job's step 17 is now listed as Aggregate lane results, started_at 16:47:25Z, so the name matches.

• The same warning appeared on more contract-only runs:

  • https://github.com/melodic-software/claude-code-plugins/actions/runs/40113296871 (16:49)
  • https://github.com/melodic-software/claude-code-plugins/actions/runs/40113578392 (16:52)
  • https://github.com/melodic-software/claude-code-plugins/actions/runs/40114755853 (17:08)

• The lookup resolved within about 1.5 s on these contract-only runs:

  • https://github.com/melodic-software/claude-code-plugins/actions/runs/40040727625 (2026-10-23 23:38)
  • https://github.com/melodic-software/claude-code-plugins/actions/runs/40110629444 (16:14)
  • https://github.com/melodic-software/claude-code-plugins/actions/runs/40112364878 (16:40)

  The workflow and the action pin were the same throughout, so the failures depend on GitHub-side state. They are not caused by a code change.

Step data in the jobs REST API (GET /repos/{owner}/{repo}/actions/jobs/{id}) lagged by minutes after the jobs finished:

Earlier the same day, re-run attempts listed all 21 steps (ci-status jobs 120920484811 and 120920481528 of runs 40110078170 and 40110077030, 16:07).

https://www.githubstatus.com listed no incident for 2026-10-24 when checked. The logs API returned HTTP 502 several times during this investigation.

Root cause

Line numbers below are in .github/actions/pr-require-checks/aggregate-results/run.sh at 5e18f3d6.

1. The own-step lookup needs the step listed as in progress within 15 s.
   • run.sh:535-545 resolves the gate job by job.check_run_id. It also requires exactly one step of that job with .name == $step and .status == "in_progress".
   • run.sh:498 caps the retries at OWN_GATE_RETRY_DELAYS='1 2 4 8' (15 s, 5 reads).
   • When the jobs API serves a stale or empty steps array, every read misses and run.sh:585 warns.
   • verify_writer_step (run.sh:608-613) then fails closed and drops the carried ci-lanes=success.
   • The job identity is already proven at this point: the job has the right REST id and run id and is in_progress. Only the step half fails.
2. The writer-step check depends on the same step data. If the own step resolved, gate_step_outcome (run.sh:652-656) would read the writer job's steps. Writer job 120937212521 listed [] nine minutes after it completed. That gives absent, which run.sh:635-638 treats as still running. The run marks the success pending and fails closed at the end of its wait. A fix to part 1 alone would move the failure here.
3. Why it loops. The full run re-runs the failed contract-only run right after it writes ci-lanes. That is when the step data for both jobs is most likely to lag, so the re-run fails the same way.

UNVERIFIED: why GitHub served stale or empty steps today. How fresh step data is in the jobs API is undocumented (see the Evidence section of #743). The lag shown here is observed, not explained.

Proposed fix

These are options for the maintainers. The order is a suggestion.

1. Own step: when the job is keyed by job.check_run_id and passes the id, run id and in_progress predicate, take STEP_NAME (the step-name input) as the step name without requiring it to be listed. Keep the in-progress-step requirement only for the runner-name fallback (no check_run_id) and for the case where step-name is unset. This removes failure 1 with no new API calls.
2. Writer step: separate "the writer job is completed and its steps is empty" (lagging data) from "the step is absent from a populated list".
   • (a) For the empty case, wait longer within the job's timeout, then fail closed. The cost: today's lag (over 9 min) can exceed any wait that fits.
   • (b) Verify the writer from a source that does not depend on step data. For example, the full run could record its verdict where a contract-only run can read it and an unrelated workflow cannot forge it. The cost: a new mechanism, which needs design.
   • (c) Accept the writer job's conclusion == success when steps is empty. The cost: this conflicts with the design note at run.sh:596-598. That note says a full run whose contract check failed still records a true lanes success, so the job conclusion is not the lanes verdict. It would need a guard against a writer run with a failing job.
3. Re-run timing: when the full run re-runs a failed contract-only sibling, it could delay the re-run until its own gate step reads completed in the jobs API. This narrows the loop but does not close it.

Add run.test.sh cases for:

• a job whose steps is []
• a job whose steps list omits the running step
• a completed writer job whose steps is []

Related

• fix(check-contract): contract-only ci-status fails closed with "could not identify this run's gate job and running step" #743 (closed): the same warning, fixed by fix(aggregate-results): resolve ci-status's own gate job by runner name #814 with a retry of at most 15 s
• aggregate-results: a superseded contract-only ci-status shows as a real failure #842: a superseded contract-only ci-status shows as a real failure. It was also seen today on https://github.com/melodic-software/claude-code-plugins/actions/runs/40114193611.
• fix(ci): a contract-only ci-status that races the full run stays red claude-code-plugins#9113: a contract-only ci-status that races the full run stays red. It records the same lookup failure during the 2026-10-22 Actions incident.
• The consumer pin is .github/workflows/pr-require-checks.yml in claude-code-plugins (aggregate-results@5e18f3d6, step-name: Aggregate lane results).

🤖 Generated with Claude Code

https://claude.ai/code/session_01Q9bziHugSS2AXBhM2ryQff