Skip to content

feat: show why CI failed — failed job log excerpts (--logs, MCP get_failed_logs) - #21

Merged
devblac merged 8 commits into
mainfrom
feat/failed-job-logs
Oct 25, 2026
Merged

devblac merged 8 commits into
mainfrom
feat/failed-job-logs

Conversation

@devblac

@devblac devblac commented Oct 24, 2026

Copy link
Copy Markdown
Owner

Roadmap #15. For each failed GitHub Actions job, prw downloads the log and shows the output of the step that failed — what you'd scroll to in the browser.

$ prw checks https://github.com/cloudposse/atmos/pull/3476 --logs
…
── [lint] quick-start-advanced · step: ./.github/actions/setup-atmos-install ──
   …
   Warning: Failed to download action '…/download-artifact/tar.gz/9000827…'. Error: Resource temporarily unavailable (codeload.github.com:443)
   Warning: Back off 21.694 seconds before retry.
   Error: Resource temporarily unavailable (codeload.github.com:443)
   Error: Failed to download archive '…' after 3 attempts.
   Full log: https://github.com/cloudposse/atmos/actions/runs/…/job/…

What

  • Extraction (github.ExtractFailure): from the step's Run … header to the first ##[error] (consecutive errors kept together), last --log-lines lines (default 40); timestamps, ANSI colors and runner markup (##[group], ##[start-action …], ##[debug]) stripped; ##[error]/##[warning]/##[notice] → Error:/Warning:/Notice:. No error marker → end of the log.
  • prw checks --logs, prw wait --logs (only on failure); logs / failed_logs in --json.
  • MCP get_failed_logs (max_lines, max 200): text has the rendered excerpts (not duplicated as JSON), structuredContent has the data. get_ci_status / wait_for_ci descriptions point to it on failure.
  • Bounded: keeps only the last 4 MiB of a log, ≤ 5 log downloads per call, skips superseded and cancelled runs, no caching of logs.
  • Clear when there's no log: non-Actions checks and failed commit statuses are listed with their link; 404/410 → "expired or not finished"; 403 → needs "Actions: read".
  • Security: the log endpoint redirects to blob storage; Go drops Authorization on the cross-host redirect — covered by a test.

Test plan

  • go test ./... -race, go vet ./...
  • Parser on a real staticcheck failure log (testdata/, from devblac/go-semver-audit), long output, no marker, consecutive errors, composite-action markup, tail reading of large streams
  • Commands: wait --logs text and JSON, checks --logs, no download without the flag, failed commit statuses listed
  • Live: fix(ci): stabilize quality reporting and CI dependencies cloudposse/atmos#3476 (action download failure), #3422 (git fetch failure + matrix result job), #2199 (failed commit status → link), MCP get_failed_logs over stdio with the real binary

Stacked on #20.

🤖 Generated with Claude Code

devblac and others added 2 commits October 24, 2026 15:40
`prw checks <PR_URL>` shows each check run and commit status of the PR's head
commit with its result, how long it took and a link: failures first, then
cancelled and running ones, then passed, neutral and skipped. Superseded runs
of the same check are left out, as they don't count towards the result. The
header has the short SHA, overall state, counts per bucket and merge state.
`--json` prints the same as an object with a `checks` array.

Exit codes match `prw wait` (0 passed, 1 failed, 3 error, 4 no CI), plus 8
while checks are still running, like `gh pr checks`.

Verified live on devblac/docker-lint#5, zephyrproject-rtos/zephyr#105183
(35 checks; the cancelled run superseded by a newer one is hidden) and
#13.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…_failed_logs`)

For each failed GitHub Actions job, prw now downloads the job log and
extracts the output of the step that failed: its last lines up to the first
`##[error]`, starting after the step's "Run ..." header, with timestamps,
ANSI colors and runner markup (groups, composite action boundaries, debug)
stripped, and errors/warnings rendered as "Error:" / "Warning:". That's what
you'd scroll to in the browser.

- `prw checks --logs` and `prw wait --logs` (`--log-lines`, default 40) print
  the excerpts after the result; `logs` / `failed_logs` in `--json`.
- MCP tool `get_failed_logs` returns them to agents (text rendered for
  reading, data in structuredContent); `get_ci_status` and `wait_for_ci`
  point to it on failure.
- Bounded: only the tail (4 MiB) of a log is kept, at most 5 logs per call,
  superseded and cancelled runs are skipped. Logs aren't cached.
- Checks from other CI systems and failed commit statuses are listed with
  their link, since the API has no log for them; expired logs (404/410) and
  a missing "Actions: read" permission get a clear message.
- The log download redirects to blob storage; Go drops the Authorization
  header on that cross-host redirect (tested).

Parser tested on a real staticcheck failure log (devblac/go-semver-audit).
Verified live on cloudposse/atmos#3476 (action download failure), #3422
(git fetch failure, matrix job), #2199 (failed commit status) and over MCP
stdio.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 24, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-24T19:00:23.780831Z 4985421 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

devblac and others added 4 commits October 24, 2026 15:58
It exited 0, so `prw checks "$PR" && ...` went ahead on an abandoned PR, and
the exit codes didn't match `prw wait` as documented.

Addresses Codex review on #20.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The previous commit only updated the README; this is the code change: a PR
closed without merging now exits 3 (like `prw wait`) instead of 0.

Addresses Codex review on #20.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 49854218e4

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread internal/github/logs.go
Comment on lines +130 to +133
for i, l := range lines {
if strings.HasPrefix(l, "##[error]") {
errAt = i
break

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Select errors from the step that actually failed

If an earlier successful step emits an error annotation, this scan selects that annotation and excludes the later step that actually caused the job to fail. GitHub's Actions toolkit documentation explicitly notes that core.error() can annotate an error while the action still succeeds. Consequently, both --logs and get_failed_logs can return an unrelated successful step as the failure explanation. Identify the failed step before choosing its error marker, rather than assuming the first annotation determines the job's outcome.

Useful? React with 👍 / 👎.

Comment thread cmd/prw/wait.go
Comment on lines +153 to +154
logCtx, cancelLogs := context.WithTimeout(context.Background(), time.Minute)
result.FailedLogs, err = fetchFailedLogs(logCtx, client, owner, repo, result.SHA, waitLogLines)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve signal cancellation while downloading failure logs

When wait --logs is downloading logs, Ctrl+C or SIGTERM only cancels the wait context handled by the signal goroutine above; this new background context remains active. Since signal.Notify is still installed, the signal no longer terminates the process either, leaving the command downloading until completion or the one-minute deadline and eventually returning the CI failure code instead of the documented interruption code. Keep signal cancellation connected to log retrieval even if its timeout must be independent of the polling timeout.

Useful? React with 👍 / 👎.

A job that failed and then passed when its workflow run was re-run is flaky:
same commit, same code, different result. `prw flaky <owner/repo | URL>`
scans the repository's last --runs workflow runs (default 300, max 1000),
fetches the jobs of every attempt of the re-run ones, and reports each job
that failed (failure / timed_out) in one attempt and succeeded in a later one,
grouped by workflow and job: how many runs it flaked in, when last, and the
latest failed attempt's link. Cancelled jobs (fail-fast victims) don't count;
jobs that kept failing aren't flaky. `--json` for scripts.

The MCP tool `get_flaky_checks` gives agents the same, so they can tell a
known-flaky failure from one their change caused.

GitHub takes seconds per runs/jobs request on large repositories, so pages
and attempts are fetched in parallel (4 at a time) and runs are listed
without their pull_requests payload: grafana/grafana went from 24s to 9s.

Verified live: grafana/grafana (4 flaky jobs in 300 runs), hashicorp/terraform
("Unit Tests"), devblac/prw (none) and over MCP stdio.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
feat: find flaky checks (prw flaky, MCP get_flaky_checks)
@devblac
devblac changed the base branch from feat/checks-command to main October 25, 2026 00:16
@devblac
devblac merged commit f7e5298 into main Oct 25, 2026
@devblac
devblac deleted the feat/failed-job-logs branch October 25, 2026 00:17
devblac added a commit that referenced this pull request Oct 25, 2026
captureStdout restored os.Stdout right after the pipe was drained, while the
goroutine running the command had read os.Stdout earlier. Closing a pipe isn't
a synchronization point for the race detector, so `go test -race` on CI
flagged TestChecksCmd_PendingExitCode (main went red after #21). os.Stdout is
now restored after receiving the command's error from the channel, which
orders the two.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant