Skip to content

Fix AI PR review workflow: inline comments never post, unhandled API errors#272

Merged
arjunsuresh merged 1 commit into
mainfrom
fix/ai-pr-review-action
Jul 24, 2026
Merged

Fix AI PR review workflow: inline comments never post, unhandled API errors#272
arjunsuresh merged 1 commit into
mainfrom
fix/ai-pr-review-action

Conversation

@anandhu-eng

Copy link
Copy Markdown
Contributor

Summary

Diagnosed against the live run history of #271 (5 pushes, 4 successful + 1 failed job, zero inline comments ever posted). Root causes:

  • Inline comments always failed (HTTP 422) — the gh api .../pulls/comments call sent subject_type="line", which is not a valid field for that endpoint. Every attempt across all runs failed this way; only the summary comment ever got through.
  • Workflow crashes on bad OpenAI responsesRun OpenAI review blindly called response.json() and indexed data["output"][0]["content"][0]["text"] with no status check or error handling. One run (commit 025f794a on Bundle the ScriptAutomation engine (migrate automation/ from mlperf-automations) #271) got a non-JSON response and the whole job died with an opaque JSONDecodeError, silently producing no PR comment — which looked like "the action didn't trigger."
  • Summary comment rendered with literal \n\n--body "...\n\n$SUMMARY" in a bash double-quoted string doesn't interpret \n as a newline. Confirmed via raw byte inspection of a posted comment.

Changes

  • Remove the invalid subject_type field from the inline-comment API call
  • Check response.ok and search for the message-type output item (tolerant of response shape) before parsing; fail with a clear ::error:: annotation instead of a raw traceback
  • Strip accidental markdown code fences before json.loads
  • Post the summary via --body-file so newlines render correctly
  • Quote the $c loop variable in the inline-comment loop

Test plan

  • Open/push to a test PR and confirm the summary comment renders with a proper blank line
  • Confirm at least one inline review comment is successfully posted (no more 422s)
  • Force an OpenAI API error (e.g. bad key) and confirm the job fails with a readable ::error:: message instead of a traceback

- Remove invalid subject_type field that caused every inline review
  comment to fail with HTTP 422 (confirmed in PR #271 run logs)
- Add status-code check and robust output parsing to the OpenAI call so
  a bad/empty API response fails loudly instead of crashing with an
  opaque JSONDecodeError
- Post the summary comment via --body-file so literal \n\n renders as
  an actual blank line instead of raw escape text

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@anandhu-eng
anandhu-eng requested a review from a team as a code owner July 23, 2026 16:09
@github-actions

Copy link
Copy Markdown

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅

@github-actions

Copy link
Copy Markdown

🤖 AI PR Review Summary\n\nThis PR improves the GitHub Actions workflow for AI-based PR review by enhancing OpenAI API response handling, adding timeout and error checks, and refining JSON parsing from the model output. It also improves the formatting and posting of PR summary and inline comments. The main risks involve potential breaking changes in response parsing logic and error handling that could cause the workflow to fail if the API response format changes unexpectedly.

@arjunsuresh
arjunsuresh merged commit f4e574b into main Jul 24, 2026
61 checks passed
@arjunsuresh
arjunsuresh deleted the fix/ai-pr-review-action branch July 24, 2026 00:48
@github-actions github-actions Bot locked and limited conversation to collaborators Jul 24, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants