Fix AI PR review workflow: inline comments never post, unhandled API errors#272
Merged
Conversation
- Remove invalid subject_type field that caused every inline review comment to fail with HTTP 422 (confirmed in PR #271 run logs) - Add status-code check and robust output parsing to the OpenAI call so a bad/empty API response fails loudly instead of crashing with an opaque JSONDecodeError - Post the summary comment via --body-file so literal \n\n renders as an actual blank line instead of raw escape text Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅ |
🤖 AI PR Review Summary\n\nThis PR improves the GitHub Actions workflow for AI-based PR review by enhancing OpenAI API response handling, adding timeout and error checks, and refining JSON parsing from the model output. It also improves the formatting and posting of PR summary and inline comments. The main risks involve potential breaking changes in response parsing logic and error handling that could cause the workflow to fail if the API response format changes unexpectedly. |
arjunsuresh
approved these changes
Jul 24, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Diagnosed against the live run history of #271 (5 pushes, 4 successful + 1 failed job, zero inline comments ever posted). Root causes:
gh api .../pulls/commentscall sentsubject_type="line", which is not a valid field for that endpoint. Every attempt across all runs failed this way; only the summary comment ever got through.Run OpenAI reviewblindly calledresponse.json()and indexeddata["output"][0]["content"][0]["text"]with no status check or error handling. One run (commit025f794aon Bundle the ScriptAutomation engine (migrate automation/ from mlperf-automations) #271) got a non-JSON response and the whole job died with an opaqueJSONDecodeError, silently producing no PR comment — which looked like "the action didn't trigger."\n\n—--body "...\n\n$SUMMARY"in a bash double-quoted string doesn't interpret\nas a newline. Confirmed via raw byte inspection of a posted comment.Changes
subject_typefield from the inline-comment API callresponse.okand search for themessage-type output item (tolerant of response shape) before parsing; fail with a clear::error::annotation instead of a raw tracebackjson.loads--body-fileso newlines render correctly$cloop variable in the inline-comment loopTest plan
::error::message instead of a traceback