Skip to content

Treat the AWF proxy's HTTP 403 max-AI-credits rejection as trusted budget-abort evidence - #55241

Merged
pelikhan merged 11 commits into
mainfrom
copilot/fix-github-actions-job-another-one
Aug 24, 2026
Merged

Treat the AWF proxy's HTTP 403 max-AI-credits rejection as trusted budget-abort evidence#55241
pelikhan merged 11 commits into
mainfrom
copilot/fix-github-actions-job-another-one

Conversation

Copilot AI commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

The agent job in run 32683896339 went red on intentional budget enforcement: the AWF API proxy stopped the run with 403 Maximum AI credits exceeded (302.111025 / 300), which the harness is supposed to convert into exit 0 ("budget control, not an error"). The conclusion job classified it correctly and filed #55239; only the agent job failed.

Two defects in handleFailure blocked the budget path:

  1. Trusted evidence was unobtainable. The sole accepted source was parseMaxAICreditsExceededFromAuditLog(), but AWF flushes the firewall audit JSONL during container teardown (02:49:14.97) — after the attempt is classified (02:49:13.17). The audit-only gate can never fire for a mid-run budget abort.
  2. The auth veto swallowed it. Claude Code surfaces the proxy's 403 as error: authentication_failed, so && !isAuthenticationFailed would have suppressed the budget path even with audit evidence.
[claude-harness] attempt 1 failed: exitCode=1 isRateLimitError=true isAuthenticationFailedError=true
[claude-harness] attempt 1: AI credits marker found in CLI output without trusted firewall audit confirmation — preserving normal failure handling
[claude-harness] attempt 1: authentication failed — not retrying (first-attempt auth failure is non-retryable)

Changes

  • harness_retry_guard.cjs — new parseAICreditsExceededProxyRejection() matching the proxy's canonical rejection, including the proxy-computed usage pair, and rejecting pairs where usage has not reached the budget:
    const AI_CREDITS_EXCEEDED_PROXY_REJECTION_RE =
      /\b403\b[^\n]{0,80}?maximum ai credits exceeded\s*\(\s*(\d+(?:\.\d+)?)\s*\/\s*(\d+(?:\.\d+)?)\s*\)/i;
    This is deliberately narrower than the existing loose max_ai_credits_exceeded markers, which remain untrusted.
  • claude / codex / copilot harnesses — accept the proxy signature as trusted evidence alongside the audit log, and scope the auth-failure veto so it only stands down for that signature:
    const trustedAICreditsExceeded = nonRetryableGuard.aiCreditsExceeded && (!!proxyAICreditsRejection || parseMaxAICreditsExceededFromAuditLog());
    const shouldTreatAICreditsExceededAsSuccess = trustedAICreditsExceeded && (!isAuthenticationFailed || !!proxyAICreditsRejection);
    A bare Authentication failed still fails the job; codex keeps its !isMissingApiKey guard.
  • Tests — parser unit tests (usage-below-budget, missing usage pair, used == max boundary) plus a claude harness regression test replaying the failing run's stream-JSON payload.

Note on steering

No [AWF TOKEN WARNING] / [AWF TIME WARNING] messages were inserted before the abort despite enableTokenSteering: true — AWF steering is token/time-budget based and does not steer on AI-credit budgets, so the agent got no warning and was cut off mid safe-output write. Credit-aware steering would need to happen upstream in gh-aw-firewall and is out of scope here.


Run: https://github.com/github/gh-aw/actions/runs/32689010109> Generated by 👨‍🍳 PR Sous Chef · pi · gpt54 · 9.37 AIC · ⌖ 10.9 AIC · ⊞ 8.7K ·

Comment /souschef to run again


Generated by 👨‍🍳 PR Sous Chef · pi · gpt54 · 11.4 AIC · ⌖ 7.95 AIC · ⊞ 8.7K ·
Comment /souschef to run again


Run: https://github.com/github/gh-aw/actions/runs/32706066137> Generated by 👨‍🍳 PR Sous Chef · pi · gpt54 · 24.6 AIC · ⌖ 14.4 AIC · ⊞ 8.7K ·

Comment /souschef to run again


Run: https://github.com/github/gh-aw/actions/runs/32726270698

Generated by 👨‍🍳 PR Sous Chef · pi · gpt54 · 6.74 AIC · ⌖ 7.96 AIC · ⊞ 8.7K ·
Comment /souschef to run again


Generated by 👨‍🍳 PR Sous Chef · pi · gpt54 · 23.8 AIC · ⌖ 8.71 AIC · ⊞ 8.7K ·
Comment /souschef to run again

Copilot AI and others added 2 commits August 24, 2026 03:08
…bort

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix failing GitHub Actions job and implement a fix Treat the AWF proxy's HTTP 403 max-AI-credits rejection as trusted budget-abort evidence Aug 24, 2026
Copilot AI requested a review from pelikhan August 24, 2026 03:10
@pelikhan
pelikhan marked this pull request as ready for review August 24, 2026 03:28
Copilot AI balanced review requested due to automatic review settings August 24, 2026 03:28

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds early recognition of AWF proxy AI-credit budget rejections so harnesses treat intentional enforcement as success.

Changes:

  • Parses canonical HTTP 403 credit-limit responses.
  • Applies trusted-budget handling across all harnesses.
  • Adds parser and Claude regression tests.
Show a summary per file
File Description
actions/setup/js/harness_retry_guard.cjs Adds proxy rejection parsing.
actions/setup/js/harness_retry_guard.test.cjs Tests parser boundaries.
actions/setup/js/claude_harness.cjs Handles proxy budget aborts.
actions/setup/js/claude_harness.test.cjs Reproduces the reported failure.
actions/setup/js/codex_harness.cjs Handles proxy budget aborts.
actions/setup/js/copilot_harness.cjs Handles proxy budget aborts.

Review details

💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

  • Files reviewed: 6/6 changed files
  • Comments generated: 1
  • Review effort level: Balanced

*/
function parseAICreditsExceededProxyRejection(output) {
const safeOutput = typeof output === "string" ? output : "";
const match = AI_CREDITS_EXCEEDED_PROXY_REJECTION_RE.exec(safeOutput);

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in d1963cf and a4cb063: parseAICreditsExceededProxyRejection() no longer treats free-text as trusted. It now only matches lines that (1) parse as standalone JSON and (2) carry an engine-set API-error marker (is_api_error_message: true or a string error field) — a source only the harness's own transport layer sets, mirroring how isInvalidRequestError validates codex turn.failed events. Plain assistant text quoting the phrase, without that structural marker, is ignored. See harness_retry_guard.test.cjs for the regression test covering that exact untrusted-text case.

@github-actions

github-actions Bot commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

🧠 Matt Pocock Skills Reviewer has completed the skills-based review. ✅

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer

@github-actions

github-actions Bot commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

PR Code Quality Reviewer completed the code quality review.

🔎 Code quality review by PR Code Quality Reviewer

@github-actions

github-actions Bot commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

Ponytail Reviewer completed successfully!

Generated by Ponytail Reviewer for #55241

@github-actions

github-actions Bot commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

Design Decision Gate 🏗️ completed the design decision gate check.

No ADR enforcement needed: PR #55241 does not have the 'implementation' label and has 0 new lines of code in business logic directories (threshold: 100).

🏗️ ADR gate enforced by Design Decision Gate 🏗️

@github-actions

github-actions Bot commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

Test Quality Sentinel completed test quality analysis.

Test Quality Sentinel skipped because pre-fetch PR data was unavailable: unable to fetch test file diff

🧪 Test quality analysis by Test Quality Sentinel

@github-actions

Copy link
Copy Markdown
Contributor

Comment Memory

reviewed_at: 2026-08-24T00:00:00Z
review_event: REQUEST_CHANGES
top_themes:
  - untrusted stdout/stderr promoted to trusted budget-abort evidence
  - auth-failure path can now be suppressed by forged proxy-looking text
files_reviewed:
  - actions/setup/js/claude_harness.cjs
  - actions/setup/js/codex_harness.cjs
  - actions/setup/js/copilot_harness.cjs
  - actions/setup/js/harness_retry_guard.cjs
  - actions/setup/js/harness_retry_guard.test.cjs
  - actions/setup/js/claude_harness.test.cjs
comment_count: 1

Note

This comment is managed by comment memory.

It stores persistent context for this thread in the code block at the top of this comment.
Edit only the text inside the backtick fences; workflow metadata and the footer are regenerated automatically.

Learn more about comment memory

🔎 Code quality review by PR Code Quality Reviewer · pi · gpt54 · 18.8 AIC · ⌖ 6.98 AIC · ⊞ 4.6K ·
Comment /review to run again

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Request changes

The main fix regresses the trust boundary: it upgrades a regex match from combined child stdout/stderr into “trusted budget-abort evidence”, so model- or tool-controlled text can now suppress a real failure and make the harness exit 0.

Blocking theme
  • parseAICreditsExceededProxyRejection() reads from result.output, which runProcess() builds by concatenating untrusted child stdout/stderr.
  • The new harness logic then lets that free-text match override the existing authentication-failure veto.
  • That means a forged string in assistant output can masquerade as an AWF proxy rejection and convert an actual error into success.

I left one inline comment on the trust upgrade point; the budget-abort classification needs to stay tied to a channel the model cannot forge.

🔎 Code quality review by PR Code Quality Reviewer · pi · gpt54 · 18.8 AIC · ⌖ 6.98 AIC · ⊞ 4.6K
Comment /review to run again

*/
function parseAICreditsExceededProxyRejection(output) {
const safeOutput = typeof output === "string" ? output : "";
const match = AI_CREDITS_EXCEEDED_PROXY_REJECTION_RE.exec(safeOutput);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change now treats a free-text 403 Maximum AI credits exceeded (used / max) substring in combined child stdout/stderr as trusted budget-abort evidence, but result.output is not a trusted channel: process_runner.cjs concatenates raw child stdout and stderr verbatim, which includes assistant/model text and any tool output. That means a prompt-injected or tool-emitted string matching this regex can suppress a real authentication failure and force the harness to exit 0. The new trust upgrade removes the audit-log boundary the old code relied on and turns untrusted model output into a success signal.

💡 Why this should block

parseAICreditsExceededProxyRejection() only checks for a textual pattern:

(b403/redacted)\b[^\n]{0,80}?maximum ai credits exceeded\s*\(\s*(\d+(?:\.\d+)?)\s*\/\s*(\d+(?:\.\d+)?)\s*\)/i

and the harnesses then do:

const trustedAICreditsExceeded = nonRetryableGuard.aiCreditsExceeded && (!!proxyAICreditsRejection || parseMaxAICreditsExceededFromAuditLog());
const shouldTreatAICreditsExceededAsSuccess = trustedAICreditsExceeded && (!isAuthenticationFailed || !!proxyAICreditsRejection);

Because runProcess() builds result.output from arbitrary child stdout/stderr, the agent itself can emit that exact phrase in normal assistant text and make the wrapper classify the run as a trusted budget abort. That is a correctness and security boundary regression: auth failures and other real errors can be hidden behind model-controlled text.

Please keep the trust decision anchored to a source the model cannot forge, e.g. the firewall/audit channel or a harness-generated side channel that captures authenticated proxy metadata separately from assistant output.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in d1963cf and a4cb063: the parser is no longer a free-text match against combined stdout/stderr. parseAICreditsExceededProxyRejection() now requires each candidate line to parse as a standalone JSON object and carry an engine-set API-error marker (is_api_error_message: true or a string error field) before the 403 usage-pair regex is applied — a marker only the harness's own transport layer sets on genuine proxy/API errors, not on assistant conversational text. An assistant turn that merely quotes the phrase in its message content (without that structural marker) no longer qualifies as trusted evidence; this exact case is covered by a new regression test in harness_retry_guard.test.cjs. The audit-log path remains available as an alternative trusted source.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One simplification opportunity stands out: the proxy-budget parser can be collapsed into a local regex match in claude_harness.cjs, removing the extra helper/export surface without changing behavior. net: -15 lines possible.

Generated by ✂️ Ponytail Reviewer for #55241 · codex · cop10 · 4.87 AIC · ⌖ 1.29 AIC · ⊞ 18.3K
Comment /ponytail to run again

return AUTHENTICATION_FAILED_PATTERNS.some(pattern => pattern.test(safeOutput));
}

/**

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actions/setup/js/harness_retry_guard.cjs:59-75: yagni: helper + exported regex for a one-off proxy budget match. Inline the regex match in claude_harness.cjs and keep the budget check local.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review: Harden + Audit

This PR correctly addresses a real gap — when Claude Code surfaces the proxy's 403 budget rejection as authentication_failed, the harness was masking intentional budget enforcement. The logic change and the new test case are both well-targeted.

Primary concern (already raised in an existing inline comment): the trust boundary of result.output. Since that output includes model-generated text, a sufficiently crafted response could match AI_CREDITS_EXCEEDED_PROXY_REJECTION_RE — the aiCredits >= maxAICredits guard helps but does not fully close this. The existing comment covers this well.

Minor observations (non-blocking):

  • codex_harness.cjs received the identical logic change but has no new integration test — only claude_harness.test.cjs has the new scenario.
  • The codex_harness path also vetoes isMissingApiKey alongside isAuthenticationFailed; the new proxy-override only exempts isAuthenticationFailed, which looks intentional and correct.
  • The parseAICreditsExceededProxyRejection unit tests in harness_retry_guard.test.cjs are thorough and cover the boundary condition (aiCredits === maxAICredits).

Overall the fix is well-scoped and the test coverage for the primary Claude Code scenario is solid. Resolving the trust-boundary concern raised in the existing inline comment before merge would be ideal.

🧵 Reviewed using Impeccable skills by Impeccable Skills Reviewer · copilot · sonnet46 · 23.8 AIC · ⌖ 9.08 AIC · ⊞ 6.2K

@github-actions github-actions Bot mentioned this pull request Aug 24, 2026

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Skills-Based Review 🧠

Applied /diagnosing-bugs and /tdd — commenting with 3 issues, no hard blockers.

📋 Key Themes & Highlights

Key Themes

  • Test coverage gap: regression test only covers the claude harness; codex and copilot harnesses have the identical fix but no integration tests
  • Trusted-source ambiguity: the existing concern (comment #3840533313) about matching against combined stdout/stderr still applies to the new regex — it's narrower but not structurally isolated from model output
  • Type contract: parseAICreditsExceededProxyRejection accepts unknown but only handles string; non-string values fail silently

Positive Highlights

  • ✅ Root-cause analysis is thorough — both defects (audit-JSONL timing and auth-veto masking) are correctly diagnosed and fixed
  • ✅ The regex intentionally requires 403 + the precise (used / max) format, making accidental matches unlikely
  • ✅ Usage-below-budget guard (aiCredits < maxAICredits) correctly rejects partial matches
  • ✅ Good boundary test: used == max accepted, used < max rejected
  • ✅ Codex's !isMissingApiKey guard is correctly preserved in the new expression

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · copilot · sonnet46 · 40.9 AIC · ⌖ 10.1 AIC · ⊞ 7.6K
Comment /matt to run again

expect(result.stderr).toContain("trusted budget-abort evidence");
expect(result.stderr).toContain("AI credits budget enforced");
});

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[/tdd] Regression test only covers the claude harness. The identical fix is applied to codex_harness.cjs and copilot_harness.cjs — a future refactor could silently regress either without parallel tests.

💡 Suggestion

Add analogous tests to codex_harness.test.cjs and copilot_harness.test.cjs replaying the same proxy 403 payload with each harness's stub, asserting status === 0 and the trusted budget-abort evidence log line. For codex, also verify the !isMissingApiKey guard doesn't interfere.

@copilot please address this.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added in a4cb063: codex_harness.test.cjs and copilot_harness.test.cjs each now have a regression test replaying the same proxy 403 payload, asserting status === 0 and the "trusted budget-abort evidence"/"AI credits budget enforced" log lines. The codex test sets CODEX_API_KEY so the !isMissingApiKey guard can't suppress the budget path.

* Returns null when the output does not carry the proxy signature, or when the reported
* usage does not actually reach the reported budget.
* @param {unknown} output
* @returns {{ aiCredits: number, maxAICredits: number } | null}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[/diagnosing-bugs] The safeOutput guard discards any non-string output silently — if result.output is ever an array or object (e.g. structured log lines), the parser returns null and the fix silently fails to fire.

💡 Suggestion

Consider a defensive coercion that mirrors the pattern used by other guards in this file:

const safeOutput = output != null ? String(output) : "";

or explicitly document that callers must pass the joined string output. A short comment noting the contract would prevent future callers from passing a raw result.output object.

@copilot please address this.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Documented the contract in d1963cf/a4cb063: added a comment on parseAICreditsExceededProxyRejection noting callers must pass the joined stdout+stderr string (matching every other guard in this file), and that non-string input is treated as empty rather than coerced via String(...), since that would produce a meaningless value here.

*/
function parseAICreditsExceededProxyRejection(output) {
const safeOutput = typeof output === "string" ? output : "";
const match = AI_CREDITS_EXCEEDED_PROXY_REJECTION_RE.exec(safeOutput);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[/diagnosing-bugs] There is an existing review comment (id 3840533313) that notes free-text matching on result.output is untrustworthy because it contains combined stdout/stderr including model-generated text. The new regex is narrower (requires 403 + the specific format), but a model could still emit an identical-looking string in its output. Please confirm whether this risk is acceptable, or consider adding an additional check (e.g. verify the line comes from a known harness prefix rather than assistant content).

@copilot please address this.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in a4cb063 (building on d1963cf): parseAICreditsExceededProxyRejection() now only trusts lines that parse as standalone JSON and carry an engine-set marker (is_api_error_message: true or a string error field) before running the 403 regex against them. Plain assistant text containing the phrase, without that structural marker, is ignored — added a regression test for that exact case in harness_retry_guard.test.cjs.

@gh-aw-bot

Copy link
Copy Markdown
Collaborator

@copilot This PR still needs forward progress before maintainer investigation is likely to be productive.

Please address the current blocking review feedback first:

Additional reviewer context worth closing out:

  • existing trust-boundary concern remains on the parser/input channel
  • add follow-up coverage for the identical codex/copilot harness paths if the fix remains shared

The branch refresh request has also been queued. After addressing the review, refresh the branch as needed, run the pr-finisher skill, and push the follow-up changes.

Generated by 👨‍🍳 PR Sous Chef · pi · gpt54 · 9.37 AIC · ⌖ 10.9 AIC · ⊞ 8.7K ·
Comment /souschef to run again

Copilot AI commented Aug 24, 2026

Copy link
Copy Markdown
Contributor Author

@copilot This PR still needs forward progress before maintainer investigation is likely to be productive.

Please address the current blocking review feedback first:...

All review threads on this PR already carry a substantive reply addressing the trust-boundary concern (see discussion_r3840533313 / discussion_r3840580081 and their replies): parseAICreditsExceededProxyRejection() (d1963cf, a4cb063) only trusts lines that parse as standalone JSON and carry an engine-set is_api_error_message/error marker — a signal set by the harness's own transport layer, not by assistant/model text — so free-form output can no longer forge the budget-abort signal. Codex/copilot regression coverage replaying the same proxy 403 payload exists and all 544 tests pass.

I don't have GitHub API/thread-resolution access in this sandbox to mark the threads "resolved" myself — that requires a maintainer or reviewer action. No further code changes are pending from my side; happy to address any new/specific technical concern if one is raised.

@gh-aw-bot

Copy link
Copy Markdown
Collaborator

gh-aw-pr-sous-chef
@copilot Thanks for the follow-up. The outstanding reviewer-facing step now is to close the loop on the unresolved trust-boundary feedback directly in the PR review threads and then rerun the pr-finisher skill if any final edits are needed.

Current blockers to clear:

  • unresolved review feedback from github-actions about trusted budget-abort evidence on combined stdout/stderr
  • direct reviewer acknowledgement that the thread can be resolved now that the parser only trusts structured engine-set JSON markers

After that, refresh the branch if needed and push any final follow-up.

Generated by 👨‍🍳 PR Sous Chef · pi · gpt54 · 21.9 AIC · ⌖ 7.95 AIC · ⊞ 8.7K ·
Comment /souschef to run again

Copilot AI commented Aug 24, 2026

Copy link
Copy Markdown
Contributor Author

gh-aw-pr-sous-chef
@copilot Thanks for the follow-up. The outstanding reviewer-facing step now is to close the loop on the unresolved trust-boundary feedback directly in the PR review threads and th...

Both trust-boundary threads (discussion_r3840533313, discussion_r3840580081) already have my direct reply in-thread explaining the fix and pointing to the exact commits/tests. I don't have thread-resolution or gh/GraphQL write access in this sandbox, so I can't mark them resolved myself — that action needs to come from a maintainer/reviewer with repo write access. No new technical concern has been raised in the threads since my last reply, so there's no additional code change to make here.

@gh-aw-bot

Copy link
Copy Markdown
Collaborator

@copilot Thanks for the follow-up here. This PR still looks blocked on unresolved reviewer feedback.

Please use the pr-finisher skill and do the following:

  • Re-engage directly on the unresolved review concerns about the trust boundary of the proxy-budget detection logic.
  • If the current implementation is correct, post concise reviewer-facing replies that explain the trusted signal and why model-controlled output cannot spoof success; otherwise push the needed fix.
  • Refresh the branch after your changes so maintainers can retry merge evaluation on the latest tip.
  • Re-run make fmt and impacted tests before pushing.

Open review themes to close out:

  • github-actions requested changes on the trust-boundary regression risk.
  • Multiple review comments note missing reviewer-thread resolution after follow-up.

Once you have either pushed the fix or provided reviewer-facing resolution in-thread, resolve the addressed review threads.

Generated by 👨‍🍳 PR Sous Chef · pi · gpt54 · 14.7 AIC · ⌖ 8.12 AIC · ⊞ 8.7K ·
Comment /souschef to run again

Copilot AI commented Aug 24, 2026

Copy link
Copy Markdown
Contributor Author

@copilot Thanks for the follow-up here. This PR still looks blocked on unresolved reviewer feedback.

Please use the pr-finisher skill and do the following:...

The current implementation is correct as-is: parseAICreditsExceededProxyRejection() only trusts lines that parse as standalone JSON and carry an engine-set is_api_error_message/error marker (a signal only the harness transport layer sets, never assistant text), so model-controlled output cannot spoof the budget-abort success path. This was already explained in-thread on both trust-boundary comments (discussion_r3840533313, discussion_r3840842060 and discussion_r3840580081, discussion_r3840842058), with regression tests covering the untrusted-text rejection case and the claude/codex/copilot harnesses.

I don't have GitHub thread-resolution or write API access in this sandbox (no gh/GraphQL credentials), so I can't call the resolve-thread mutation myself — that step requires a maintainer or reviewer with repo write access. There's no new technical concern to act on and no code change is pending.

@gh-aw-bot

Copy link
Copy Markdown
Collaborator

gh-aw-pr-sous-chef
@copilot Thanks for the follow-up. Since the reviewer-facing replies are already in place and no further code changes are pending, the remaining unblocker is to have the addressed review threads resolved or otherwise acknowledged by a maintainer/reviewer.

Please do one focused pass with the pr-finisher skill and:

  • verify the in-thread replies still point reviewers to the exact trust-boundary fix and regression tests
  • push only if any wording or code follow-up is still needed
  • otherwise leave the branch as-is and wait for maintainer/reviewer resolution of the addressed threads

Outstanding context:

  • the blocking github-actions review still appears open
  • repeated nudges have not changed branch contents since your last reply

Generated by 👨🍳 PR Sous Chef
Comment /souschef to run again

Generated by 👨‍🍳 PR Sous Chef · pi · gpt54 · 6.74 AIC · ⌖ 7.96 AIC · ⊞ 8.7K ·
Comment /souschef to run again

@github-actions

Copy link
Copy Markdown
Contributor

Triage: PR #55241 (re-check)

Category: bug · Risk: high · Score: 75/100 (Impact 40, Urgency 25, Quality 15)

Recommended action: fast_track

CI green (copilot ×4, CodeQL analyze go/js all passing). 8 files changed (+243/-10), covers harness_retry_guard.cjs + claude/codex/copilot harnesses + tests. Fixes a genuine budget-abort misclassification bug.

Status: Still blocked — github-actions[bot] requested changes on the untrusted-stdout/stderr trust-boundary concern (raw child output can theoretically forge a budget-abort match). Author has replied to the threads multiple times (04:29, 05:11, 08:35, 11:13 UTC) but the CHANGES_REQUESTED review has not been dismissed/re-reviewed. codex_harness.cjs still lacks its own integration test (only claude has one), per multiple independent reviewer comments.

Next action: Needs a maintainer/reviewer pass to either resolve the trust-boundary thread or explicitly accept the residual risk, then dismiss the stale CHANGES_REQUESTED review to unblock merge.

Generated by 🔧 PR Triage Agent · copilot · auto · 28.3 AIC · ⌖ 5.85 AIC · ⊞ 8.4K ·

@gh-aw-bot

Copy link
Copy Markdown
Collaborator

gh-aw-pr-sous-chef
@copilot Thanks for the repeated follow-up. From the current PR state, the branch looks stable and the remaining blocker appears to be stale review state rather than missing implementation work.

Please use the pr-finisher skill for one final maintainer-facing pass:

  • verify the existing in-thread replies still point to the exact trust-boundary fix and the codex/copilot regression coverage
  • push only if a concrete reviewer concern still requires code or wording changes
  • otherwise leave the branch unchanged and wait for maintainer/reviewer resolution of the addressed blocking review

Run: https://github.com/github/gh-aw/actions/runs/32750412251

Generated by 👨‍🍳 PR Sous Chef · pi · gpt54 · 23.8 AIC · ⌖ 8.71 AIC · ⊞ 8.7K ·
Comment /souschef to run again

Copilot AI commented Aug 24, 2026

Copy link
Copy Markdown
Contributor Author

gh-aw-pr-sous-chef
@copilot Thanks for the repeated follow-up. From the current PR state, the branch looks stable and the remaining blocker appears to be stale review state rather than missing imple...

Verified: the trust-boundary fix from d1963cf/a4cb063 is unchanged and stable — parseAICreditsExceededProxyRejection() still requires a standalone JSON line with an engine-set is_api_error_message/error marker, so free-form assistant text still can't forge the signal. Ran the full suite (harness_retry_guard.test.cjs, claude_harness.test.cjs, codex_harness.test.cjs, copilot_harness.test.cjs): 544/544 passing. No concrete new reviewer concern is present, so no code changes were needed this pass.

@pelikhan

Copy link
Copy Markdown
Collaborator

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot AI commented Aug 24, 2026

Copy link
Copy Markdown
Contributor Author

@copilot lint js https://github.com/github/gh-aw/actions/runs/32751356025/job/97523134014?pr=55241

Fixed in 40e8a9d's follow-up: harness_retry_guard.cjs's JSDoc type-cast (/** @type {Record<string, unknown>} */ (parsed)) was missing a // prettier-ignore, so prettier stripped the parens and broke the assertion (matching the existing pattern in daily_aic_workflow_helpers.cjs). Added the ignore comment; make lint-cjs and tsc --noEmit now pass clean.

@pelikhan
pelikhan merged commit a3fcb6f into main Aug 24, 2026
12 checks passed
@pelikhan
pelikhan deleted the copilot/fix-github-actions-job-another-one branch August 24, 2026 17:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants