ci(security): make the security-fixes routine observable and stagger its schedules - #338
Open
PMerlet wants to merge 4 commits into
Open
ci(security): make the security-fixes routine observable and stagger its schedules#338PMerlet wants to merge 4 commits into
PMerlet wants to merge 4 commits into
Conversation
First real runs: toolbelt and agent-ruby produced valid security PRs; forestadmin-server and forestadmin failed with '1 turn, $0.00, is_error: true' after ~180s (no token ever billed → the first API request never completed), and agent-nodejs was cancelled by hand after 28 minutes of zero visible output. - show_full_output: true — the action hides all agent output by default, making a working run indistinguishable from a hung one and the failures impossible to diagnose (secrets stay masked by GitHub) - stagger the five schedules 30 min apart: five concurrent Opus sessions on one API key, the two largest-context repos starting 53s apart, is the likely trigger of the unbilled retry loop Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
actions/checkout@v4 and actions/setup-node@v4 ship the Node 20 runtime, which runners now warn about and force onto Node 24. v6 targets Node 24 natively and is already used elsewhere in these repos. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
GitHub renders '- [ ]' as an interactive checkbox only at the start of a
list item. Inside a table cell it stays literal text ('<td>- [ ]</td>',
verified against GitHub's own /markdown API), and a raw
<input type="checkbox"> is stripped by the sanitizer — so the Done and
Dismissed columns rendered as '- []' text in every row.
The tables are now purely informational and the PR body ends with a
'Review checklist' section holding real task lists: fixes to verify, and
alerts to dismiss.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ilure Macroscope is right that show_full_output publishes the whole agent transcript — tool results, file reads, command output — to logs any repo member can download, and GitHub only masks registered secret values. On toolbelt it would also copy the full Dependabot alert JSON into logs readable by people without security_events access. Its diagnostic value was real (it is what revealed the 401 on the two repos with a stale repo-level ANTHROPIC_API_KEY) but it is far wider than needed. Replaced by a failure-only step that extracts just the terminal error fields (subtype, is_error, terminal_reason, api_error_status, turns, cost, and the first 400 chars of the result string) from the execution log. Enough to tell an auth/API failure from an application one, without publishing the transcript. Also: the workflow-file push fallback now rewrites the commit (reset --soft + restore + re-commit) instead of merely unstaging — the files were already inside the single commit, so unstaging changed nothing and the retry was rejected identically. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
First real runs of the new routine (2026-07-30) gave mixed results:
1 turn, $0.00, is_error: trueafter ~180 sTwo problems to address, neither in the agent instructions themselves:
1. No observability.
claude-code-actionhides all agent output unlessshow_full_output: trueis set. A run that is working normally and a run that is stuck look identical in the logs — which is exactly why the agent-nodejs run was cancelled by hand, and why the two failures can't be diagnosed beyond "the first API request never completed" (total_cost_usd: 0= nothing was ever billed).2. Five concurrent Opus sessions on one API key. All five workflows shared the
0 11 * * 4slot, so five sessions started within ~3 minutes. The two repos with the largest initial context (forestadmin-server, forestadmin — the biggest monorepos, both with a sizeableCLAUDE.md) started 53 s apart and both burned ~3 minutes of retries before giving up without a single billed token; the smaller repos, running with less overlap, succeeded. That is the signature of an input-token rate limit, not of a bug in the routine.What
show_full_output: true— agent progress becomes visible in the workflow log (GitHub still masks secrets).forestadmin-server11:00 ·forestadmin11:30 ·agent-nodejs12:00 ·toolbelt12:30 ·agent-ruby13:00 UTC.workflow_dispatchstays available; when triggering several repos by hand, space them out.Next
Re-run the three non-green repos manually one at a time after merging. With full output on, a repeat failure will name the actual API error — if it is a rate limit, the options are a higher API tier or
--model claude-sonnet-5for the two big monorepos.🤖 Generated with Claude Code
Note
Make the security-fixes workflow observable on failure and stagger its schedule
subtype,is_error,terminal_reason,api_error_status,num_turns,total_cost_usd) viajqto make failures diagnosable.actions/checkoutfrom@v4to@v6.Macroscope summarized 539a66a.