Repository navigation
fix(save): LRU prune survives an early break; failed saves are loud (#3) - #4
Merged
Merged
Conversation
…ud (#3) The prune piped `find | sort -n` into a `while` loop that breaks once the budget is met. On a large target/ that leaves sort still writing: it takes SIGPIPE (or, on the runner, EPIPE: "sort: write failed: 'standard output': Broken pipe", exit 2), pipefail fails the pipeline, and set -e kills the step before tar/push. Callers' continue-on-error hid it, so every save that took the prune path published nothing. - scripts/prune.sh: cc_prune_lru writes the sorted list to a temp file and reads it, so there is no writer for the break to kill. Same LRU semantics; the report now shows before→after instead of a subshell value. - save/action.yml: inputs reach the push script through env (no ${{ }} splices). Each stage sets STAGE; an EXIT trap turns any failure into a ::warning:: (fail-open, exit 0) or, with the new `strict: "true"` input, an ::error:: that fails the step. New outputs `published`, `reason`, `key`. A failure after the push keeps published=true. `nothing to cache` is a warning too. _cc_timeout's kill of an exited watchdog no longer trips set -e and masks the 124. - selftest: strict save, and assert `published`. - README: the outputs, strict, and why to gate on `published`. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016kQb87339uTdFWivk3j2hF
- tests/prune_test.sh: a 3000-file target/ over budget with controlled mtimes, pruned under `bash -e -o pipefail` with SIGPIPE default and ignored. Asserts exit 0, the step reaches tar, no broken pipe, the kept set is exactly the newest files within budget with minimal deletion, and the loop broke early with >64 KiB of list unread. Also checks save/action.yml sources and calls cc_prune_lru. - tests/save_step_test.sh: runs the push step's script verbatim with a stub oras: good save through the prune, login/prune/push failures, strict mode, nothing to cache, retag failure, and a failure after the push. Asserts exit code, annotation, and the published/reason outputs. - .github/workflows/test.yml runs both. Mutation: restoring the pipe-into-while form fails prune_test in both SIGPIPE modes (exit 141 and exit 2 with the runner's "Broken pipe" line). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016kQb87339uTdFWivk3j2hF
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
…e.sh (#3) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016kQb87339uTdFWivk3j2hF
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #3
Cause (measured)
save/action.ymlpipedfind | sort -ninto awhileloop thatbreaks once target/ is undermax-size-mb. On a large target/ the break leavessortstill writing, sosortgets SIGPIPE. Underpipefailthe pipeline fails,set -ekills the step beforetar/oras push, and the caller'scontinue-on-error: truereported success. Measured on CIRISPersist (details on CIRISAI/CIRISPersist#1022): no Linux blob has been published since at least v53.1.3.Reproduced locally with a 3000-file fixture (12356 KB, 8192 KB budget), running the original snippet under
bash -e -o pipefail:sort: write failed: 'standard output': Broken pipe, exit 2, tar never reachedpruned target/ 12356KB→8192KB, reaches tarFix
scripts/prune.sh(cc_prune_lru): the sorted list goes to a temp file and the loop reads from it, so thebreakhas no writer to kill. LRU semantics are unchanged: oldest mtime goes first, and pruning stops at the budget. The log line now shows real before→after numbers (the old one printed a value from the pre-subshell).env(no${{ }}spliced into the script). Each stage setsSTAGE. An EXIT trap turns any failure into:::warning::CIRISCache: save NOT published for <ref> — <stage> failed (exit N), then exit 0strict: "true"(new input):::error::, and the step failspublished,reasonandkey. A failure after the push (the alias tail) keepspublished=true.nothing to cacheis now a warning too._cc_timeout: killing a watchdog that has already exited no longer tripsset -eand hides the 124 behind a plain 1.strict: "true"and assertspublished. The README documents the outputs,strict, and why callers should checkpublishedand not the step outcome.Tests
tests/prune_test.shruns the prune on a fixture over budget, underbash -e -o pipefail, with SIGPIPE both default and ignored. It asserts exit 0 and that the step reaches tar with no broken pipe. It also asserts that the kept set is exactly the newest files within budget with minimal deletion, and that the loop broke early with more than 64 KiB of the list unread. It also checks that the action sources the function and calls it.tests/save_step_test.shruns the push step's script verbatim with a stuboras. Cases: a good save through the prune, login fails, prune fails, push fails, push fails with strict, nothing to cache, retag fails, and a failure after the push. It asserts the exit code, the annotation, andpublished/reason..github/workflows/test.ymlruns both on every PR.Mutation results (all run locally):
published=truebefore the push: SURVIVED. This mutant is equivalent if GITHUB_OUTPUT is last-write-wins, which is BELIEVED.Claim status
published/reasonas describedbash -nand shellcheck are clean on the scripts and tests; actionlint is clean on the workflowsfind --version, intended).outputs:still carrypublished=falsewhen the inner step fails understrictpublished=falseoverrides nothing written earlierpublished=trueafter the push)$GITHUB_ACTION_PATH/../scripts/prune.shresolves on all three runner OSes (cygpath on Windows)max-size-mb: "1024"(within budget, so the script is sourced but nothing is pruned). macOS has no GNU find, so the prune is skipped there by design.v1has not moved)Not done here: moving
v1, or tagging.🤖 Generated with Claude Code
https://claude.ai/code/session_016kQb87339uTdFWivk3j2hF