Repository navigation
chore: recover from the Rekor 409 that keeps failing releases - #408
Merged
Merged
Conversation
The last two releases died on the same provenance error: TLOG_CREATE_ENTRY_ERROR error creating tlog entry - (409) an equivalent entry already exists in the transparency log Run 39601015349 lost @argos-ci/cypress@7.4.0. Run 38902693584 lost @argos-ci/browser@6.5.1, and with it every package that depends on it. It is the retry-after-success from #364. Rekor commits the entry but answers after sigstore's 5s timeout, sigstore resubmits it, and Rekor deduplicates with a 409. sigstore hardcodes `fetchOnConflict: false`, so the 409 is fatal (sigstore/sigstore-js#1708, still the case in 5.0.0). The timing matches: each failure lands ~7.5s after the previous publish, one 5s timeout plus retry backoff. #366 showed the timeout cannot be raised through lerna, and called this patch the proper fix. - Patch sigstore@4.1.1 to set `fetchOnConflict: true`. On a 409 it reads the existing entry from the Location header and uses it, which is the entry the first attempt wrote. Against a mock Rekor that commits the first POST, replies after 6s and 409s the retry, stock sigstore fails with TLOG_CREATE_ENTRY_ERROR and the patched one builds the bundle. lerna's libnpmpublish resolves the patched copy. - Add a `from-package` release type running `lerna publish from-package`. A failed release has already tagged and created the GitHub releases, so a normal rerun sees nothing to bump and 7.4.0 would never ship. Against a mock registry, it picked only @argos-ci/cypress@7.4.0 on `main` and left no commit or tag behind. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
The last two releases failed on the same provenance error:
@argos-ci/cypress@7.4.0was not published. The other 7 packages were.@argos-ci/browser@6.5.1was not published, and lerna skipped every package that depends on it.This is the retry-after-success described in #364:
fetchOnConflict: false, so the 409 is fatal. Upstream issue: Recover Rekor duplicate-entry conflicts by default sigstore/sigstore-js#1708. It's still hardcoded in sigstore 5.0.0.The timing matches: each failure lands about 7.5s after the previous publish, which is one 5s timeout plus the retry backoff. #366 showed the timeout can't be raised through lerna, and named this patch as the proper fix.
Changes
sigstore@4.1.1(patches/sigstore@4.1.1.patch): one line,fetchOnConflict: true. On a 409, Sigstore fetches the existing entry using theLocationheader and uses it. That entry is the one our first attempt wrote, because Rekor only calls an entry "equivalent" when it has the same signature and key. The patch is registered inpnpm-workspace.yamlwith a comment. If an upgrade removessigstore@4.1.1from the dependency tree, pnpm errors on the unused patch, so someone has to look at it again.from-packagerelease type (workflow option,release-from-packagescript,RELEASING.md). It runslerna publish from-package. A failed release has already tagged the packages and created their GitHub releases, so a normal rerun finds nothing to bump and@argos-ci/cypress@7.4.0would never ship. This publishes only the versions that npm is missing.Testing
sigstorefails withTLOG_CREATE_ENTRY_ERROR, as in CI.POST,POST,GET /api/v1/log/entries/<uuid>and builds the bundle.pnpm install --frozen-lockfileaccepts the patch.from-package: I ranlerna publish from-package --yeson a checkout ofmainagainst a mock npm registry on localhost, so nothing reached npm. It reportedFound 1 package to publish: @argos-ci/cypress => 7.4.0, sent only that one, and created no commit or tag.After merge
Dispatch Release with release type
from-packageto publish@argos-ci/cypress@7.4.0. All other package versions onmainare already on npm.@argos-ci/cypress@7.3.0from Oct 16 was never published either, but 7.4.0 replaces it.Type of changes
bugChecklist
RELEASING.md)🤖 Generated with Claude Code