Verify release attestation offline against published bundle asset - #421
Verify release attestation offline against published bundle asset#421evandowning wants to merge 8 commits into
Conversation
install.sh ran `gh attestation verify --repo`, which fetches the Sigstore bundle from the GitHub attestations API. `gh` always attaches its stored credential, so a token without an SSO session for the trailofbits org 403s on public data — external users cannot install even though the artifact is public and its SHA256SUMS check already passed. Publish the provenance bundle as a release asset (attestations.jsonl) and verify it offline with `gh attestation verify --bundle`, which makes no API call and needs no credential. The --repo identity constraint and the artifact digest match are still enforced, so transporting the already-signed bundle over an unauthenticated download does not weaken the guarantee. - release.yml: give the attest step an id, normalize its bundle output to one-per-line with `jq -c`, and publish attestations.jsonl with the release. - install.sh: download the bundle asset and pass --bundle; releases before the asset existed fail closed with a clear message. - README.md / RELEASING.md: document the offline path and the credential-free manual recipe; strip credentials in the release smoke test. src/update.rs has the identical bug (coop update shells out to the same API-fetching verify) and needs the same fix in a follow-up. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Review of #421 found two problems with the installer change. install.sh failed closed when a release published no attestations.jsonl. Since README serves the installer from main, merging would have broken `curl … | bash` for every user until the next release: latest is v0.5.4, which has no bundle asset, and the error told users to "install a newer version" that does not exist. It also broke `VERSION=v0.5.x` pinned installs permanently. Fall back to the API path in that case instead — exactly what every release did before — so the bundle is a strict improvement and never a regression. A bundle that fails to verify still refuses the install; only a missing bundle falls back. src/update.rs had the same API-only verification, so `coop update` still 403'd for the users this fixes. It now downloads the release's attestations.jsonl and passes --bundle, with the same fallback. The bundle is skipped when gh is absent or the API base is overridden, so a fallible download cannot fail an update whose attestation step is a no-op anyway. Also: install.sh's no-gh hint and update.rs's equivalent advised the credential-requiring recipe; both now show --bundle. README leads with the offline recipe, RELEASING.md pins VERSION in the smoke test, and docs/trust-model.md records that --bundle changes transport, not the guarantee — --repo still pins signer identity and the bundle is signed. Verified against v0.5.4 with GH_CONFIG_DIR empty and GH_TOKEN/GITHUB_TOKEN unset: offline bundle path installs (exit 0), corrupt bundle refuses (exit 1), missing bundle falls back to the API, and a multi-subject bundle — which is what the release actually produces — verifies. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
hbrodin
left a comment
There was a problem hiding this comment.
Thanks for tracking this down — the diagnosis is right and this is a genuinely useful catch. I'm looking into the bigger picture around this separately.
I verified the central claims independently rather than taking the description on faith, against the real v0.5.4 release with gh 2.93.0:
- The attestations API is readable anonymously (
200, 2 attestations). It really is onlyghattaching its token that converts public data into a 403 — same class as cli/cli#6675. I also confirmed the obvious workaround does not exist: stripping the token and running the API path exits 4, so--bundleis the only credential-free route. - Credential-free
--bundleverification works: correct artifact → exit 0 withGH_CONFIG_DIRempty and both token vars unset. Negative controls all fail closed — tampered tarball → 1, wrong--repo→ 1, garbage bundle → 1, empty bundle → 1, raw API JSON as--bundle→ 1. - So the guarantee is unchanged: Sigstore trust root,
--reposigner identity, and subject-digest match are all still enforced offline. Agreed this is not a softening of the chain. bundle-pathis declared at the exact pinned action SHA, and--bundledocuments and accepts JSON Lines — so thejq -cnormalization is sound whichever shape the action emits.
One thing the description undersells in the other direction: this fixes a second, independent failure. On main, anyone with gh installed but not logged in gets exit 4 ("please run gh auth login") → die → the install is refused outright. That's a larger population than SAML users, and today their only escape is uninstalling gh, which silently downgrades them to checksum-only. Coverage genuinely goes up here.
Comments inline. The only one I'd want resolved before merge is the failure messaging in install.sh. Worth stating plainly though: the fix is latent — install.sh is served from main and latest is v0.5.4 with no asset, so the reported symptom persists for every user until the next release ships, and permanently for VERSION=v0.5.x pinned installs.
- install.sh: gate the credential explanation on gh actually reporting a 403 / SAML / "gh auth login" symptom. A network error, a gh too old for the command, or a genuine provenance mismatch previously got described as an SSO problem and pointed the user at a different release. - install.sh: confirm a successful verify on both paths — the offline bundle path printed nothing, which read the same as a skip. - update.rs: a failed bundle download now falls back to the API instead of failing the update, matching install.sh. One policy, one behavior. fetch_attestation_bundle is infallible, so it returns Option directly. - update.rs: the asset-name tripwire now asserts on the `gh release create` line. Dropping the asset from publication alone left the test green while the asset silently stopped shipping. - install.sh: record why a bundle that fails to verify is refused rather than retried — it is not stricter on integrity, it surfaces a broken download or an unusable gh. - Docs: trust-model records the failed-download fallback and the refuse-don't-retry rule; RELEASING checks for the positive confirmation line. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Thanks for the independent verification — especially the negative controls and the Changed
Deferred to #423 — making the API fallback itself credential-free, with your verified Gates on the merged tree: |
|
13 findings posted inline. Three claims I verified directly rather than reasoning about, since the review turns on them:
Out of diff, so not inline: No diff noise. Checked and deliberately not raised: the All seven prior inline threads were confirmed genuinely addressed at Coverage: all eight agents ran — |
Second review round on #421. - `update.rs`: replace `Option<PathBuf>` with `Provenance`, which carries why there is no bundle instead of re-deriving it from `None`. A failed download no longer reports "the release publishes no attestations.jsonl" on a release that publishes it. Split the decision from the IO as `bundle_decision(release, api_overridden, gh_present)`, so all four outcomes are unit-testable; this also drops the guard that duplicated `verify_attestation`'s own skip checks by convention. - Reject an empty bundle in both clients and in `release.yml`. `jq -c .` exits 0 on empty input, so the step could publish a 0-byte asset that blocks every `gh`-equipped install of an immutable release; `gh` before 2.56.0 reports success on one, having verified nothing. - Harden the asset-name tripwire on the installer half. `install.sh` names the asset literally only in its `BUNDLE=` declaration, so the old `contains` check survived deletion of the download and the `--bundle` verify. Now keyed on those lines. - `install.sh`: the fallback message no longer states "not published" as the only cause of a silenced probe failure, and the credential explanation moves to stderr alongside the `gh` output and the `die`. - Docs: `--bundle` means no attestations-API call and no credential, not offline — `gh` still fetches the Sigstore trust root, and the bundle download itself uses a credential when one exists. Record that `--repo` pins the certificate's source repository, not the signer, and that the subject-digest binding is what defeats a substituted bundle. Correct the RELEASING.md diagnosis and the `coop update` asset list in ARCHITECTURE. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
All 13 inline findings addressed in 1d649b4 and the threads resolved. Branch was already current with Three of these were bugs in things the previous round added or claimed, which is a fair hit rate on a round that was itself review follow-up:
On the two documentation corrections: "offline" and " Coverage note. Still deferred to #423: the credential-free API fallback. Unchanged reasoning — it changes fallback transport rather than the bundle mechanism, and it is the piece that reaches Verification for the behavior changes, beyond the gates:
|
|
Evan — this is a long-ish list and I want to say up front why. It's the release-verification path, so a mistake here gets baked into an immutable tag and can't be re-cut. I've been deliberately picky as a result, and that's about the blast radius, not about the work. I checked the claims in the description rather than taking them on trust, and they held up:
The round-2 fixes all landed. And the premise is still current: cli/cli#11803 (unauthenticated verify) is open, so no newer Two things are worth your time — both posted inline:
Happy to push those two fixes myself if you'd rather not spin another round on this. Optional — take or leaveNone of these block. Listing once rather than as separate threads:
Not in the diff, so no line to hang them on
DeferredThe signer-identity gap (neither client passes On keeping the assetWorth recording, since it argues for the approach you took over the #423 shortcut: |
|
Filed the signer-identity item as #436 rather than growing this PR — it's a release-process policy decision, not a defect here, and it needs a couple of research questions answered first (notably an audit of the cert identity of every past release, since a strict pin could break Two things I turned up while writing it up that are worth knowing for this PR:
|
The bundle download went through `download_asset`, which prefers `gh` and then falls back to curl carrying `$GITHUB_TOKEN`. That re-attached the credential `--bundle` exists to avoid: on a SAML-restricted token the fetch was the one step in the chain that could still 403, dropping back to the API path and failing with the original error. Both clients now fetch the asset with a bare, unauthenticated request — `download_bundle` in `install.sh`, `curl_download(url, dest, None)` in `update.rs` — so the path is credential-free end to end, as the README already claimed. Correct the credential attribution throughout: the attestations API is anonymously readable for a public repo. What needs a credential is `gh attestation verify` without `--bundle`, which refuses to run unless logged in and then attaches its token. Record the two ways the transports are not interchangeable in `docs/trust-model.md`: the `--bundle` path accepts a superset (any correctly-signed bundle in a release needs only `contents: write`, while registering one in the attestation store needs `attestations: write`), and it is unrevocable (Fulcio certs carry no CRL/OCSP and `gh` consults no revocation source, so a saved bundle keeps verifying after the attestation is deleted). Also state that `--repo` constrains the signer SAN to that repo, just not to a workflow file or ref. The installer tripwire keyed on `download_asset`, so it would have stayed green on both regressions. It now asserts the call site uses `download_bundle` and that the helper's body is a bare curl with no `GITHUB_TOKEN` or `gh release`; each assertion was confirmed to fail on the regression it names. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The comment claimed an empty asset would leave clients failing verification "with no API fallback". Both clients fall back on it: `install.sh` gates on `[ -s ]` and `resolve_provenance` returns `ApiReason::EmptyBundle`. The unwarned risk is the opposite one — a non-empty degenerate bundle. A whitespace-only file passes `[ -s ]` and `len() > 0` alike, and `gh`'s JSON-Lines loader then yields zero attestations, which `gh` before 2.56.0 (cli/cli#9541) reports as success. `jq -c .` is what makes that unreachable, so it has to stay upstream of publication. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…station-verify # Conflicts: # .github/workflows/release.yml
Problem
curl -fsSL …/install.sh | bashfails for users outside thetrailofbitsorg:install.shandcoop updateboth rangh attestation verify … --repo trailofbits/coop,which fetches the Sigstore bundle from the GitHub attestations API.
ghalways attaches itsstored credential and refuses to run without one. Once a token is attached,
trailofbitsSAMLenforcement rejects it unless it carries an SSO session for the org, so the request 403s even
though the data is public. The affected user cannot fix it if they have no SSO access. The
failure is unrelated to artifact integrity — the
SHA256SUMScheck already passed beforeverification aborted.
The same code path also fails for anyone who has
ghinstalled but is not logged in:ghexits 4 ("please run gh auth login") and the install is refused outright. That is a larger
population than SAML users, and their only escape today is uninstalling
gh, which silentlydowngrades them to checksum-only. Coverage goes up here, not just error quality.
Fix
Publish the provenance bundle as a release asset (
attestations.jsonl) and verify itwith
gh attestation verify --bundle, which makes no attestations-API call and needs nocredential. Both install paths stop depending on the caller's GitHub identity for the
verification step.
--bundleis credential-free, not offline:ghstill fetches the Sigstore trust root fromtuf-repo.github.comunless--custom-trusted-rootis supplied, and downloading the bundleasset itself still uses
gh/GITHUB_TOKENwhen one exists..github/workflows/release.yml— give the attest step anid, normalize its bundleoutput to one-per-line with
jq -c(runs on the runner only; not a user-facing dep), andpublish
attestations.jsonlalongside the tarballs andSHA256SUMS.install.sh— download the bundle asset via the existingdownload_asset(gh → curlfallback) and pass
--bundle. Both verification paths print a confirmation on success.src/update.rs— same treatment forcoop update: look up theattestations.jsonlasset in the release metadata, download it, and pass
--bundle. The lookup is skipped whenghis absent orCOOP_UPDATE_API_BASE_URLis overridden, so an update whose attestationstep is a no-op does no pointless work.
RELEASING.mdadds the asset tothe release checklist and pins
VERSIONin the credentials-stripped smoke test;docs/trust-model.mdrecords that--bundlechanges transport, not the guarantee.Releases without the asset still install
Verification falls back to the API path — exactly what every release did before — when a
release publishes no bundle, or when the bundle download fails. Failing closed instead would
have broken the documented
curl … | bashone-liner for every user between merge and thenext release (README serves
install.shfrommain, andlatestis v0.5.4, which has noasset), and would have broken
VERSION=v0.5.xpinned installs permanently. So the bundle isa strict improvement and never a regression.
install.shandupdate.rstreat "no asset published" and "asset download failed" the sameway, because for the client they are the same situation: no bundle to read.
A bundle that downloads but fails to verify is refused, not retried through the API. To be
clear about what that does and does not buy: it is not stricter on integrity — a
substituted bundle fails
--bundleand would then verify correctly against the genuineattestation, and a tampered artifact fails both checks. The reason is diagnostic: a bundle
that downloaded and will not verify means a broken download or a
ghthat cannot read it,and silently switching transports would hide both.
Verification strength is unchanged
This is not a softening of the update-verification chain. Verified against v0.5.4 with
credentials stripped (
GH_CONFIG_DIRempty,GH_TOKEN/GITHUB_TOKENunset):--repo(trailofbits/not-coop)--bundleThe bundle is signed and verified against Sigstore's trust root, and what defeats a
substituted bundle is the subject-digest binding —
ghdigests the artifact and requiresa matching subject.
--repopins the certificate's source repository, not the signer: anyworkflow in
trailofbits/coopwithid-token: write+attestations: writesatisfies it.The API path is keyed by digest against the same repo-scoped store and is equally unpinned,
so fetching an already-signed object over an unauthenticated download changes the transport
and nothing else.
Review follow-ups (fd28783)
install.shfailure messaging — the credential explanation is now gated onghactually reporting a
403/SAML/gh auth loginsymptom. Previously a network error, aghtoo old for the command, or a genuine provenance mismatch was all described as an SSOproblem and pointed the user at a different release. Exercised with a stubbed
ghacrosssix outcomes: the explanation appears for the two credential cases and for neither the
network error nor the "no matching attestations found" mismatch.
update.rsfalls back to the APIinstead of failing the update, matching
install.sh.fetch_attestation_bundleis nowinfallible and returns
Option<PathBuf>directly.gh release createline, notjust on the file containing the string. Confirmed it now fails when the asset is dropped
from publication while the
jqstep still creates it; before, that regression stayed green.release.ymlstep name —Normalize attestation bundle, which is what the step does;publishing happens in
gh release create.verify with no credential) is filed as Make the attestation-API fallback credential-free for pre-bundle releases #423 rather than grown into this PR.
Review follow-ups (1d649b4)
Second round; all 13 inline threads addressed.
fetch_attestation_bundle'sOption<PathBuf>collapsed fouroutcomes and the error text then re-derived a reason from the surviving
None, so a faileddownload reported "the release publishes no
attestations.jsonl" on a release that publishesit. Replaced with a
Provenanceenum that carries the reason; the failure branch is now unit-tested in both directions.
bundle_decision(release, api_overridden, gh_present)ispure and covered over all four outcomes. This was the only new decision logic in the diff and
had no coverage of any kind:
src/update.rsis whole-module excluded in.cargo/mutants.tomland
integration-update.shshort-circuits it viaCOOP_UPDATE_API_BASE_URL. It also removesthe guard that duplicated
verify_attestation's own skip checks by convention.jq -c .exits 0 on empty input, so the normalize step could publisha 0-byte asset green, blocking every
gh-equipped install of an immutable release (and, onghbefore 2.56.0, reporting success having verified nothing). Guarded in
release.ymlwith[ -s ]and in both clients, which now fall back to the API rather than pass an empty file to
--bundle.install.shnames the asset literally only in itsBUNDLE=declaration, so
installer.contains(BUNDLE_ASSET)survived deleting the download and the--bundleverify. Now keyed on those lines; each of the three assertions was confirmed to failon the regression it names, where the old check stayed green.
failed download, so the fallback line no longer asserts the first. The gated credential
explanation also moved to stderr, alongside the
ghoutput it explains and thediethatfollows.
RELEASING.md's release-acceptance step carries the same correction.--repodocumented as pinning the certificate's source repository rather than the signer; theSSO/403 rationale reduced from four copies to
docs/trust-model.mdplus a short one in thestandalone
install.sh.docs/ARCHITECTURE.mdupdated for the third asset (raised out-of-diff).Gates:
shellcheck,bash -n,cargo fmt --check,cargo clippy --all-targets --all-features -D warningsclean;cargo test1065 passed;./tests/integration-update.sh5 passed.actionlintreports only the pre-existing SC2086 onmainin the unrelated Package step.Review follow-ups (ff03f8a)
Third round; all 5 inline threads addressed.
download_asset, whichprefers
ghand then falls back to curl carrying$GITHUB_TOKEN— so on aSAML-restricted token the bundle download was the one step in the chain that could
still 403, and when it did, verification fell back to the API and failed with exactly
the original error. Both clients now fetch the asset with a bare unauthenticated
request:
download_bundleininstall.sh,curl_download(url, dest, None)inupdate.rs.download_assetis unchanged for the tarball andSHA256SUMS, where acredential is harmless.
for a public repo; what needs a credential is
gh attestation verifywithout--bundle, which refuses to run unlessghis logged in and then attaches its token.The API path was described as the thing requiring the credential in README,
install.sh,update.rs(theProvenance::Apidoc and threetracingmessages), anddocs/trust-model.md. All corrected.trust-model.mdnow says so insteadof concluding "the transport change costs nothing here". The
--bundlepath accepts asuperset (any correctly-signed bundle in a release needs only
contents: write, whileregistering one in the attestation store needs
attestations: write) and is unrevocable(Fulcio certificates carry no CRL/OCSP and
ghconsults no revocation source, so asaved bundle keeps verifying after the attestation is deleted). The conclusion is
unchanged; it now rests on the subject-digest binding rather than on equivalence.
--repois also documented as constraining the signer SAN to that repo, just not to aspecific workflow file or ref.
download_asset&&$BUNDLE, so itwould have stayed green on the change above. It now asserts the call site uses
download_bundleand that the helper's body is a bare curl with noGITHUB_TOKENorgh release; each assertion was confirmed to fail on the regression it names.release.ymlcomment corrected (commit001326e). It claimed an empty asset wouldleave clients failing with "no API fallback", which contradicts the clients in this PR —
both fall back on it. The unwarned risk is the opposite: a non-empty degenerate bundle,
since a whitespace-only file passes
[ -s ]andlen() > 0alike andgh's JSON-Linesloader then yields zero attestations, reported as success by
ghbefore 2.56.0.Gates:
bash -n install.shclean;cargo fmt --check,cargo clippy --all-targets --all-features -D warningsclean;cargo test1065 passed;./tests/integration-update.sh5 passed.
shellcheckandactionlintare unavailable in this environment and were notre-run for this round.
Testing
shellcheck install.sh,bash -n install.sh— clean.actionlintreports only thepre-existing SC2086 on
mainin the unrelated Package step.cargo fmt --check,cargo clippy --all-targets --all-features -- -D warnings,cargo test(1012 passed) — clean../tests/integration-update.sh— 5 passed.install.shwith zero credentials against a v0.5.4 release with the bundleasset shimmed in: installs cleanly. Corrupt bundle: refuses. No bundle: falls back to the
API and reports the SSO cause on failure.
gh attestation verifyargument list (with and without--bundle)and guard the
attestations.jsonlname against drift betweenrelease.yml,install.sh,and
update.rs. Both were confirmed to fail when the behavior is reverted.Stage 2 (a real release carrying the asset) can only run after the next release publishes
attestations.jsonl.🤖 Generated with Claude Code