Skip to content

Registry: a pull no longer keeps a deleted image's blobs pullable - #907

Merged
schmitch merged 2 commits into
mainfrom
claude/576-deleted-image-blobs
Nov 17, 2026
Merged

schmitch merged 2 commits into
mainfrom
claude/576-deleted-image-blobs

Conversation

@schmitch

Copy link
Copy Markdown
Member

Fixes #576

The problem

After an image, manifest or tag is deleted, its config and layers stay linked to the image until the registry sweeper finds the links unused for docker.unreferenced_blob_ttl_seconds (24 h). Every GET or HEAD of such a blob renewed the link (serve_blob → oci_dao::touch_link), so anyone who could read the repository and knew a digest kept the content of a deleted image pullable for as long as they fetched it once a day.

The change

Of the issue's two directions, this takes the first: a fetch does not renew a link, except for a client that is pushing.

  • Registry::confirms_link (crates/server/src/images/pull.rs) decides whether serving a blob records a use of its link. It does only for a caller who may push the image the link is in: a credential a push is accepted from, a registry token whose scope has push, and write permission on the repository a push to that name lands in (the repository itself, or a group's writable member; never a proxy).
  • That keeps the case the renewal exists for (Docker sweeper can release a layer the client just confirmed: the hourly touch throttle equals the minimum TTL, and re-uploads or mounts of a linked blob never refresh it #218): docker push asks with a HEAD whether the image has a layer, skips the upload, and names the layer in a manifest afterwards. Such a caller could push the blob again anyway, so the renewal gives it nothing new.
  • Every other fetch records nothing. The sweeper releases an unreferenced blob at most the TTL after it was last pushed, mounted or confirmed by a pusher, however often it is pulled. For an image pushed longer ago than the TTL, that is the next sweep after the delete.
  • Pushing a blob again and mounting it record the use as before.

Why not the second direction (purge the image's unreferenced links with a whole-image delete): it would also remove the layers of a push in progress to that image, whose manifest then fails with BLOB_UNKNOWN; it covers only the whole-image delete, not a manifest deleted through the registry API or retention; and it changes the admin delete that #560 is reworking.

What this does not change: a deleted image's blobs are still pullable by digest until the sweeper releases them (up to the TTL plus a sweep interval). The docs and the dialog now say so.

Docs and UI

  • docs/docker.md, Retention and garbage collection: who records a use, what stays pullable after a delete and for how long, and that a leaked secret is to be rotated.
  • The image page's delete dialog (image, manifest and chart version; not the tag delete, which says the manifest stays) has a notice saying the config and layers can still be pulled by digest until the registry releases them. It is an element of its own below the description, to keep clear of Image delete dialog: counts from the capped lists, and a chart version's delete does not say it takes the digest's other tags #546's rewording of the descriptions; a rebase over that is expected to be small.
  • docs/changes/2.2.0/576.security.md. The issue was in Backlog; it is moved to the 2.2.0 milestone, the next release.

No admin API changes, so no new admin screen.

Verification

Run locally, with Postgres and versitygw from testcontainers (pulled through mirror.gcr.io):

  • New fetching_the_blobs_of_a_deleted_image_does_not_keep_them (crates/server/tests/registry.rs): after a manifest is deleted, a reader and a writer GET/HEAD its layer and config, a reader also with a token requested for pull,push; the sweep releases both and they answer 404. The layer the other manifest names and that manifest still pull. A push in progress (a layer pushed before its manifest, a cross-repository mount) confirmed with a push-token HEAD survives the sweep and its manifests are accepted. A layer nobody confirms goes after the TTL although it is read.
  • confirming_a_blob_records_the_use_of_its_link extended: a pull token's HEAD and GET record nothing, a push token's HEAD does (throttled as before), also through a group whose writable member holds the link.
  • Both tests fail with the old always-renew behaviour (checked by forcing the renewal on).
  • cargo xtask check (fmt, clippy -D warnings), cargo xtask changelog --check, cargo xtask docs --check, cargo xtask unused, cargo xtask doc-check.
  • forge-server unit tests and the integration binaries registry, import, admin_retention, format_contract, workers, browse: all pass.
  • Web: pnpm run tsc, pnpm run knip, pnpm run test (635 tests).

Not run locally:

  • The rest of the workspace's test binaries and forge-e2e (the session's disk did not hold a full workspace test build); CI's test and e2e jobs run them.
  • oci-e2e (real docker push, crane, helm, cosign and the conformance suite): crane, helm and cosign are not installed in the session. This is the check that real clients' pushes still confirm their layers, so its CI result matters.
  • web-e2e (Playwright).

Generated by Claude Code

Every GET or HEAD of a blob renewed its link's grace period
(serve_blob -> oci_dao::touch_link), so after an image, manifest or tag
was deleted, anyone who could read the repository and knew a digest kept
its config and layers pullable for as long as they fetched them once per
docker.unreferenced_blob_ttl_seconds.

The grace period is for uploads before their manifest. A fetch now
records a use of the link only for a caller who may push the image the
link is in (Registry::confirms_link): a credential a push is accepted
from, a registry token whose scope has push, write permission on the
repository the push would land in (the repository itself, or a group's
writable member). That is the HEAD docker push sends before it skips a
layer's upload. Every other fetch records nothing, so the sweeper
releases an unreferenced blob at most that TTL after it was last pushed,
mounted or confirmed by a pusher.

docs/docker.md and the image page's delete dialog say what stays
pullable after a delete and for how long.

Fixes #576

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012e6DJbNZS4UTo7eDJvykUw
@schmitch
schmitch added this pull request to the merge queue Nov 17, 2026
Merged via the queue into main with commit b9876a3 Nov 17, 2026
20 checks passed
@schmitch
schmitch deleted the claude/576-deleted-image-blobs branch November 17, 2026 10:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Deleted images: their config and layers stay pullable by digest for as long as someone keeps fetching them

2 participants