Skip to content

Deploys remove the old images they leave behind - #36

Merged
DevinMB merged 1 commit into
mainfrom
image-prune
Nov 16, 2026
Merged

DevinMB merged 1 commit into
mainfrom
image-prune

Conversation

@DevinMB

@DevinMB DevinMB commented Nov 16, 2026

Copy link
Copy Markdown
Owner

Deploys no longer leave old images behind. That gap filled deals-lab's 30 GB disk twice in two weeks (RCA, 2026-11-16).

Root cause: every deploy that moves a tag left the image it replaced, and nothing removed it. Argus's companion had the same gap as the old stack-sync. The only backstop was a disk alert at 95%, which on a 30 GB host left less room than one more pull of a 6 GB image.

What changes (design decision 24)

  • After a healthy deploy, the companion removes images of the repositories that stack runs which no container uses. It keeps the images the stack ran just before, so going back one version needs no download.
  • After a healthy Update Argus, the same happens for Argus's own image, keeping the previous release.
  • Once when a companion starts, every stack it deploys is cleaned the same way. Hosts with a backlog today are cleared when their companion updates to this release, not at each stack's next deploy.
  • Safe by construction: it never forces a removal, so Docker refuses to remove an image any container uses, running or stopped. It never touches another repository, so a locally built image is never removed (the Kafka case in the RCA). Nothing is removed after a deploy that failed. docker image prune -a was not used for exactly those reasons.
  • The deploy result says what went: "removed 12 old images, freeing 13.0 GB". The space is measured on the disk, because image sizes count shared layers.
  • Disks now alert at 90% used (was 95%) and show as filling from 80% (was 85%). Today no host is above 74%, so this adds no alerts now.

Docs: design.md (decision 24), guide.md (Deploys), security.md (what the companion may remove).

Tests: Go (which images go and which stay; an unhealthy deploy removes nothing; the deploy result; cleanup on start), UI (disk tones), Python (the 90% alert).

🤖 Generated with Claude Code

https://claude.ai/code/session_01NaZdmEt5VCJtgF27kW7pbJ

@DevinMB
DevinMB merged commit a474c2e into main Nov 16, 2026
4 checks passed
@DevinMB
DevinMB deleted the image-prune branch November 16, 2026 21:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant