feat: Slice 6 — server audit, route remove, releases/server prune

AUDIT_OPEN.md

@@ -1,5 +1,68 @@
 # Open audit items
 
+## 2026-11-05 — Dash auto-config Slice 6 CLI: cleanup / audit (P5)
+
+Plan: `../_internal/DASH_AUTOCONFIG_PLAN_2026-11-03.md` (section 9). Branch
+`slice6-cleanup`. `teploy server audit` (read-only findings + remedy argv),
+`route remove` (exact-block CAS), `releases prune --keep N`,
+`server prune --build-cache --dangling`, keep_versions D3 (unset → 5 after a
+recorded 7-day notice, `init` writes 3, 0 keeps everything), contracts rev 19.
+
+Deliberately report-only (design principle 4: hand-written config and other
+people's containers are never edited by teploy):
+- Hand-written Caddy blocks: dead upstream, non-local address. Evidence only.
+- Large container writable layers (managed ≥ 5 GB) and large unmanaged
+  containers (≥ 1 GB): a layer's data belongs to whoever wrote it; the
+  evidence names the move-to-volume fix.
+- Dangling volumes: may hold the only copy; there is never a remedy argv.
+- Unmanaged containers (portainer et al.): not teploy's to remove.
+- A managed block that still serves files (a `root` that exists), carries the
+  autodeploy webhook, or whose app is live: `route remove` refuses; the audit
+  reports the finding with the refusal as its evidence and a null remedy.
+- An upstream that is not this app's (a `caddy_extra` proxy to another app's
+  dead container): this app's redeploy cannot fix it, so no redeploy remedy.
+
+Deferred surface:
+- **Backup output sizes in the audit** (plan lesson: "backup jobs whose
+  outputs are implausibly small"): covered at write time by the engine's
+  minimum-size refusal (Slice 2, `internal/backup/engine.go`) and
+  `backup status`'s per-target last success/failure; the audit does not
+  re-walk buckets, which would need target credentials it must not use.
+- **`server audit` on a server with no caddyfile at all** reports no route
+  findings by construction; a Caddyfile with no Caddy container is flagged
+  (`caddyfile_without_caddy`, compute-2's case).
+- **Disk attribution beyond docker + /deployments**: `du` of other trees
+  (logs, /var) is not attempted; the pressure finding attributes what teploy
+  can name and links the reclaimable findings.
+- **Audit does not pin the notice file's mtime against clock skew**: the
+  notice is a RFC 3339 server-side timestamp written by the deploy that
+  starts it, same trust domain as state.json.
+
+Decisions worth reviewing:
+- `route remove` protects running-container versions of an app's name even
+  without state (`ownedContainers`, unlabelled `<app>-*` names included):
+  an older deploy's container keeps the name alive, so the route is not
+  proven dead until it is gone.
+- `releases prune --keep 0` (and the audit never emits it) prunes everything
+  the protections do not name — deliberately the opposite reading of
+  `keep_versions: 0`, documented in the command's help, because "retain none
+  beyond the protected set" is a coherent manual operation.
+- Automatic pruning under the default keep_versions runs only inside a
+  deploy of that app (never from the audit, never fleet-wide), so the notice
+  is always written by an operation that already holds the app lock.
+- `retentionProtected` includes any version with a running container, beyond
+  the plan's pins/current/previous: `docker rm -f` of a running stranger is
+  not something a cleanup command should do even inside the keep window.
+
+Verified: unit/hermetic only (mock SSH executors, the same discipline as
+`domains check`); every audit finding kind, all `route remove` refusal
+paths, the CAS refusal on a concurrent rewrite, prune selections (pins/
+current/previous/running survive, `--keep 0`, unreadable pins refuse),
+the notice-period state machine, and the read-only pin (no mutating verb,
+docker subcommand or upload in any command `server audit` issues). Not yet
+exercised live: the four fleet servers' first real audit run and the Dash
+cleanup page (Dash side, next slice).
+
 ## 2026-11-05 — Dash auto-config Slice 4 CLI: fleet backup engine (G1, G3, G4, G5)
 
 Plan: `../_internal/DASH_AUTOCONFIG_PLAN_2026-11-03.md` (sections 2.1-2.3, 5). Branch

CHANGELOG.md

@@ -4,6 +4,46 @@ All notable changes to teploy are documented here. Format follows [Keep a Change
 
 ## [Unreleased]
 
+### Added
+
+- `teploy server audit <server> --json`: what is left over on a server, each
+  finding with its evidence and — only where a CLI command fixes it safely —
+  the remedy argv. Stale and empty managed routes, managed routes with dead
+  upstreams, app state with no containers, versions beyond `keep_versions`,
+  reclaimable build cache and dangling images, large container writable
+  layers, large containers teploy does not manage, dangling volumes, a
+  Caddyfile with no Caddy container, scheduled jobs with no cron daemon, and
+  disk pressure with attribution. Read-only: every command the audit runs on
+  the server only reads (a test pins that). A source that could not be read
+  is an `unknown` check and yields no finding — never a clean bill.
+- `teploy route remove --app X`: removes a managed Caddy block proven dead
+  (no app state, no containers carrying the name, no upstream that still
+  resolves, no files it still serves, nothing unreadable), under the app's
+  lock and an exact-block compare-and-swap, so a concurrent rewrite is
+  refused rather than undone. Recorded as a `route.remove` event.
+- `teploy releases prune --app X --keep N`: removes old versions' containers
+  and images beyond `--keep`. Pinned, current, previous and running versions
+  are never removed whatever `--keep` says; an unreadable pin set, state or
+  container list refuses the prune. Recorded as a `releases.prune` event.
+- `teploy server prune --host S --build-cache --dangling`: reclaims docker's
+  unused build cache and dangling images — nothing else (no tagged image, no
+  container, no volume). Recorded as a `server.prune` event under `_fleet`.
+- Contracts: `server-audit-envelope` (schema + generated fixture, corpus rev
+  19); version-handshake advertises `server-audit-v1`.
+
+### Changed
+
+- `keep_versions` is now optional (decision D3): unset means 5, pruned by
+  deploys only after a 7-day notice recorded on the server (the deploy
+  output and `server audit` warn first); `teploy init` writes 3; `0` keeps
+  everything. Applied manifests record unset and N as before, so existing
+  digests hold; an explicit 0 records `keep_all_versions`.
+- `teploy remove` on an app whose directory is already gone (an earlier
+  removal finished) succeeds instead of failing its state deletion.
+- `teploy doctor` disk remediations point at `teploy server audit`, which
+  attributes the space; one named a `releases prune` command that did not
+  exist.
+
 ## [0.1.52] - 2026-11-05
 
 ### Added

README.md

@@ -134,7 +134,9 @@ drain_seconds: 10             # request drain: keep the old container serving in
                               # the drain policy — size it to your longest normal request.
 memory: 2g                    # cgroup RAM cap (docker units: 512m, 2g). Unset = unlimited.
 cpu: "1.5"                    # CPU cap in cores. Unset = unlimited.
-keep_versions: 3              # auto-prune older versions after deploy (0 = keep all, default)
+keep_versions: 3              # versions retained after a deploy. Unset = 5, pruned only after a
+                              # 7-day notice (teploy init writes 3); 0 keeps everything.
+                              # The current, previous and pinned versions are never pruned.
 backup_before_hooks: true     # back up running databases before a deploy whose pre_deploy hook will run (default off)
                               # `teploy pin <version>` protects a version from this prune
 
@@ -354,6 +356,7 @@ teploy plan                               # preview what a deploy would change (
 teploy plan --out plan.json               # write a bound plan record (config digest + target identity + generation)
 teploy apply plan.json                    # execute a reviewed plan; refuses naming what drifted since it was made
 teploy scale <count>                      # multi-server deploy + LB update
+teploy server audit <server>              # read-only: what is left over and what fixes it
 teploy version / update                   # version info and self-update
 ```
 
@@ -595,6 +598,40 @@ serves. Read-only, run from your machine over SSH, nothing new on the server:
 `ingress: external` and an unfronted `ingress: host` domain are labels: they
 carry a `skip` reason. A server that cannot be reached is reported as such.
 
+### Fleet cleanup & audit
+
+```bash
+teploy server audit <server>              # what is left over, with evidence and the fix
+teploy server audit <server> --json       # contracts/schema/server-audit-envelope.schema.json
+teploy route remove --app X --host S      # remove a managed Caddy block proven dead (CAS)
+teploy releases prune --app X --host S --keep N   # remove versions beyond N (pins/current/previous survive)
+teploy server prune --host S --build-cache --dangling   # reclaim unused build cache and dangling images
+```
+
+`server audit` changes nothing: every command it runs on the server only
+reads (a test pins that). Each finding says what it is, the evidence for it,
+and — only where a CLI command can fix it safely — the remedy:
+
+| Finding | Remedy |
+|---|---|
+| stale/empty managed route | `teploy route remove --app X` |
+| managed route, dead upstream | `teploy redeploy` (confirm) |
+| app state, no containers | `teploy remove --app X -y` (keeps volumes; confirm) |
+| versions beyond `keep_versions` | `teploy releases prune --keep N` (confirm) |
+| build cache / dangling images | `teploy server prune --build-cache --dangling` |
+| hand-written blocks, big writable layers, dangling volumes, unmanaged containers, disk pressure, a Caddyfile with no Caddy, scheduled jobs with no cron daemon | report only |
+
+A source that could not be read shows as `unknown` in `checks[]` and the
+findings depending on it are missing — a server is never reported clean
+because it could not be inspected. With `--json` the exit code is 0 whenever
+the envelope was produced; read `state` (`ok` / `partial` / `unreachable`).
+
+`route remove` only takes a block proven dead — no app state, no containers
+carrying the name, no upstream that still resolves (asked inside Caddy's
+network), no files it still serves, nothing unreadable — and commits under
+an exact-block compare-and-swap, so a deploy that rewrites the block in
+between is never undone.
+
 ### Accessories
 ```
 teploy accessory list              # list running accessories

contracts/MANIFEST.md

@@ -10,6 +10,7 @@ Neutron/Nucleus dependency and a public mirror.
 
 | Corpus rev | Emitting CLI | Machine Interface | Notes |
 |---|---|---|---|
+| 19 | main (Dash auto-config Slice 6: cleanup / audit) | 2 | Additive. New artifact server-audit-envelope (`teploy server audit <server> --json`, read-only: findings with stable `id` = kind:subject, severity, evidence, and remedy {argv (null when nothing the CLI can do safely), destructive, confirm}, per-source `checks` where unknown means findings depending on it are missing — never clean — and disk attribution), fixture GENERATED from the real `auditServer` over a mock executor (one of each finding incl. null-remedy, one unknown check). New remedy commands `route remove --app X` (exact-block CAS), `releases prune --keep N` (pins/current/previous/running never pruned) and `server prune --build-cache --dangling`; events `route.remove`, `releases.prune`, `server.prune` (the last under `_fleet`). keep_versions becomes optional: unset → 5 pruned only after a 7-day notice recorded server-side (`deploy/retention.go`), `init` writes 3, explicit 0 keeps everything and records `keep_all_versions` in the applied manifest (unset and N record what they always did, so existing digests hold). version-handshake advertises `server-audit-v1` (regenerated mi2.json). No MI bump. |
 | 18 | main (Dash auto-config Slice 4: fleet backup engine) | 2 | Additive. New artifacts backup-status-envelope (`backup status --json`: every DB-like container and app volume with coverage state, last success/failure, newest object, last verify, plus target and policy summaries, the aws CLI's `aws_cli_problem` and the target's `incoming_lifecycle`; no secrets), backup-run-envelope (`backup run --all --json`) and backup-policy-envelope (`server backup-policy show/set --json`), fixtures GENERATED from the real DTOs. backup-list rows gain optional `encrypted`, `previous_prefix`, and `database` only with `--databases` or `--adopted` (a caller restoring by id alone would replace the primary); backup-create gains `encrypted` and `objects`; backup-restore gains `pre_backup_id`; backup-target gains `previous_prefixes`. Event kinds `backup.all`, `backup_policy.set` and `backup_target.owners_clear` (a marker removal or refusal) under the pseudo-app `_fleet`; adopted containers log under `_adopted`. version-handshake advertises `backup-fleet-v1` (regenerated mi2.json). No MI bump. |
 | 17 | main (dash auto-config slice 5: domain health) | 2 | Additive. New artifact domains-check-envelope (`teploy domains check [--server] [--app] [--log-window] --json`: per (server, app, host) DNS via the fixed public resolvers, Caddy certificate failures from its logs — `failures` is null, never 0, on short or unavailable logs — and the route with upstream resolution; per-server state so an unreachable server is explicit), fixture GENERATED from the real `sweepDomains` over a mock executor. version-handshake advertises `domains-check-v1` (regenerated mi2.json). release-record's `caddy` gains optional `tls_http_only` (absent on older records; every caddy release since records true or false). No MI bump. |
 | 16 | main (server-authority spec P3 / plan D4, git sources) | 2 | Additive. New artifacts source-show-envelope (`config source show --json`: repository/branch/poll, public credential facts — type, deploy-key public half + fingerprint, token username, pinned host-key fingerprints — and the per-ref admission state), source-key-envelope (`config source key --json` for ssh/token and `host-key --json`) and trigger-poll-envelope (`trigger poll --json`: one result object per app — ref, head, admitted, reason vocabulary incl. force-push/retry-limit/not-due, deployed_release), all fixtures GENERATED from the real DTOs. version-handshake advertises `git-source-v1` (git: code source, per-app credentials outside /deployments, build_secrets) and `trigger-poll-v1` (the one admitter + reset/retry/install-timer) (regenerated mi2.json). No MI bump. |
@@ -60,6 +61,7 @@ Neutron/Nucleus dependency and a public mirror.
 | backup-run-envelope | yes (rev 18) | valid x2 (partial, skipped; real backupRunEnvelope) | teploy-cli |
 | backup-policy-envelope | yes (rev 18) | valid x2 (configured, unconfigured; real backupPolicyDTO) | teploy-cli |
 | domains-check-envelope | yes (rev 17) | valid (one server with each finding, one unreachable; real `sweepDomains`) | teploy-cli |
+| server-audit-envelope | yes (rev 19) | valid (one of each finding incl. null-remedy, one unknown check; real `auditServer` over a mock executor) | teploy-cli |
 | releases-envelope | yes (rev 14) | valid x2 (container app, static app; real releasesDTO) | teploy-cli |
 | rollback-result | yes (rev 14) | valid (real rollbackResult) | teploy-cli |
 | redeploy-result | yes (rev 14) | valid x2 (teploy-home, folder-home; real redeployDTO) | teploy-cli |