Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
52 changes: 52 additions & 0 deletions AUDIT_OPEN.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,57 @@
# Open audit items

## 2026-11-05 — Dash auto-config Programme B CLI: server siblings

Plan: `../_internal/DASH_AUTOCONFIG_PLAN_2026-11-03.md` section 10. Branch
`programme-b-siblings`. `teploy server siblings <server> --json` (read-only
discovery: dev.teploy.product container/image label, forgejo via OCI labels,
version hints, published-port hints, unknown values passed through),
capability `siblings-v1`, contracts rev 20.

Deliberately out of scope (the plan puts them on the consumer):
- Probing, DNS, and anything URL-shaped: the command only observes labels
and port binds. Dash probes `/.well-known/teploy.json` and forgejo's
`/api/v1/version`; credential minting and env writes need a confirmed
`instance_id` + admin consent, neither of which exists here.
- Grouping a product's containers: one teploy app with web + worker
containers yields two rows (both carry the product label). The consumer
groups by its own probe; a CLI-side dedupe would guess which container
is "the" instance — exactly the substring guessing the plan forbids.

Open / deferred:
- Forgejo detection is substring-based by design (`forgejo/forgejo` or
`gitea` in the image's OCI title/source labels): upstream forgejo/gitea
images carry no teploy label, so their OCI labels are the only signal. A
third-party image that merely mentions gitea in its source label becomes
a candidate — it costs the consumer one failed probe, never a wrong
connection. `teploy.engine=`-style override (a manual label pinning the
product) is the escape hatch and is already honoured: dev.teploy.product
on the container wins over everything the image says.
- Image facts come from the image as the host knows it: a container run
from an image that was since re-tagged reports the CURRENT tag's labels.
Same trust domain as `docker ps`; not solvable without a registry read
this read-only command will not do.
- A stopped sibling is listed with its docker state (`state: exited`): a
candidate that is down is information, not noise. Consumers should read
`state` before probing.
- The port-hint read includes every container's binds (one batched inspect
for the whole host) rather than only the candidates': image-label
detection lands in the same round trip, and a third trip to re-inspect
just-discovered candidates would double the command's cost for no
privacy or safety gain. Both reads are inspects.
- Version gates: Dash must read `siblings-v1` from `teploy version --json`
before relying on the envelope (plan: version-gated). Not yet exercised
live against the fleet: first real run happens with the Dash slice.

Verified: unit/hermetic only (mock SSH executors, the same discipline as
`server audit`): every detection source and fallback, version priority,
port-hint extraction (dual-stack, loopback, tailnet, udp,
exposed-but-unpublished), pass-through of unknown products, partial vs ok
states (unlistable server, wholesale inspect failure), determinism,
JSON round-trip, ID-created image tag resolution, and the read-only pin
(no mutating verb or docker subcommand, no stdin, no uploads, ≤2 remote
commands). Schemas validated against the generated fixtures.

## 2026-11-05 — Dash auto-config Slice 6 CLI: cleanup / audit (P5)

Plan: `../_internal/DASH_AUTOCONFIG_PLAN_2026-11-03.md` (section 9). Branch
Expand Down
16 changes: 16 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,22 @@ All notable changes to teploy are documented here. Format follows [Keep a Change

## [Unreleased]

### Added

- `teploy server siblings <server> --json`: read-only sibling discovery
(Programme B) — teploy products (dash, observe, ship) by the
`dev.teploy.product` label (the container's own, else its image's) and the
forgejo family by the OCI labels on its image, each row with version hints,
the teploy app name when managed, and published-port hints where the
product's `/.well-known/teploy.json` may live. An unrecognised
`dev.teploy.product` value passes through verbatim under `unknown_labeled`
(a product newer than the CLI stays visible; the consumer decides).
Observes only — no probing, no DNS, no env or credentials — in at most two
remote commands, both pinned read-only by test. An unobserved server
reports `state: partial|unreachable`, never a clean "no siblings".
- Contracts: `siblings-envelope` (schema + generated fixtures, corpus rev
20); version-handshake advertises `siblings-v1`.

## [0.1.53] - 2026-11-05

### Added
Expand Down
38 changes: 38 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -357,6 +357,7 @@ teploy plan --out plan.json # write a bound plan record (config di
teploy apply plan.json # execute a reviewed plan; refuses naming what drifted since it was made
teploy scale <count> # multi-server deploy + LB update
teploy server audit <server> # read-only: what is left over and what fixes it
teploy server siblings <server> # read-only: teploy products and forgejo running on a server
teploy version / update # version info and self-update
```

Expand Down Expand Up @@ -632,6 +633,43 @@ network), no files it still serves, nothing unreadable — and commits under
an exact-block compare-and-swap, so a deploy that rewrites the block in
between is never undone.

### Sibling discovery

```bash
teploy server siblings <server> # teploy products + forgejo on that server
teploy server siblings <server> --json # contracts/schema/siblings-envelope.schema.json
```

What Dash (or you) runs before offering to connect the products that share a
server. Each teploy product image carries
`LABEL dev.teploy.product=<dash|observe|ship>` and serves an unauthenticated
`GET /.well-known/teploy.json`; this command finds the candidates without
probing anything:

- **product**: the container's `dev.teploy.product` label (`source: label`),
else its image's (`source: image_label`), else — for the forgejo family,
whose images carry no teploy label — the image's OCI labels
(`org.opencontainers.image.title`/`source` naming forgejo or gitea →
`source: oci_label`). A `dev.teploy.product` value the CLI does not
recognise passes through verbatim under `unknown_labeled`: a product newer
than the CLI stays visible, and the consumer decides what it knows.
- **version / app**: hints only — `dev.teploy.version`, else the teploy
release label when the container is teploy-managed, else the image's
`org.opencontainers.image.version`.
- **port_hints**: the container's published host binds (loopback and
tailnet/mesh binds included as-is) — where
`/.well-known/teploy.json` (or forgejo's `/api/v1/version`) may live.

One row per container: an app with web + worker containers yields two rows,
grouped by the consumer's probe. Observation only — no DNS, no HTTP, no URL
guessing, no env or credentials — and read-only end to end (a test pins every
command it issues only reads; the whole thing is at most two SSH round
trips). Confirming a candidate, minting credentials or writing env needs its
confirmed `instance_id` and an admin's consent — that is the consumer's
surface, never this command's. With `--json` the exit code is 0 whenever the
envelope was produced; read `state` (`ok` / `partial` / `unreachable`) — an
unobserved server is never a clean "no siblings".

### Accessories
```
teploy accessory list # list running accessories
Expand Down
2 changes: 2 additions & 0 deletions contracts/MANIFEST.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ Neutron/Nucleus dependency and a public mirror.

| Corpus rev | Emitting CLI | Machine Interface | Notes |
|---|---|---|---|
| 20 | main (Dash auto-config Programme B: siblings) | 2 | Additive. New artifact siblings-envelope (`teploy server siblings <server> --json`, read-only sibling discovery: teploy products by the `dev.teploy.product` label — the container's own, else its image's — and the forgejo family by the OCI `org.opencontainers.image.title`/`source` labels; per row version hints (dev.teploy.version, the teploy release label for a teploy app, else the image's `org.opencontainers.image.version`), the teploy app name, and `port_hints` from docker inspect's published binds; unrecognised `dev.teploy.product` values pass through verbatim under `unknown_labeled`; state ok/partial/unreachable so an unobserved server is never a clean "no siblings"). Observation only — no probing, no DNS, no env/credentials; at most two remote commands (the shared batched `docker ps`, then one batched inspect for port binds + image labels), pinned read-only by test. Fixtures GENERATED from the real `collectServerSiblings` over a mock executor (mixed: one of each detection class + pass-through; none). version-handshake advertises `siblings-v1` (regenerated mi2.json). No MI bump. |
| 19 | main (Dash auto-config Slice 6: cleanup / audit) | 2 | Additive. New artifact server-audit-envelope (`teploy server audit <server> --json`, read-only: findings with stable `id` = kind:subject, severity, evidence, and remedy {argv (null when nothing the CLI can do safely), destructive, confirm}, per-source `checks` where unknown means findings depending on it are missing — never clean — and disk attribution), fixture GENERATED from the real `auditServer` over a mock executor (one of each finding incl. null-remedy, one unknown check). New remedy commands `route remove --app X` (exact-block CAS), `releases prune --keep N` (pins/current/previous/running never pruned) and `server prune --build-cache --dangling`; events `route.remove`, `releases.prune`, `server.prune` (the last under `_fleet`). keep_versions becomes optional: unset → 5 pruned only after a 7-day notice recorded server-side (`deploy/retention.go`), `init` writes 3, explicit 0 keeps everything and records `keep_all_versions` in the applied manifest (unset and N record what they always did, so existing digests hold). version-handshake advertises `server-audit-v1` (regenerated mi2.json). No MI bump. |
| 18 | main (Dash auto-config Slice 4: fleet backup engine) | 2 | Additive. New artifacts backup-status-envelope (`backup status --json`: every DB-like container and app volume with coverage state, last success/failure, newest object, last verify, plus target and policy summaries, the aws CLI's `aws_cli_problem` and the target's `incoming_lifecycle`; no secrets), backup-run-envelope (`backup run --all --json`) and backup-policy-envelope (`server backup-policy show/set --json`), fixtures GENERATED from the real DTOs. backup-list rows gain optional `encrypted`, `previous_prefix`, and `database` only with `--databases` or `--adopted` (a caller restoring by id alone would replace the primary); backup-create gains `encrypted` and `objects`; backup-restore gains `pre_backup_id`; backup-target gains `previous_prefixes`. Event kinds `backup.all`, `backup_policy.set` and `backup_target.owners_clear` (a marker removal or refusal) under the pseudo-app `_fleet`; adopted containers log under `_adopted`. version-handshake advertises `backup-fleet-v1` (regenerated mi2.json). No MI bump. |
| 17 | main (dash auto-config slice 5: domain health) | 2 | Additive. New artifact domains-check-envelope (`teploy domains check [--server] [--app] [--log-window] --json`: per (server, app, host) DNS via the fixed public resolvers, Caddy certificate failures from its logs — `failures` is null, never 0, on short or unavailable logs — and the route with upstream resolution; per-server state so an unreachable server is explicit), fixture GENERATED from the real `sweepDomains` over a mock executor. version-handshake advertises `domains-check-v1` (regenerated mi2.json). release-record's `caddy` gains optional `tls_http_only` (absent on older records; every caddy release since records true or false). No MI bump. |
Expand Down Expand Up @@ -62,6 +63,7 @@ Neutron/Nucleus dependency and a public mirror.
| backup-policy-envelope | yes (rev 18) | valid x2 (configured, unconfigured; real backupPolicyDTO) | teploy-cli |
| domains-check-envelope | yes (rev 17) | valid (one server with each finding, one unreachable; real `sweepDomains`) | teploy-cli |
| server-audit-envelope | yes (rev 19) | valid (one of each finding incl. null-remedy, one unknown check; real `auditServer` over a mock executor) | teploy-cli |
| siblings-envelope | yes (rev 20) | valid x2 (mixed detection classes + pass-through, none; real `collectServerSiblings` over a mock executor) | teploy-cli |
| releases-envelope | yes (rev 14) | valid x2 (container app, static app; real releasesDTO) | teploy-cli |
| rollback-result | yes (rev 14) | valid (real rollbackResult) | teploy-cli |
| redeploy-result | yes (rev 14) | valid x2 (teploy-home, folder-home; real redeployDTO) | teploy-cli |
Expand Down
93 changes: 93 additions & 0 deletions contracts/fixtures/siblings-envelope/valid/mixed.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
{
"machine_interface": 2,
"server": "infra-home",
"host": "192.0.2.10",
"observed_at": "2026-11-05T12:00:00Z",
"state": "ok",
"siblings": [
{
"container": "dash-web-1a2b3c4",
"product": "dash",
"version": "v0.3.1",
"state": "running",
"image": "ghcr.io/useteploy/dash:v0.3.1",
"port_hints": [
{
"host_ip": "127.0.0.1",
"host_port": 49221,
"container_port": 3000,
"proto": "tcp"
},
{
"host_ip": "::",
"host_port": 49221,
"container_port": 3000,
"proto": "tcp"
}
],
"source": "image_label"
},
{
"container": "forgejo",
"product": "forgejo",
"version": "11.0.1",
"state": "running",
"image": "codeberg.org/forgejo/forgejo:11",
"port_hints": [
{
"host_ip": "127.0.0.1",
"host_port": 3000,
"container_port": 3000,
"proto": "tcp"
}
],
"source": "oci_label"
},
{
"container": "observe-web-7cf98cae",
"product": "observe",
"version": "7cf98cae",
"app": "observe",
"state": "running",
"image": "ghcr.io/useteploy/observe:7cf98cae",
"port_hints": [
{
"host_ip": "100.108.123.49",
"host_port": 3000,
"container_port": 3000,
"proto": "tcp"
}
],
"source": "label"
},
{
"container": "observe-worker-7cf98cae",
"product": "observe",
"version": "7cf98cae",
"app": "observe",
"state": "running",
"image": "ghcr.io/useteploy/observe:7cf98cae",
"port_hints": [],
"source": "label"
}
],
"unknown_labeled": [
{
"container": "lullmail-web-9f8e7d6",
"product": "lullmail",
"version": "9f8e7d6",
"app": "lullmail",
"state": "running",
"image": "ghcr.io/fylun/lullmail:latest",
"port_hints": [
{
"host_ip": "0.0.0.0",
"host_port": 8080,
"container_port": 8080,
"proto": "tcp"
}
],
"source": "label"
}
]
}
9 changes: 9 additions & 0 deletions contracts/fixtures/siblings-envelope/valid/none.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
{
"machine_interface": 2,
"server": "infra-home",
"host": "192.0.2.10",
"observed_at": "2026-11-05T12:00:00Z",
"state": "ok",
"siblings": [],
"unknown_labeled": []
}
1 change: 1 addition & 0 deletions contracts/fixtures/version-handshake/valid/mi2.json
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@
"server-rename",
"server-status-machine",
"server-update",
"siblings-v1",
"template-var-stdin",
"trigger-convergence-v1",
"trigger-poll-v1",
Expand Down
54 changes: 54 additions & 0 deletions contracts/schema/siblings-envelope.schema.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://teploy.github.io/contracts/schema/siblings-envelope.schema.json",
"title": "`teploy server siblings <server> --json` envelope: sibling product candidates on a server, as the containers and their images report them (rev 20). Observation only — no probing, no DNS, no credentials; the consumer confirms via each product's own discovery endpoint.",
"type": "object",
"required": ["machine_interface", "server", "host", "observed_at", "state", "siblings", "unknown_labeled"],
"properties": {
"machine_interface": {"type": "integer", "minimum": 1},
"server": {"type": "string", "minLength": 1},
"host": {"type": "string"},
"observed_at": {"type": "string", "format": "date-time"},
"state": {"enum": ["ok", "partial", "unreachable"], "description": "partial means the container listing or the batched inspect failed: empty arrays are 'unobserved', never 'no siblings'"},
"error": {"type": "string"},
"siblings": {
"type": "array",
"items": {"$ref": "#/$defs/sibling"},
"description": "recognised candidates: teploy products (dash, observe, ship) and the forgejo family, one row per container (a multi-container product app yields several rows; the consumer groups them by its own probe)"
},
"unknown_labeled": {
"type": "array",
"items": {"$ref": "#/$defs/sibling"},
"description": "containers carrying a dev.teploy.product value this CLI does not recognise, passed through verbatim — a product newer than the CLI stays visible; the consumer decides what it knows"
}
},
"$defs": {
"sibling": {
"type": "object",
"required": ["container", "product", "state", "image", "port_hints", "source"],
"properties": {
"container": {"type": "string", "minLength": 1},
"product": {"type": "string", "minLength": 1},
"version": {"type": "string", "description": "hint, in priority order: the container's dev.teploy.version label, the teploy release label when the container belongs to a teploy app, the image's org.opencontainers.image.version; absent when nothing says"},
"app": {"type": "string", "description": "the teploy app name when the container is teploy-managed; absent otherwise"},
"state": {"type": "string", "description": "docker's container state (running, exited, …)"},
"image": {"type": "string", "description": "the image ref docker reports, with a 12-hex image ID resolved to its first repo tag when the facts hold one"},
"port_hints": {
"type": "array",
"items": {
"type": "object",
"required": ["host_ip", "host_port", "container_port", "proto"],
"properties": {
"host_ip": {"type": "string", "description": "as docker reports the bind: 0.0.0.0 (every interface), :: (v6), 127.0.0.1 (loopback only), or a mesh/tailnet address"},
"host_port": {"type": "integer", "minimum": 1, "maximum": 65535},
"container_port": {"type": "integer", "minimum": 1, "maximum": 65535},
"proto": {"enum": ["tcp", "udp"]}
}
},
"description": "published host binds where the product's discovery endpoint may live; hints, not endpoints"
},
"source": {"enum": ["label", "image_label", "oci_label"], "description": "label: the container's own dev.teploy.product; image_label: the image's; oci_label: the image's org.opencontainers.image.title/source names the forgejo family"}
}
}
}
}
21 changes: 21 additions & 0 deletions internal/cli/contracts_golden_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -624,6 +624,27 @@ func TestContractsServerAuditGolden(t *testing.T) {
writeFixture(t, "server-audit-envelope/valid/findings.json", auditKitchenSink().audit(t))
}

// TestContractsSiblingsGolden pins `teploy server siblings <server> --json`
// (rev 20, siblings-v1): one server carrying each detection class — a teploy
// product by its container label (a two-container app), one by its IMAGE's
// label, forgejo by its OCI labels, an unrecognised product passed through
// verbatim — and one with nothing on it. Generated from the real
// `collectServerSiblings` over a mock executor.
func TestContractsSiblingsGolden(t *testing.T) {
writeFixture(t, "siblings-envelope/valid/mixed.json", siblingsKitchenSink().collect(t))

none := siblingsFixture{
containers: strings.Join([]string{
auditContainer("caddy", "caddy:2", "running", "2026-05-01 00:00:00 +0000 UTC", nil),
auditContainer("blog-web-v3", "blog:v3", "running", "2026-11-04 10:00:00 +0000 UTC",
map[string]string{"teploy.app": "blog", "teploy.version": "v3"}),
}, "\n"),
inspectPorts: "/blog-web-v3\t{}\n/caddy\t{}",
inspectImgs: siblingsImageID("blog") + "\t" + `["blog:v3"]` + "\t" + `{}`,
}
writeFixture(t, "siblings-envelope/valid/none.json", none.collect(t))
}

func TestContractsBackupFleetGolden(t *testing.T) {
at := time.Date(2026, 11, 4, 3, 17, 4, 0, time.UTC)
failedAt := at.Add(-time.Hour)
Expand Down
Loading
Loading