From 46accae750995807f0c81c8d9cccd91d6ba517a0 Mon Sep 17 00:00:00 2001 From: mintaka Date: Sun, 23 Aug 2026 23:15:07 -0400 Subject: [PATCH] docs(design): compass release-bundling + distribution surface (RIG-1746, RIG-2477, RIG-2608) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two tightly-coupled RIG-1746 design records, consolidated into one PR (Matt ruled 2026-08-25 they ship together; supersedes standalone PR #460): - compass-release-bundling.md — the GitHub Releases MECHANISM: per-build `build-` prereleases + the reserved `v*` whole-product-semver lane on compass GHA, per-arch binary assets, GHCR image by digest, nix-outputs manifest. Load-bearing forks (OQ-1/2/3) and the whole-product semver policy (OQ-7) ruled by Matt; freezes as the contract its T1-T5 execute against. - compass-distribution/design.md — the install/distribution SURFACE: the client app's full per-OS matrix (Linux x86_64 tarball + macOS arm64 signed/notarized .app-in-dmg), install channels (homebrew tap, nix flake, raw release assets), the headless self-host stack's host-level KVM bring-up, and postgres provisioned as a zero-config container. Consumes the release mechanism as its publishing rail; extends its Fork 2(i) asset boundary, never redesigns it. Status: distribution record carries four load-bearing Open Questions for Matt at the design-PR gate (RIG-2666: Apple Developer account, postgres image mechanism, container-teardown architecture, darwin CI cadence); ledger rows DL-241..244 land in this PR at freeze once the OQs are ruled. Ledger-impact: proposes DL-241..244 (client per-OS matrix, install channels, host-KVM stack, postgres-as-container); rows land at freeze after the load-bearing OQs are ruled. No status flips (DL-217 already Superseded by DL-238). Co-authored-by: Matt Wilkinson --- docs/designs/DECISIONS.md | 7 + .../platform/compass-distribution/design.md | 852 ++++++++++++++++++ .../platform/compass-release-bundling.md | 530 +++++++++++ 3 files changed, 1389 insertions(+) create mode 100644 docs/designs/platform/compass-distribution/design.md create mode 100644 docs/designs/platform/compass-release-bundling.md diff --git a/docs/designs/DECISIONS.md b/docs/designs/DECISIONS.md index dbb03a29..3cae2417 100644 --- a/docs/designs/DECISIONS.md +++ b/docs/designs/DECISIONS.md @@ -249,6 +249,13 @@ check enforces the mechanical half. Full rationale: | DL-236 | `compass-stack` survives the app's retirement of embedded mode as the standalone headless single-user bring-up CLI (`up`/`down`/`status`, DL-183 teardown intact); the app never spawns, supervises, or tears down a stack. Refines DL-108 (which stays Active — the supervisor design lives; only the app's shell-spawn invocation retires) | Active (Matt, 2026-08-23) | [client-only §A1](product/compass-native-client-only/design.md#a1--thesis-compass-app-is-a-native-client-only-binary) | | DL-237 | app.toml is client-only: absent or `mode="client"` → client (`server_url` required); `mode="embedded"` is a legible retirement error, never a compatibility arm; the `--mode`/`$COMPASS_APP_MODE` override retires with the second mode. Supersedes the embedded-default mode-selection half of DL-109 (whose keychain-first bearer clause stays Active) | Active (Matt, 2026-08-23) | [client-only §A3](product/compass-native-client-only/design.md#a3--appconfig-the-clean-cut-client-is-the-only-mode) | | DL-238 | The app bundle is the THIN CLIENT: `compass-app` + `dist` + `.desktop` + LICENSE, no sidecar binaries, no postgres tooling; headless-stack distribution (`compass-stack`+`compass-postgres` onto the dedicated machine) is a releases-lane follow-up, and RIG-2477 re-scopes to client-app release artifacts with macOS as the first new target. Supersedes DL-215 (sidecar `bin/` + PATH threading) and DL-217 (bundled postgres tooling) | Active (Matt, 2026-08-23) | [client-only §A4](product/compass-native-client-only/design.md#a4--cutover-plan-sequenced-clean-no-shims) | +| DL-254 | The client app ships a full per-OS matrix from day one — Linux x86_64 (thin-client tarball, DL-238 content) AND macOS arm64 (signed+notarized `.app` in a dmg, built on a macOS runner, never cross-compiled) — attached to the RIG-1746 release lanes, extending the release-bundling record's Fork 2(i) asset boundary and resolving its OQ-1 deferral (RIG-2477) | Active (Matt, 2026-08-24) | [distribution §S1](platform/compass-distribution/design.md#s1--client-app-per-os-matrix) | +| DL-255 | Client install channels are homebrew (RigelBuild/homebrew-tap: cask for the macOS app + a cross-OS CLI formula, semver-bumped by the release workflow) and a repo `flake.nix` (client + stack packages pinned to the devenv.lock nixpkgs), over the raw release assets; distro-native packages (deb/rpm/AUR) deferred | Active (Matt, 2026-08-24) | [distribution §S2](platform/compass-distribution/design.md#s2--client-app-install-channels) | +| DL-256 | The self-host stack stays a host-level bring-up on a KVM-capable Linux machine (`compass-stack up`; microVM D3 hard-fail consumed, no compose/Swarm packaging); `compass-stack` joins the release binary matrix, and the flake + preflight + self-host doc are its install surface — resolving the client-only record's OQ-3 (RIG-2608) | Active (Matt, 2026-08-24) | [distribution §S3](platform/compass-distribution/design.md#s3--self-host-stack-host-level-kvm-bring-up) | +| DL-257 | Postgres leaves the installed stack's host-process tree: the zero-config default is a dedicated postgres container run by the supervisor via rootless podman (socket-dir bind-mounted so the DSN contract is unchanged); a user-supplied DSN (`--database-external`) opts out. The image is the STOCK upstream `postgres:18` pinned by DIGEST — NOT a nix2container build of nixpkgs `postgresql` (that is the agent-shell mechanism, not a bundled stock service) and NOT a custom wrapper-entrypoint image; the official entrypoint does initdb+createdb+SIGTERM drain, so the `compass-postgres` wrapper collapses into env config (`POSTGRES_DB`, `POSTGRES_HOST_AUTH_METHOD=trust`, PGDATA volume, `unix_socket_directories`) on the container path and stays the host/dev-path bring-up. The postgres-as-container split was ruled 2026-08-24; the stock-`postgres:18`-by-digest image mechanism 2026-08-25. Supersedes the host-prerequisite interim answer (client-only OQ-3) atop the already-superseded DL-217; resolves distribution OQ-5 | Active (Matt, 2026-08-25) | [distribution §S4](platform/compass-distribution/design.md#s4--postgres-as-container-the-dl-217-supersession) | +| DL-258 | macOS signing identity provisioning waits for the company rename (Sealed Security Inc → Rigel AI Software Inc, same DE entity so its D-U-N-S carries over as an update, not a new request); enroll the Apple Developer Program org clean as Rigel AI Software Inc post-rename. Nothing pre-GA needs public signing, so dev/internal builds self-sign meanwhile (ad-hoc `codesign --sign -`, mandatory on Apple Silicon, or a local self-signed cert — no Apple account); the Developer ID cert + notarization (the distribute-to-other-Macs leg) is GA-gated on the enrollment. Resolves distribution OQ-2 | Active (Matt, 2026-08-25) | [distribution §GC6](platform/compass-distribution/design.md#global-constraints) | +| DL-259 | Containerized-postgres teardown extends the DL-183 pgid record to format v2: `pgidFileVersion` `"1"`→`"2"`, entries become a kind-tagged discriminated union (`proc ` torn down by group signal as today; `ctr ` torn down by `podman stop`/`rm -f`), `readPgidFile` dispatches on the tag and a shipped v1 binary hard-errors on a v2 entry line by the entry-grammar (not a header check) under the unchanged signal-off-a-half-understood-record discipline. The per-agent microVMs are the runner's sandbox one layer below the stack and are untouched (the stack tears the runner down by pgid as a plain host process); what forces v2 is postgres containerizing, not microVMs. Extends DL-183 (which stays Active); resolves distribution OQ-7 | Active (Matt, 2026-08-25) | [distribution §S4](platform/compass-distribution/design.md#s4--postgres-as-container-the-dl-217-supersession) | +| DL-260 | darwin CI cadence mirrors the shipped affected-on-PR + full-sweep-on-main + nightly shape (`ci.yml:25-36`): a `macos-14` compile+bundle sweep runs on every push to main AND nightly ALWAYS (the backstop), plus an affected-on-PR leg gated by a small ubuntu pre-job asking moon which projects the PR affects (moon's own affected-detection — the signal `moon ci` uses — never a GitHub `paths:` filter, so moon stays the single source of affected-truth per the ci.yml header's rejection of a YAML project list), so a scarce mac runner spins up on a PR only when a darwin-relevant project is affected. Resolves distribution OQ-8 | Active (Matt, 2026-08-25) | [distribution §T3](platform/compass-distribution/design.md#t3--macos-app-bundle--ci-lane-unsigned) | ## Agent batteries diff --git a/docs/designs/platform/compass-distribution/design.md b/docs/designs/platform/compass-distribution/design.md new file mode 100644 index 00000000..64593936 --- /dev/null +++ b/docs/designs/platform/compass-distribution/design.md @@ -0,0 +1,852 @@ +# Compass distribution & install surface + +Status: Draft +Linear: RIG-2477 (client app per-OS matrix + install channels), RIG-2608 +(headless self-host stack distribution). Scope expansion of RIG-1746 ruled by +Matt 2026-08-24. + +> **Companion record, same PR — composes, never duplicates:** +> [`../compass-release-bundling.md`](../compass-release-bundling.md) (the +> binary/image Release lane — per-build `build-` prereleases + the +> reserved `v*` semver lane on compass GHA, per-arch binary assets, GHCR image +> by digest, nix-outputs manifest). The two records ship and freeze together +> in one PR because they are tightly coupled: this record CONSUMES that lane as +> its publishing rail and decides what the release-bundling record deliberately +> deferred: its OQ-1 ruling ("the desktop lane (compass-app, compass-stack) is +> DEFERRED … decided in the native-packaging lane", filed as RIG-2477) is +> resolved HERE. (That pointer names the *native-packaging* lane as the +> deciding record; the client-only pivot DL-238 re-homed app distribution into +> THIS record, so the deferral chain is repaired via DL-238 — the decision +> lands here, not in `compass-native-packaging/design.md`.) +> Where this record extends a release-bundling decision (the release asset-set +> boundary, Fork 2(i)), it says so explicitly. + +## Problem / Intent + +Compass has a release lane (the release-bundling record) but no install surface: the thin client app +(`compass-app`, post client-only pivot DL-235/DL-238) has no per-OS artifacts +and no install channel beyond a Linux tarball built locally, and the headless +self-host stack (`compass-stack` + `compass-server` + `compass-runner` + +postgres) has no distribution home at all — the client-only record's OQ-3 +recorded exactly this orphan ("`compass-stack`/`compass-postgres` have no +release-artifact home today", +`../../product/compass-native-client-only/design.md:447-462`). This record +designs both surfaces: the client's full OS matrix (Linux AND macOS together) +with the install channels users actually reach for (homebrew, nix flakes, a +tarball), and the self-hoster's host-level KVM-stack bring-up with postgres +provisioned as a dedicated container image out of the box. + +## Decisions encoded (Matt's rulings — not open questions) + +Two rulings from Matt (2026-08-24) are the frozen premises of this record; +nothing below re-litigates them: + +1. **Client app (RIG-2477): full per-OS matrix + install channels.** The + client ships on Linux AND macOS together (not Linux-first), and beyond a + tarball it ships through OS install channels — **nix flakes** and + **homebrew (especially for macOS)**. This is an app-distribution surface, + not a bare artifact attach. +2. **Self-host stack (RIG-2608): host-level KVM stack + postgres-as-container.** + The headless stack STAYS a host-level bring-up on a KVM-capable machine — + it cannot ship as a plain compose/Swarm config droppable on any host, + because the microVM runtime hard-requires `/dev/kvm` + (`../compass-elastic-session-runtime/microvm-runner.md:230-236`, quoted in + Global Constraint 1). **Postgres moves OUT of the stack's host-process + tree**: the simple path runs a **dedicated postgres container image** out + of the box with zero thought; a user can always point the stack at their + own Postgres instead. This SUPERSEDES DL-217's bundle answer (already + `Superseded by DL-238`) AND the client-only record's interim OQ-3 + recommendation of "postgres tooling = host prerequisite on a dedicated + machine" (`compass-native-client-only/design.md:456-460`). + +## Global Constraints + +1. **The KVM floor is consumed, not designed.** The stack host MUST expose + `/dev/kvm` to the Runner uid; the runtime hard-fails without it — + `../compass-elastic-session-runtime/microvm-runner.md:230-233`: "**KVM-absent + ⇒ hard-fail (D3):** with no container fallback, `/dev/kvm` absence (or any + preflight failure) aborts Runner startup with an error naming the missing + capability". Consequence: every stack install channel targets **Linux + x86_64 with KVM**; there is no macOS or no-KVM stack story, ever, in this + record. +2. **The publishing rail is the release-bundling record's Release lane.** All downloadable artifacts + attach to the `build-` prerelease / `v*` semver Releases on + `RigelBuild/compass` GHA (`../compass-release-bundling.md` Forks 1/3); + this record adds assets and channels, never a second Release mechanism. + Its posture rules inherit: no PR-event trigger on any workflow holding + `contents: write`; immutable identities first, moving pointers last; + `v*` tag creation enforced by ruleset + ancestry guard. +3. **Module path + toolchain.** One Go module + `github.com/RigelBuild/compass/go` (`go/go.mod:13`), Go pinned via + `tools/toolchain/versions/go.nix`; builds use the pinned toolchain, never + a `setup-go` drift (release-bundling record Global Constraint 6). +4. **One version stamp across every stack binary in a bundle/release.** The + attach check hard-fails on mismatch — `go/cmd/compass-stack/main.go:35-38`: + "It feeds Deps.ExpectedVersion, so the attach mismatch check compares a + live server's version against this build's". Every channel MUST install + stack binaries from the SAME release tag. +5. **macOS does not cross-compile from an ubuntu runner.** The app on macOS + links the system WebKit framework — `devenv.nix:237-238`: "on macOS the app + links the system WebKit framework, so the closure is Linux's alone" — so + the macOS lane runs on a **macOS arm64 runner** (GHA `macos-14`/`macos-15`) + with cgo against system frameworks. +6. **macOS artifacts are signed + notarized before any PUBLIC channel + carries them.** Gatekeeper quarantines an unsigned downloaded `.app`/dmg; + homebrew-cask installs of unsigned apps are a broken first-run. Public + signing = a Developer ID Application certificate + `codesign --options + runtime`; notarization = `xcrun notarytool submit` + staple. Secrets (cert + p12 + App Store Connect API key) live as GHA repo secrets on the release + workflow only (never a PR-triggered workflow, per Constraint 2). The + Developer ID identity is GA-gated on Apple Developer Program org enrollment + (OQ-2 / DL-258): dev/internal builds self-sign in the interim — ad-hoc + `codesign --sign -` (mandatory on Apple Silicon) or a local self-signed + cert, neither needing an Apple account — so T2/T3 proceed unblocked and + only the public-channel T4/T5-cask leg waits on the enrollment secrets. +7. **The postgres major is pinned and matches the repo's parity pin.** The + dev shell pins bare `postgresql` "for strict parity" (`devenv.nix:123-128`, + postgresql 18.x at the current lock); the postgres container image is the + stock upstream `postgres:18` pinned BY DIGEST (OQ-5 / DL-257), the same + major, so the private cluster's on-disk format never skews between a + dev-box stack and an installed stack — and the digest pin (not a mutable + tag) matches the discipline `ci.yml:138-144` already applies to the pgtest + service image, since generated-tsvector behavior is major-version-sensitive. +8. **Rootless podman is the stack host's container runtime for the postgres + container.** The stack host already speaks podman (the agent image is + `podman pull`ed, DL-112) and the repo's container tooling is podman + throughout; the postgres container reuses it — no docker, no compose file + as the product interface. +9. **Thin-client bundle content is frozen by DL-238**: `compass-app` + `dist` + + `.desktop` + LICENSE on Linux (`app-bundle/build.sh:2-5`: "the thin + CLIENT bundle: the gtk3 shell (compass-app) + the UI dist + the desktop + file + LICENSE. No sidecar binaries, no postgres tooling"); the macOS + bundle is the `.app` equivalent (binary + dist inside the app bundle). + This record distributes that content; it does not reopen its shape. +10. **Scripts with real logic are TypeScript, not bash** + (`rule://scripts-ts-over-bash`); nix + build orchestration glue may stay + bash per the `build.sh:7-10` precedent. + +## Approach + +Two surfaces, one publishing rail. + +### S1 — Client app: per-OS matrix + +**Linux (x86_64).** The artifact exists: the thin client tarball +`compass-app--linux-amd64.tar.gz`, store-rpathed against the +devenv.lock-pinned GTK closure (DL-214 stays Active for it; +`app-bundle/build.sh:54-60` builds it with the pinned cc + PKG_CONFIG_PATH). +S1 attaches it to the release-bundling Releases: the release workflow gains an +`app-bundle` build step (or invokes `app-bundle/build.sh`) and uploads the +tarball + its checksum. **This extends the release-bundling record's Fork 2(i) asset boundary** — +that record excluded `compass-app` from v1 "to keep the asset set meaning +something" and deferred it to exactly this record (its OQ-1 ruling). The +honest limit DL-214 named stays: a box with no nix store cannot run the +tarball's rpathed binary — which is precisely why the *channels* below (nix +flake first) matter more than the raw asset on Linux. + +**macOS (arm64).** The new target. Three facts shape it: + ++ The shell today is gated `//go:build unix && gtk3` + (`go/cmd/compass-app/main.go:1`) and the non-gtk3 entrypoint is an + exit-1 stub (`main_nogtk3.go:24-28`); GTK3 is "the only Linux stack + available in this repo's frozen toolchain" (`main_nogtk3.go:5-7`). A darwin + build therefore needs a **build-tag restructure**: the Wails v3 shell + compiled on darwin without the gtk3 tag (Wails selects the system WebKit + backend on darwin), i.e. the gate becomes + `(linux && gtk3) || darwin` on the real entrypoint, with the stub covering + the remaining untagged-linux module build. The client-only pivot already + removed every Linux-only runtime dependency from the app (the runner was + the Linux-only piece, `compass-native-client-only/design.md:216-220`: + "the Linux-only-ness was the runner's … and the runner no longer ships in + the app"), so this is a build-surface change, not a feature port. ++ No cross-compile (Global Constraint 5): a `macos-14` (arm64) GHA job in the + release workflow builds the `.app` bundle — `Compass.app/Contents/MacOS/ + compass-app` + `Contents/Resources/dist/` (the same beside-the-executable + dist resolution the shell already does, `main.go:313-321` per the packaging + record) + `Info.plist` — then signs, notarizes, staples, and wraps it in a + `.dmg` (the homebrew-cask-native format). ++ Signing is a hard gate for the public channels (Global Constraint 6); the + lane lands in two steps — unsigned ad-hoc build proving the runner + bundle + (T3), then signing/notarization (T4) — so the mac build isn't blocked on + Apple-account provisioning. + +Arch matrix: `linux-amd64` + `darwin-arm64` only, matching the release-bundling record's darwin +choice for the CLI ("shipping darwin daemon builds would assert a support +surface nothing consumes" — same logic here for darwin-amd64; OQ-4). + +### S2 — Client app: install channels + +Three channels in priority order; each consumes the SAME release assets +(never rebuilds from divergent sources): + +1. **Homebrew (macOS-first, the ruling's emphasis).** A + `RigelBuild/homebrew-tap` repo carrying: + + a **cask** `compass` installing the notarized `.dmg` (macOS app), and + + a **formula** `compass-cli` installing the per-arch CLI binaries the release-bundling record + already attaches (`compass__darwin-arm64` / `linux-amd64`) — so + `brew install rigelbuild/tap/compass-cli` works on macOS AND Linux with + zero new build machinery. + The semver (`v*`) release workflow bumps the tap automatically (a small + step templating version + sha256 into the cask/formula and pushing to the + tap repo with a tap-scoped fine-grained PAT — the one new standing + *write* credential to an external repo this record adds, scoped to + `RigelBuild/homebrew-tap` contents only; the Apple signing secrets + Constraint 6 / T4 add are read-only material consumed on the release + workflow, not push credentials). + Per-build prereleases do NOT bump the tap: brew users track semver. +2. **Nix flake.** The compass repo gains a `flake.nix` (it has none today — + the repo is devenv-based, `devenv.yaml:1-9`) exposing packages: + `compass` (CLI), `compass-app` (Linux gtk3 client, reusing + `tools/toolchain/gtk-closure.nix` — the same 13-package set the bundle + links against), and the stack set (S3). `nix profile install + github:RigelBuild/compass#compass-app` (or a NixOS/home-manager module + consuming the flake) is the first-class Linux channel — it sidesteps + DL-214's no-nix-store limit entirely, because the flake install IS the nix + store. The flake pins nixpkgs to the devenv.lock revision (the + `gtk-e2e-env.nix:9-13` precedent: "Pins nixpkgs to the SAME devenv.lock + revision the dev shell and gate-tools.nix resolve") so flake-built and + bundle-built binaries link one closure. +3. **Tarball / dmg direct download.** The raw release assets (S1) remain the + channel of last resort and the substrate the other two wrap. + +Native Linux distro packages (deb/rpm/AUR) are explicitly deferred (OQ-1): +flake + brew-on-Linux + tarball cover the Linux install matrix without +taking on per-distro packaging debt. + +### S3 — Self-host stack: host-level KVM bring-up + +The stack stays exactly what the client-only record kept it as: "the +standalone headless single-user bring-up CLI" +(`compass-native-client-only/design.md:112-119`), running host-level because +the Runner opens `/dev/kvm` and drives cloud-hypervisor/virtiofsd/passt as +ordinary user processes (`devenv.nix:198-201`: "All three are ordinary user +binaries that open(2) /dev/kvm but need no capability or device node of their +own; the host-level /dev/kvm enablement is a separate, out-of-repo concern"). +A Docker-Swarm/compose packaging of the stack is rejected structurally: it +would either demand privileged `/dev/kvm` device passthrough into a container +(a worse posture than host processes) or silently lose the microVM boundary — +and D3 forbids degrading (Global Constraint 1). + +Distribution, three legs: + ++ **Release binaries.** `compass-stack` joins the release-bundling binary asset set + (linux-amd64), **extending Fork 2(i)'s boundary** the same way S1 does for + the app: the v1 exclusion of `compass-stack` was the OQ-1 deferral this + record resolves. `compass-server`/`compass-runner` are already attached. + `compass-postgres` (the Go wrapper) is NOT attached as a host binary — it + moves into the postgres container image (S4), which is its only shipped + home; the host-level LookPath spawn remains only for the dev-box/devenv + path. ++ **Nix flake (the recommended channel).** The same `flake.nix` (S2 leg 2) + exposes `compass-stack`, `compass-server`, `compass-runner`, and a + `compass-stack-env` convenience (the microVM userspace trio — + cloud-hypervisor, virtiofsd, passt — at the devenv.lock pin, which the + Runner preflight version-floors: `devenv.nix:204-206`: "the shell provides + one pinned version from devenv.lock; the runtime preflight (V2a+) will + enforce the floor"). A NixOS module (`services.compass-stack`) wrapping + `compass-stack up` in a systemd unit is the polished endgame for this leg + (OQ-3 sizes it). ++ **Bring-up UX.** `compass-stack up` stays the one entry point; this record + adds (a) a host preflight surfacing the KVM/podman prerequisites at + install-time rather than first-`up` (its own minimal checks — no Runner + preflight function exists to reuse today; T9 carries the dependency + honesty), and (b) an operator doc + (`docs/self-host.md`) covering the two supported shapes — dedicated + KVM machine, or one-box localhost-TLS (sanctioned by the client-only + record's OQ-6 ruling). + +### S4 — Postgres-as-container (the DL-217 supersession) + +**Decision (Matt's ruling 2):** the simple path provisions postgres as a +dedicated container image the stack runs out of the box; a user-supplied DSN +opts out entirely. + +Mechanics, grounded in the current seams: + ++ **The image (Matt's OQ-5 ruling): the stock upstream `postgres` image, + pinned to `postgres:18` by digest — NOT a nix2container build of nixpkgs + `postgresql`, and NOT a custom wrapper-entrypoint image.** nix2container is + the repo's mechanism for the NixOS+devenv agent shell, not for bundling a + stock third-party service; the community `postgres` image is the boring, + correct base. `18` is the current stable major (19 is beta until ~GA). The + pin is by DIGEST, not tag — the same discipline `ci.yml:138-144` applies to + the pgtest service image, because a mutable tag would ship unreviewed + database behavior and generated-tsvector / `websearch_to_tsquery` behavior + is major-version-sensitive. The official entrypoint already does what the + `compass-postgres` wrapper does on the host path — initdb-if-needed + + createdb + a SIGTERM smart-shutdown drain — so on the container path the + wrapper collapses into it, configured by environment: `POSTGRES_DB=compass` + (the createdb), `POSTGRES_HOST_AUTH_METHOD=trust` (the loopback-only + private-store posture, `compass-postgres/main.go:16-18`), PGDATA on the + bind-mounted `/postgres` volume, and `unix_socket_directories` + pointed at the bind-mounted DSN socket dir so `compass-server` opens the + identical `host=` DSN. The `compass-postgres` wrapper stays the + host/dev-path (`ProcessSupervisor` LookPath) bring-up unchanged; only the + container path swaps it for the stock entrypoint + env. No custom image + build lane is needed: the release body references the upstream digest + directly (an optional GHCR mirror for pull-reliability is a convenience, + not a build — the same way the agent image is `podman pull`ed, DL-112). ++ **The supervisor seam.** The stack starts postgres through + `stack.ProcessSupervisor` today (`go/internal/stack/stack.go:193-196`: + `Component: ComponentPostgres, Args: ["--state-dir", …, "--database", …]`), + resolved by bare name on PATH (`adapters/process.go:31-44`). S4 adds a + **container-backed postgres adapter**: when configured with a postgres + image ref (default: the pinned `compass-postgres` image), the supervisor + runs `podman run` with the state dir and the DSN's socket directory + bind-mounted, instead of LookPathing a host binary. The DSN shape is + unchanged — `host= port= dbname=compass sslmode=disable` + (`compass-postgres/main.go:9-11`) — because the unix socket directory is + bind-mounted host↔container, so `compass-server` opens the identical DSN + and the cluster stays loopback-free and network-invisible (the + "socket-only (no TCP), trust auth on the local socket" posture, + `main.go:16-18`, survives containerization intact). *Readiness* reuses the + existing probe unchanged: `waitPostgres` polls the full DSN + (`stack.go:306-311`: "polls DBProber.ProbeDB until postgres accepts + connections on the full DSN") and the bind-mounted socket is + byte-identical to it. ++ **Teardown is NOT process-shaped — the container needs its own identity + (pgid format v2).** The naive claim "the adapter satisfies `stack.Process` + so teardown needs no change" is FALSE for the fresh-`down` (linger) path, + and this record designs the fix rather than papering over it. Today's + teardown identity is a *process-group* record: `recordChild` persists + `pgidEntry{Component, Pgid, StartTime}` with "pgid == pid, plus the leader + start-time token read at spawn" (`go/internal/stack/stack.go:258-269`; + `pgidfile.go:30-34`), and a fresh `compass-stack down` reads + `/stack.pgids` and tears each entry down by group signal — + `syscall.Kill(-pgid, sysSig)` (`adapters/groupsignal.go:54`). But a + `podman run` client's `Pid()` does not describe the container: under + rootless podman the containerized postgres runs beneath `conmon`, + *outside* the podman-client's process group, so group-signalling the + recorded pgid would orphan the container — postgres survives, + `DownDetached`'s socket-quiescence confirm ("postgres stops accepting on + the DSN socket", `downdetached.go:188-191`) reports a genuine survivor, + and `down` fails while leaking the container. **Resolution (Matt ruled + OQ-7 2026-08-25, DL-259 — the contract change is blessed):** the containerized + postgres stays a *supervised stack component* — preserving the + one-command up/down lifecycle that is the reason postgres is a supervised + child at all — and the pgid record format grows a first-class **container + teardown identity**: + + **Format v2.** `pgidFileVersion` bumps `"1"` → `"2"` + (`pgidfile.go:20-22`, the frozen DL-183 contract). Entries become a + discriminated union, tagged by a kind field on the entry line: + + *process entry* — today's `{Component, Pgid, StartTime}`, torn down by + group signal exactly as now (`proc `); + + *container entry* — `{Component, ContainerName}`, torn down by + `podman stop -t ` with `podman rm -f ` as the + SIGKILL-tier escalation (`ctr `). + `readPgidFile` dispatches on the tag; the hard-error-on-malformed + discipline is unchanged ("signaling off a half-understood record is + exactly the blast radius the design forbids", `pgidfile.go:100-103`). + + **Cross-version rule.** A v1-only binary never half-parses a v2 record — + but by the *entry-line grammar*, not a header-version check: shipped v1 + `readPgidFile` stores `header[0]` as `Version` and never compares it to + `pgidFileVersion` (`pgidfile.go:124`), so the refusal comes from + `parsePgidLine` hard-erroring on a v2 entry line (a `proc` line is 4 + fields where v1 demands exactly 3, `pgidfile.go:144`; a `ctr` line's + leading token is not a known component, `pgidfile.go:149`) under the same + hard-error-on-malformed discipline ("signaling off a half-understood + record is exactly the blast radius the design forbids"). The T8a + `refuse-unknown-version` clause is the *forward* guard — it protects a v2 + reader from a future v3, not v1 from v2. A v2 binary reads v1 records as + all-process entries (v2 is a strict superset). In practice the window is + nil: Global Constraint 4's one-version-stamp invariant means every + channel installs a matched build set, so the up that wrote the record + and the down that reads it are the same build — but the rule is stated + so a mixed-build accident degrades to a legible refusal, not a blind + signal. + + **`DownDetached` dispatch.** `liveTargets`/`drainTargets` dispatch per + entry kind: process entries keep the identity-checked group-signal path; + the container entry's SIGTERM tier is `podman stop` (bounded by the + existing `postgresDrainBudget`, `downdetached.go:31`) and its SIGKILL + tier is `podman rm -f`. The socket-quiescence confirm survives intact: + the socket dir is bind-mounted from the host, so the DSN socket goes + quiet when the container's wrapper stops — the confirm channel needs no + change, only the signal delivery does. + + **Stable container name.** The name is derived from the state dir alone + (e.g. `compass-postgres-`), so a fresh `down` with + no in-memory handle reconstructs it from config — and it is *also* + persisted in the v2 container entry, which is the authoritative copy the + teardown uses (derivation is the collision-avoidance scheme for + concurrent state dirs, the record is the teardown identity). ++ **The container run contract (interface, not implementation).** Two seams + break the zero-thought default if left unstated, so they are pinned here: + + *uid mapping.* postgres refuses to run as uid 0, and the bind-mounted + `/postgres` data dir is created 0700 host-user-owned by the + wrapper's initdb path; under rootless podman the container root maps to + the host user, so the run MUST use `--userns=keep-id` (host uid ↔ same + uid in-container) with the image running the wrapper as that non-root + user — otherwise the data dir is unreadable across the uid map. + + *stop timeout — two distinct knobs, not one.* postgres never force-kills + itself: the wrapper's drain grace is 30s — + `compass-postgres/main.go:256-259`: "shutdownGrace bounds how long we + wait after forwarding SIGTERM … We never force-kill; escalation is the + supervisor's job" (`const shutdownGrace = 30 * time.Second`) — while + `podman stop`'s default SIGKILLs after 10s. So (1) the `podman run` pins + `--stop-timeout` ≥ the 30s wrapper grace, the safe default for any stop + that passes no explicit `-t` (podman never hard-kills before the wrapper + would). (2) The detached-`down` teardown path instead passes an explicit + `podman stop -t ` = 10s (`downdetached.go:31`): this + is deliberate behavior parity with today's process model, where the + detached path already caps the postgres drain at `postgresDrainBudget` + and group-SIGKILLs at that bound rather than waiting the wrapper's full + 30s. The two knobs serve different callers; they are not equal and must + not be conflated. + + *mounts.* `/postgres` (data dir) and the DSN's socket dir are + bind-mounted read-write; nothing else from the host is mounted. ++ **Bring-your-own postgres.** `Config.DatabaseDSN` is already caller-provided + (`go/internal/stack/config.go:29-31`); a new `--database-external` (or + DSN-shape detection: a `host=` pointing outside the state dir) makes the + supervisor skip the postgres component entirely and just `waitPostgres` the + given DSN. Zero-thought default = container; escape hatch = your DSN. ++ **Dev-box path unchanged.** devenv keeps providing host postgres tooling + for the e2e suites (`devenv.nix:114-128`); the container is the *installed* + stack's default, selected by config, not a repo-wide replacement of the + process adapter. + +## Alternatives considered + ++ **Compose/Swarm-packaged stack** — rejected in S3: `/dev/kvm` + microVM + userspace as host processes is the designed boundary + (`microvm-runner.md:230-236` hard-fail; `devenv.nix:198-201`); a + containerized Runner needs privileged device passthrough and still can't + degrade (D3). ++ **Postgres as a host prerequisite** (the client-only OQ-3 interim + recommendation) — superseded by Matt's ruling: "install postgres 18 + yourself" fails the zero-thought bar; the container path is one `podman + pull` the stack performs itself, on a host that already runs podman + (DL-112). ++ **Vanilla `docker.io/postgres` image + wrapper on the host** — rejected: + it splits the private-cluster brain (wrapper) from the server binary + (postgres) across a host/container seam, reintroducing the host binary the + ruling removes; the dedicated image keeps wrapper+postgres one artifact + with one version. ++ **TCP loopback instead of bind-mounted socket for the container cluster** — + rejected: it would change the DSN contract, expose a port, and forfeit the + socket-only/trust-auth posture (`compass-postgres/main.go:16-18`); the + bind-mount keeps every consumer byte-identical. ++ **systemd/quadlet-managed postgres container outside the child tree** (the + stack only probes the DSN, never owns the container) — rejected: it would + sidestep the pgid-format-v2 change (the container would carry no stack + teardown identity at all), but at the cost of adding unit provisioning + (a quadlet file, `systemctl --user` wiring, lingering) to the + zero-thought install story and breaking the stack's + up-starts-all / down-tears-all lifecycle — `compass-stack up` would bring + up a stack whose database it neither started nor can stop. The supervised + container + v2 teardown identity keeps the one-command lifecycle; the + format change it costs is the DL-259 pgid-v2 extension Matt blessed (OQ-7). ++ **homebrew-core / distro-official packages** — rejected for v1: core/distro + inclusion has review latency and policy floors (notarization, popularity) + that a young project fails; an org tap ships today and migrates later + without user-visible change beyond the tap prefix. ++ **goreleaser for the matrix** — re-rejected (same grounds as the release-bundling record's Fork + 2(i)): tag-driven model fights the per-build lane; the repo convention is + dependency-free glue. ++ **Electron-style auto-update in the app** — out of scope; deferred with + macOS follow-ups per `compass-native-app/design.md:275-277` ("Deferred to + follow-up issues: … signed installers with auto-update"). Channels here are + pull-based (brew upgrade / nix profile upgrade / re-download). + +## Plan + +Dependency order: T1 → T2 → T3 → T4 → T5; T6 → T7 → T8; T8a → T8 +(T8a is the pgid-format-v2 teardown identity T8's fresh-`down` needs, per the +DL-259 ruling); T9 after T6+T8 +(T9's preflight ships its own checks — see T9 — so it does not block on the +runtime lane); T10 last. S1/S2 (client) and S3/S4 (stack) are independent +lanes until T9. **Cross-lane impl order:** T1-T5 hard-depend on the +release-bundling record's T1 impl landing first — T1 edits +`.github/workflows/release.yml`, which is that record's T1 deliverable and does +not exist until its impl lane merges. (Both records freeze together in this +PR; the dependency is between their *impl* lanes, not their design records.) + +### T1 — Linux client bundle joins the Release asset set + ++ **Do:** extend `.github/workflows/release.yml` (the release-bundling record's T1 workflow): a + step running `app-bundle/build.sh` (nix + gtk3 link, per its own header) + and uploading it renamed to the release-bundling asset grammar + `compass-app__linux-amd64.tar.gz` (the produced asset, see Interfaces) + to the same Release; + fold its checksum into `SHA256SUMS`. Trigger paths gain `app-bundle/**`, + `apps/ui/**`, `tools/toolchain/gtk-closure.nix`, `devenv.lock` (the bundle's + moon inputs, packaging record §A4). ++ **Interfaces:** consumes `app-bundle/build.sh` (existing; emits the tarball + in `app-bundle/`), the release workflow's tag + upload step. Produces one + new asset per Release: `compass-app__linux-amd64.tar.gz` (renamed to + the release-bundling asset grammar `__-`). ++ **Test cycle:** a main push touching `app-bundle/**` mints a prerelease + whose tarball unpacks, `bin/compass-app --version` prints the stamp, and + the DL-238 smoke (`app-bundle/SMOKE.md`) passes from the downloaded asset. + +### T2 — Darwin build path for compass-app + ++ **Do:** restructure the shell's build tags so darwin compiles the real + entrypoint AND does not double-compile the non-gtk3 stubs. The stub pairs + are load-bearing: each gtk3 file that gains darwin forces its `!gtk3` stub + pair to narrow to `linux && !gtk3` — otherwise a darwin build (which is + `unix` with no gtk3 tag) compiles BOTH the retagged real file and its + `unix && !gtk3` stub, a duplicate-symbol compile break (e.g. + `windowFromContext` is defined in `bridge_service_window_gtk3.go:28` AND + `bridge_service_window_nogtk3.go:19`). Full per-file table (current tags + verified against every `go/cmd/compass-app/*.go:1` header): + + | File | Before | After | + | --- | --- | --- | + | `main.go` | `unix && gtk3` | `(linux && gtk3) \|\| darwin` | + | `client.go` | `unix && gtk3` | `(linux && gtk3) \|\| darwin` | + | `window_set.go` | `unix && gtk3` | `(linux && gtk3) \|\| darwin` | + | `bridge_service_window_gtk3.go` | `unix && gtk3` | `(linux && gtk3) \|\| darwin` | + | `main_nogtk3.go` (stub pair of `main.go`) | `unix && !gtk3` | `linux && !gtk3` | + | `bridge_service_window_nogtk3.go` (stub pair of `bridge_service_window_gtk3.go`) | `unix && !gtk3` | `linux && !gtk3` | + | `bridge_service.go` | `unix` | `unix` (unchanged — compiles on darwin already) | + | `version.go` | untagged | untagged (unchanged) | + | `client_test.go`, `window_set_test.go`, `window_name_test.go`, `multiwindow_e2e_test.go`, `multiwindow_e2e_helpers_test.go` | `unix && gtk3` | `unix && gtk3` (unchanged — Linux-only suite, see note) | + | `bridge_service_test.go`, `bridge_service_connect_test.go` | `unix` | `unix` (unchanged) | + + Wails v3 selects the system-WebKit backend on darwin (no pkg-config, no + gtk closure — `devenv.nix:110-111`: "harmless on macOS, where the app + links the system WebKit framework and pkg-config goes unused"). Verify + the keychain tokenstore path (DL-109) uses the darwin keychain backend. + **Honesty note:** the gtk3-tagged test suite stays `unix && gtk3` — the + darwin binary ships with ZERO of the shell's gtk3-tagged tests compiled + for it; darwin coverage is the manual launch smoke (below) plus whatever + the untagged/`unix` suites exercise. ++ **Interfaces:** consumes the existing gtk3-tagged shell sources + Wails v3 + (`v3.0.0-beta.0`, `go/go.mod:29` per the packaging record). Produces a + darwin-arm64 `compass-app` binary buildable on a mac with + `go build -o compass-app ./cmd/compass-app` (cgo, system frameworks, no + tag). ++ **Test cycle:** `go build ./...` (untagged, on Linux) still green — this + is what a Linux box CAN verify; a full darwin compile is NOT runnable on + Linux (the darwin entrypoint's cgo needs the macOS SDK), so the real + darwin compile gate is T3's `macos-14` job (cadence per DL-260: main + + nightly always, plus a moon-affected-gated PR leg). On a mac: the build + launches, loads `dist`, and completes the + client connect flow against a live stack. + +### T3 — macOS app bundle + CI lane (unsigned) + ++ **Do:** a `scripts/macos-bundle.ts` (TypeScript per Global Constraint 10 — + it templates Info.plist, stages `Compass.app/Contents/{MacOS,Resources}`, + ad-hoc signs `codesign -s -`, wraps a `.dmg` via `hdiutil`); a + `macos-14` job in `release.yml` building it per Release (paths-gated like + T1). Asset: `compass-app__darwin-arm64.dmg`. ++ **Interfaces:** consumes T2's darwin binary + `apps/ui` dist (built by + `bunx vite build`, `apps/ui/moon.yml:19-23` per the packaging record). + Produces the `.app`-in-`.dmg` asset (ad-hoc signed; NOT yet a public + channel input — T4 gates that). ++ **Test cycle:** the CI-built dmg mounts, the app launches on a mac + (quarantine cleared manually — expected pre-notarization), connects to a + stack, board renders. + +### T4 — macOS signing + notarization + ++ **Do:** add Developer ID signing to T3's lane: import the cert p12 from a + GHA secret into a throwaway keychain, `codesign --options runtime + --timestamp` the app, `xcrun notarytool submit --wait` with an App Store + Connect API-key secret, `xcrun stapler staple`, re-wrap the dmg and sign + it. Secrets scoped to the release workflow (no PR trigger exists on it — + Global Constraint 2). ++ **Interfaces:** consumes T3's bundle step + three GHA secrets + (`MACOS_CERT_P12`, `MACOS_CERT_PASSWORD`, `NOTARY_API_KEY`). Produces a + Gatekeeper-clean dmg: `spctl -a -t open --context context:primary-signature` + passes; first launch shows no quarantine dialog. ++ **Test cycle:** download the release dmg on a clean mac (no dev tools), + drag-install, launch — zero Gatekeeper friction. `codesign -dv` + + `stapler validate` in CI as the mechanical gate. ++ **Prerequisite:** an Apple Developer Program membership + Developer ID + certificate (OQ-2 — the one human-action prerequisite in this record). + +### T5 — Homebrew tap + auto-bump + ++ **Do:** create `RigelBuild/homebrew-tap`: `Casks/compass.rb` (installs the + T4 dmg) + `Formula/compass-cli.rb` (installs the release-bundling CLI binaries, + per-arch url/sha blocks). Add a semver-lane step to `release.yml` that + renders both files from templates (version + asset sha256s) and pushes to + the tap via a fine-grained PAT scoped to that one repo (`TAP_PUSH_TOKEN`). ++ **Interfaces:** consumes the `vX.Y.Z` Release's asset URLs + `SHA256SUMS`. + Produces `brew install rigelbuild/tap/compass` (macOS app) and + `brew install rigelbuild/tap/compass-cli` (macOS/Linux CLI). ++ **Test cycle:** `brew install` + `brew upgrade` against a cut `v*` release + on both a mac and a Linux box; `brew audit --cask compass` clean; a + prerelease `build-*` demonstrably does NOT bump the tap. + +### T6 — Nix flake + ++ **Do:** add `flake.nix` at the repo root: inputs pinned to the devenv.lock + nixpkgs revision (the `gtk-e2e-env.nix:9-13` single-pin discipline); + packages `compass`, `compass-server`, `compass-runner`, `compass-stack` + (buildGoModule over `go/`, `-ldflags -X main.version=` stamped from the + flake's `self.rev`), `compass-app` (Linux: the gtk3 cgo build against + `gtk-closure.nix`), and `compass-stack-env` (cloud-hypervisor + virtiofsd + + passt at the pinned rev). **Pin-parity gate:** a flake has its OWN + `flake.lock`, so the repo would carry TWO independent nixpkgs locks — the + "one closure" claim (flake-built ≡ bundle-built binaries) holds only if + they resolve the same rev, and nothing enforces that by construction. So + T6 adds a named parity check (the `parity.ts` discipline the repo already + runs — `devenv.nix:118-119`: CI's gate-tools are "fed by `parity.ts + --print-nix-attrs` off THIS list") verifying `flake.lock`'s nixpkgs rev == + `devenv.lock`'s nixpkgs rev, failing CI on skew. A CI check (`nix flake + check` + the parity check, moon-registered, affected-gated on + `flake.nix`/`flake.lock`/`devenv.lock`/`go/**` — `devenv.lock` is in the + trigger set precisely because a devenv pin bump is the event that causes + the drift) keeps it green. ++ **Interfaces:** consumes `go/`, `tools/toolchain/gtk-closure.nix`, + `devenv.lock`'s nixpkgs rev. Produces + `nix profile install github:RigelBuild/compass#` for every package; + `nix run .#compass-stack -- status` works from a bare checkout. ++ **Test cycle:** `nix build .#compass-stack .#compass-app` from a clean + clone; the built `compass-stack --version` matches the flake rev stamp; + the four stack binaries carry ONE stamp (Global Constraint 4); the parity + check red on a deliberately skewed `flake.lock`, green after `nix flake + lock --override-input` back to the devenv.lock rev. + +### T7 — Pin the postgres container image (stock `postgres:18` by digest) + ++ **Do (Matt's OQ-5 ruling):** pin the stock upstream `postgres:18` image by + digest as the default `PostgresImage` — NOT a nix2container or custom + wrapper-entrypoint build. Record the digest beside the pgtest image pin so + the two stay legible together, and document the bump procedure (advance the + digest when the postgres minor/major moves, re-running the T8 integration + test). No `publish-*-image` workflow: there is no image to build. An + OPTIONAL GHCR re-tag/mirror of the pinned digest (for pull-rate reliability + on self-host hosts) may ride the release lane later; it copies, never + builds, and is not load-bearing for v1. ++ **Interfaces:** produces the pinned `postgres:18@sha256:…` reference the S4 + container-backed adapter (T8) runs, configured by the env contract in S4 + (`POSTGRES_DB`, `POSTGRES_HOST_AUTH_METHOD=trust`, PGDATA volume, + `unix_socket_directories`). Consumes nothing from `go/` — the stock image + needs no compass code on the container path. ++ **Test cycle:** `podman run` the pinned image with a tmp state dir + a + socket-dir bind-mount and the S4 env; `psql 'host= + dbname=compass'` connects; a forwarded SIGTERM drains cleanly (postgres + smart-shutdown). + +### T8 — Stack: container-backed postgres component + external-DSN opt-out + ++ **Do:** in `go/internal/stack`: a container-backed postgres start path — + config gains `PostgresImage string` (default: the pinned stock + `postgres:18` digest, T7; empty on the dev path = today's + `ProcessSupervisor` LookPath spawn of the `compass-postgres` wrapper) and + `ExternalDatabase bool` (skip the postgres component, probe + `Config.DatabaseDSN` as-is). The container adapter runs the S4 run + contract (`--userns=keep-id`; the S4 env — `POSTGRES_DB=compass`, + `POSTGRES_HOST_AUTH_METHOD=trust`, PGDATA + `unix_socket_directories` on + the bind-mounts; `--stop-timeout` covering postgres's smart-shutdown drain; + bind-mounts of `/postgres` and the DSN's `host=` socket dir) with + the stable per-state-dir container name (S4). Readiness is unchanged: + `waitPostgres` probes the full DSN (`stack.go:306-311`) over the + bind-mounted socket. + Teardown for the *attached/in-process* stop path maps to `podman stop`; + the fresh-`down` (linger) path is NOT covered by the `stack.Process` + contract and is T8a's pgid-format-v2 work — T8 depends on T8a for a + correct `down`. `compass-stack up` flags: `--postgres-image`, + `--database-external`. ++ **Interfaces:** consumes T7's image ref, T8a's container-entry teardown, + `Config.DatabaseDSN` (`config.go:29-31`), rootless podman on the host + (Global Constraint 8). Produces: default `compass-stack up` on a clean + KVM host brings up containerized postgres with zero flags; + `--database-external --database ` runs against user postgres. ++ **Test cycle:** unit tests on the config/dispatch logic (fake supervisor); + a podman-tagged integration test mirroring + `compass-stack/cross_process_podman_test.go`'s harness: up → probe DSN → + fresh-process `down` → container gone (`podman ps -a` empty) → pgid file + removed; the external-DSN path green against a throwaway host postgres. + +### T8a — pgid record format v2: container teardown identity + ++ **Do:** implement the S4 teardown design in `go/internal/stack`: + `pgidFileVersion` `"1"` → `"2"` (`pgidfile.go:20-22`), the discriminated + entry kinds (`proc ` / + `ctr `), `readPgidFile` v1-compat (v1 records parse as + all-process entries) + refuse-unknown-version, `recordChild` gains a + container-entry variant, and `DownDetached`'s `liveTargets`/`drainTargets` + dispatch per kind — container entries: liveness by `podman container + exists `, SIGTERM tier `podman stop -t `, SIGKILL + tier `podman rm -f `, confirm unchanged (DSN socket quiescence, + `downdetached.go:188-191`). ++ **Interfaces:** consumes the DL-183 pgid file seams (`pgidfile.go`, + `downdetached.go`) and podman on the host. Produces: a v2 record format; + `writePgidFile`/`readPgidFile` round-tripping both entry kinds; a fresh + `down` that tears down a containerized postgres by name. Implements the + DL-259 format extension (OQ-7, blessed by Matt 2026-08-25). ++ **Test cycle:** unit tests: v2 round-trip both kinds; a v1 record parses + (compat); an unknown-version record refuses legibly (the + `ErrNoTeardownRecord` posture, `downdetached.go:47-51`); the existing + `pgidfile_test.go` prefix/atomicity suites green on v2. Integration: + T8's up → fresh down → container gone. + +### T9 — Self-host bring-up surface: preflight + docs + ++ **Do:** `compass-stack preflight` (or fold into `up`'s first phase): KVM + openable, podman present + rootless-capable, microVM userspace trio found + at/above floors, legible per-check pass/fail. **Dependency honesty:** the + Runner has NO `VerifyMicroVMSupport` function today — the name occurs only + in a forward-looking comment (`go/internal/runtime/microvm.go:126-128`: + "the default collapses to microVM guarded by a VerifyMicroVMSupport hard + gate at startup") and in the elastic-session design record, not in code. + T9 therefore ships its own minimal checks — open `/dev/kvm`, + `exec.LookPath` the microVM userspace trio (cloud-hypervisor, virtiofsd, + passt), `podman info` — explicitly marked in-code as to-be-replaced by + the runtime lane's eventual preflight gate when it lands. This keeps T9 + off the runtime lane's critical path (no cross-lane blocks-on). + `docs/self-host.md`: the dedicated-KVM-machine shape and the one-box + localhost-TLS shape (client-only OQ-6), each channel's install one-liner + (flake, release tarball), postgres default + BYO-DSN, systemd unit example. ++ **Interfaces:** consumes T6 (flake install path), T8 (postgres default). + Produces the documented, preflight-guarded install story RIG-2608 asks for. ++ **Test cycle:** follow `docs/self-host.md` verbatim on a clean KVM VM: + flake install → preflight green → `compass-stack up` → client connects + from another machine over TLS → one agent session runs. + +### T10 — Docs, ledger, tracker + ++ **Do:** update `docs/architecture/build-and-ci.md` with the distribution + surfaces; land the §Ledger delta rows + status flips in + `docs/designs/DECISIONS.md` (driver lands them in the design PR); + re-scope/close RIG-2477 and RIG-2608 against the frozen record; file the + per-task impl issues per the freeze→file→dispatch gate. ++ **Interfaces:** consumes the frozen record. Produces ledger rows + + dispatched impl issues. ++ **Test cycle:** `tools/design-ledger-gate` green on the design PR; every + DL id verified free at landing. + +## Tasks + ++ [ ] T1 — Linux client bundle attached to Releases (release.yml + build.sh) ++ [ ] T2 — darwin build path for compass-app (build-tag restructure) ++ [ ] T3 — macOS `.app`/dmg bundle + macos-14 CI lane (ad-hoc signed) ++ [ ] T4 — Developer ID signing + notarization + stapling in the release lane ++ [ ] T5 — RigelBuild/homebrew-tap (cask + CLI formula) + semver auto-bump ++ [ ] T6 — `flake.nix`: client + stack + stack-env packages, `nix flake check` CI ++ [ ] T7 — pin stock `postgres:18` container image by digest (no build lane) ++ [ ] T8 — stack container-backed postgres component + `--database-external` ++ [ ] T8a — pgid record format v2 (container teardown identity, DL-259) ++ [ ] T9 — `compass-stack` preflight + `docs/self-host.md` ++ [ ] T10 — build-and-ci docs, ledger rows, RIG-2477/RIG-2608 follow-through + +## Ledger delta (intended) + +Ledger-impact: new rows (DL-254..260) + no status flips (DL-217 is already +`Superseded by DL-238`; DL-238/DL-214/DL-183 stay Active — this record +*extends* them, per the DL-213 partial-supersession-by-citation pattern). +Rows land in `docs/designs/DECISIONS.md` in this PR, written by the driver. +IDs verified free at landing: the observed highest on main is DL-240, so +DL-254..260 are the next free block. + +| ID | Decision | Status | Record | +| --- | --- | --- | --- | +| DL-254 | The client app ships a full per-OS matrix from day one — Linux x86_64 (thin-client tarball, DL-238 content) AND macOS arm64 (signed+notarized `.app` in a dmg, built on a macOS runner, never cross-compiled) — attached to the RIG-1746 release lanes, extending the release-bundling record's Fork 2(i) asset boundary and resolving its OQ-1 deferral (RIG-2477) | Active (Matt, 2026-08-24) | this record §S1 | +| DL-255 | Client install channels are homebrew (RigelBuild/homebrew-tap: cask for the macOS app + a cross-OS CLI formula, semver-bumped by the release workflow) and a repo `flake.nix` (client + stack packages pinned to the devenv.lock nixpkgs), over the raw release assets; distro-native packages (deb/rpm/AUR) deferred | Active (Matt, 2026-08-24) | this record §S2 | +| DL-256 | The self-host stack stays a host-level bring-up on a KVM-capable Linux machine (`compass-stack up`; microVM D3 hard-fail consumed, no compose/Swarm packaging); `compass-stack` joins the release binary matrix, and the flake + preflight + self-host doc are its install surface — resolving the client-only record's OQ-3 (RIG-2608) | Active (Matt, 2026-08-24) | this record §S3 | +| DL-257 | Postgres leaves the installed stack's host-process tree: the zero-config default is a dedicated postgres container run by the supervisor via rootless podman (socket-dir bind-mounted so the DSN contract is unchanged); a user-supplied DSN (`--database-external`) opts out. The image is the STOCK upstream `postgres:18` pinned by DIGEST — NOT a nix2container build of nixpkgs `postgresql` (that is the agent-shell mechanism, not a bundled stock service) and NOT a custom wrapper-entrypoint image; the official entrypoint does initdb+createdb+SIGTERM drain, so the `compass-postgres` wrapper collapses into env config (`POSTGRES_DB`, `POSTGRES_HOST_AUTH_METHOD=trust`, PGDATA volume, `unix_socket_directories`) on the container path and stays the host/dev-path bring-up. Supersedes the host-prerequisite interim answer (client-only OQ-3) atop the already-superseded DL-217; resolves OQ-5 | Active (Matt, 2026-08-24; image mechanism 2026-08-25) | this record §S4, §T7 | +| DL-258 | macOS signing identity provisioning waits for the company rename (Sealed Security Inc → Rigel AI Software Inc, same DE entity so its D-U-N-S carries over as an update, not a new request); enroll the Apple Developer Program org clean as Rigel AI Software Inc post-rename. Nothing pre-GA needs public signing, so dev/internal builds self-sign meanwhile (ad-hoc `codesign --sign -`, mandatory on Apple Silicon, or a local self-signed cert — no Apple account); the Developer ID cert + notarization (the distribute-to-other-Macs leg) is GA-gated on the enrollment. Resolves OQ-2 | Active (Matt, 2026-08-25) | this record §GC6, §T4 | +| DL-259 | Containerized-postgres teardown extends the DL-183 pgid record to format v2: `pgidFileVersion` `"1"`→`"2"`, entries become a kind-tagged discriminated union (`proc ` torn down by group signal as today; `ctr ` torn down by `podman stop`/`rm -f`), `readPgidFile` dispatches on the tag and a shipped v1 binary hard-errors on a v2 entry line by the entry-grammar (not a header check) under the unchanged signal-off-a-half-understood-record discipline. The per-agent microVMs are the runner's sandbox one layer below the stack and are untouched (the stack tears the runner down by pgid as a plain host process); what forces v2 is postgres containerizing, not microVMs. Extends DL-183 (which stays Active); resolves OQ-7 | Active (Matt, 2026-08-25) | this record §S4, §T8a | +| DL-260 | darwin CI cadence mirrors the shipped affected-on-PR + full-sweep-on-main + nightly shape (`ci.yml:25-36`): a `macos-14` compile+bundle sweep runs on every push to main AND nightly ALWAYS (the backstop), plus an affected-on-PR leg gated by a small ubuntu pre-job asking moon which projects the PR affects (moon's own affected-detection — the signal `moon ci` uses — never a GitHub `paths:` filter, so moon stays the single source of affected-truth per the ci.yml header's rejection of a YAML project list), so a scarce mac runner spins up on a PR only when a darwin-relevant project is affected. Resolves OQ-8 | Active (Matt, 2026-08-25) | this record §T3 | + +## Open Questions + +Per the batched-clarifications rule; each carries a recommendation. Matt's +two rulings above are NOT here — they are decided. The **load-bearing set** +— OQ-2 (Apple account), OQ-5 (postgres image mechanism), OQ-7 (pgid-v2 +contract change), OQ-8 (darwin CI cadence) — is now RESOLVED (Matt, +2026-08-25; DL-257..260). Each carries its resolution inline below. + +### OQ-1 [non-load-bearing] — native Linux distro packages (deb/rpm/AUR) + +Flake + brew-on-Linux + tarball cover the Linux matrix; distro packages add +per-format packaging debt (postinst scripts, repo hosting/signing) for +uncertain reach. **Recommendation:** defer; revisit on demand signal. The +channels in S2 are additive — a later `.deb` wraps the same bundle content. + +### OQ-2 [load-bearing] — Apple Developer account provisioning + +T4 requires an Apple Developer Program membership (org or individual), a +Developer ID Application certificate, and an App Store Connect API key — a +human/console prerequisite no agent can perform. **RESOLVED (Matt, +2026-08-25 — DL-258):** WAIT for the company rename (Sealed Security Inc → +Rigel AI Software Inc, same DE entity, D-U-N-S carries over as an update), +then enroll the org clean as Rigel AI Software Inc. Nothing pre-GA needs +public signing, so dev/internal builds self-sign in the interim (ad-hoc +`codesign --sign -`, mandatory on Apple Silicon, or a local self-signed +cert — no Apple account); the Developer ID cert + notarization is GA-gated +on the enrollment. T2/T3 proceed unblocked; the public-channel T4/T5-cask +leg gates on the post-rename enrollment secrets. A human-action issue files +the enrollment runbook at GA. + +### OQ-3 [non-load-bearing] — NixOS module (`services.compass-stack`) + +The flake's packages are the load-bearing channel; a NixOS module wrapping +`up` in systemd (with the KVM group + podman socket wiring declared) is the +polished self-host endgame but not required for T9's documented systemd-unit +example. **Recommendation:** defer to a follow-up on the flake once T6/T9 +land; the module is additive. + +### OQ-4 [non-load-bearing] — darwin-amd64 (Intel mac) support + +The release-bundling record ships darwin-arm64 only for the CLI; the app matrix here matches. +Intel macs lack nested-virt relevance (client-only anyway) but are a +shrinking install base. **Recommendation:** arm64-only now; brew cask can +grow an `on_intel` block later without a design change (GHA `macos-13` for +an amd64 leg if demanded). + +### OQ-5 [load-bearing] — postgres image build mechanism + +T7 leaves nix2container-vs-Containerfile to the executor, but the fork has a +real tradeoff: nix2container reuses the `agent-image/` publish machinery and +the repo's pinning discipline (`agent-image/publish.sh` + the vendored +skopeo), while a Containerfile over the official `postgres:18` base is +simpler and inherits upstream security updates by rebuild. +**RESOLVED (Matt, 2026-08-25 — DL-257):** the STOCK upstream `postgres:18` +image pinned by DIGEST — NOT nix2container (that mechanism is for the +NixOS+devenv agent shell, not a bundled stock service). The official +entrypoint does initdb+createdb+SIGTERM drain, so the `compass-postgres` +wrapper collapses into env config on the container path (`POSTGRES_DB`, +`POSTGRES_HOST_AUTH_METHOD=trust`, PGDATA volume, `unix_socket_directories`) +and stays the host/dev-path bring-up; no custom image build lane is needed. +See S4 and T7. Non-blocking for every other task. + +### OQ-6 [non-load-bearing] — prerelease channel exposure + +Should `build-` prereleases feed any channel beyond raw assets (e.g. +a `compass-cli@head` formula or a flake `packages.head`)? The flake tracks +`main` by construction (`github:RigelBuild/compass` is a moving ref), which +already IS the head channel. **Recommendation:** no extra head channels; +brew stays semver-only, flake-at-main is the sanctioned bleeding edge. + +### OQ-7 [load-bearing] — containerized postgres teardown extends the frozen DL-183 pgid format to v2 + +The container has no process-group teardown identity (S4: a rootless +`podman run` client's `Pid()` does not describe the container, which runs +under conmon outside the client's group), so the fresh-`down` path needs a +container entry kind in the `stack.pgids` record — a `"1"` → `"2"` bump of +`pgidFileVersion` (`pgidfile.go:20-22`), the format DL-183 froze. The +alternative that avoids the format change is the systemd/quadlet-managed +container (§Alternatives): the stack only probes the DSN, but the install +story gains unit provisioning and `up`/`down` stop owning the database +lifecycle. **RESOLVED (Matt, 2026-08-25 — DL-259):** the pgid-format-v2 +contract change (the S4/T8a design), NOT the systemd/quadlet lifecycle +change. The one-command up/down lifecycle is the reason postgres is a +supervised child at all; the v2 format is a strict superset (v1 records +parse as all-process entries), and Global Constraint 4's matched-build-set +invariant makes the cross-version window nil in practice. The per-agent +microVMs are the RUNNER's sandbox lifecycle one layer below the stack and +are untouched — the stack tears the runner down by pgid as a plain host +process; what forces v2 is postgres containerizing, not microVMs. + +### OQ-8 [load-bearing] — darwin CI cadence (recurring macOS runner spend) + +Nothing keeps the darwin build green between releases: the per-PR gate is +`go build ./...` untagged on Linux, which never compiles the darwin arm — +a PR can silently break darwin (exactly the T2 duplicate-symbol class of +error) and it surfaces on release day inside the signing-gated semver lane, +the worst place. macOS runners cost ~10x Linux minutes, so this is a +recurring-spend policy call. **RESOLVED (Matt, 2026-08-25 — DL-260):** +mirror the shipped CI shape (`ci.yml:25-36`, affected-on-PR + full-sweep-on- +main + nightly): a `macos-14` compile+bundle sweep on every push to main AND +nightly, ALWAYS (the backstop Matt wants for everything), PLUS an +affected-on-PR leg gated by a small ubuntu pre-job asking moon which projects +the PR affects (moon's own affected-detection, the signal `moon ci` uses — +never a GitHub `paths:` filter, so moon stays the single source of +affected-truth, per the ci.yml header's rejection of a YAML project list). A +scarce mac runner spins up on a PR only when a darwin-relevant project is +affected; main + nightly catch a mac-only regression regardless. diff --git a/docs/designs/platform/compass-release-bundling.md b/docs/designs/platform/compass-release-bundling.md new file mode 100644 index 00000000..fda74c3f --- /dev/null +++ b/docs/designs/platform/compass-release-bundling.md @@ -0,0 +1,530 @@ +# Bundle nix outputs, images, and binaries as GitHub Releases + +> **Design record.** Citations name paths in `compass` = RigelBuild/compass at +> `c25ce94f` (this repo, main at authoring). Line numbers drift as code evolves; +> resolve against that revision. This is a public repo: the internal CD monorepo +> that consumes these artifacts is referred to by role, never by name, path, PR, +> or quoted source. +> +> **Companion record, same PR:** +> [`compass-distribution/design.md`](compass-distribution/design.md) (the +> install/distribution surface — per-OS client matrix, install channels, the +> headless self-host stack). This record designs the Release *mechanism*; the +> distribution record consumes it as its publishing rail. The two are tightly +> coupled and ship + freeze together in one PR. + +Status: Active — the load-bearing forks (OQ-1/2/3, ruled 2026-08-22) and the +whole-product semver policy (OQ-7, ruled 2026-08-24) were ruled by Matt; +freezes on merge as the contract T1–T5 execute against. +Tracking: RIG-1746 (whole-product semver + GitHub Releases). RIG-2103 is +retargeted onto T4 (OQ-2); the OQ-1 desktop deferral is RIG-2477. + +## Problem / Intent + +CI/CD publishes container images to GHCR and an internal config-publish job +`go install`s a pinned compass CLI from source on every run — but nothing +produces a durable, versioned bundle of what a build actually shipped. This +record designs a GitHub Releases lane: one Release per qualifying build, +carrying the compiled binaries (compass CLI per-arch), the container image +identity (GHCR ref + digest), and the nix build output identity (store paths + +hashes), so consumers pin and download a verified artifact instead of +rebuilding from source. + +**The record lives in the compass repo** because Fork 1 below concludes compass +owns the surface: GitHub Releases attach to a GitHub repo, and the public +`RigelBuild/compass` repo is where the release artifacts' sources live, where +the existing GHA publish lane runs, and where an in-repo `GITHUB_TOKEN` can +create Releases with zero new credentials. The internal CD monorepo is +Woodpecker-driven and would need a cross-repo PAT to write Releases anywhere +(§Fork 1). + +## Approach + +Four design forks, each with options, a recommendation, and tradeoffs. + +### Fork 1: which repo + which surface owns Release creation + +**Options:** + +- **(a) compass repo, GitHub Actions** — a new `.github/workflows/release.yml` + beside the existing publish lane. +- **(b) the internal CD monorepo, Woodpecker CD** — a push→main CD job like its + image-publish lane, writing Releases on some GitHub repo via a PAT. +- **(c) hybrid** — compass GHA creates the Release; the internal CD appends its + own assets to it. + +**Recommendation: (a) compass GHA.** Grounds: + +1. *The artifacts are compass-built.* The compass CLI is + `go/cmd/compass` in the compass Go module — after the RIG-1930/RIG-2025 + module-path rename (compass PR #458, in review at authoring; a Global + Constraint below), `compass go/go.mod:13` on that branch reads: + + ```text + module github.com/RigelBuild/compass/go + ``` + + (main at `c25ce94f` still carries the pre-rename + `github.com/sealedsecurity/compass/go`; the remaining in-flight half is the + internal config-publish job's install target, still on the old path — §Fork 2). + The agent image builds from + `compass agent-image/` via the vendored devenv/nix2container forks — + `compass agent-image/publish.sh:51`: + + ```text + BUILD_OUT="$(nix run path:../forks/devenv#devenv -- container build agent)" + ``` + + A Release must attach to the repo whose commits it versions; releasing + compass artifacts on the internal monorepo would version them against the + wrong history. + +2. *The GHA precedent already exists and is the right shape.* The compass repo + deliberately runs artifact publishing as a separate least-privilege GHA + workflow, not a CI-gate step — `compass + .github/workflows/publish-agent-image.yml:3-9`: + + ```text + # WHY A SEPARATE WORKFLOW, NOT A STEP IN THE CI GATE — a deliberate, principled + # exception to this repo's ONE-JOB doctrine (see ci.yml's header): + # + # - Least privilege. A GHCR push needs `packages: write`; the gate job runs + # `contents: read` only. + ``` + + A Release workflow is the same pattern with `contents: write` instead of + `packages: write`. + +3. *Auth is free in-repo.* GHA's ephemeral `GITHUB_TOKEN` can create Releases + and upload assets in its own repo with a `permissions: contents: write` + grant — no standing secret. The internal Woodpecker CD authenticates GHCR + pushes with a provisioned `GHCR_TOKEN` — a packages-scoped credential; + writing Releases from Woodpecker would need + a *new* repo-scoped PAT to provision, rotate, and fork-gate (§Fork 4). + +4. *The public repo is the consumption point.* `RigelBuild/compass` is public, + so Release assets download anonymously — which is exactly what the internal + config-publish consumer needs (§Fork 3). The internal monorepo is not the + public surface. + +**Tradeoffs accepted:** (b) would co-locate the Release cut with the internal +CD's other push→main GHCR legs, keeping one CD brain; but none of those legs +produce compass artifacts, and the cross-repo PAT + Woodpecker↔GitHub seam +outweighs the co-location. (c) hybrid is +deferred as non-load-bearing: today no internally-built artifact belongs in a +compass Release (the internal CD's images are CI step images, not product +artifacts). If that changes, the internal CD can append assets to an existing +Release with a narrowly-scoped PAT without reopening this record. + +### Fork 2: what artifacts, and how each is produced + +Three artifact classes. The principle: **attach what consumers download; +reference what registries already serve durably.** + +**(i) Binaries — attach, per-arch.** + +- *Which:* `compass` (the CLI — the one binary with a known external consumer: + the internal config-publish job pins it, §Fork 3), plus `compass-server` and + `compass-runner` (the deployable daemons — + `compass go/cmd/` also carries `compass-stack`, `compass-app`, + `compass-gen-cert`, `compass-mint-runner-token`, `compass-postgres`; those + are dev/desktop tooling built on demand and are *excluded* from v1 to keep + the asset set meaning something — an OQ records the boundary). +- *Arch matrix:* `linux-amd64` for all three binaries (the dogfood/CD target — + the agent-image record froze linux/amd64 as the dogfood platform, `compass + docs/designs/platform/compass-agent-image-publish.md:54-67`), plus + `darwin-arm64` for the **CLI only** (Matt's macOS dev machines run the CLI + against remote/dogfood stacks; the daemons deploy on Linux, and shipping + darwin daemon builds would assert a support surface nothing consumes). + Go cross-compiles the darwin CLI from the same ubuntu runner + (`GOOS=darwin GOARCH=arm64 CGO_ENABLED=0`; the CLI is pure-Go — CGO is + needed only by the pgtest suites, `compass .github/workflows/ci.yml:263` + sets `CGO_ENABLED: '1'` for tests, not for the CLI build). Asset set: + 3 × linux-amd64 + 1 × darwin-arm64 = 4 binaries. +- *How:* plain `go build` in the release workflow, named + `compass__-` + a single `SHA256SUMS` asset. No goreleaser: + the repo's convention is zero-extra-tooling glue (`compass + agent-image/publish.sh:5-11` argues exactly this for bash-over-frameworks), + and goreleaser's tag-driven model fights the per-build lane in Fork 3. + +**(ii) Container image — reference by digest, do NOT attach a tarball.** + +The agent image is already published durably to GHCR with an immutable +per-commit tag — `compass .github/workflows/publish-agent-image.yml:33`: + +```text +podman pull ghcr.io/rigelbuild/compass-agent:git- +``` + +and the publish enforces tag immutability + digest coherence +(`publish-agent-image.yml:143-178`, the verify step). A `docker save`/`skopeo +copy dir:` tarball would duplicate a multi-GB closure GitHub caps and GHCR +already serves, and would create a *second* image identity to keep coherent. +Instead the Release **body** records `ghcr.io/rigelbuild/compass-agent@sha256:…` +(the config digest the publish verify step already computes, +`publish-agent-image.yml:173`) — a durable pointer to an immutable artifact. +*Tradeoff:* if GHCR retention ever deletes the package, the Release's image +pointer dangles; accepted because the package is public + repo-linked and +nothing in the fleet deletes it, and a tarball's cost is paid on every release +for a disaster that has a rebuild path (`publish.sh` re-mints any sha via +`workflow_dispatch`, `publish-agent-image.yml:69`). + +**(iii) Nix build outputs — attach the identity manifest, not the closure.** + +The repo's nix outputs are the agent-image nix2container spec and the +toolchain derivations (`compass devenv.nix:29-36` — go from go-overlay, +bun/node/moon vendored as derivations). A full closure export +(`nix-store --export` / `nix copy` to a file) of the agent image is the same +multi-GB payload rejected in (ii), and CI already declares binary caches as +the closure-serving mechanism (`compass .github/workflows/ci.yml:165-168`, +the pinned substituters). **Attach instead a `nix-outputs.json` manifest**: +per built derivation, the store path + narHash from +`nix path-info --json` (the image spec path `publish.sh:52` captures, plus the +toolchain `langs` set `ci.yml:183-184` already evaluates). That makes every +Release a verifiable statement of *which* nix outputs the build produced — +reproducible byte-for-byte from the tagged commit — without shipping the +bytes twice. *Tradeoff:* a consumer without nix or cache access cannot +materialize the closure from the Release alone; accepted — no such consumer +exists (the runtime consumer pulls the GHCR image; developers have nix), and +an OQ records closure-attachment as a possible later add for offline/archival +use. + +### Fork 3: versioning + trigger + +**Options:** + +- **(a) Release per push→main**, tagged by commit (`build-` or + CalVer+sha). +- **(b) Release per semver tag push** (`vX.Y.Z`), manual cut. +- **(c) Two lanes:** an automatic **prerelease** per qualifying main push + (tag `build-`, `prerelease: true`) + a **semver release** on `v*` tag + push (RIG-1746's whole-product semver), both from the same workflow. + +**Recommendation: (c).** The directive asks for "a durable, versioned Release +per build" — that is lane one. But RIG-1746 already reserves whole-product +semver as its own deferred concern (RIG-1746: "Whole-product semantic +versioning + GitHub Releases is DEFERRED to its own product-versioning +record"), and the agent-image record explicitly left the tag/release trigger +open for it (`compass docs/designs/platform/compass-agent-image-publish.md: +185-186`: "No tag/release trigger yet; that arrives with the GA +release-version tag"). Lane two implements that reserved slot: the same +workflow, triggered on `push: tags: ['v*']`, cuts a non-prerelease Release and +additionally pushes the `:vX.Y.Z` image tag via the positional-args seam the +agent-image record forward-planned (`publish.sh:38-39`: "A GA add is one more +positional arg (`./publish.sh git- v latest`)") — **invoked as +`./publish.sh git- v`, deliberately WITHOUT `latest`.** +`publish-agent-image.yml` serializes every `:latest` move under its own +concurrency group (`publish-agent-image.yml:76-82`: "Publishes SERIALIZE — an +in-flight `:latest` move must never be half-superseded") and hard-fails its +verify when `:git-`/`:latest` digests diverge +(`publish-agent-image.yml:171-178`). A release-lane invocation moving `:latest` +under a *different* concurrency group (`release`) would break that +serialization: two workflows could interleave `:latest` moves, and the image +lane's verify would red on a divergence the release lane caused. Alternatives +weighed: *(a) share one concurrency group across both workflows* — GitHub +concurrency groups do span workflows, but that serializes every release cut +behind every image build (and vice versa) and couples the two lanes' latencies +for no gain; *(b) drop `latest` from the release invocation* — `:latest` is the +per-push image lane's pointer and a semver cut needs only `:vX.Y.Z`; *(c) +`workflow_dispatch` the image workflow to add the tag* — indirection plus a +second run for one manifest write. **(b) chosen**: the semver lane mints +`:vX.Y.Z` only; `:latest` stays owned, moved, and verified exclusively by +`publish-agent-image.yml`. What `vX.Y.Z` *means* across the product — cadence, +changelog, and what bumps major — is RIG-1746's core semver policy; Matt ruled +it (2026-08-24), recorded as OQ-7 below. This lane gives that policy its +mechanical rail. + +- *Per-build tag scheme:* `build-` (12-hex short sha, matching the image + tag derivation `publish.sh:43`). Git-sha, not date: it is the identity every + existing pin already speaks (the pinned-rev constant, the `:git-` image + tag), and it is collision-free where CalVer needs a disambiguator. +- *Qualifying push:* a `paths:` filter over the binary-affecting inputs only — + `go/**`, `tools/toolchain/versions/go.nix`, and the workflow file — NOT + unioned with the image lane's closure paths (OQ-3, ruled). It reuses the + `on.push.paths` native-scoping technique the image publish uses + (`publish-agent-image.yml:44-68`) but not that lane's path *set*: an + image-only sha (`agent-image/**`, `forks/**`, `bun.lock`) republishes the + image with its GHCR per-sha identity but mints no Release, and a docs-only + push mints nothing. Unioning the two path sets would mint binary-identical + Releases on image-only shas — the exact outcome OQ-3 rejects. + Prereleases are **published, not draft**: a draft is invisible to the + anonymous asset-download consumer, and the per-build lane has no human step. +- *Semver releases:* draft-then-publish is unnecessary too — the tag push *is* + the human act (only Matt pushes `v*` tags). That is a convention, not an + enforcement, so Fork 4 adds two hard controls on the tag lane. +- *Consumer cutover (the RIG-2103 overlap):* the internal config-publish job + currently provisions the CLI by compiling it (`go install` of the pinned rev), + which requires a Go toolchain in the CD step image (the exact RIG-2025 + regression: the job runs in `ci-base`, which has none) and go-get module + resolution (the exact module-path regression). **Ordering is explicit:** the + in-flight RIG-2025 fixes (route the job to `ci-go`; rename the module path, + compass PR #458 + the internal job's install-target repoint) land NOW and + are independent of this record — T4 is a LATER revert of that routing once a + Release exists to download, never a substitute for the P0 fix. With this + lane, the job instead downloads + `https://github.com/RigelBuild/compass/releases/download/build-/compass_<…>_linux-amd64` + and verifies against `SHA256SUMS` — anonymous (public repo), toolchain-free, + and still pinned by the same pinned-rev constant. No Release covers the current + pin `a61d0caf` retroactively, so T4's landing path is a **pin advance**: move + the pinned-rev constant to the first released sha as its own reviewed change, + then cut the sourcing over. RIG-2103 proposed a nix derivation for the same + problem; the Release asset is the lighter answer and T4 below proposes + retargeting RIG-2103 to this cutover rather than running both. Matt ruled + RETARGET (2026-08-22, OQ-2); RIG-2103's Linear scope was updated to match. + +### Fork 4: auth + fork-safety + +**Options:** (a) the workflow's own `GITHUB_TOKEN` with `contents: write`; +(b) a releases-scoped fine-grained PAT; (c) reuse/widen the internal CD's `GHCR_TOKEN`. + +**Recommendation: (a).** Same-repo Release creation is exactly what +`GITHUB_TOKEN` covers; the agent-image record already weighed and rejected a +PAT for the analogous GHCR case (`compass +docs/designs/platform/compass-agent-image-publish.md:245-247`: "a PAT/deploy +token — needed only for cross-repo or cross-org pushes, which this is not; a +standing secret to rotate for zero benefit"). (c) is a category error: +`GHCR_TOKEN` is the internal Woodpecker CD's packages credential, provisioned +for image pushes — it never touches this GHA surface, and widening it to +`contents: write` would raise the blast radius of the internal CD's +most-injected secret for nothing. + +**Fork-safety:** the workflow triggers only on `push: branches: [main]`, +`push: tags: ['v*']`, and `workflow_dispatch` — no `pull_request` event, so +fork-PR code never executes with the write token (the identical posture to +`publish-agent-image.yml:44-69`, which has no PR trigger for the same reason). +The dispatch lane carries the same main-ref guard the image publish uses +(`publish-agent-image.yml:88-91`: "guard so a dispatch from a feature branch +can never mint a `:git-` for unmerged code"). Workflow-level +`permissions:` block grants `contents: write` and nothing else; the CI gate +keeps its `contents: read` (`compass .github/workflows/ci.yml:90-91`). + +**Tag-lane hardening (the `v*` injection hole).** A `push: tags: ['v*']` +trigger fires at whatever commit the tag names, wherever it lives — "agents +never push tags" (Global Constraint 5) is a convention, not an enforcement, +and the push-guard gates agent pushes, not every credential that could mint a +tag. Two hard controls close it: + +1. **A GitHub tag ruleset** on `RigelBuild/compass` restricting `v*` tag + creation to Matt (repo-admin bypass only) — the platform-level control, so + a leaked or over-scoped credential cannot create a release-triggering tag. +2. **An in-workflow ancestry guard** on the tag lane: fail unless the tagged + commit is on main — + `git merge-base --is-ancestor "$GITHUB_SHA" origin/main` — mirroring the + image workflow's dispatch main-ref guard (`publish-agent-image.yml:88-91`), + so even a ruleset mis-configuration cannot cut a non-prerelease Release + (and a `:vX.Y.Z` image tag) from unmerged code. + +## Alternatives considered + +- **The internal Woodpecker CD as the release surface** — rejected in Fork 1: wrong + repo for the versioned history, needs a new cross-repo PAT, and the public + download point is compass. +- **Image tarball as a Release asset** — rejected in Fork 2(ii): duplicates a + registry-served multi-GB artifact and creates a second identity to keep + coherent; the digest pointer is the durable record. +- **Full nix closure export as an asset** — rejected in Fork 2(iii): same size + argument; the narHash manifest gives verifiability, caches give + materialization. +- **goreleaser** — rejected in Fork 2(i): tag-only model fights the per-build + prerelease lane; the repo convention is dependency-free glue scripts. +- **Semver-only releases (no per-build lane)** — rejected in Fork 3: fails the + directive's "Release per build" and leaves the config-publish consumer with + no per-rev asset to pin. +- **Signed provenance for v1 (attestations / cosign)** — weighed, deferred. + `SHA256SUMS` uploaded from the same origin as the binaries is + transfer-integrity, not provenance: a compromised workflow forges both. + GitHub artifact attestations (`permissions: attestations: write` + + `gh attestation verify`) or cosign signing would bind assets to the building + workflow identity. Rejected for v1 because the only consumer is our own + sha-pinned CD (T4), which already trusts the repo it pins; recorded as + OQ-6 so a future external consumer triggers the add, not a redesign. + +## Global Constraints + +1. **Module path rename is a prerequisite.** The compass-side rename + `github.com/sealedsecurity/compass/go` → `github.com/RigelBuild/compass/go` + (RIG-1930 / RIG-2025) lands via compass PR #458 (in review at authoring); + the internal job's install-target repoint is its in-flight other half. Every + task here lands *after* the rename and uses the `RigelBuild` path + exclusively; nothing in this design may reintroduce a + `sealedsecurity/compass` reference. +2. **Release names are `build-` (prerelease) and `vX.Y.Z` (release)**; + binary assets are `compass__-` — three `linux-amd64` + binaries + one `darwin-arm64` CLI (Fork 2(i)) — plus one `SHA256SUMS` per + Release. +3. **No PR-event trigger, ever**, on any workflow holding `contents: write` — + the E1 fork posture of every publish lane in both repos. +4. **The tag pair ordering rule extends:** immutable identities first + (Release + `:git-`), moving pointers last (`:vX.Y.Z` image tag on the + semver lane) — per `publish.sh:37-39`. `:latest` is exclusively the image + lane's pointer; the release lane never moves it (Fork 3). +5. **`v*` tag creation is enforced, not assumed.** A GitHub tag ruleset + restricts `v*` creation to Matt, and the tag lane hard-fails on a + non-main-ancestor sha (Fork 4's two controls). "Agents never push tags" + remains the convention those controls back. The per-build lane needs no + tag-push actor (GHA creates the tag via the Release API at the built sha). +6. **Go version floor** tracks `tools/toolchain/versions/go.nix` (the same + pin CI resolves, `ci.yml:170-195`); the release build must use the pinned + toolchain, not a `setup-go` drift. + +## Plan + +**T1 — Release workflow: per-build prerelease lane.** +New `compass .github/workflows/release.yml`: trigger `push: branches: [main]` +with a `paths:` filter over `go/**`, `tools/toolchain/versions/go.nix`, and the +workflow itself; `permissions: {contents: write}`; concurrency group +`release`, `cancel-in-progress: false` (serialize, like +`publish-agent-image.yml:80-82`). Steps: install nix (the pinned +`cachix/install-nix-action` + substituter block copied from `ci.yml:150-168`), +resolve the pinned Go toolchain, build the four binaries (three × +`linux-amd64` + the CLI × `darwin-arm64`, `CGO_ENABLED=0` for the darwin +cross-build), generate `SHA256SUMS`, `gh release create build- +--prerelease` with the assets and a generated body carrying the image digest + +nix-outputs manifest (T2). +*Interfaces:* consumes HEAD sha + `go/cmd/{compass,compass-server,compass-runner}`; +produces Release `build-` with 6 assets (4 binaries + `SHA256SUMS` + +`nix-outputs.json`). Acceptance: a main push touching `go/**` yields a +downloadable, checksum-verifying `compass_build-_linux-amd64`. + +**T2 — Release body + nix-outputs manifest generator.** +A small script (bash or bun, matching `publish.sh` conventions) that emits: +(a) the GHCR image ref + config digest for the sha (querying the tag with the +fork skopeo exactly as `publish-agent-image.yml:160-173` does; "image not yet +published for this sha" degrades to a recorded absence, not a failure — the +image lane is paths-filtered independently); (b) `nix-outputs.json` from +`nix path-info --json` over the toolchain `langs` set and, when built, the +agent-image spec path. +*Interfaces:* consumes `git rev-parse HEAD`, GHCR (read, anonymous), +`tools/toolchain/gate-tools.nix`; produces the release body markdown + +`nix-outputs.json`. Acceptance: unit-testable pure formatting; a dry-run mode +prints both without any GitHub write. + +**T3 — Semver lane + image release-tag.** +Extend `release.yml` with `push: tags: ['v*']`: same build steps, Release is +non-prerelease named `vX.Y.Z`, guarded by the Fork 4 ancestry check +(`git merge-base --is-ancestor "$GITHUB_SHA" origin/main` — fail otherwise), +and additionally invokes `./publish.sh git- v` (the +forward-planned positional add, `publish.sh:38-39`; deliberately WITHOUT +`latest` — Fork 3's concurrency resolution) so the GHCR image gains the +matching `:vX.Y.Z` tag. Requires `packages: write` on this lane only +(job-level permissions split). Includes provisioning the `v*` tag ruleset +(Fork 4 control 1) — an IaC change if the github Pulumi stack manages this +repo, else a documented one-time setting. +*Interfaces:* consumes the pushed tag ref; produces Release `vX.Y.Z` + GHCR +`:vX.Y.Z`. Acceptance: pushing a `v0.0.1-test` tag on a throwaway ref cuts a +coherent Release + image tag (then deleted); a tag on a non-main sha fails the +ancestry guard; documented in the workflow header. + +**T4 — Consumer cutover: the internal config-publish job downloads the Release asset.** +Sequenced strictly AFTER the in-flight RIG-2025 fixes (ci-go routing + module +rename), which land now and independently — T4 is the later revert of that +routing, not its substitute. In the internal CD monorepo, two reviewed steps: +(1) **pin advance** — move the pinned-rev constant to the first released sha (no +Release covers the current pin `a61d0caf` retroactively); (2) replace the +`go install` provisioning with a download-and-verify of +`releases/download/build-/compass_…_linux-amd64` + `SHA256SUMS` check, +then drop the Go-toolchain requirement from the CD job (reverting the RIG-2025 +ci-go routing). Keep the single pinned-rev constant. RIG-2103 is retargeted onto +this task (Matt ruled RETARGET, 2026-08-22, OQ-2); the exact file targets live +in RIG-2103. +*Interfaces:* consumes Release assets (anonymous HTTPS); produces the same +`binDir/compass` provisioning contract the CD job returns today. Acceptance: +the existing unit tests' injected-runner seam re-covers the new argv; a CD run +on main provisions the CLI with no `go` in the image. + +**T5 — Docs + record freeze.** +Update `docs/architecture/build-and-ci.md` (the doc the image publish workflow +already cross-references, `publish-agent-image.yml:39-40`) with the release +lane; append the DECISIONS.md ledger rows for the choices in Forks 1-4 and the +OQ-7 semver policy. + +Ordering: T1 → T2 fold into one PR if small; T3 independent after T1; T4 lands +in the internal CD monorepo only after the RIG-2025 fixes are green on main AND +T1 has minted a Release at the advanced pin (the pin advance is T4 step 1); T5 +with the record freeze. + +## Tasks + +- [ ] T1 — `release.yml` per-build prerelease lane (binaries + SHA256SUMS) +- [ ] T2 — release body generator (image digest pointer + nix-outputs.json) +- [ ] T3 — semver `v*` lane + `:vX.Y.Z` image tag via publish.sh +- [ ] T4 — internal config-publish cutover: go install → Release asset download +- [ ] T5 — docs/architecture update + DECISIONS.md ledger rows (Forks 1–4 + the OQ-7 semver policy) + +## Resolved decisions and deferred questions + +The load-bearing questions were put to Matt and ruled — OQ-1/2/3 (2026-08-22) +and OQ-7 (the whole-product semver policy, 2026-08-24); the rulings below are +the frozen contract. OQ-4/5/6 stay deferred (non-load-bearing). + +- **OQ-1 (load-bearing): binary set boundary. RESOLVED (Matt, 2026-08-22): DEFER + the desktop lane.** v1 attaches `compass`, `compass-server`, `compass-runner` + only. `compass-stack` / `compass-app` (the Wails desktop lane — platform-linked, + `compass devenv.nix:189-190` notes the app links the system WebKit framework on + macOS, so it does NOT cross-compile from an ubuntu runner) are out of scope + here; their release-artifact distribution is decided in the native-packaging + lane (`compass docs/designs/product/compass-native-packaging/design.md`, RIG-1687 + umbrella). The deferral is filed as **RIG-2477** (parented under RIG-1687) so it + has a tracked home rather than living only as a design-record note. + - *Cross-ref (companion record, same PR):* the client-only pivot's **DL-238** + subsequently re-homed the desktop-lane distribution decision into + `compass-distribution/design.md` (this PR's companion record), so the current + deciding record for `compass-app`/`compass-stack` distribution is that record, + not `compass-native-packaging/design.md`. This OQ-1 pointer names the lane as + it stood at the 2026-08-22 ruling; the frozen prose is left intact. +- **OQ-2 (load-bearing): RIG-2103 disposition. RESOLVED (Matt, 2026-08-22): + RETARGET.** RIG-2103 (formerly "compass CLI via a nix derivation") is retargeted + onto T4's Release-asset download — one sourcing mechanism, checksum-verified, + toolchain-free; a nix derivation adds hermeticity the CD job does not need. + RIG-2103 now IS this record's T4 consumer-cutover task (its Linear scope was + updated 2026-08-22 to match), gated on T1 minting a Release at the advanced pin. +- **OQ-3 (load-bearing): per-build lane trigger breadth. RESOLVED (Matt, + 2026-08-22): every qualifying push, binary-affecting scope.** The per-build + prerelease lane triggers on every qualifying main push (cheap — binaries only, + no nix image build), filtered by the binary-affecting paths (`go/**` + the + toolchain pin), NOT unioned with the image lane's paths. An image-only sha + (`agent-image/**`, `forks/**`, `bun.lock`) republishes the image but mints NO + Release: its image identity stays GHCR-only (`:git-` + digest) and is + recorded in the *next* qualifying Release's body. Rationale: the Release lane + versions downloadable binaries; the image already has an immutable per-sha GHCR + identity, and unioning the filters would mint binary-identical Releases on + image-only shas. +- **OQ-7 (load-bearing): whole-product semver policy. RESOLVED (Matt, + 2026-08-24): the ruling Fork 3 deferred.** What `vX.Y.Z` means across the + product, on four points: + - *Pre-GA scheme.* Stay `0.MINOR.PATCH` now — matching the current `0.1.0` + stamp (`compass go/cmd/compass-stack/main.go:38`) and the GA milestone. Cut + `v1.0.0` **at GA**; that is when the + "MAJOR gates a breaking user-facing contract" rule below turns on. + - *Bump mechanism.* **Manual now** — Matt pushes `vX.Y.Z` and the T3 tag lane + cuts the Release + `:vX.Y.Z` image. **At GA, adopt conventional-commit + automation** (release-please: derive the bump + CHANGELOG from the + Conventional Commit subjects the repo already writes), replacing the manual + tag push. + - *One whole-product version, architecturally forced.* A release stamps ONE + version across every binary from the tag: `compass go/cmd/compass-stack/main.go` + (`version` at :38, fed to `Deps.ExpectedVersion` at :232) hard-fails + the attach when a live server's version != the build's, so client and server + are strict same-version (shipped together; the attach-check enforces it). + This is DISTINCT from the wire-protocol interop axis — the app pins + `clientAPIVersion = "compass.v1"` (`compass go/cmd/compass-app/bridge_service.go:468`) + against the server's `apiVersion = "compass.v1"` + (`compass go/cmd/compass-server/main.go:33`), a separate cross-version + contract the release semver does not track. + - *Post-1.0 bump rule.* MAJOR = a breaking change to a user-facing contract + (the CLI / its flags, the operator config + DSN + stack layout, or the wire + protocol); MINOR = backward-compatible features; PATCH = fixes. +- **OQ-4 (non-load-bearing, deferred): closure attachment.** Whether an + archival lane should ever attach full nix closures (`nix copy --to file://`) + for offline reproduction. No consumer today; caches + pinned substituters + (`ci.yml:165-168`) serve materialization. Revisit if an air-gapped consumer + appears. +- **OQ-5 (non-load-bearing, deferred): prerelease retention.** `build-*` + prereleases accumulate one per qualifying push. GitHub has no Release quota + pressure; a later janitor (keep last N + all semver) can be added without + contract change. Deferred. +- **OQ-6 (non-load-bearing, deferred): signed provenance.** GitHub artifact + attestations (`attestations: write` + `gh attestation verify`) or cosign + would bind assets to the building workflow identity; `SHA256SUMS` alone is + transfer-integrity, not provenance (§Alternatives). No external consumer + yet — revisit when one appears.