fix(self-hosting): stop publishing Postgres and require POSTGRES_PASSWORD

README.md

@@ -7,7 +7,7 @@
 
 <p align="center">
   <a href="https://deepwiki.com/simstudioai/sim" target="_blank" rel="noopener noreferrer"><img src="https://img.shields.io/badge/Ask-DeepWiki-E6E6E6?labelColor=C3C3C3&color=E6E6E6" alt="Ask DeepWiki"></a>
-  <a href="https://cursor.com/link/prompt?text=Help%20me%20set%20up%20Sim%20locally.%20Follow%20these%20steps%3A%0A%0A1.%20First%2C%20verify%20Docker%20is%20installed%20and%20running%3A%0A%20%20%20docker%20--version%0A%20%20%20docker%20info%0A%0A2.%20Clone%20the%20repository%3A%0A%20%20%20git%20clone%20https%3A%2F%2Fgithub.com%2Fsimstudioai%2Fsim.git%0A%20%20%20cd%20sim%0A%0A3.%20Generate%20required%20secrets%20%28the%20stack%20will%20not%20start%20without%20them%29%3A%0A%20%20%20cat%20%3E%20.env%20%3C%3C%20EOF%0A%20%20%20BETTER_AUTH_SECRET%3D%24%28openssl%20rand%20-hex%2032%29%0A%20%20%20ENCRYPTION_KEY%3D%24%28openssl%20rand%20-hex%2032%29%0A%20%20%20INTERNAL_API_SECRET%3D%24%28openssl%20rand%20-hex%2032%29%0A%20%20%20CRON_SECRET%3D%24%28openssl%20rand%20-hex%2032%29%0A%20%20%20EOF%0A%0A4.%20Start%20the%20services%20with%20Docker%20Compose%3A%0A%20%20%20docker%20compose%20-f%20docker-compose.prod.yml%20up%20-d%0A%0A4.%20Wait%20for%20all%20containers%20to%20be%20healthy%20(this%20may%20take%201-2%20minutes)%3A%0A%20%20%20docker%20compose%20-f%20docker-compose.prod.yml%20ps%0A%0A5.%20Verify%20the%20app%20is%20accessible%20at%20http%3A%2F%2Flocalhost%3A3000%0A%0AIf%20there%20are%20any%20errors%2C%20help%20me%20troubleshoot%20them.%20Common%20issues%3A%0A-%20Port%203000%2C%203002%2C%20or%205432%20already%20in%20use%0A-%20Docker%20not%20running%0A-%20Insufficient%20memory%20(needs%2012GB%2B%20RAM)%0A%0AFor%20local%20AI%20models%20with%20Ollama%2C%20use%20this%20instead%20of%20step%203%3A%0A%20%20%20docker%20compose%20-f%20docker-compose.ollama.yml%20--profile%20setup%20up%20-d"><img src="https://img.shields.io/badge/Set%20Up%20with-Cursor-E6E6E6?logo=cursor&logoColor=1A1A1A&labelColor=C3C3C3&color=E6E6E6" alt="Set Up with Cursor"></a>
+  <a href="https://cursor.com/link/prompt?text=Help%20me%20set%20up%20Sim%20locally.%20Follow%20these%20steps%3A%0A%0A1.%20First%2C%20verify%20Docker%20is%20installed%20and%20running%3A%0A%20%20%20docker%20--version%0A%20%20%20docker%20info%0A%0A2.%20Clone%20the%20repository%3A%0A%20%20%20git%20clone%20https%3A%2F%2Fgithub.com%2Fsimstudioai%2Fsim.git%0A%20%20%20cd%20sim%0A%0A3.%20Generate%20required%20secrets%20%28the%20stack%20will%20not%20start%20without%20them%29%3A%0A%20%20%20cat%20%3E%20.env%20%3C%3C%20EOF%0A%20%20%20BETTER_AUTH_SECRET%3D%24%28openssl%20rand%20-hex%2032%29%0A%20%20%20ENCRYPTION_KEY%3D%24%28openssl%20rand%20-hex%2032%29%0A%20%20%20INTERNAL_API_SECRET%3D%24%28openssl%20rand%20-hex%2032%29%0A%20%20%20CRON_SECRET%3D%24%28openssl%20rand%20-hex%2032%29%0A%20%20%20POSTGRES_PASSWORD%3D%24%28openssl%20rand%20-hex%2024%29%0A%20%20%20EOF%0A%0A4.%20Start%20the%20services%20with%20Docker%20Compose%3A%0A%20%20%20docker%20compose%20-f%20docker-compose.prod.yml%20up%20-d%0A%0A4.%20Wait%20for%20all%20containers%20to%20be%20healthy%20(this%20may%20take%201-2%20minutes)%3A%0A%20%20%20docker%20compose%20-f%20docker-compose.prod.yml%20ps%0A%0A5.%20Verify%20the%20app%20is%20accessible%20at%20http%3A%2F%2Flocalhost%3A3000%0A%0AIf%20there%20are%20any%20errors%2C%20help%20me%20troubleshoot%20them.%20Common%20issues%3A%0A-%20Port%203000%20or%203002%20already%20in%20use%0A-%20Docker%20not%20running%0A-%20Insufficient%20memory%20(needs%2012GB%2B%20RAM)%0A%0AFor%20local%20AI%20models%20with%20Ollama%2C%20use%20this%20instead%20of%20step%203%3A%0A%20%20%20docker%20compose%20-f%20docker-compose.ollama.yml%20--profile%20setup%20up%20-d"><img src="https://img.shields.io/badge/Set%20Up%20with-Cursor-E6E6E6?logo=cursor&logoColor=1A1A1A&labelColor=C3C3C3&color=E6E6E6" alt="Set Up with Cursor"></a>
 </p>
 
 <p align="center">

apps/docs/content/docs/platform/self-hosting/docker.mdx

@@ -43,13 +43,15 @@ EOF
 
 <Callout type="info">
   Do not set `DATABASE_URL` or `BETTER_AUTH_URL` in `.env` — `docker-compose.prod.yml` composes both on the service definition, and a value set here is ignored. Change `POSTGRES_*` and `NEXT_PUBLIC_APP_URL` instead.
+
+  Because `DATABASE_URL` is composed from them, keep `POSTGRES_USER`, `POSTGRES_PASSWORD`, and `POSTGRES_DB` URL-safe — letters, digits, and `.` `_` `~` `-`. They are inserted into the connection string as written, so a value containing `@`, `/`, `?`, `#`, `%`, or a space can initialize the database while leaving the app and migrations unable to connect. `openssl rand -hex` output is always safe.
 </Callout>
 
 <Callout type="error">
   Save `ENCRYPTION_KEY` and `API_ENCRYPTION_KEY` somewhere outside this server. `ENCRYPTION_KEY` encrypts workspace and personal environment variables, stored provider API keys, MCP OAuth credentials, and deployment/chat secrets; `API_ENCRYPTION_KEY` encrypts user-generated Sim API keys. Neither can be regenerated — a database restore paired with a different key leaves the data it protected permanently unreadable.
 </Callout>
 
-The compose file refuses to start if `BETTER_AUTH_SECRET`, `ENCRYPTION_KEY`, or `INTERNAL_API_SECRET` is missing, rather than booting with empty values. `CRON_SECRET` is treated more gently: without it the `cron` service prints what to set and exits, leaving the rest of the stack running — so upgrading from a compose file that predates the scheduler still works.
+The compose file refuses to start if `BETTER_AUTH_SECRET`, `ENCRYPTION_KEY`, `INTERNAL_API_SECRET`, or `POSTGRES_PASSWORD` is missing, rather than booting with empty or well-known values. Postgres applies `POSTGRES_PASSWORD` only when it first creates the database volume — see [Postgres on Compose](/platform/self-hosting/security#postgres-on-compose) before changing it on an existing install. `CRON_SECRET` is treated more gently: without it the `cron` service prints what to set and exits, leaving the rest of the stack running — so upgrading from a compose file that predates the scheduler still works.
 
 Images track `latest` unless you pin them. For production, see [Upgrades](/platform/self-hosting/upgrades).
 
@@ -65,7 +67,7 @@ Six services start:
 |---|---|---|
 | `simstudio` | 3000 | Main application (8 GB memory limit) |
 | `realtime` | 3002 | WebSocket server (1 GB memory limit) |
-| `db` | 5432 | PostgreSQL 17 with pgvector |
+| `db` | internal | PostgreSQL 17 with pgvector — not published to the host |
 | `redis` | internal | Pub/sub and shared cache — not published to the host |
 | `cron` | — | Runs the [background jobs](/platform/self-hosting/background-jobs) on a schedule |
 | `migrations` | — | Applies schema migrations once, then exits |
@@ -206,5 +208,5 @@ npx sim-setup update
   { question: "Do scheduled workflows work on Docker Compose?", answer: "Yes. The cron service runs the same jobs the Helm chart schedules as Kubernetes CronJobs, using the schedules in docker/crontab. It needs CRON_SECRET — without it the service prints what to set and exits, and the rest of the stack keeps running."},
   { question: "Why is there a Redis container?", answer: "Redis backs pub/sub for live Chat task status and table events, plus shared caches. Pub/sub has no fallback that works across processes, so live status would not stream without it. The port is deliberately not published so it cannot collide with a local Redis."},
   { question: "How do I back up and restore the database?", answer: "Back up with: docker compose -f docker-compose.prod.yml exec -T db pg_dump -U postgres simstudio > backup.sql. The -T matters — without it exec allocates a TTY and corrupts the redirected dump. Restore with: docker compose -f docker-compose.prod.yml exec -T db psql -U postgres simstudio < backup.sql. The database data is persisted in a Docker volume named postgres_data."},
-  { question: "Can I customize the PostgreSQL credentials?", answer: "Yes. The docker-compose.prod.yml uses environment variable defaults: POSTGRES_USER (default: postgres), POSTGRES_PASSWORD (default: postgres), POSTGRES_DB (default: simstudio), and POSTGRES_PORT (default: 5432). Set these in your .env file to override them." },
+  { question: "Can I customize the PostgreSQL credentials?", answer: "Yes. Set POSTGRES_USER (default: postgres) and POSTGRES_DB (default: simstudio) in .env before the first start. POSTGRES_PASSWORD has no default — the compose file will not start without it. Postgres applies all three only when it creates the database volume, so changing them later does not change the existing database; to rotate the password, see Postgres on Compose in the security guide." },
 ]} />

apps/docs/content/docs/platform/self-hosting/security.mdx

@@ -271,19 +271,29 @@ networkPolicy:
 
 The service bundles ~2.2 GB of spaCy models, so first start takes around three minutes and it needs at least 4 GB of memory.
 
-## The shipped Compose file publishes Postgres
+## Postgres on Compose
 
-<Callout type="error">
-  `docker-compose.prod.yml` maps the database to the host: `${POSTGRES_PORT:-5432}:5432`, with `POSTGRES_USER` and `POSTGRES_PASSWORD` both defaulting to `postgres`. A plain `docker compose up -d` against that file, on a machine with a public interface, therefore exposes an open Postgres on 5432 with credentials anyone can guess. The local and Ollama stacks map the database the same way, so apply the fix to whichever file started your install.
+The Compose files do not publish the `db` service to the host: `simstudio`, `realtime`, and `migrations` reach it over the Compose network as `db:5432`, and nothing outside the stack can. `docker-compose.prod.yml` also refuses to start without `POSTGRES_PASSWORD`, the same way it refuses to start without `BETTER_AUTH_SECRET`.
 
-  The [Docker guide](/platform/self-hosting/docker#1-configure-environment) tells you to generate `POSTGRES_PASSWORD` before the first start — do that, and additionally close the port:
+<Callout type="error">
+  Installs created from an earlier Compose file published the database on every interface of the host (`5432:5432`), and a `POSTGRES_PASSWORD` left unset fell back to `postgres`. A Docker `ports:` mapping writes its own iptables rules, so a host firewall that looks like it blocks 5432 usually does not. Update the Compose file, then check what your database was created with:
 
-  - **Do not need host access.** Delete the `ports:` block from the `db` service. Every other service reaches it over the Compose network by name.
-  - **Need host access.** Bind it to loopback only — `127.0.0.1:${POSTGRES_PORT:-5432}:5432` — and reach it over an SSH tunnel.
+  - **You set `POSTGRES_PASSWORD` before the first start.** Nothing else to do — updating the file closes the port.
+  - **You never set it.** Postgres applies `POSTGRES_PASSWORD` only when it creates the data volume, so the database still uses `postgres`. Set `POSTGRES_PASSWORD=postgres` in `.env` so the stack starts, then rotate it: run `docker compose -f docker-compose.prod.yml exec db psql -U postgres -c "ALTER ROLE CURRENT_USER PASSWORD '<new password>'"` (with your `POSTGRES_USER` in place of `postgres` if you set one), set `POSTGRES_PASSWORD` to the same value, and run `docker compose -f docker-compose.prod.yml up -d`. Setting a new value in `.env` alone does not change the database's password and locks the app out.
 
-  A Docker `ports:` mapping writes its own iptables rules, so a host firewall that looks like it blocks 5432 usually does not.
+  `npx sim-setup start` and `npx sim-setup update` write the right value for you and print the rotation steps.
 </Callout>
 
+To reach the database from the host — `psql`, a desktop client, a backup job — use `docker compose -f docker-compose.prod.yml exec db psql -U postgres simstudio`, or pass a second Compose file that publishes it on loopback only and connect over an SSH tunnel:
+
+```yaml
+# db-port.yml — docker compose -f docker-compose.prod.yml -f db-port.yml up -d
+services:
+  db:
+    ports:
+      - '127.0.0.1:5432:5432'
+```
+
 ## Pre-launch checklist
 
 - All five secrets generated fresh, stored in a secret manager, and **`ENCRYPTION_KEY` backed up separately**
@@ -297,7 +307,7 @@ The service bundles ~2.2 GB of spaCy models, so first start takes around three m
 - NetworkPolicy enabled and `ingressFrom` scoped to the ingress controller
 - Namespace labelled `pod-security.kubernetes.io/enforce=restricted`
 - Object storage buckets private, with CORS limited to your Sim origin
-- Database reachable only from the deployment — on Compose, the `db` service's host `ports:` mapping removed or bound to `127.0.0.1` — with a generated `POSTGRES_PASSWORD`
+- Database reachable only from the deployment — on Compose, no host `ports:` mapping on the `db` service, or one bound to `127.0.0.1` — with a generated `POSTGRES_PASSWORD`
 - TLS enforced (`sslMode: require`) on an externally managed database, or on the bundled one once you have configured it for TLS — the shipped Compose database does not enable it
 - Backups configured **and a restore rehearsed**
 - Sandbox strategy decided for user code

apps/docs/content/docs/platform/self-hosting/upgrades.mdx

@@ -193,11 +193,13 @@ install uses:
 
 | Install | What it runs |
 |---|---|
-| `docker-compose.prod.yml` | Refreshes its managed copy of the Compose file, then `docker compose pull` — the versions configured by `SIM_VERSION`, or `latest` when unset |
+| `docker-compose.prod.yml` | Refreshes its managed copy of the Compose file, writes `POSTGRES_PASSWORD` to `.env` if it is missing, then `docker compose pull` — the versions configured by `SIM_VERSION`, or `latest` when unset |
 | `docker-compose.local.yml` | `docker compose build --pull` — rebuilds from source against refreshed base images, no pull of published images |
 
 Inspect the result with `npx sim-setup logs`, which targets whichever Compose file the install uses.
 
+`docker-compose.prod.yml` requires `POSTGRES_PASSWORD`. If you manage the file yourself and Compose stops with `required variable POSTGRES_PASSWORD is missing a value`, your database was created with the password `postgres` — set exactly that in `.env`, not a new value, then see [Postgres on Compose](/platform/self-hosting/security#postgres-on-compose) to rotate it.
+
 The CLI detects only those two files. An install started from `docker-compose.ollama.yml` is invisible to it: `update`, `logs`, and `status` report no install, or — in a source checkout that also carries per-application env files — report that checkout's dev install instead. Upgrade that stack directly:
 
 ```bash

docker-compose.local.yml

@@ -111,11 +111,13 @@ services:
       timeout: 5s
       retries: 5
 
+  # Not published to the host: the other services reach it over the Compose
+  # network as db:5432. A host `ports:` mapping binds every interface, and the
+  # password below defaults to a well-known value. For host access use
+  # `docker compose exec db psql -U postgres simstudio`.
   db:
     image: pgvector/pgvector:pg17
     restart: always
-    ports:
-      - '${POSTGRES_PORT:-5432}:5432'
     environment:
       - POSTGRES_USER=${POSTGRES_USER:-postgres}
       - POSTGRES_PASSWORD=${POSTGRES_PASSWORD:-postgres}

docker-compose.ollama.yml

@@ -87,12 +87,14 @@ services:
     command: ['bun', 'run', 'db:migrate']
     restart: 'no'
 
-  # PostgreSQL Database with Vector Extension
+  # PostgreSQL Database with Vector Extension.
+  # Not published to the host: the other services reach it over the Compose
+  # network as db:5432. A host `ports:` mapping binds every interface, and the
+  # password below defaults to a well-known value. For host access use
+  # `docker compose exec db psql -U postgres simstudio`.
   db:
     image: pgvector/pgvector:pg17
     restart: always
-    ports:
-      - '${POSTGRES_PORT:-5432}:5432'
     environment:
       - POSTGRES_USER=${POSTGRES_USER:-postgres}
       - POSTGRES_PASSWORD=${POSTGRES_PASSWORD:-postgres}

docker-compose.prod.yml

@@ -15,7 +15,7 @@ services:
           memory: 8G
     environment:
       - NODE_ENV=production
-      - DATABASE_URL=postgresql://${POSTGRES_USER:-postgres}:${POSTGRES_PASSWORD:-postgres}@db:5432/${POSTGRES_DB:-simstudio}
+      - 'DATABASE_URL=postgresql://${POSTGRES_USER:-postgres}:${POSTGRES_PASSWORD:?is required. For a new install set it in .env to the output of: openssl rand -hex 24. An install created before it was required uses the password postgres - set POSTGRES_PASSWORD=postgres to keep its data.}@db:5432/${POSTGRES_DB:-simstudio}'
       - BETTER_AUTH_URL=${NEXT_PUBLIC_APP_URL:-http://localhost:3000}
       - NEXT_PUBLIC_APP_URL=${NEXT_PUBLIC_APP_URL:-http://localhost:3000}
       # TRUSTED_ORIGINS: comma-separated public origins to trust for auth in
@@ -85,7 +85,7 @@ services:
           memory: 1G
     environment:
       - NODE_ENV=production
-      - DATABASE_URL=postgresql://${POSTGRES_USER:-postgres}:${POSTGRES_PASSWORD:-postgres}@db:5432/${POSTGRES_DB:-simstudio}
+      - 'DATABASE_URL=postgresql://${POSTGRES_USER:-postgres}:${POSTGRES_PASSWORD:?is required. For a new install set it in .env to the output of: openssl rand -hex 24. An install created before it was required uses the password postgres - set POSTGRES_PASSWORD=postgres to keep its data.}@db:5432/${POSTGRES_DB:-simstudio}'
       - NEXT_PUBLIC_APP_URL=${NEXT_PUBLIC_APP_URL:-http://localhost:3000}
       # Derived from NEXT_PUBLIC_APP_URL, matching the simstudio service — a
       # single public-origin variable keeps the two from disagreeing.
@@ -109,7 +109,7 @@ services:
     image: ghcr.io/simstudioai/migrations:${SIM_VERSION:-latest}
     working_dir: /app/packages/db
     environment:
-      - DATABASE_URL=postgresql://${POSTGRES_USER:-postgres}:${POSTGRES_PASSWORD:-postgres}@db:5432/${POSTGRES_DB:-simstudio}
+      - 'DATABASE_URL=postgresql://${POSTGRES_USER:-postgres}:${POSTGRES_PASSWORD:?is required. For a new install set it in .env to the output of: openssl rand -hex 24. An install created before it was required uses the password postgres - set POSTGRES_PASSWORD=postgres to keep its data.}@db:5432/${POSTGRES_DB:-simstudio}'
     depends_on:
       db:
         condition: service_healthy
@@ -151,14 +151,21 @@ services:
       simstudio:
         condition: service_healthy
 
+  # Not published to the host: app, realtime and migrations reach it over the
+  # Compose network as db:5432. A host `ports:` mapping binds every interface and
+  # Docker's own iptables rules bypass host firewalls, so it would put the
+  # database on the network. For host access (psql, a desktop client), use
+  # `docker compose exec db psql -U postgres simstudio`, or pass a second -f
+  # file that publishes '127.0.0.1:5432:5432' (-f disables override files).
   db:
     image: pgvector/pgvector:pg17
     restart: unless-stopped
-    ports:
-      - '${POSTGRES_PORT:-5432}:5432'
     environment:
       - POSTGRES_USER=${POSTGRES_USER:-postgres}
-      - POSTGRES_PASSWORD=${POSTGRES_PASSWORD:-postgres}
+      # Postgres applies this only when it creates the data volume; later
+      # changes are ignored. An install created before this was required got
+      # the password `postgres` — set exactly that to keep it working.
+      - 'POSTGRES_PASSWORD=${POSTGRES_PASSWORD:?is required. For a new install set it in .env to the output of: openssl rand -hex 24. An install created before it was required uses the password postgres - set POSTGRES_PASSWORD=postgres to keep its data.}'
       - POSTGRES_DB=${POSTGRES_DB:-simstudio}
     volumes:
       - postgres_data:/var/lib/postgresql/data

packages/cli/src/index.ts

@@ -280,8 +280,6 @@ async function main() {
     'POSTGRES_DB=simstudio',
     '-v',
     `${dataDir}/postgres:/var/lib/postgresql/data`,
-    '-p',
-    '5432:5432',
     'pgvector/pgvector:pg17',
   ])