Skip to content

fix: set explicit max_connections floor in postgresql.conf - #1

Draft
posthog[bot] wants to merge 1 commit into
developfrom
posthog-self-driving/fixrails-size-postgres-connection-pools-fa745a
Draft

fix: set explicit max_connections floor in postgresql.conf#1
posthog[bot] wants to merge 1 commit into
developfrom
posthog-self-driving/fixrails-size-postgres-connection-pools-fa745a

Conversation

@posthog

@posthog posthog Bot commented Aug 10, 2026

Copy link
Copy Markdown

What kind of change does this PR introduce?

Bug fix — server-side capacity floor for Postgres connection slots.

What is the current behavior?

  • On 2026-08-10 three Postgres hosts refused all non-superuser connections for about two minutes, and every app on each host lost database access at once. See the inbox report.
  • ansible/files/postgresql_config/postgresql.conf.j2 left max_connections and superuser_reserved_connections commented out, so the box fell back to the compiled-in stock 100 slots.
  • The database-optimizations service sizes the box by writing generated-optimizations.conf, but it runs with ExecStart=- and ignores its own failures. When optimize db fails, the placeholder file stays empty and the ceiling silently stays at 100 — no floor.

What is the new behavior?

  • The template now sets an explicit max_connections = 200 floor.
-#max_connections = 100			# (change requires restart)
-#superuser_reserved_connections = 3	# (change requires restart)
+max_connections = 200			# floor above the stock 100; generated-optimizations.conf below raises this to fit the box (change requires restart)
+superuser_reserved_connections = 3	# (change requires restart)
  • generated-optimizations.conf is included later in the file, so a successful optimize db run still overrides the value with box-appropriate sizing. The floor applies only when sizing is absent, which removes the stock-100 failure mode.

Additional context

  • Scope: This PR is the server-side half of the fix and lives in this repo. The root cause of the incident also has a client-side half — an invalid max_connections: pool key in york_factory/config/database.yml (ActiveRecord expects pool:) and pool: 100 in two other apps — which lives in application repositories not present here. Those pool trims must land separately.
  • Agent context: I confirmed the build runs setup-supabase-internal (nixpkg_mode: true), which does uncomment the generated-optimizations.conf include at build time. So the fragility is not the commented include but the swallowed optimize db failure that leaves the box at stock 100. An explicit floor is resilient to that failure, so I chose it over touching the already-effective include.

Created with PostHog Desktop from this inbox report.

The template left max_connections and superuser_reserved_connections
commented out, so the box fell back to the compiled-in stock 100 slots.
The database-optimizations service that sizes the box writes
generated-optimizations.conf, but it runs with ExecStart=- and ignores
its own failures. When optimize db fails, the placeholder file stays
empty and the ceiling silently stays at 100.

Set an explicit max_connections floor of 200. The generated-optimizations.conf
include sits later in the file, so a successful optimize run still
overrides this value with box-appropriate sizing. The floor only applies
when sizing is absent, which removes the stock-100 failure mode.

Generated-By: PostHog Code
Task-Id: 66ac8c8b-209d-4387-af52-a7e9cc83e447
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants