feat(proxy): multiplex HTTP and Postgres on one port when their ports coincide - #56
Conversation
… coincide
Ships as v0.19.0. When postgres.port equals proxy.port (and their hosts
resolve to the same address), gatekeeper now multiplexes the HTTP/CONNECT
proxy and the Postgres data-plane listener onto one real listener instead
of binding two — no new config flag; the port equality itself is the
declaration. Distinct ports (today's default, and every existing config)
keep the two-listener path byte-for-byte unchanged.
Mechanism: proxy.Demux (proxy/demux.go) owns the one real listener and
classifies each accepted connection by its first 8 bytes in its own
goroutine — never in the shared accept loop, so a silent or slow client
can't stall Accept for other connections. Classification positively
matches Postgres startup signatures (SSLRequest, GSSENCRequest, v3
StartupMessage) and defaults everything else to HTTP, mirroring Caddy-L4's
postgres matcher. The sniffed bytes are replayed on the classified
connection (mirroring proxyProtoLogConn's hold-then-replay pattern) before
it's pushed onto one of two in-memory virtual listeners; http.Server.Serve
and PostgresServer.StartListener then run completely unmodified against
those.
Config validation (resolveListenTopology, config.go) runs at New() and
Start() so the two can never disagree: equal ports with different hosts,
or with proxy.proxy_protocol != postgres.proxy_protocol, are fatal
startup errors — a shared listener has exactly one PROXY protocol
setting, owned by proxy.proxy_protocol. Port 0 on both sides never
triggers multiplexing, since two independent ephemeral binds aren't "the
same port" — this preserves every existing test and config that leaves
the port unset on both listeners.
Docs: config reference ("Sharing one listener with the HTTP proxy"),
load-balancer guide ("Single load balancer, one shared port", including
health-check behavior on the shared listener), a concepts note, and a
commented example in examples/gatekeeper-postgres.yaml.
📦 Previous Review (superseded by newer review)Claude finished @andybons's task in 5m 8s —— View job Review: single-port multiplexing (proxy/demux.go)
SummaryThis is a well-scoped, well-documented change. I traced the full lifecycle carefully:
FindingOne inline comment on Note on verificationI was not able to run |
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
…f exiting Demux.acceptLoop is the sole caller of Accept on the real socket in multiplex mode — http.Server.Serve only ever sees the virtual listener, which never surfaces an OS-level error, so http.Server's own accept-retry loop can neither see nor recover from a transient failure on the real socket. The loop previously returned on ANY Accept error, so one transient error (EMFILE/ENFILE under fd exhaustion, ECONNABORTED — realistic for a proxy holding many long-lived CONNECT tunnels and Postgres relays) permanently killed accept for BOTH planes until process restart. Before multiplexing, http.Server.Serve tolerated exactly these errors, so this silently removed the HTTP plane's prior resilience. Mirror net/http.Server.Serve's accept-error handling: exit cleanly only when the demux is shutting down (Close/StopAccepting set the closed flag before closing the listener), otherwise back off with a capped exponential delay (5ms doubling to a 1s cap, logged at WARN) and retry while the listener is live, resetting the delay after a successful Accept. Unlike net/http, the retry is not gated on the deprecated, unreliable net.Error.Temporary(): any error while the listener is open is retried; a genuinely dead-but-unclosed listener retries once per second forever with a WARN each time — the same visible, capped pathological case net/http tolerates, not a zero-delay spin. PostgresServer.acceptLoop (proxy/postgres.go) has the same unconditional-exit pattern, but its blast radius is unchanged by this PR (non-multiplex Postgres plane only), so it is left out of scope here.
|
Claude finished @andybons's task in 58s —— View job Review: transient Accept-error fix (commit
|
…nstead of exiting (#57) PostgresServer.acceptLoop had the identical unconditional-exit-on-any- Accept-error bug that PR #56 fixed in Demux.acceptLoop, and was intentionally left out of that PR's scope. This is the sibling fix: acceptLoop now mirrors Demux.acceptLoop (and, in turn, net/http.Server.Serve) by retrying any non-shutdown Accept error with a capped exponential backoff (reusing demux.go's demuxAcceptRetryBaseDelay/demuxAcceptRetryMaxDelay constants so the two loops can't drift), logging at WARN, and only exiting cleanly on an intentional Stop/Shutdown.
1 participant
What
Serves the HTTP/CONNECT proxy and the Postgres data plane on a single listener when
postgres.port == proxy.port(and the same host), so one GCP TCP Proxy load balancer — one forwarding rule, one backend service, one health check, one PROXY config — can front both planes. A TCP Proxy LB is L4 and can't route the two apart itself, so gatekeeper demultiplexes them.Design
Trigger — no flag. Multiplexing is expressed by the ports coinciding, mirroring how Traefik/Caddy attach multiple protocols to one entrypoint (no
multiplex: trueswitch). Different ports → two listeners, exactly as today.port: 0(OS-assigned) never triggers it — two ephemeral binds aren't "the same port."Mechanism — hand-rolled cmux-style, no new dependency. One accept loop owns the real (optionally proxyproto-wrapped) listener. Each accepted conn is peeked in its own goroutine (never the accept loop — preserves the don't-block-Accept discipline), classified, and pushed onto one of two in-memory virtual listeners.
http.Server.ServeandPostgresServer.StartListenerrun unmodified on them.Classification. Positive-match Postgres (
SSLRequest/GSSENCRequest/ v3StartupMessage), everything else → HTTP. A short read can't match (requires 8 bytes with a zero at offset 4, which no HTTP/TLS opening has). A misroute lands on the wrong parser and fails — it can't bypass a control, since each plane still authenticates after demux.PROXY protocol. The wrapper stays outermost; the header is stripped before classification and the advertised client IP reaches both planes. One shared listener ⇒ one setting: equal ports require
proxy.proxy_protocol == postgres.proxy_protocol(fatal error on mismatch).Review
Opus-reviewed (SHIP): mutation-checked the classifier and byte-replay tests non-vacuous, verified the accept loop never blocks on a slow client, byte-replay is exact through both planes, shutdown drains without goroutine/fd leaks, and demux only routes (auth unchanged). One doc/code wording mismatch it flagged (host match is string equality, not address resolution) is corrected in the config reference.
Tests
Test-first: classifier unit tests incl. adversarial inputs (short read, garbage, length-only, all HTTP verbs, h2 preface, GSSENC), shared-port E2E (real CONNECT+TLS interception AND real Postgres handshake on one port), PROXY-on-shared-port for both planes, silent-client-doesn't-block-Accept, backlog-full-drops. Live-verified end-to-end.
go test -race ./..., vet, gofmt clean. Distinct-port path is the original code moved verbatim — all pre-existing tests untouched and green.CHANGELOG
v0.19.0.