fix(deploy): let the cluster own the Gateway, keep only the route

[themightychris]

Draft — one assumption to verify against a real cluster first (see the end).

Follow-up to #517. It was chasing a real bug, but the fix pointed at the wrong layer, and the layer it pointed at can't run on our cluster.

The problem

This repo cannot know the cluster it lands in, and gateway-listeners.yaml was trying to. Between the ListenerSet and its routes, it hardcoded four facts about cfp-live-cluster — the shared Gateway's name and namespace, the ClusterIssuer, the TLS Secret name, and the ListenerSet API itself. Three of them are wrong there:

• cert-manager.io/cluster-issuer: letsencrypt-prod is dead. It solves ACME through ingress-nginx, which requires PROXY protocol on every connection and is therefore unreachable from inside the cluster. No certificate has issued through it since May — that's what expired nine certs on 2026-07-12 (cfp-live-cluster#144).

• certificateRefs: balancer-tls is the legacy Ingress Secret. Gateway certs are <app>-gw-tls.

• ListenerSet is not reconciled by Envoy Gateway v1.7.3. It watches XListenerSet:

  XListenerSet CRD not found, skipping XListenerSet watch
  

  The object applies cleanly to the API server and is then silently ignored. balancerproject.org would simply have no listener, with nothing in any log to say why. (It lands in EG v1.8.)

None of that is a mistake anyone could have avoided from this repo — it isn't knowable here. So the fix is to stop trying to know it.

The split

This repo owns	The cluster owns
HTTPRoute — paths, backend Services, ports	Gateway — listeners, hostnames, TLS
Deployment, Service, app config	Certificates, ClusterIssuers, GatewayClass

Paths and ports are app facts: they change when the app changes, and they should land in the same commit as that change. (Case in point — v1.1.6 moved the Service from 8000 → 80, and the cluster's copy of the route still says 8000. Under this split that mismatch cannot happen.)

Hostnames, TLS, and issuers are cluster facts: they differ per environment and are negotiated with every other app sharing the ingress.

The contract

The cluster provides a Gateway named after the app, in the app's namespace. That's the entire interface.

spec:
  parentRefs:
    - name: balancer     # the Gateway the cluster provides
  # no hostnames — inherited from the Gateway's listeners
  rules:
    - matches: [{path: {type: PathPrefix, value: /}}]
      backendRefs: [{name: balancer, port: 80}]

A route that omits hostnames inherits them from the listeners it attaches to. One file serves balancerproject.org in live and balancer.sandbox.k8s.phl.io in sandbox — no patches, no placeholders, no environment-specific routing. The HOSTNAME_PLACEHOLDER substitutions and the overlay routing patches are all gone.

HOSTNAME stays in the ConfigMap: the application legitimately needs its own public name for Django ALLOWED_HOSTS / CSRF_TRUSTED_ORIGINS. That's app config, not routing.

Also drops balancer-http-redirect

The cluster already redirects every hostname reaching it on port 80. An app-level redirect is worse than redundant: a route claiming an exact hostname on the shared HTTP listener outranks the catch-all redirect under Gateway API precedence, and can shadow cert-manager's ACME solver route.

That's what produced the apex 404 #517 set out to fix. The 404 was a stuck certificate — cert-manager's solver route had been sitting on the hostname for 57 days because DNS hadn't cut over. It cleared the instant the cert issued, and balancerproject.org now returns 301 → https with no code change at all.

Changes

• Delete base/gateway-listeners.yaml
• base/httproute.yaml — one route, parentRefs by convention, no hostnames, no redirect route
• base/kustomization.yaml — drop the ListenerSet
• overlays/{production,sandbox} — drop all routing patches
• Add deploy/manifests/balancer/README.md stating the contract

All three kustomizations build clean (base, overlays/production, overlays/sandbox).

Before merging — verify the inheritance

The design rests on a route with no hostnames inheriting its Gateway listener's hostname. That's what the Gateway API spec says, and Envoy Gateway should implement it — but today's whole lesson is that a manifest can look correct and be silently ignored, so I don't want to assert it. Worth confirming against a real Gateway (sandbox's echo-http is a zero-stakes probe) before this merges.

If inheritance doesn't behave, the fallback is one explicit hostname patch per overlay — still no Gateway, no issuer, no TLS in this repo.

Unrelated, but noticed

overlays/sandbox sets HOSTNAME=sandbox.balancerproject.org, but the sandbox cluster actually serves balancer.sandbox.k8s.phl.io. Left alone here — flagging it in case it's stale.

Refs CodeForPhilly/cfp-live-cluster#166, CodeForPhilly/cfp-live-cluster#168