From 33e1758a15a55e749133a3ae7ca794b5c8d2f7c9 Mon Sep 17 00:00:00 2001 From: Robert Gingras Date: Fri, 31 Jul 2026 15:42:45 -0400 Subject: [PATCH 1/2] feat(agent): serve a friendly 504 page for paused or timing-out upstreams A frozen (cgroup.freeze) or OOMing-but-alive container keeps its TCP socket open but never responds, so NGINX hits proxy_read_timeout and returns a bare 504 Gateway Timeout. Only a 502 page existed. - Add error-pages/504.html (paused/under-load messaging), styled to match the existing pages. - Wire it via error_page 504 in the agent nginx template alongside the existing 502 handling (default server, per-service, and bare-domain blocks) plus the internal error-pages socket server. - Clarify 502 copy as connection-refused (stopped/crashed) so 502 vs 504 read distinctly for the memory-thrash case. Part of #431. Closes #435 --- agent/templates/nginx.conf.ejs | 34 ++++++++ error-pages/502.html | 3 +- error-pages/504.html | 151 +++++++++++++++++++++++++++++++++ 3 files changed, 187 insertions(+), 1 deletion(-) create mode 100644 error-pages/504.html diff --git a/agent/templates/nginx.conf.ejs b/agent/templates/nginx.conf.ejs index 5238054c..c57395a5 100644 --- a/agent/templates/nginx.conf.ejs +++ b/agent/templates/nginx.conf.ejs @@ -90,6 +90,7 @@ http { location /403.html { } location /404.html { } location /502.html { } + location /504.html { } location /auth-unavailable.html { } } @@ -141,6 +142,17 @@ http { proxy_pass http://error_pages; } + <%_ /* A frozen/paused or OOMing-but-alive upstream keeps the socket open + but never replies, so NGINX hits proxy_read_timeout and returns 504. */ -%> + error_page 504 @504; + location @504 { + rewrite ^ /504.html break; + proxy_method GET; + proxy_pass_request_body off; + proxy_set_header Content-Length ""; + proxy_pass http://error_pages; + } + <%_ if (httpServices.length === 0) { _%> location / { proxy_pass http://localhost:3000; @@ -210,6 +222,17 @@ http { proxy_pass http://error_pages; } + <%_ /* A frozen/paused or OOMing-but-alive upstream keeps the socket open + but never replies, so NGINX hits proxy_read_timeout and returns 504. */ -%> + error_page 504 @504; + location @504 { + rewrite ^ /504.html break; + proxy_method GET; + proxy_pass_request_body off; + proxy_set_header Content-Length ""; + proxy_pass http://error_pages; + } + <%_ if (authRequired && authServer) { _%> <%_ /* oauth2-proxy runs on its own address (this domain's "oauth2-proxy URL"). @@ -394,6 +417,17 @@ http { proxy_pass http://error_pages; } + <%_ /* A frozen/paused or OOMing-but-alive upstream keeps the socket open + but never replies, so NGINX hits proxy_read_timeout and returns 504. */ -%> + error_page 504 @504; + location @504 { + rewrite ^ /504.html break; + proxy_method GET; + proxy_pass_request_body off; + proxy_set_header Content-Length ""; + proxy_pass http://error_pages; + } + <%_ /* Serve documentation site statically */ -%> location / { root /opt/opensource-server/mie-opensource-landing/site; diff --git a/error-pages/502.html b/error-pages/502.html index 547c52b9..0a12bccd 100644 --- a/error-pages/502.html +++ b/error-pages/502.html @@ -130,7 +130,8 @@
502

Service Unavailable

- A service is configured for this hostname, but the upstream is not responding. + A service is configured for this hostname, but the upstream refused the + connection. The service is likely stopped or has crashed.

diff --git a/error-pages/504.html b/error-pages/504.html new file mode 100644 index 00000000..484a3e78 --- /dev/null +++ b/error-pages/504.html @@ -0,0 +1,151 @@ + + + + + + 504 — Service Paused + + + +
+
504
+

Service Paused

+

+ This service accepted the connection but did not respond in time. It is + likely paused or under heavy load — it should come back on its own shortly. +

+
+
+

What's happening?

+
    +
  • Wait a few moments and reload — the service may just be temporarily paused or busy.
    • +
  • The container may be under memory pressure; the host can pause it to protect other services on the same machine.
    • +
  • If you're in charge of this service, check its memory usage and look for a request that may be taking too long to complete.
    • +
  • Requests that take longer than the proxy's proxy_read_timeout are ended with this error.
    • +
+
+
+ + + From be3ef3c11f35fb0b8f5e9c11caa0c3f4028d817d Mon Sep 17 00:00:00 2001 From: Robert Gingras Date: Fri, 31 Jul 2026 15:45:52 -0400 Subject: [PATCH 2/2] Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> --- error-pages/504.html | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/error-pages/504.html b/error-pages/504.html index 484a3e78..5e4014c8 100644 --- a/error-pages/504.html +++ b/error-pages/504.html @@ -140,7 +140,7 @@

What's happening?

  • Wait a few moments and reload — the service may just be temporarily paused or busy.
  • The container may be under memory pressure; the host can pause it to protect other services on the same machine.
  • If you're in charge of this service, check its memory usage and look for a request that may be taking too long to complete.
    • -
  • Requests that take longer than the proxy's proxy_read_timeout are ended with this error.
    • +
  • Requests that exceed the proxy timeouts (for example, proxy_read_timeout) can end with this error.