fix: relaunch Chromium on disconnect and fail health check when it is gone

[HarshMN2345]

Problem

Chromium is launched once at module load and never re-checked:

export const browser = await chromium.launch({ ... });

When it dies, the Bun server keeps listening. Every request after that fails with Target page, context or browser has been closed, and it stays that way until something unrelated restarts the container.

/v1/health already reports the right thing — browser.isConnected() — but returns it with HTTP 200, so no probe can act on it.

Observed in production: sustained failures at roughly this rate for days, each failing in well under a second, with zero pod restarts. It stopped only when a rollout happened to restart the pods.

Change

• getBrowser() resolves the browser and relaunches it when the instance is disconnected. Concurrent callers share a single in-flight relaunch, and the guard is cleared on failure so a failed relaunch is retried on the next request.
• /v1/health returns 503 when disconnected, so an httpGet liveness probe can restart the pod.
• screenshots, reports and test resolve through getBrowser() instead of importing the singleton.

Verification

Ran the service and killed Chromium out from under it.

Step	Before	After
Health, browser alive	200 pass	200 pass
Kill Chromium → health	200 fail	503 fail
Screenshot request	400 "context: Target page, context or browser has been closed"	200, valid PNG
Health after	200 fail, stays broken	200 pass, self-healed

The "before" column reproduces the production error string exactly.

bun run type-check and bun run lint pass.

Tests

tests/recovery/browser.test.ts, running against the real image as its own CI step:

1. Recovery — SIGKILL headless-shell (what the OOM killer sends in production), assert /v1/health returns 503, a capture succeeds (relaunch), and health returns 200 again.
2. Shared relaunch — kill Chromium, fire five concurrent captures, assert all succeed and the container logs show exactly one relaunch. Verified the assertion has teeth: with the sharing guard removed, five concurrent launches wedged the service past the test timeout.

No mocks, no test-only surface in the image. The suite runs separately from the other e2e tests because killing the browser mid-flight fails whatever else is using it, and skips when no container is resolvable.

Note for deployment

The 503 only has an effect once the liveness probe actually calls /v1/health. The chart currently uses tcpSocket, which only checks that Bun is listening — that needs a separate change to httpGet on this path, plus a readiness probe so a pod with a dead browser leaves the Service.

[github-actions]

Docker Image Stats

Metric	Value
Image Size	494MB
Memory Usage	156.6MiB
Cold Start Time	1.04s
Screenshot Time	2.82s

_{Screenshot benchmark: Average of 3 runs on https://appwrite.io}

[greptile-apps]

Greptile Summary

This PR adds on-demand Chromium recovery and makes browser disconnection visible through the health endpoint.

• Shares one in-flight relaunch among concurrent browser requests and permits retries after launch failures.
• Routes screenshot, report, and test operations through the recoverable browser accessor.
• Returns HTTP 503 from the health endpoint while Chromium is disconnected.
• Adds container-level recovery and concurrent-relaunch tests to CI.

Confidence Score: 5/5

The PR appears safe to merge because no blocking failure remains within the eligible follow-up-review scope.

No blocking failure remains.

Important Files Changed

Filename	Overview
src/config/browser.ts	Introduces a shared, retryable Chromium relaunch promise and exposes the current browser instance for health checks.
src/routes/health.ts	Changes disconnected browser health responses from HTTP 200 to HTTP 503 while retaining the existing JSON status.
src/routes/screenshots.ts	Resolves the shared browser through the recovery accessor before creating each screenshot context.
src/routes/reports.ts	Resolves the shared browser through the recovery accessor before creating each Lighthouse context.
src/routes/test.ts	Uses the recovery accessor before creating the diagnostic route's browser context.
tests/recovery/browser.test.ts	Adds real-container coverage for recovery after SIGKILL and for sharing one relaunch among concurrent requests.
.github/workflows/test.yml	Adds the browser recovery suite as a separate CI step.
package.json	Adds a dedicated command for running the recovery tests.

_{Reviews (9): Last reviewed commit: "test: poll for the disconnect instead of..." | Re-trigger Greptile}