From b1832329ee1846fdfb56a410263e835499c26d02 Mon Sep 17 00:00:00 2001 From: mavneox Date: Fri, 19 Jun 2026 00:50:46 +0000 Subject: [PATCH 1/2] fix(GEN-3647): restore api.gen.pro docs routing after CloudFront exit api.gen.pro serves both the Rails API (/v1, /up) and the Astro docs site (GitHub Pages, poweredbygen.github.io/api-docs). CloudFront used to split the paths and rewrite the Pages origin to /api-docs. The Hetzner migration removed CloudFront and pointed api.gen.pro entirely at Rails, so every docs path (/, /llms.txt, /openapi.yaml, /guides/*, /reference/*) now 404s. Adds edge-routing artifacts (no code redeploy of the docs needed): - cloudflare/api-gen-pro-router.worker.js: Worker that routes API prefixes to the Rails origin and all other paths to GitHub Pages (prepending /api-docs, rewriting Pages redirects back to api.gen.pro). - cloudflare/DEPLOY.md: Worker deploy + verify steps. - cloudflare/DEPLOY-no-worker.md: native Origin Rules + URL Rewrite alternative. Co-Authored-By: Claude Opus 4.8 --- cloudflare/DEPLOY-no-worker.md | 55 ++++++++++++ cloudflare/DEPLOY.md | 53 +++++++++++ cloudflare/api-gen-pro-router.worker.js | 111 ++++++++++++++++++++++++ 3 files changed, 219 insertions(+) create mode 100644 cloudflare/DEPLOY-no-worker.md create mode 100644 cloudflare/DEPLOY.md create mode 100644 cloudflare/api-gen-pro-router.worker.js diff --git a/cloudflare/DEPLOY-no-worker.md b/cloudflare/DEPLOY-no-worker.md new file mode 100644 index 0000000..c3fcf40 --- /dev/null +++ b/cloudflare/DEPLOY-no-worker.md @@ -0,0 +1,55 @@ +# Alternative fix (NO Worker): Cloudflare Origin Rules + URL Rewrite + +Use this if you prefer not to run a Worker. It reproduces the CloudFront path-split +with native Cloudflare rules. Slightly more moving parts than the Worker, but no code. + +> Goal is identical: `api.gen.pro/v1`,`/up`,… → Rails; everything else → GitHub Pages +> `poweredbygen.github.io/api-docs/*` (with the `/api-docs` base-path prepend). + +## Prereq DNS +Add an **unproxied** record so Rails is reachable as a distinct origin: +- `origin-api.gen.pro` → `5.161.246.2`, **DNS only (grey cloud)**. + +## Step 1 — Send API paths to the Rails origin (Origin Rule) +Rules → **Origin Rules** → Create rule: +- **When incoming requests match:** + `(http.host eq "api.gen.pro" and (starts_with(http.request.uri.path, "/v1") or http.request.uri.path eq "/up" or starts_with(http.request.uri.path, "/users") or starts_with(http.request.uri.path, "/rails") or starts_with(http.request.uri.path, "/cable") or starts_with(http.request.uri.path, "/auth")))` +- **Then — Override:** DNS record / Host header → `origin-api.gen.pro` + (keep the visitor Host `api.gen.pro` so Rails routing/cookies/CORS are unchanged). + +## Step 2 — Send everything else to GitHub Pages (Origin Rule) +Origin Rules → Create rule (place AFTER step 1 so API wins): +- **When:** `(http.host eq "api.gen.pro")` *(catch-all; step 1's rule, ordered first, has already peeled off API paths)* +- **Then — Override:** + - **Host header** → `poweredbygen.github.io` + - **(SNI)** → `poweredbygen.github.io` + +## Step 3 — Prepend `/api-docs` to the docs path (URL Rewrite / Transform Rule) +Rules → **Transform Rules → Rewrite URL** → Create rule: +- **When:** same catch-all as step 2 (api.gen.pro, non-API). To be safe, exclude API: + `(http.host eq "api.gen.pro" and not starts_with(http.request.uri.path, "/v1") and http.request.uri.path ne "/up" and not starts_with(http.request.uri.path, "/users") and not starts_with(http.request.uri.path, "/rails") and not starts_with(http.request.uri.path, "/cable") and not starts_with(http.request.uri.path, "/auth"))` +- **Then — Rewrite to… Path → Dynamic:** + `concat("/api-docs", http.request.uri.path)` + This turns `/_astro/x.css` → `/api-docs/_astro/x.css` and `/llms.txt` → `/api-docs/llms.txt`, matching how Pages serves the project. + +## Step 4 — SSL mode +Ensure the zone (or a Configuration Rule for api.gen.pro) uses **Full** SSL so CF +talks HTTPS to both origins. GitHub Pages presents a valid cert; the Rails origin +presents a self-signed/CF-Origin cert (Full, not Full-Strict, for that origin — +or install a CF Origin Certificate on the box and use Full-Strict). + +## Caveat vs the Worker +GitHub Pages issues **trailing-slash 301 redirects** whose `Location` points at +`poweredbygen.github.io/api-docs/...`. The Worker rewrites those back to +`api.gen.pro/...`; native rules do **not**. If you see the Pages hostname leak in a +redirect, add a **Response Header Transform** rewriting `Location`, or just use the +Worker (`api-gen-pro-router.worker.js`) which handles it in one artifact. + +## Verify (same as Worker path) +```bash +for p in / /llms.txt /llms-full.txt /openapi.yaml /reference/agents/; do + curl -s -o /dev/null -w "$p -> %{http_code}\n" "https://api.gen.pro$p"; done # all 200 +curl -s -o /dev/null -w "/up -> %{http_code}\n" https://api.gen.pro/up # 200 +curl -s -o /dev/null -w "/v1 -> %{http_code}\n" https://api.gen.pro/v1/templates/projects # 200 +curl -s -o /dev/null -w "/agents -> %{http_code}\n" https://api.gen.pro/v1/agents # 401 +``` diff --git a/cloudflare/DEPLOY.md b/cloudflare/DEPLOY.md new file mode 100644 index 0000000..2157b0e --- /dev/null +++ b/cloudflare/DEPLOY.md @@ -0,0 +1,53 @@ +# Fix: api.gen.pro docs site 404 (post-AWS-migration routing gap) + +## What broke +`api.gen.pro` serves **two** things off one hostname: +- API paths (`/v1`, `/up`, ...) → Rails (`gen-backend-v2`, now on Hetzner `5.161.246.2`) +- Docs (`/`, `/llms.txt`, `/openapi.yaml`, `/guides/*`, `/reference/*`, `/cards/*`) → Astro Starlight on **GitHub Pages** (`poweredbygen.github.io/api-docs/`) + +A **CloudFront** distribution used to split these paths and rewrite the Pages origin +path to `/api-docs`. The AWS migration removed CloudFront and pointed `api.gen.pro` +**entirely at Rails**. Rails has no docs routes → every docs URL 404s. + +The API itself was never down. Docs content is healthy at `poweredbygen.github.io/api-docs/`. + +## The fix (Cloudflare Worker — no AWS, no redeploy) +`api-gen-pro-router.worker.js` restores the split at the CF edge. + +### Steps (Cloudflare dashboard, zone `gen.pro`) +1. **Create an unproxied origin record for Rails** so the Worker can reach the + backend without looping through `api.gen.pro/*`: + - DNS → add `origin-api` → `5.161.246.2`, **Proxy status: DNS only (grey cloud)**. + - (The box presents the `api.gen.pro` cert / a CF Origin cert; `fetch()` to + `https://origin-api.gen.pro` from the Worker resolves to the same box.) +2. **Workers & Pages → Create Worker** → paste `api-gen-pro-router.worker.js` → Deploy. +3. **Workers Routes** → add route `api.gen.pro/*` → select this Worker. +4. Verify (below). Total time: a few minutes. + +### If you prefer NOT to add a Worker +Equivalent with **Origin Rules + a Redirect/Transform Rule** is possible but messier +because the `/api-docs` base-path rewrite + asset paths need a URL Rewrite rule. +The Worker is the clean single-artifact fix. + +## Verify after deploy +```bash +# Docs must be 200 again: +for p in / /llms.txt /llms-full.txt /openapi.yaml /reference/agents/; do + curl -s -o /dev/null -w "$p -> %{http_code}\n" "https://api.gen.pro$p" +done +# API must still work: +curl -s -o /dev/null -w "/up -> %{http_code}\n" https://api.gen.pro/up # 200 +curl -s -o /dev/null -w "/v1 -> %{http_code}\n" https://api.gen.pro/v1/templates/projects # 200 +curl -s -o /dev/null -w "/v1/agents -> %{http_code}\n" https://api.gen.pro/v1/agents # 401 +``` + +## Permanent hardening (separate follow-up) +The underlying fragility is `astro.config` has `site: 'https://api.gen.pro'` with **no +`base:`** and **no `CNAME`** — it only ever worked because something rewrote the origin +path. Long-term, either: +- give the docs their **own subdomain** (`docs.gen.pro`) with a Pages custom-domain + CNAME (cleanest — removes the dual-purpose host entirely), or +- set Astro `base: '/api-docs'` so published asset paths match the Pages path + (lets you drop the prepend logic). + +Both are larger changes; the Worker unblocks production now. diff --git a/cloudflare/api-gen-pro-router.worker.js b/cloudflare/api-gen-pro-router.worker.js new file mode 100644 index 0000000..0d9d5c7 --- /dev/null +++ b/cloudflare/api-gen-pro-router.worker.js @@ -0,0 +1,111 @@ +/** + * api.gen.pro edge router (Cloudflare Worker) + * + * WHY THIS EXISTS + * --------------- + * `api.gen.pro` is a dual-purpose hostname: + * - API paths (/v1, /up, ...) -> Rails backend (gen-backend-v2, Phusion Passenger) + * - everything else (docs) -> Astro Starlight docs site on GitHub Pages + * (poweredbygen.github.io/api-docs/*) + * + * Before the AWS migration, a CloudFront distribution split these paths and + * rewrote the GitHub Pages origin path to `/api-docs`. When CloudFront was torn + * down, api.gen.pro was pointed entirely at Rails, so every docs URL started + * 404ing (Rails has no route for /llms.txt, /, /guides/*, etc.). + * + * This Worker restores that split at the Cloudflare edge — no AWS, no redeploy + * of the docs, no change to Rails. + * + * BIND THIS WORKER TO: api.gen.pro/* (Workers Routes) + * + * IMPORTANT NOTES + * --------------- + * 1. Rails origin is reached by IP with Host: api.gen.pro because the public + * cert is terminated at the CF edge and the origin uses a self-signed/CF + * Origin cert. Fetching the proxied hostname directly from the Worker would + * loop back through this same route. We therefore target the origin via a + * Cloudflare "Origin Rule" / a dedicated unproxied hostname instead. See + * RAILS_ORIGIN below. + * 2. The published docs HTML uses ROOT-RELATIVE asset paths (/_astro/...), but + * GitHub Pages only serves them under /api-docs/_astro/... . So we prepend + * /api-docs to ALL non-API paths (including /_astro and /assets), which is + * exactly what CloudFront used to do. + */ + +// API path prefixes that must go to Rails. Everything else -> docs. +const API_PREFIXES = [ + '/v1', + '/up', + '/users', + '/rails', + '/cable', + '/admin', + '/sidekiq', + '/auth', +]; + +// Rails origin. Use an UNPROXIED DNS record (grey-cloud) for the backend so the +// Worker can reach it without re-entering the api.gen.pro/* route. +// Recommended: create origin-api.gen.pro -> 5.161.246.2 (DNS only, grey cloud) +// Cloudflare's fetch() will validate its edge cert normally for that hostname. +// If you instead front the box with a CF Origin Certificate on api.gen.pro, +// you can keep RAILS_ORIGIN = 'https://origin-api.gen.pro'. +const RAILS_ORIGIN = 'https://origin-api.gen.pro'; + +// GitHub Pages docs origin + project base path. +const DOCS_ORIGIN = 'https://poweredbygen.github.io'; +const DOCS_BASE = '/api-docs'; + +function isApiPath(pathname) { + return API_PREFIXES.some( + (p) => pathname === p || pathname.startsWith(p + '/') || pathname.startsWith(p + '?'), + ); +} + +export default { + async fetch(request) { + const url = new URL(request.url); + + // --- API traffic -> Rails (unchanged behavior) --- + if (isApiPath(url.pathname)) { + const originUrl = new URL(RAILS_ORIGIN); + originUrl.pathname = url.pathname; + originUrl.search = url.search; + + const apiReq = new Request(originUrl.toString(), request); + // Preserve the public Host so Rails routing/cookies/CORS behave identically. + apiReq.headers.set('Host', 'api.gen.pro'); + apiReq.headers.set('X-Forwarded-Host', 'api.gen.pro'); + apiReq.headers.set('X-Forwarded-Proto', 'https'); + return fetch(apiReq); + } + + // --- Everything else -> GitHub Pages docs, with /api-docs prepended --- + const docsUrl = new URL(DOCS_ORIGIN); + docsUrl.pathname = DOCS_BASE + url.pathname; // /_astro/x -> /api-docs/_astro/x + docsUrl.search = url.search; + + const docsReq = new Request(docsUrl.toString(), { + method: request.method, + headers: request.headers, + redirect: 'manual', + }); + + const resp = await fetch(docsReq); + + // Rewrite GitHub Pages redirects (e.g. trailing-slash 301s point at + // poweredbygen.github.io/api-docs/...) back to api.gen.pro/... so the user + // never sees the Pages hostname or the /api-docs prefix. + if (resp.status >= 300 && resp.status < 400 && resp.headers.has('location')) { + const loc = resp.headers.get('location'); + const rewritten = loc + .replace(DOCS_ORIGIN + DOCS_BASE, 'https://api.gen.pro') + .replace(DOCS_BASE + '/', '/'); + const headers = new Headers(resp.headers); + headers.set('location', rewritten); + return new Response(resp.body, { status: resp.status, headers }); + } + + return resp; + }, +}; From a29c1d40558ede82f835e0c29b1c0f267e69fa20 Mon Sep 17 00:00:00 2001 From: mavneox Date: Fri, 19 Jun 2026 01:05:31 +0000 Subject: [PATCH 2/2] fix(GEN-3647): point docs router Worker at existing origin-app1.gen.pro api.gen.pro is a proxied CNAME to origin-app1.gen.pro (grey-cloud, 5.161.246.2, Rails). Use that existing unproxied backend hostname as the Worker origin instead of a new origin-api record. Matches what is deployed live as the gen-api-docs-router Worker. Co-Authored-By: Claude Opus 4.8 --- cloudflare/api-gen-pro-router.worker.js | 12 +++++------- 1 file changed, 5 insertions(+), 7 deletions(-) diff --git a/cloudflare/api-gen-pro-router.worker.js b/cloudflare/api-gen-pro-router.worker.js index 0d9d5c7..adcc0c5 100644 --- a/cloudflare/api-gen-pro-router.worker.js +++ b/cloudflare/api-gen-pro-router.worker.js @@ -44,13 +44,11 @@ const API_PREFIXES = [ '/auth', ]; -// Rails origin. Use an UNPROXIED DNS record (grey-cloud) for the backend so the -// Worker can reach it without re-entering the api.gen.pro/* route. -// Recommended: create origin-api.gen.pro -> 5.161.246.2 (DNS only, grey cloud) -// Cloudflare's fetch() will validate its edge cert normally for that hostname. -// If you instead front the box with a CF Origin Certificate on api.gen.pro, -// you can keep RAILS_ORIGIN = 'https://origin-api.gen.pro'. -const RAILS_ORIGIN = 'https://origin-api.gen.pro'; +// Rails origin. Uses the existing UNPROXIED (grey-cloud) backend hostname +// `origin-app1.gen.pro` (-> 5.161.246.2) so the Worker reaches Rails directly +// without re-entering the api.gen.pro/* route. (`api.gen.pro` itself is a proxied +// CNAME to this same host.) +const RAILS_ORIGIN = 'https://origin-app1.gen.pro'; // GitHub Pages docs origin + project base path. const DOCS_ORIGIN = 'https://poweredbygen.github.io';