From 596fe8b949e4eaada0b9dedd3ccdbac0c44fd373 Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 21 Oct 2026 03:28:25 +0000 Subject: [PATCH 1/3] ci(website): Serve each build's own 404 page Vercel answers every miss with the root's 404.html, so a missing page under /next/ or a locale showed the root site's 404, and on an archive host it loaded no styles. docs deploy now adds an error-phase route per 404.html in the build, so each archive host, /next/ and locale serves its own. Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_01YJ3T6rbteN6Ra4Kg1m8P1s --- .claude/rules/ci-config.md | 2 +- .cursor/rules/ci-config.mdc | 2 +- .github/workflows/site-preview.yml | 20 ++++++++++++++++++++ website/scripts/snapshot/README.md | 5 ++++- 4 files changed, 26 insertions(+), 3 deletions(-) diff --git a/.claude/rules/ci-config.md b/.claude/rules/ci-config.md index 6c55fb055f15..f6b0608b5640 100644 --- a/.claude/rules/ci-config.md +++ b/.claude/rules/ci-config.md @@ -36,7 +36,7 @@ paths: - `agent-rules.yml` runs `scripts/agent-rules.mjs --check` and the agent hook tests (`node --test '.cursor/hooks/*.test.js' '.claude-plugin/*.test.mjs'`; a bare directory runs nothing on Node 22) with no install (node and git only). Its push `paths` must cover every input and output of that script, `.cursor/hooks/`, and everything the Claude Code plugin's tests read (`.claude-plugin/README.md` lists it). - `regenerate.yml` reruns `yarn copy:websitetypes` and `yarn build:skills`, fails if their committed output changes (CI never pushes fixes), then runs `yarn check:doc-examples`. Its push `paths` must cover every input of `scripts/copywebsitetypes.sh`, `skillReferences.mjs`, `scripts/pack-skills.mjs --check` (`.agents/skill-packages.json`) and `checkExamples.mjs`, including the editor types, `Playground/DesignSystem/index.ts` (global components), `Playground/monaco/typeCheckFlags.json` (the editor's compiler flags) and the root `package.json`. It needs the `website` workspace (e.g. `bignumber.js`), which CircleCI's `setup` drops. - `site-preview.yml` builds the site once, in the required `website` job (one install for the typecheck, the website lint (CircleCI's `lint` covers only `packages/*/src`) and the build, only the packages it imports via `ci:build:website`). `VERCEL_ENV` is `preview` for pull requests (builds drafts) and `production` otherwise, including the merge queue and master, so the queue checks the build master deploys (a link to a draft fails there instead of as a red master commit). It restores Docusaurus Faster's Rspack cache (only master pushes save it, so PRs share one entry) only from the same `yarn.lock` hash, since a cache built from other dependencies breaks the build. It fails on any `[WARNING]`/`[ERROR]` line. Each locale is a full build (most of the build step), so a pull request builds only English (`--locale en`, noted in a run notice and the preview comment) unless its diff versus the base touches `website/i18n`, `website/translate`, `website/framework-docs` or `docusaurus.config.ts`, or deletes or renames a file under `docs/`, `website/src` or `website/static` (a translation may link to or import it; other locale-only breaks surface in the queue); the merge queue and master always build every locale. After the build, `website/scripts/check-lazy-chunks.mjs` fails unless the Playground Store inspector ships only in the lazy `PreviewWithScope` chunk, and `website/scripts/check-build-base.mjs` fails when a page or stylesheet loads a static file from outside its build's baseUrl (`/next/`, `/es/`), which that build's installed offline app can't serve (a PR's English build has base `/`, so English docs are checked by locale builds and the queue's `/next/` build). Broken links are `warn` in `docusaurus.config.ts`, so this check catches them without failing the build. When a deploy follows, the job uploads `website/build` (with `include-hidden-files`; `static/.circleci` ships with the site). -- Archived docs (`.dataclient.io`, `website/scripts/snapshot/README.md`): `docs-snapshot.yml`'s build job runs an old release's install and build, which is third-party code, so it gets no write token or dependency cache (`persist-credentials: false`). Only its `publish` job writes, and it installs nothing. Assets are named by a sha256 prefix, so a rebuild adds an asset instead of replacing the listed one, and `fetch.sh` refuses one that doesn't match `versionsArchived.json`. Each archive serves at its own host through one `vercel.json` rewrite to `archives//`; Vercel runs rewrites after its file check, where the root's pages would answer, so `docs deploy` moves that route ahead of the check in `.vercel/output/config.json` and fails if it can't find it; the root's own `/archives//` paths redirect to that host. Production deploys add each listed host to the Vercel project (`vercel domains add`). Every version's site reads the root's `site-versions.json` (written by master's build) for its version menu, and the menu checks (`HEAD`) whether the page exists in each version, linking that version's section home (like `/docs`) when it 404s, so `vercel.json` lets other origins read every path but `/api`. +- Archived docs (`.dataclient.io`, `website/scripts/snapshot/README.md`): `docs-snapshot.yml`'s build job runs an old release's install and build, which is third-party code, so it gets no write token or dependency cache (`persist-credentials: false`). Only its `publish` job writes, and it installs nothing. Assets are named by a sha256 prefix, so a rebuild adds an asset instead of replacing the listed one, and `fetch.sh` refuses one that doesn't match `versionsArchived.json`. Each archive serves at its own host through one `vercel.json` rewrite to `archives//`; Vercel runs rewrites after its file check, where the root's pages would answer, so `docs deploy` moves that route ahead of the check in `.vercel/output/config.json` and fails if it can't find it; Vercel also answers every miss with the root's `404.html`, so `docs deploy` adds an error-phase route per `404.html` in the build (each archive host's and its locales' first, then `/next/` and locales, longest prefix first) serving that build's own 404 page, and fails if the config has no error phase; the root's own `/archives//` paths redirect to that host. Production deploys add each listed host to the Vercel project (`vercel domains add`). Every version's site reads the root's `site-versions.json` (written by master's build) for its version menu, and the menu checks (`HEAD`) whether the page exists in each version, linking that version's section home (like `/docs`) when it 404s, so `vercel.json` lets other origins read every path but `/api`. - Current and next (`scripts/maintenance/README.md`): while `website/versionCurrent.json` exists (`cut.sh` adds it; the archive's listing PR deletes it), master's push, dispatch and queue builds go to `/next/` (they build the blog to check its links, but `/next/blog` loads the root's), and the `current` job builds the `.x` branch at the root, blog included (no write token, as above). Blog URLs never change: `/blog` is always the root's. Only the branch's `maintenance.yml` dispatch rebuilds the root after a branch push. `docs deploy` needs both, fails unless both found the file, and fails unless `vercel.json` has exactly one `/next{/:path...}` redirect to drop. With no next build, `website` puts `website/scripts/snapshot/retiredSw.js` at `next/sw.js` and `next//sw.js`, which that redirect leaves out, so a worker installed from `/next` unregisters instead of serving its frozen precache. The site reads both only through `website/siteVersions.ts`, which checks their invariants. `scripts/maintenance/version-line.sh` defines release lines and which one a release replaces (`maintenance-scripts.yml` runs `version-line.test.sh` through `paths-gate.yml`, so it can be required); `siteVersions.ts` mirrors its `is_line`. `website/scripts/snapshot/docusaurus.snapshot.config.ts` and `withoutBlog.ts` also run against old releases' configs, so read only fields every release has. A maintenance branch's workflows, `publish.sh`, `version-line.sh` and `create-github-releases.mjs` are copies made by `cut.sh`: changing them here doesn't reach an existing branch. - `website-port.yml` is the reverse of the branch's `maintenance.yml` (`scripts/maintenance/README.md#website-fixes`): while `versionCurrent.json` exists, each master push opens an auto-merging PR to its `ref` branch picking the pushed commits that change only `website/src` or `website/static`, but not the editor types or the demos' code. Its push `paths` must match the step's `site` pattern. It uses the port app's token (`PORT_APP_CLIENT_ID` variable, `PORT_APP_PRIVATE_KEY` secret) when set, so the PR runs CI and its merge starts the branch's `maintenance.yml`; with the workflow token instead, neither starts workflows, so it dispatches `site-preview.yml` itself when the PR merged at once (hence `actions: write`). - `site-preview.yml` runs `blog-publish.mjs --check`: posts published in the PR must be dated within 3 days by filename. The merge queue rechecks, since a PR can wait there past the limit. diff --git a/.cursor/rules/ci-config.mdc b/.cursor/rules/ci-config.mdc index 58c232f0a407..a0686fa07916 100644 --- a/.cursor/rules/ci-config.mdc +++ b/.cursor/rules/ci-config.mdc @@ -34,7 +34,7 @@ alwaysApply: false - `agent-rules.yml` runs `scripts/agent-rules.mjs --check` and the agent hook tests (`node --test '.cursor/hooks/*.test.js' '.claude-plugin/*.test.mjs'`; a bare directory runs nothing on Node 22) with no install (node and git only). Its push `paths` must cover every input and output of that script, `.cursor/hooks/`, and everything the Claude Code plugin's tests read (`.claude-plugin/README.md` lists it). - `regenerate.yml` reruns `yarn copy:websitetypes` and `yarn build:skills`, fails if their committed output changes (CI never pushes fixes), then runs `yarn check:doc-examples`. Its push `paths` must cover every input of `scripts/copywebsitetypes.sh`, `skillReferences.mjs`, `scripts/pack-skills.mjs --check` (`.agents/skill-packages.json`) and `checkExamples.mjs`, including the editor types, `Playground/DesignSystem/index.ts` (global components), `Playground/monaco/typeCheckFlags.json` (the editor's compiler flags) and the root `package.json`. It needs the `website` workspace (e.g. `bignumber.js`), which CircleCI's `setup` drops. - `site-preview.yml` builds the site once, in the required `website` job (one install for the typecheck, the website lint (CircleCI's `lint` covers only `packages/*/src`) and the build, only the packages it imports via `ci:build:website`). `VERCEL_ENV` is `preview` for pull requests (builds drafts) and `production` otherwise, including the merge queue and master, so the queue checks the build master deploys (a link to a draft fails there instead of as a red master commit). It restores Docusaurus Faster's Rspack cache (only master pushes save it, so PRs share one entry) only from the same `yarn.lock` hash, since a cache built from other dependencies breaks the build. It fails on any `[WARNING]`/`[ERROR]` line. Each locale is a full build (most of the build step), so a pull request builds only English (`--locale en`, noted in a run notice and the preview comment) unless its diff versus the base touches `website/i18n`, `website/translate`, `website/framework-docs` or `docusaurus.config.ts`, or deletes or renames a file under `docs/`, `website/src` or `website/static` (a translation may link to or import it; other locale-only breaks surface in the queue); the merge queue and master always build every locale. After the build, `website/scripts/check-lazy-chunks.mjs` fails unless the Playground Store inspector ships only in the lazy `PreviewWithScope` chunk, and `website/scripts/check-build-base.mjs` fails when a page or stylesheet loads a static file from outside its build's baseUrl (`/next/`, `/es/`), which that build's installed offline app can't serve (a PR's English build has base `/`, so English docs are checked by locale builds and the queue's `/next/` build). Broken links are `warn` in `docusaurus.config.ts`, so this check catches them without failing the build. When a deploy follows, the job uploads `website/build` (with `include-hidden-files`; `static/.circleci` ships with the site). -- Archived docs (`.dataclient.io`, `website/scripts/snapshot/README.md`): `docs-snapshot.yml`'s build job runs an old release's install and build, which is third-party code, so it gets no write token or dependency cache (`persist-credentials: false`). Only its `publish` job writes, and it installs nothing. Assets are named by a sha256 prefix, so a rebuild adds an asset instead of replacing the listed one, and `fetch.sh` refuses one that doesn't match `versionsArchived.json`. Each archive serves at its own host through one `vercel.json` rewrite to `archives//`; Vercel runs rewrites after its file check, where the root's pages would answer, so `docs deploy` moves that route ahead of the check in `.vercel/output/config.json` and fails if it can't find it; the root's own `/archives//` paths redirect to that host. Production deploys add each listed host to the Vercel project (`vercel domains add`). Every version's site reads the root's `site-versions.json` (written by master's build) for its version menu, and the menu checks (`HEAD`) whether the page exists in each version, linking that version's section home (like `/docs`) when it 404s, so `vercel.json` lets other origins read every path but `/api`. +- Archived docs (`.dataclient.io`, `website/scripts/snapshot/README.md`): `docs-snapshot.yml`'s build job runs an old release's install and build, which is third-party code, so it gets no write token or dependency cache (`persist-credentials: false`). Only its `publish` job writes, and it installs nothing. Assets are named by a sha256 prefix, so a rebuild adds an asset instead of replacing the listed one, and `fetch.sh` refuses one that doesn't match `versionsArchived.json`. Each archive serves at its own host through one `vercel.json` rewrite to `archives//`; Vercel runs rewrites after its file check, where the root's pages would answer, so `docs deploy` moves that route ahead of the check in `.vercel/output/config.json` and fails if it can't find it; Vercel also answers every miss with the root's `404.html`, so `docs deploy` adds an error-phase route per `404.html` in the build (each archive host's and its locales' first, then `/next/` and locales, longest prefix first) serving that build's own 404 page, and fails if the config has no error phase; the root's own `/archives//` paths redirect to that host. Production deploys add each listed host to the Vercel project (`vercel domains add`). Every version's site reads the root's `site-versions.json` (written by master's build) for its version menu, and the menu checks (`HEAD`) whether the page exists in each version, linking that version's section home (like `/docs`) when it 404s, so `vercel.json` lets other origins read every path but `/api`. - Current and next (`scripts/maintenance/README.md`): while `website/versionCurrent.json` exists (`cut.sh` adds it; the archive's listing PR deletes it), master's push, dispatch and queue builds go to `/next/` (they build the blog to check its links, but `/next/blog` loads the root's), and the `current` job builds the `.x` branch at the root, blog included (no write token, as above). Blog URLs never change: `/blog` is always the root's. Only the branch's `maintenance.yml` dispatch rebuilds the root after a branch push. `docs deploy` needs both, fails unless both found the file, and fails unless `vercel.json` has exactly one `/next{/:path...}` redirect to drop. With no next build, `website` puts `website/scripts/snapshot/retiredSw.js` at `next/sw.js` and `next//sw.js`, which that redirect leaves out, so a worker installed from `/next` unregisters instead of serving its frozen precache. The site reads both only through `website/siteVersions.ts`, which checks their invariants. `scripts/maintenance/version-line.sh` defines release lines and which one a release replaces (`maintenance-scripts.yml` runs `version-line.test.sh` through `paths-gate.yml`, so it can be required); `siteVersions.ts` mirrors its `is_line`. `website/scripts/snapshot/docusaurus.snapshot.config.ts` and `withoutBlog.ts` also run against old releases' configs, so read only fields every release has. A maintenance branch's workflows, `publish.sh`, `version-line.sh` and `create-github-releases.mjs` are copies made by `cut.sh`: changing them here doesn't reach an existing branch. - `website-port.yml` is the reverse of the branch's `maintenance.yml` (`scripts/maintenance/README.md#website-fixes`): while `versionCurrent.json` exists, each master push opens an auto-merging PR to its `ref` branch picking the pushed commits that change only `website/src` or `website/static`, but not the editor types or the demos' code. Its push `paths` must match the step's `site` pattern. It uses the port app's token (`PORT_APP_CLIENT_ID` variable, `PORT_APP_PRIVATE_KEY` secret) when set, so the PR runs CI and its merge starts the branch's `maintenance.yml`; with the workflow token instead, neither starts workflows, so it dispatches `site-preview.yml` itself when the PR merged at once (hence `actions: write`). - `site-preview.yml` runs `blog-publish.mjs --check`: posts published in the PR must be dated within 3 days by filename. The merge queue rechecks, since a PR can wait there past the limit. diff --git a/.github/workflows/site-preview.yml b/.github/workflows/site-preview.yml index f380bc0fbf31..e35386ec2953 100644 --- a/.github/workflows/site-preview.yml +++ b/.github/workflows/site-preview.yml @@ -405,6 +405,26 @@ jobs: else $rest[:$fs] + $archive + $rest[$fs:] end)' "$config" > config.json.tmp || { echo "::error::Couldn't move $config's archive host rewrite"; exit 1; } mv config.json.tmp "$config" + # A miss gets the root's 404.html, which is another site under /next/ + # or a locale and has no styles on an archive's host; answer it with + # the 404.html of the build it's in. Archives first, as their hosts' + # paths look like the root's; then the longest prefix wins + dirs="$(cd website/build && find . -mindepth 2 -name 404.html -printf '%h\n' | cut -c3- | jq -R . | jq -s .)" + jq --argjson dirs "$dirs" ' + ($dirs | sort_by([(startswith("archives/") | not), -length]) | map( + if test("^archives/") then + capture("^archives/(?[^/]+)(/(?.+))?$") + | (.version | gsub("\\."; "\\.")) as $v + | { src: ("^(?:/archives/\($v))?/" + if .sub then "\(.sub)/" else "(?!api)" end), + has: [{ type: "host", value: "\($v)\\.dataclient\\.io" }], + dest: "/archives/\(.version)\(if .sub then "/\(.sub)" else "" end)/404" } + else { src: "^/\(.)/", dest: "/\(.)/404" } end + | . + { status: 404 })) as $pages + | (.routes | map(.handle == "error") | index(true)) as $error + | if $error == null then error("no error phase to add them to") + else .routes |= .[:$error + 1] + $pages + .[$error + 1:] end' "$config" > config.json.tmp || + { echo "::error::Couldn't add each build's 404 page to $config"; exit 1; } + mv config.json.tmp "$config" meta=() # Ties the preview to its branch instead of the PR merge ref [ -n "$HEAD_REF" ] && meta=(--meta githubDeployment=1 --meta "githubCommitRef=$HEAD_REF") diff --git a/website/scripts/snapshot/README.md b/website/scripts/snapshot/README.md index 345bf7c47235..8add2934b9d3 100644 --- a/website/scripts/snapshot/README.md +++ b/website/scripts/snapshot/README.md @@ -30,7 +30,10 @@ type-check, as they did when that release's CI passed. Archives add nothing to t `site-preview.yml` runs it before uploading a build for deploy. `vercel.json` rewrites each archive's host (`.dataclient.io`) to its directory, except `/api` (the site's functions), and redirects that directory's own paths to the host. Vercel applies rewrites after its file check, where the root's pages would answer - first, so `docs deploy` moves that route ahead of the check in `vercel build`'s output. Production + first, so `docs deploy` moves that route ahead of the check in `vercel build`'s output. Vercel + also answers every miss with the root's `404.html`, which on an archive's host loads no styles + (its `/assets/` map to the archive's), so `docs deploy` routes misses on each archive host, + under `/next/` and under each locale to that build's own `404.html`. Production deploys then add each listed host to the Vercel project as a domain (dataclient.io's DNS, on Cloudflare, already sends every subdomain to Vercel). The hostname scheme lives in `vercel.json`, `docusaurus.snapshot.config.ts` and that deploy step. From a1c5f419faf9b346f2f279555de9043ec5d6556d Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 21 Oct 2026 03:31:50 +0000 Subject: [PATCH 2/3] ci(website): Build the 404 routes in one jq pass from find's paths Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_01YJ3T6rbteN6Ra4Kg1m8P1s --- .github/workflows/site-preview.yml | 22 ++++++++++------------ 1 file changed, 10 insertions(+), 12 deletions(-) diff --git a/.github/workflows/site-preview.yml b/.github/workflows/site-preview.yml index e35386ec2953..4f695f0d2e6f 100644 --- a/.github/workflows/site-preview.yml +++ b/.github/workflows/site-preview.yml @@ -409,20 +409,18 @@ jobs: # or a locale and has no styles on an archive's host; answer it with # the 404.html of the build it's in. Archives first, as their hosts' # paths look like the root's; then the longest prefix wins - dirs="$(cd website/build && find . -mindepth 2 -name 404.html -printf '%h\n' | cut -c3- | jq -R . | jq -s .)" - jq --argjson dirs "$dirs" ' - ($dirs | sort_by([(startswith("archives/") | not), -length]) | map( - if test("^archives/") then - capture("^archives/(?[^/]+)(/(?.+))?$") - | (.version | gsub("\\."; "\\.")) as $v - | { src: ("^(?:/archives/\($v))?/" + if .sub then "\(.sub)/" else "(?!api)" end), - has: [{ type: "host", value: "\($v)\\.dataclient\\.io" }], - dest: "/archives/\(.version)\(if .sub then "/\(.sub)" else "" end)/404" } - else { src: "^/\(.)/", dest: "/\(.)/404" } end - | . + { status: 404 })) as $pages + jq ' + ($ARGS.positional | sort_by((startswith("archives/") | not), -length) | map( + { src: "^/\(.)/", dest: "/\(.)/404", status: 404 } + (( + split("/") | select(.[0] == "archives") + | (.[1] | gsub("\\."; "\\.")) as $v + | { src: "^(?:/archives/\($v))?/\(.[2:] | map(. + "/") | join("") | if . == "" then "(?!api)" end)", + has: [{ type: "host", value: "\($v)\\.dataclient\\.io" }] } + ) // {}))) as $pages | (.routes | map(.handle == "error") | index(true)) as $error | if $error == null then error("no error phase to add them to") - else .routes |= .[:$error + 1] + $pages + .[$error + 1:] end' "$config" > config.json.tmp || + else .routes |= .[:$error + 1] + $pages + .[$error + 1:] end' "$config" --args \ + $(find website/build -mindepth 2 -name 404.html -printf '%h\n' | cut -d/ -f3-) > config.json.tmp || { echo "::error::Couldn't add each build's 404 page to $config"; exit 1; } mv config.json.tmp "$config" meta=() From 2ca231cf35a0c188941ed4a21d252fdb104969ef Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 21 Oct 2026 03:37:02 +0000 Subject: [PATCH 3/3] ci(website): Bound the archive 404's /api exclusion like the host rewrite Co-Authored-By: Claude Opus 5.5 (1M context) Claude-Session: https://claude.ai/code/session_01YJ3T6rbteN6Ra4Kg1m8P1s --- .github/workflows/site-preview.yml | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/.github/workflows/site-preview.yml b/.github/workflows/site-preview.yml index 4f695f0d2e6f..3ce032cdc0b3 100644 --- a/.github/workflows/site-preview.yml +++ b/.github/workflows/site-preview.yml @@ -408,18 +408,19 @@ jobs: # A miss gets the root's 404.html, which is another site under /next/ # or a locale and has no styles on an archive's host; answer it with # the 404.html of the build it's in. Archives first, as their hosts' - # paths look like the root's; then the longest prefix wins + # paths look like the root's; then the longest prefix wins. An archive's + # route replaces src and adds its host jq ' ($ARGS.positional | sort_by((startswith("archives/") | not), -length) | map( { src: "^/\(.)/", dest: "/\(.)/404", status: 404 } + (( split("/") | select(.[0] == "archives") | (.[1] | gsub("\\."; "\\.")) as $v - | { src: "^(?:/archives/\($v))?/\(.[2:] | map(. + "/") | join("") | if . == "" then "(?!api)" end)", + | { src: "^(?:/archives/\($v))?/\(.[2:] | map(. + "/") | join("") | if . == "" then "(?!api(?:/|$))" end)", has: [{ type: "host", value: "\($v)\\.dataclient\\.io" }] } - ) // {}))) as $pages - | (.routes | map(.handle == "error") | index(true)) as $error - | if $error == null then error("no error phase to add them to") - else .routes |= .[:$error + 1] + $pages + .[$error + 1:] end' "$config" --args \ + ) // {}))) as $notFound + | (.routes | map(.handle == "error") | index(true)) as $errorPhase + | if $errorPhase == null then error("no error phase to add them to") + else .routes |= .[:$errorPhase + 1] + $notFound + .[$errorPhase + 1:] end' "$config" --args \ $(find website/build -mindepth 2 -name 404.html -printf '%h\n' | cut -d/ -f3-) > config.json.tmp || { echo "::error::Couldn't add each build's 404 page to $config"; exit 1; } mv config.json.tmp "$config"