diff --git a/.claude/rules/ci-config.md b/.claude/rules/ci-config.md index 6c55fb055f15..f6b0608b5640 100644 --- a/.claude/rules/ci-config.md +++ b/.claude/rules/ci-config.md @@ -36,7 +36,7 @@ paths: - `agent-rules.yml` runs `scripts/agent-rules.mjs --check` and the agent hook tests (`node --test '.cursor/hooks/*.test.js' '.claude-plugin/*.test.mjs'`; a bare directory runs nothing on Node 22) with no install (node and git only). Its push `paths` must cover every input and output of that script, `.cursor/hooks/`, and everything the Claude Code plugin's tests read (`.claude-plugin/README.md` lists it). - `regenerate.yml` reruns `yarn copy:websitetypes` and `yarn build:skills`, fails if their committed output changes (CI never pushes fixes), then runs `yarn check:doc-examples`. Its push `paths` must cover every input of `scripts/copywebsitetypes.sh`, `skillReferences.mjs`, `scripts/pack-skills.mjs --check` (`.agents/skill-packages.json`) and `checkExamples.mjs`, including the editor types, `Playground/DesignSystem/index.ts` (global components), `Playground/monaco/typeCheckFlags.json` (the editor's compiler flags) and the root `package.json`. It needs the `website` workspace (e.g. `bignumber.js`), which CircleCI's `setup` drops. - `site-preview.yml` builds the site once, in the required `website` job (one install for the typecheck, the website lint (CircleCI's `lint` covers only `packages/*/src`) and the build, only the packages it imports via `ci:build:website`). `VERCEL_ENV` is `preview` for pull requests (builds drafts) and `production` otherwise, including the merge queue and master, so the queue checks the build master deploys (a link to a draft fails there instead of as a red master commit). It restores Docusaurus Faster's Rspack cache (only master pushes save it, so PRs share one entry) only from the same `yarn.lock` hash, since a cache built from other dependencies breaks the build. It fails on any `[WARNING]`/`[ERROR]` line. Each locale is a full build (most of the build step), so a pull request builds only English (`--locale en`, noted in a run notice and the preview comment) unless its diff versus the base touches `website/i18n`, `website/translate`, `website/framework-docs` or `docusaurus.config.ts`, or deletes or renames a file under `docs/`, `website/src` or `website/static` (a translation may link to or import it; other locale-only breaks surface in the queue); the merge queue and master always build every locale. After the build, `website/scripts/check-lazy-chunks.mjs` fails unless the Playground Store inspector ships only in the lazy `PreviewWithScope` chunk, and `website/scripts/check-build-base.mjs` fails when a page or stylesheet loads a static file from outside its build's baseUrl (`/next/`, `/es/`), which that build's installed offline app can't serve (a PR's English build has base `/`, so English docs are checked by locale builds and the queue's `/next/` build). Broken links are `warn` in `docusaurus.config.ts`, so this check catches them without failing the build. When a deploy follows, the job uploads `website/build` (with `include-hidden-files`; `static/.circleci` ships with the site). -- Archived docs (`.dataclient.io`, `website/scripts/snapshot/README.md`): `docs-snapshot.yml`'s build job runs an old release's install and build, which is third-party code, so it gets no write token or dependency cache (`persist-credentials: false`). Only its `publish` job writes, and it installs nothing. Assets are named by a sha256 prefix, so a rebuild adds an asset instead of replacing the listed one, and `fetch.sh` refuses one that doesn't match `versionsArchived.json`. Each archive serves at its own host through one `vercel.json` rewrite to `archives//`; Vercel runs rewrites after its file check, where the root's pages would answer, so `docs deploy` moves that route ahead of the check in `.vercel/output/config.json` and fails if it can't find it; the root's own `/archives//` paths redirect to that host. Production deploys add each listed host to the Vercel project (`vercel domains add`). Every version's site reads the root's `site-versions.json` (written by master's build) for its version menu, and the menu checks (`HEAD`) whether the page exists in each version, linking that version's section home (like `/docs`) when it 404s, so `vercel.json` lets other origins read every path but `/api`. +- Archived docs (`.dataclient.io`, `website/scripts/snapshot/README.md`): `docs-snapshot.yml`'s build job runs an old release's install and build, which is third-party code, so it gets no write token or dependency cache (`persist-credentials: false`). Only its `publish` job writes, and it installs nothing. Assets are named by a sha256 prefix, so a rebuild adds an asset instead of replacing the listed one, and `fetch.sh` refuses one that doesn't match `versionsArchived.json`. Each archive serves at its own host through one `vercel.json` rewrite to `archives//`; Vercel runs rewrites after its file check, where the root's pages would answer, so `docs deploy` moves that route ahead of the check in `.vercel/output/config.json` and fails if it can't find it; Vercel also answers every miss with the root's `404.html`, so `docs deploy` adds an error-phase route per `404.html` in the build (each archive host's and its locales' first, then `/next/` and locales, longest prefix first) serving that build's own 404 page, and fails if the config has no error phase; the root's own `/archives//` paths redirect to that host. Production deploys add each listed host to the Vercel project (`vercel domains add`). Every version's site reads the root's `site-versions.json` (written by master's build) for its version menu, and the menu checks (`HEAD`) whether the page exists in each version, linking that version's section home (like `/docs`) when it 404s, so `vercel.json` lets other origins read every path but `/api`. - Current and next (`scripts/maintenance/README.md`): while `website/versionCurrent.json` exists (`cut.sh` adds it; the archive's listing PR deletes it), master's push, dispatch and queue builds go to `/next/` (they build the blog to check its links, but `/next/blog` loads the root's), and the `current` job builds the `.x` branch at the root, blog included (no write token, as above). Blog URLs never change: `/blog` is always the root's. Only the branch's `maintenance.yml` dispatch rebuilds the root after a branch push. `docs deploy` needs both, fails unless both found the file, and fails unless `vercel.json` has exactly one `/next{/:path...}` redirect to drop. With no next build, `website` puts `website/scripts/snapshot/retiredSw.js` at `next/sw.js` and `next//sw.js`, which that redirect leaves out, so a worker installed from `/next` unregisters instead of serving its frozen precache. The site reads both only through `website/siteVersions.ts`, which checks their invariants. `scripts/maintenance/version-line.sh` defines release lines and which one a release replaces (`maintenance-scripts.yml` runs `version-line.test.sh` through `paths-gate.yml`, so it can be required); `siteVersions.ts` mirrors its `is_line`. `website/scripts/snapshot/docusaurus.snapshot.config.ts` and `withoutBlog.ts` also run against old releases' configs, so read only fields every release has. A maintenance branch's workflows, `publish.sh`, `version-line.sh` and `create-github-releases.mjs` are copies made by `cut.sh`: changing them here doesn't reach an existing branch. - `website-port.yml` is the reverse of the branch's `maintenance.yml` (`scripts/maintenance/README.md#website-fixes`): while `versionCurrent.json` exists, each master push opens an auto-merging PR to its `ref` branch picking the pushed commits that change only `website/src` or `website/static`, but not the editor types or the demos' code. Its push `paths` must match the step's `site` pattern. It uses the port app's token (`PORT_APP_CLIENT_ID` variable, `PORT_APP_PRIVATE_KEY` secret) when set, so the PR runs CI and its merge starts the branch's `maintenance.yml`; with the workflow token instead, neither starts workflows, so it dispatches `site-preview.yml` itself when the PR merged at once (hence `actions: write`). - `site-preview.yml` runs `blog-publish.mjs --check`: posts published in the PR must be dated within 3 days by filename. The merge queue rechecks, since a PR can wait there past the limit. diff --git a/.cursor/rules/ci-config.mdc b/.cursor/rules/ci-config.mdc index 58c232f0a407..a0686fa07916 100644 --- a/.cursor/rules/ci-config.mdc +++ b/.cursor/rules/ci-config.mdc @@ -34,7 +34,7 @@ alwaysApply: false - `agent-rules.yml` runs `scripts/agent-rules.mjs --check` and the agent hook tests (`node --test '.cursor/hooks/*.test.js' '.claude-plugin/*.test.mjs'`; a bare directory runs nothing on Node 22) with no install (node and git only). Its push `paths` must cover every input and output of that script, `.cursor/hooks/`, and everything the Claude Code plugin's tests read (`.claude-plugin/README.md` lists it). - `regenerate.yml` reruns `yarn copy:websitetypes` and `yarn build:skills`, fails if their committed output changes (CI never pushes fixes), then runs `yarn check:doc-examples`. Its push `paths` must cover every input of `scripts/copywebsitetypes.sh`, `skillReferences.mjs`, `scripts/pack-skills.mjs --check` (`.agents/skill-packages.json`) and `checkExamples.mjs`, including the editor types, `Playground/DesignSystem/index.ts` (global components), `Playground/monaco/typeCheckFlags.json` (the editor's compiler flags) and the root `package.json`. It needs the `website` workspace (e.g. `bignumber.js`), which CircleCI's `setup` drops. - `site-preview.yml` builds the site once, in the required `website` job (one install for the typecheck, the website lint (CircleCI's `lint` covers only `packages/*/src`) and the build, only the packages it imports via `ci:build:website`). `VERCEL_ENV` is `preview` for pull requests (builds drafts) and `production` otherwise, including the merge queue and master, so the queue checks the build master deploys (a link to a draft fails there instead of as a red master commit). It restores Docusaurus Faster's Rspack cache (only master pushes save it, so PRs share one entry) only from the same `yarn.lock` hash, since a cache built from other dependencies breaks the build. It fails on any `[WARNING]`/`[ERROR]` line. Each locale is a full build (most of the build step), so a pull request builds only English (`--locale en`, noted in a run notice and the preview comment) unless its diff versus the base touches `website/i18n`, `website/translate`, `website/framework-docs` or `docusaurus.config.ts`, or deletes or renames a file under `docs/`, `website/src` or `website/static` (a translation may link to or import it; other locale-only breaks surface in the queue); the merge queue and master always build every locale. After the build, `website/scripts/check-lazy-chunks.mjs` fails unless the Playground Store inspector ships only in the lazy `PreviewWithScope` chunk, and `website/scripts/check-build-base.mjs` fails when a page or stylesheet loads a static file from outside its build's baseUrl (`/next/`, `/es/`), which that build's installed offline app can't serve (a PR's English build has base `/`, so English docs are checked by locale builds and the queue's `/next/` build). Broken links are `warn` in `docusaurus.config.ts`, so this check catches them without failing the build. When a deploy follows, the job uploads `website/build` (with `include-hidden-files`; `static/.circleci` ships with the site). -- Archived docs (`.dataclient.io`, `website/scripts/snapshot/README.md`): `docs-snapshot.yml`'s build job runs an old release's install and build, which is third-party code, so it gets no write token or dependency cache (`persist-credentials: false`). Only its `publish` job writes, and it installs nothing. Assets are named by a sha256 prefix, so a rebuild adds an asset instead of replacing the listed one, and `fetch.sh` refuses one that doesn't match `versionsArchived.json`. Each archive serves at its own host through one `vercel.json` rewrite to `archives//`; Vercel runs rewrites after its file check, where the root's pages would answer, so `docs deploy` moves that route ahead of the check in `.vercel/output/config.json` and fails if it can't find it; the root's own `/archives//` paths redirect to that host. Production deploys add each listed host to the Vercel project (`vercel domains add`). Every version's site reads the root's `site-versions.json` (written by master's build) for its version menu, and the menu checks (`HEAD`) whether the page exists in each version, linking that version's section home (like `/docs`) when it 404s, so `vercel.json` lets other origins read every path but `/api`. +- Archived docs (`.dataclient.io`, `website/scripts/snapshot/README.md`): `docs-snapshot.yml`'s build job runs an old release's install and build, which is third-party code, so it gets no write token or dependency cache (`persist-credentials: false`). Only its `publish` job writes, and it installs nothing. Assets are named by a sha256 prefix, so a rebuild adds an asset instead of replacing the listed one, and `fetch.sh` refuses one that doesn't match `versionsArchived.json`. Each archive serves at its own host through one `vercel.json` rewrite to `archives//`; Vercel runs rewrites after its file check, where the root's pages would answer, so `docs deploy` moves that route ahead of the check in `.vercel/output/config.json` and fails if it can't find it; Vercel also answers every miss with the root's `404.html`, so `docs deploy` adds an error-phase route per `404.html` in the build (each archive host's and its locales' first, then `/next/` and locales, longest prefix first) serving that build's own 404 page, and fails if the config has no error phase; the root's own `/archives//` paths redirect to that host. Production deploys add each listed host to the Vercel project (`vercel domains add`). Every version's site reads the root's `site-versions.json` (written by master's build) for its version menu, and the menu checks (`HEAD`) whether the page exists in each version, linking that version's section home (like `/docs`) when it 404s, so `vercel.json` lets other origins read every path but `/api`. - Current and next (`scripts/maintenance/README.md`): while `website/versionCurrent.json` exists (`cut.sh` adds it; the archive's listing PR deletes it), master's push, dispatch and queue builds go to `/next/` (they build the blog to check its links, but `/next/blog` loads the root's), and the `current` job builds the `.x` branch at the root, blog included (no write token, as above). Blog URLs never change: `/blog` is always the root's. Only the branch's `maintenance.yml` dispatch rebuilds the root after a branch push. `docs deploy` needs both, fails unless both found the file, and fails unless `vercel.json` has exactly one `/next{/:path...}` redirect to drop. With no next build, `website` puts `website/scripts/snapshot/retiredSw.js` at `next/sw.js` and `next//sw.js`, which that redirect leaves out, so a worker installed from `/next` unregisters instead of serving its frozen precache. The site reads both only through `website/siteVersions.ts`, which checks their invariants. `scripts/maintenance/version-line.sh` defines release lines and which one a release replaces (`maintenance-scripts.yml` runs `version-line.test.sh` through `paths-gate.yml`, so it can be required); `siteVersions.ts` mirrors its `is_line`. `website/scripts/snapshot/docusaurus.snapshot.config.ts` and `withoutBlog.ts` also run against old releases' configs, so read only fields every release has. A maintenance branch's workflows, `publish.sh`, `version-line.sh` and `create-github-releases.mjs` are copies made by `cut.sh`: changing them here doesn't reach an existing branch. - `website-port.yml` is the reverse of the branch's `maintenance.yml` (`scripts/maintenance/README.md#website-fixes`): while `versionCurrent.json` exists, each master push opens an auto-merging PR to its `ref` branch picking the pushed commits that change only `website/src` or `website/static`, but not the editor types or the demos' code. Its push `paths` must match the step's `site` pattern. It uses the port app's token (`PORT_APP_CLIENT_ID` variable, `PORT_APP_PRIVATE_KEY` secret) when set, so the PR runs CI and its merge starts the branch's `maintenance.yml`; with the workflow token instead, neither starts workflows, so it dispatches `site-preview.yml` itself when the PR merged at once (hence `actions: write`). - `site-preview.yml` runs `blog-publish.mjs --check`: posts published in the PR must be dated within 3 days by filename. The merge queue rechecks, since a PR can wait there past the limit. diff --git a/.github/workflows/site-preview.yml b/.github/workflows/site-preview.yml index f380bc0fbf31..3ce032cdc0b3 100644 --- a/.github/workflows/site-preview.yml +++ b/.github/workflows/site-preview.yml @@ -405,6 +405,25 @@ jobs: else $rest[:$fs] + $archive + $rest[$fs:] end)' "$config" > config.json.tmp || { echo "::error::Couldn't move $config's archive host rewrite"; exit 1; } mv config.json.tmp "$config" + # A miss gets the root's 404.html, which is another site under /next/ + # or a locale and has no styles on an archive's host; answer it with + # the 404.html of the build it's in. Archives first, as their hosts' + # paths look like the root's; then the longest prefix wins. An archive's + # route replaces src and adds its host + jq ' + ($ARGS.positional | sort_by((startswith("archives/") | not), -length) | map( + { src: "^/\(.)/", dest: "/\(.)/404", status: 404 } + (( + split("/") | select(.[0] == "archives") + | (.[1] | gsub("\\."; "\\.")) as $v + | { src: "^(?:/archives/\($v))?/\(.[2:] | map(. + "/") | join("") | if . == "" then "(?!api(?:/|$))" end)", + has: [{ type: "host", value: "\($v)\\.dataclient\\.io" }] } + ) // {}))) as $notFound + | (.routes | map(.handle == "error") | index(true)) as $errorPhase + | if $errorPhase == null then error("no error phase to add them to") + else .routes |= .[:$errorPhase + 1] + $notFound + .[$errorPhase + 1:] end' "$config" --args \ + $(find website/build -mindepth 2 -name 404.html -printf '%h\n' | cut -d/ -f3-) > config.json.tmp || + { echo "::error::Couldn't add each build's 404 page to $config"; exit 1; } + mv config.json.tmp "$config" meta=() # Ties the preview to its branch instead of the PR merge ref [ -n "$HEAD_REF" ] && meta=(--meta githubDeployment=1 --meta "githubCommitRef=$HEAD_REF") diff --git a/website/scripts/snapshot/README.md b/website/scripts/snapshot/README.md index 345bf7c47235..8add2934b9d3 100644 --- a/website/scripts/snapshot/README.md +++ b/website/scripts/snapshot/README.md @@ -30,7 +30,10 @@ type-check, as they did when that release's CI passed. Archives add nothing to t `site-preview.yml` runs it before uploading a build for deploy. `vercel.json` rewrites each archive's host (`.dataclient.io`) to its directory, except `/api` (the site's functions), and redirects that directory's own paths to the host. Vercel applies rewrites after its file check, where the root's pages would answer - first, so `docs deploy` moves that route ahead of the check in `vercel build`'s output. Production + first, so `docs deploy` moves that route ahead of the check in `vercel build`'s output. Vercel + also answers every miss with the root's `404.html`, which on an archive's host loads no styles + (its `/assets/` map to the archive's), so `docs deploy` routes misses on each archive host, + under `/next/` and under each locale to that build's own `404.html`. Production deploys then add each listed host to the Vercel project as a domain (dataclient.io's DNS, on Cloudflare, already sends every subdomain to Vercel). The hostname scheme lives in `vercel.json`, `docusaurus.snapshot.config.ts` and that deploy step.