Pull blueprints at build time; drop the live branch deploy path

.claude/CLAUDE.md

@@ -29,11 +29,13 @@ This is scoped to `.njk` on purpose and does not extend to `.md`: new content in
 ### Production build order
 
 ```
-clean:nuxt → build:js:nuxt → prod:postcss-nuxt → prod:eleventy-nuxt → prod:nuxt
+clean:nuxt → build:js:nuxt → blueprints → prod:postcss-nuxt → prod:eleventy-nuxt → prod:nuxt
 ```
 
 The `docs-source` Nuxt module runs automatically during `prod:nuxt` and calls `nuxt/lib/docs-sync.mjs` to resolve `docs/` from `flowfuse/flowfuse` (see **Local docs development** below). 11ty outputs to `nuxt/public/` so Nuxt can serve 11ty-generated assets. `nuxt/public/` is gitignored (fully build-generated).
 
+The `blueprints` step resolves `src/blueprints/` from `FlowFuse/blueprint-library` (private) via `nuxt/lib/blueprints-sync.mjs` — same local → sibling → clone precedence as docs, but the clone step authenticates with a minted GitHub App installation token (`GH_BOT_APP_ID`/`GH_BOT_APP_KEY`) since that repo isn't public. Netlify deploys straight from `main`; there is no `live` branch anymore.
+
 ## Dev commands
 
 ```bash

.github/workflows/build.yml

@@ -1,51 +1,20 @@
 name: Build Site
 on:
+  # FlowFuse/flowfuse's `Publish Documentation` workflow dispatches this by name after a
+  # docs PR merges to main, since a docs change there doesn't push anything to this repo
+  # for Netlify to notice on its own.
   workflow_dispatch:
-  push:
-    branches:
-      - main
   schedule:
+    # Blueprint-library changes don't push here either, so rebuild periodically to pick
+    # those up even if nothing else triggers a deploy in between.
     - cron: "30 9 * * *"
     - cron: "5 16 * * 3,4"
 jobs:
-  build_deploy:
+  trigger_rebuild:
     if: ${{ github.repository == 'FlowFuse/website' }}
     runs-on: ubuntu-latest
     steps:
-      - name: Check out website repository
-        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
-        with:
-          path: 'website'
-      - name: Generate a token
-        id: generate_token
-        uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0
-        with:
-          client-id: ${{ secrets.GH_BOT_APP_ID }}
-          private-key: ${{ secrets.GH_BOT_APP_KEY }}
-          owner: ${{ github.repository_owner }}
-      - name: Check out FlowFuse/blueprint-library repository (to access the blueprints)
-        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
-        with:
-          repository: 'FlowFuse/blueprint-library'
-          ref: main
-          path: 'blueprint-library'
-          token: ${{ steps.generate_token.outputs.token }}
-      - name: Install jq
-        run: sudo apt-get -qy install jq
-      - uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
-        with:
-          node-version: 24
-          cache: 'npm'
-          cache-dependency-path: './website/package-lock.json'
-      - run: npm run blueprints
-        working-directory: 'website'
-      - name: Commit Latest Blueprints
-        run: |
-          cd ./website
-          git config --local user.email "41898282+github-actions[bot]@users.noreply.github.com"
-          git config --local user.name "github-actions[bot]"
-          git add src/blueprints/* -A -f
-          git commit -a -m "Bot: update blueprints"
-      - name: Push to 'live' branch
-        run: git push --force origin HEAD:live
-        working-directory: './website'
+      - name: Trigger a Netlify production rebuild
+        run: curl --fail --silent --show-error -X POST "$NETLIFY_BUILD_HOOK"
+        env:
+          NETLIFY_BUILD_HOOK: ${{ secrets.NETLIFY_BUILD_HOOK }}

README.md

@@ -4,11 +4,16 @@
 
 This repository contains the source of the FlowFuse website.
 
-It is hosted on Netlify with each commit to the `main` branch being automatically deployed to the live site.
-This works by the [Build Site](.github/workflows/build.yml) action updating the `live` branch, committing onto it the
-blueprints pulled from [FlowFuse/blueprint-library](https://github.com/FlowFuse/blueprint-library).
+It is hosted on Netlify, which watches the `main` branch directly and deploys on every commit to it.
+Netlify's own build resolves everything it needs at build time — product documentation from `main` of
+[FlowFuse/flowfuse](https://github.com/FlowFuse/flowfuse), and blueprints from
+[FlowFuse/blueprint-library](https://github.com/FlowFuse/blueprint-library) (see `npm run blueprints` /
+`nuxt/lib/blueprints-sync.mjs`) — so nothing needs to be pre-fetched and committed to a separate branch first.
 
-Netlify is then configured to watch the `live` branch for any changes, once detected, it will automatically pull the contents of this branch and deploy to our production site. Product documentation is not part of that snapshot — Netlify clones it directly from `main` of [FlowFuse/flowfuse](https://github.com/FlowFuse/flowfuse) during its own build.
+A commit to `flowfuse/flowfuse` or `blueprint-library` doesn't push anything to this repo, so it wouldn't otherwise
+trigger a Netlify rebuild on its own. The [Build Site](.github/workflows/build.yml) action covers that gap: it's
+dispatched by `flowfuse/flowfuse`'s `Publish Documentation` workflow after a docs PR merges, and also runs on a
+schedule to pick up blueprint-library changes — either way it just calls a Netlify build hook to rebuild `main`.
 
 ## Repository structure
 
@@ -107,6 +112,27 @@ Nothing needs configuring for that to happen. Every build resolves the docs in t
 
 `npm run dev` and `npm start` also watch the resolved docs and re-sync each file as it changes, so an edit appears without restarting. `npm run dev:nuxt` on its own does not include that watcher; run `npm run dev:docs` beside it if you want one.
 
+### Running Blueprints
+
+Blueprints are maintained in the (private) [FlowFuse/blueprint-library](https://github.com/FlowFuse/blueprint-library) repo. To work with them locally, clone that repository alongside this one, the same way as `flowfuse` above:
+
+```
+/<parent_directory>
+    /website
+    /blueprint-library
+```
+
+Every build resolves blueprints in this order, and logs which one it used:
+
+| Order | Source | Used when |
+|-------|--------|-----------|
+| 1 | `BLUEPRINTS_LOCAL=/path/to/blueprint-library` | The env var is set. A path that does not exist is an error, not a fallback. |
+| 2 | A sibling checkout: `../blueprint-library` | It exists. This is what local development relies on. |
+| 3 | A clone, authenticated with a minted GitHub App installation token (`GH_BOT_APP_ID`/`GH_BOT_APP_KEY`) | Nothing above applied, and those env vars are set. This is what Netlify production deploys use — `blueprint-library` is private, so this can't clone anonymously the way docs does. |
+| 4 | Skipped | Nothing above applied. Matches the previous behaviour for contributors without access to the private repo. |
+
+`npm run blueprints` runs that resolution on its own, writing `src/blueprints`, which is gitignored. `npm start` re-runs it whenever the resolved source changes (`scripts/watch_blueprints.js`), but only when it found a source to watch in the first place — no sibling checkout means no watching.
+
 ## llms.txt
 
 `/llms.txt` (and `/llms-full.txt`) are generated by the [`nuxt-llms`](https://github.com/nuxtlabs/nuxt-llms) module, configured in `nuxt/nuxt.config.ts` under the `llms` key. Sections are built from `@nuxt/content` collections (`docs`, `blog`, `changelog`, `ebooks`, `whitepapers`) plus a small hardcoded list of standalone Nuxt routes (pricing, integrations, etc.) that aren't backed by a collection.

nuxt/lib/blueprints-sync.mjs

@@ -0,0 +1,256 @@
+// Resolves the FlowFuse blueprint library for a build and copies its content into
+// src/blueprints (11ty's blueprint source - see .eleventy.js's setUseGitIgnore(false)
+// note). Mirrors nuxt/lib/docs-sync.mjs's local -> sibling -> clone precedence, but the
+// source repo (FlowFuse/blueprint-library) is private, so the clone step authenticates
+// with a minted GitHub App installation token instead of cloning anonymously.
+
+import { execFileSync } from 'node:child_process'
+import { existsSync, mkdirSync, readFileSync, readdirSync, rmSync, writeFileSync, cpSync } from 'node:fs'
+import { basename, join, relative } from 'node:path'
+import { tmpdir } from 'node:os'
+
+// Imported lazily (inside cloneBlueprints, not here) because it pulls in @octokit/auth-app.
+// CI checks out blueprint-library as a sibling and calls `npm run blueprints` before
+// `npm install` runs - see nuxt/lib/docs-sync.mjs's own note on staying dependency-free -
+// so a static import here would crash a build that never even takes the clone path. Only
+// Netlify's production build (no sibling checkout) reaches the clone path, and by then
+// npm install has already completed.
+
+const REPO_OWNER = 'FlowFuse'
+const REPO_NAME = 'blueprint-library'
+const REPO_URL = `https://github.com/${REPO_OWNER}/${REPO_NAME}.git`
+const DEFAULT_REF = 'main'
+const CLONE_ATTEMPTS = 3
+const CLONE_BACKOFF_MS = 2000
+
+const sleep = (ms) => new Promise(resolve => setTimeout(resolve, ms))
+
+/**
+ * Decide where the blueprints come from. Pure: touches nothing, so the precedence is
+ * testable.
+ *
+ * 1. `BLUEPRINTS_LOCAL` - an explicit checkout path
+ * 2. a sibling checkout of blueprint-library
+ * 3. a clone, authenticated with the GitHub App - only if credentials are configured
+ * 4. skip - matches the previous copy_blueprints.js behaviour for contributors without
+ *    access to the (private) blueprint-library repo
+ */
+export function resolveSource ({ repoRoot, env = process.env, exists = existsSync }) {
+    const local = env.BLUEPRINTS_LOCAL
+    if (local) {
+        if (!exists(local)) {
+            throw new Error(`BLUEPRINTS_LOCAL is set but ${local} does not exist`)
+        }
+        return { kind: 'local', dir: local }
+    }
+
+    const sibling = join(repoRoot, '..', 'blueprint-library')
+    if (exists(sibling)) {
+        return { kind: 'sibling', dir: sibling }
+    }
+
+    if (env.GH_BOT_APP_ID && env.GH_BOT_APP_KEY) {
+        return { kind: 'clone', ref: env.BLUEPRINTS_REF || DEFAULT_REF }
+    }
+
+    return { kind: 'skip' }
+}
+
+/**
+ * Sparse-clone the blueprint library into a temp dir and return its path.
+ *
+ * A transient network failure here would otherwise fail the entire production deploy, so
+ * each attempt gets a clean temp dir and the network steps are retried with backoff. The
+ * caller owns cleanup of the returned dir. Never let the minted token reach a thrown
+ * error's message - execFileSync embeds the full command (URL included) in its own
+ * error, so failures are reported from stderr text with the token stripped out.
+ */
+async function cloneBlueprints (ref, env, logger) {
+    const { mintInstallationToken } = await import('./github-app-token.mjs')
+
+    let lastMessage = 'unknown error'
+    for (let attempt = 1; attempt <= CLONE_ATTEMPTS; attempt++) {
+        const tmpDir = join(tmpdir(), `blueprint-library-${process.pid}-${attempt}`)
+        if (existsSync(tmpDir)) rmSync(tmpDir, { recursive: true, force: true })
+
+        // Minted fresh each attempt so a transient failure here gets the same retry +
+        // redaction as the clone/checkout below, rather than failing the build outright.
+        let token
+        try {
+            token = await mintInstallationToken({
+                appId: env.GH_BOT_APP_ID,
+                privateKey: env.GH_BOT_APP_KEY,
+                owner: REPO_OWNER,
+                repo: REPO_NAME,
+            })
+        } catch (err) {
+            lastMessage = err?.message || String(err)
+            if (attempt === CLONE_ATTEMPTS) break
+
+            const wait = CLONE_BACKOFF_MS * attempt
+            logger.warn(`Blueprint token mint attempt ${attempt}/${CLONE_ATTEMPTS} failed, retrying in ${wait}ms`)
+            await sleep(wait)
+            continue
+        }
+
+        const authedUrl = REPO_URL.replace('https://', `https://x-access-token:${token}@`)
+        const redact = (text) => text.split(token).join('***')
+
+        try {
+            // Blobless but not shallow: dating a blueprint page needs that page's history,
+            // and a --depth=1 clone stamps every page with the same commit date.
+            execFileSync('git', ['clone', '--filter=blob:none', '--no-checkout', authedUrl, tmpDir], { stdio: 'pipe' })
+            execFileSync('git', ['checkout', ref], { cwd: tmpDir, stdio: 'pipe' })
+            return tmpDir
+        } catch (err) {
+            lastMessage = redact(err?.stderr?.toString() || err?.message || String(err))
+            if (existsSync(tmpDir)) rmSync(tmpDir, { recursive: true, force: true })
+
+            if (attempt === CLONE_ATTEMPTS) break
+
+            const wait = CLONE_BACKOFF_MS * attempt
+            logger.warn(`Blueprint clone attempt ${attempt}/${CLONE_ATTEMPTS} failed, retrying in ${wait}ms`)
+            await sleep(wait)
+        }
+    }
+
+    throw new Error(`Failed to clone ${REPO_OWNER}/${REPO_NAME} after ${CLONE_ATTEMPTS} attempts: ${lastMessage}`)
+}
+
+function gitOutput (cwd, args) {
+    try {
+        return execFileSync('git', args, { cwd, encoding: 'utf8' }).trim()
+    } catch {
+        return ''
+    }
+}
+
+/**
+ * Copy one blueprint markdown file, stamping it with its last-commit date and rewriting
+ * its `image:` frontmatter path to match where it lands under src/blueprints. Ported
+ * as-is from the previous scripts/copy_blueprints.js.
+ */
+function writeBlueprintMarkdown ({ sourceRoot, srcPath, destPath, inputRelDir }) {
+    const relPath = relative(sourceRoot, srcPath)
+    const updated = gitOutput(sourceRoot, ['log', '-1', '--pretty=format:%ci', '--', relPath])
+
+    const content = readFileSync(srcPath, 'utf8')
+    let body = `---\nupdated: ${updated}\n---\n${content}`
+    if (/^---/.test(content)) {
+        // The original file starts with yaml front-matter, so remove the double-delimiter
+        // we've just introduced.
+        body = body.replace(/---\r?\n---\r?\n/s, '')
+    }
+
+    // tileImage's shortcode (.eleventy.js) resolves item.data.image relative to 11ty's
+    // input folder (src/), not as a filesystem or site-root path - so this stays relative,
+    // e.g. "blueprints/foo/bar/img.png", never "src/blueprints/..." or "/blueprints/...".
+    const imageRegex = /^image:\s*(\S.+)$/m
+    if (imageRegex.test(body)) {
+        body = body.replace(imageRegex, (match, p1) => {
+            const relImage = p1.replace(/^"\.\//, '').replace(/"$/, '')
+            return `image: ${join(inputRelDir, relImage)}`
+        })
+    }
+
+    writeFileSync(destPath, body)
+}
+
+function copyTree (srcDir, destDir, sourceRoot, inputRelDir) {
+    mkdirSync(destDir, { recursive: true })
+    for (const entry of readdirSync(srcDir, { withFileTypes: true })) {
+        if (entry.name.startsWith('.')) continue
+
+        const srcPath = join(srcDir, entry.name)
+        if (entry.isDirectory()) {
+            const lowerCaseName = entry.name.toLowerCase()
+            copyTree(srcPath, join(destDir, lowerCaseName), sourceRoot, join(inputRelDir, lowerCaseName))
+            continue
+        }
+
+        const destPath = join(destDir, entry.name.replace(/README/, 'index'))
+        if (entry.name.endsWith('.md')) {
+            writeBlueprintMarkdown({ sourceRoot, srcPath, destPath, inputRelDir })
+        } else {
+            cpSync(srcPath, destPath)
+        }
+    }
+}
+
+// Removes only the entries under destDir that no longer exist in the source - never
+// submit.njk (this repo's own "Submit Your Own" page, not something blueprint-library
+// provides) and never an entry copyTree is about to repopulate anyway. Deliberately
+// narrower than docs-sync.mjs's full wipe: copyTree already overwrites every file in
+// place on each sync (11ty sees a cheap "changed" event), so wiping unaffected blueprints
+// too would turn that into a delete+recreate of the entire tree on every sync - noisy for
+// 11ty's watcher and briefly 404s a page mid-rebuild for no reason.
+function clearOrphans (destDir, currentNames) {
+    if (!existsSync(destDir)) return
+    for (const entry of readdirSync(destDir, { withFileTypes: true })) {
+        if (entry.name === 'submit.njk' || currentNames.has(entry.name)) continue
+        rmSync(join(destDir, entry.name), { recursive: true, force: true })
+    }
+}
+
+function directoryNames (dir) {
+    return readdirSync(dir, { withFileTypes: true })
+        .filter(entry => entry.isDirectory() && !entry.name.startsWith('.'))
+}
+
+/**
+ * Populate src/blueprints from `dir` (one category folder per top-level entry, one
+ * blueprint per folder below that) and return the manifest describing what was published.
+ * Pruned one level at a time - at the category level and again inside each still-current
+ * category - since that's the actual unit that gets renamed or removed upstream, without
+ * having to fully tree-diff every nested asset to catch it.
+ */
+function writeBlueprints ({ dir, websiteRoot, kind, ref }) {
+    const destRoot = join(websiteRoot, 'src', 'blueprints')
+    const categories = directoryNames(dir)
+    clearOrphans(destRoot, new Set(categories.map(entry => entry.name)))
+
+    for (const category of categories) {
+        const categorySrcDir = join(dir, category.name)
+        const categoryDestDir = join(destRoot, basename(categorySrcDir))
+        clearOrphans(categoryDestDir, new Set(directoryNames(categorySrcDir).map(entry => entry.name)))
+        copyTree(categorySrcDir, categoryDestDir, dir, join('blueprints', basename(categorySrcDir)))
+    }
+
+    return {
+        source: kind,
+        ref: ref || gitOutput(dir, ['rev-parse', '--abbrev-ref', 'HEAD']),
+        sha: gitOutput(dir, ['rev-parse', 'HEAD']),
+        syncedAt: new Date().toISOString(),
+    }
+}
+
+/**
+ * Populate src/blueprints and return the manifest describing what was published, or null
+ * if there was no source to sync from (matches the previous copy_blueprints.js's
+ * "skipping" behaviour for contributors without access to blueprint-library).
+ */
+export async function syncBlueprints ({ repoRoot, env = process.env, logger = console } = {}) {
+    const source = resolveSource({ repoRoot, env })
+
+    if (source.kind === 'skip') {
+        logger.info('Blueprint library not found and no GH_BOT_APP_ID/GH_BOT_APP_KEY configured - skipping')
+        return null
+    }
+
+    let manifest
+    if (source.kind === 'clone') {
+        logger.info(`Cloning ${REPO_OWNER}/${REPO_NAME} from ${source.ref}...`)
+        const tmpDir = await cloneBlueprints(source.ref, env, logger)
+        try {
+            manifest = writeBlueprints({ dir: tmpDir, websiteRoot: repoRoot, kind: source.kind, ref: source.ref })
+        } finally {
+            if (existsSync(tmpDir)) rmSync(tmpDir, { recursive: true, force: true })
+        }
+    } else {
+        logger.info(`Using ${source.kind} blueprints from ${source.dir}`)
+        manifest = writeBlueprints({ dir: source.dir, websiteRoot: repoRoot, kind: source.kind })
+    }
+
+    logger.info(`Blueprints synced from ${manifest.source} (${manifest.ref} ${manifest.sha.slice(0, 8) || 'unknown'})`)
+    return manifest
+}