Audit remediation: quality gates with teeth, pinned toolchain, content and runtime fixes

.gitattributes

@@ -1,7 +1,11 @@
-# `src/asset_manifest.py` is generated by `scripts/fingerprint_assets.py`.
+# `src/asset_manifest.py` is generated by `scripts/fingerprint_assets.py`,
+# `src/example_sources_data.py` by `scripts/embed_example_sources.py`, and
+# `src/editorial_registry_data.py` by `scripts/embed_editorial_registry.py`.
 # On merge/rebase, keep our side of the conflict — the post-merge and
-# post-rewrite hooks regenerate the file deterministically afterwards.
+# post-rewrite hooks regenerate these files deterministically afterwards.
 # This works once `scripts/install-git-hooks.sh` has been run locally,
 # which registers `merge.ours.driver = true` and points `core.hooksPath`
 # at `.githooks/`.
 src/asset_manifest.py merge=ours
+src/example_sources_data.py merge=ours
+src/editorial_registry_data.py merge=ours

.githooks/post-merge

@@ -1,9 +1,10 @@
 #!/usr/bin/env bash
-# Regenerate the asset manifest after a merge or pull so the digest
-# reflects the merged tree, not whichever parent won the conflict.
+# Regenerate generated files after a merge or pull so embedded data,
+# asset manifests, fingerprints, and prototype pages reflect the merged
+# tree, not whichever parent won the conflict.
 set -e
 cd "$(git rev-parse --show-toplevel)"
-uv run python scripts/fingerprint_assets.py >/dev/null
-if ! git diff --quiet src/asset_manifest.py public/_headers; then
-  echo "post-merge: asset manifest regenerated; stage and amend if needed"
+make build >/dev/null
+if ! git diff --quiet src/example_sources_data.py src/editorial_registry_data.py src/asset_manifest.py public/_headers public/prototyping; then
+  echo "post-merge: generated files regenerated; stage and amend if needed"
 fi

.githooks/post-rewrite

@@ -1,9 +1,10 @@
 #!/usr/bin/env bash
-# Regenerate the asset manifest after rebase/amend so the digest matches
-# the rewritten history, not whichever commit happened to win each step.
+# Regenerate generated files after rebase/amend so embedded data,
+# asset manifests, fingerprints, and prototype pages match the rewritten
+# history, not whichever commit happened to win each step.
 set -e
 cd "$(git rev-parse --show-toplevel)"
-uv run python scripts/fingerprint_assets.py >/dev/null
-if ! git diff --quiet src/asset_manifest.py public/_headers; then
-  echo "post-rewrite: asset manifest regenerated; stage and amend if needed"
+make build >/dev/null
+if ! git diff --quiet src/example_sources_data.py src/editorial_registry_data.py src/asset_manifest.py public/_headers public/prototyping; then
+  echo "post-rewrite: generated files regenerated; stage and amend if needed"
 fi

.github/pull_request_template.md

@@ -5,7 +5,6 @@
 ## Verification
 
 - [ ] `make verify`
-- [ ] `scripts/check_example_migration_parity.py`
 - [ ] `scripts/format_examples.py --check`
 - [ ] `make verify-python-version VERSION=3.13`
 - [ ] `git diff --check`

.github/workflows/preview.yml

@@ -38,21 +38,29 @@ jobs:
         env:
           CLOUDFLARE_API_TOKEN: ${{ secrets.CLOUDFLARE_API_TOKEN }}
           CLOUDFLARE_ACCOUNT_ID: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }}
-        run: npx --yes wrangler whoami
+        run: npx --yes wrangler@4.90.0 whoami
       - name: Sync Python Workers vendor
         run: uv run pywrangler sync
+      # Workflow inputs are passed through env vars rather than interpolated
+      # into the script, so a crafted input cannot inject shell commands into
+      # steps that hold the Cloudflare API token.
       - name: Upload Cloudflare Preview
         env:
           CLOUDFLARE_API_TOKEN: ${{ secrets.CLOUDFLARE_API_TOKEN }}
           CLOUDFLARE_ACCOUNT_ID: ${{ secrets.CLOUDFLARE_ACCOUNT_ID }}
+          PREVIEW_NAME: ${{ inputs.name || 'preview' }}
+          PREVIEW_MESSAGE: ${{ github.sha }}
         run: |
           set -x
           uv run pywrangler preview \
-            --name "${{ inputs.name || 'preview' }}" \
-            --message "${{ github.sha }}" \
+            --name "$PREVIEW_NAME" \
+            --message "$PREVIEW_MESSAGE" \
             --json
       - name: Smoke test deployed Preview
-        run: scripts/smoke_deployment.py "https://${{ inputs.name || 'preview' }}-pythonbyexample.adewale-883.workers.dev"
+        env:
+          PREVIEW_NAME: ${{ inputs.name || 'preview' }}
+          PBE_SMOKE_BYPASS_SECRET: ${{ secrets.PBE_SMOKE_BYPASS_SECRET }}
+        run: scripts/smoke_deployment.py "https://${PREVIEW_NAME}-pythonbyexample.adewale-883.workers.dev"
       - name: Dump wrangler logs on failure
         if: failure()
         run: |

.github/workflows/regenerate-generated-files.yml

@@ -1,11 +1,12 @@
 name: Regenerate generated files
 
 # Merges made through the GitHub UI bypass the local git hooks that keep
-# src/asset_manifest.py and friends in sync, so a merge to main can land with
-# stale generated files (and a red Verify run). This workflow regenerates them
-# and pushes a fix-up commit. Pushes made with GITHUB_TOKEN do not trigger
-# other workflows, so the fix-up commit is not re-verified by CI; it only ever
-# contains deterministic `make build` output.
+# src/asset_manifest.py, src/example_sources_data.py, and
+# src/editorial_registry_data.py in sync, so a merge to main can land with
+# stale generated files (and a red Verify run). This
+# workflow regenerates them and pushes a fix-up commit. Pushes made with
+# GITHUB_TOKEN do not trigger other workflows, so the fix-up commit is not
+# re-verified by CI; it only ever contains deterministic `make build` output.
 
 on:
   push:
@@ -24,20 +25,38 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - uses: actions/checkout@v4
+      - uses: astral-sh/setup-uv@v5
+        with:
+          enable-cache: false
       - uses: actions/setup-python@v5
         with:
           python-version: '3.13'
-      - name: Regenerate embedded examples and fingerprinted assets
-        run: make build
+      # Pull before building so the regenerated output reflects the latest
+      # main, not the triggering SHA — building first then rebasing could
+      # push stale generated files when another commit lands mid-run.
+      # merge.ours.driver backs the merge=ours attributes on generated files
+      # so the rebase never hits conflict markers in them.
       - name: Commit and push regenerated files if they drifted
         run: |
-          git add -A src/example_sources_data.py src/asset_manifest.py public
-          if git diff --cached --quiet; then
-            echo "Generated files are current."
-            exit 0
-          fi
           git config user.name "github-actions[bot]"
           git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
-          git commit -m "Regenerate fingerprinted assets"
-          git pull --rebase origin main
-          git push origin main
+          git config merge.ours.driver true
+          for attempt in 1 2 3; do
+            git pull --rebase origin main
+            make build
+            git add -A src/example_sources_data.py src/editorial_registry_data.py src/asset_manifest.py public
+            if git diff --cached --quiet; then
+              echo "Generated files are current."
+              exit 0
+            fi
+            git commit -m "Regenerate fingerprinted assets"
+            if git push origin main; then
+              exit 0
+            fi
+            echo "Push raced with another commit; retrying (attempt $attempt)."
+            # Drop the unpushed generated commit and its worktree changes;
+            # the next iteration rebases a clean checkout, then rebuilds.
+            git reset --hard HEAD~1
+          done
+          echo "Failed to push regenerated files after 3 attempts." >&2
+          exit 1

.github/workflows/verify.yml

@@ -2,8 +2,17 @@ name: Verify
 
 on:
   push:
+    branches:
+      - main
   pull_request:
 
+permissions:
+  contents: read
+
+concurrency:
+  group: verify-${{ github.ref }}
+  cancel-in-progress: true
+
 jobs:
   verify:
     runs-on: ubuntu-latest
@@ -42,7 +51,6 @@ jobs:
           CHROME_PATH: /usr/bin/google-chrome
         run: |
           make verify
-          scripts/check_example_migration_parity.py
           scripts/format_examples.py --check
           make verify-python-version VERSION=3.13
           git diff --check

CHANGELOG.md

@@ -18,11 +18,16 @@ The format is inspired by [Keep a Changelog](https://keepachangelog.com/en/1.1.0
 - Skip-to-content link on every page.
 - Per-example social-card images composed from each example's marginalia figure (`make social-cards`), referenced by `og:image`/`twitter:card` on home and example pages and checked by the SEO linter.
 - Learner-behavior report (`scripts/learner_report.py` + `docs/learner-analytics.md`) aggregating exported Worker wide events into most-read pages, most-run examples with edited/error shares and execution percentiles, journey traffic, and missing-example 404s.
+- Content gates for program/cell parity, prose duplication, and internal inline links, wired into `make quality-checks`.
+- Shared inline-prose rendering, figure XML/caption contracts, regenerated figure-review prototypes, and behavioral tests for the Turnstile and ASGI paths.
 
 ### Changed
 
 - Journeys now reference every example: `iterator-vs-iterable`, `classmethods-and-staticmethods`, `bound-and-unbound-methods`, `abstract-base-classes`, and `structured-data-shapes` joined their natural sections, with journey-outcome support lists updated.
 - Journey meta descriptions no longer claim gap placeholders; all previously declared gaps are filled.
+- Python tooling is pinned to 3.13 through `uv`; CI and preview workflows are hardened, generated-file recovery retries from a clean worktree, and quality gates enforce live registry/score invariants.
+- Worker runtime hardening adds body limits, cache-key normalization, constant-time smoke-bypass comparison, fail-closed Turnstile modes, latin-1 ASGI headers, and once-per-isolate lifespan startup.
+- Content corrections across 29 example pages improve Python accuracy, executable demonstrations, sandbox-boundary framing, and link/prose quality.
 
 ## 2026-05-16
 

CONTRIBUTING.md

@@ -49,7 +49,6 @@ Then run:
 
 ```bash
 make verify
-scripts/check_example_migration_parity.py
 scripts/format_examples.py --check
 make verify-python-version VERSION=3.13
 git diff --check
@@ -64,17 +63,38 @@ make check-generated
 
 ## Quality checks
 
-Five scripts enforce the catalog-level rules from `docs/example-quality-rubric.md`. Run them together with `make quality-checks`.
+These scripts enforce the catalog-level rules from `docs/example-quality-rubric.md`. Run them together with `make quality-checks`.
 
 | Script | What it gates |
 | --- | --- |
-| `scripts/check_registry_integrity.py` | Every owner slug in `docs/quality-registries.toml` exists in `manifest.toml`; tokens are present. |
-| `scripts/check_confusable_pairs.py` | Each confusable pair's owning page contains every token that signals the contrast. |
-| `scripts/check_broad_surface_tours.py` | Each broad-title page either covers every required form or sets `scope_first_pass = true` with `see_also` links to focused neighbors. |
+| `scripts/check_registry_integrity.py` | Every owner slug in `docs/quality-registries.toml` exists in `manifest.toml`; tokens are present; each `paired_pages` pair is discoverable through `see_also` and has registry-named cell tokens demonstrated inside a teaching cell. |
+| `scripts/check_confusable_pairs.py` | Each confusable pair's owning page contains every token inside teaching cells; a token shadowed inside a longer sibling token (plain `def` inside `async def`) does not count. |
+| `scripts/check_broad_surface_tours.py` | Each broad-title page either covers every required form or sets `scope_first_pass = true` with registry-named `focused_neighbors` linked through `see_also`. |
 | `scripts/check_footgun_coverage.py` | Each canonical Python footgun has a page that contains both broken-form and fixed-form tokens. |
-| `scripts/check_notes_supported.py` | Every `:::note` bullet shares at least one keyword with the page body, so notes cannot assert behavior the page never demonstrates. |
-
-The single source of truth for the registries is `docs/quality-registries.toml`. Add a new pair, broad tour, or footgun there, then update the owning page so the tokens appear in cells or prose.
+| `scripts/check_notes_supported.py` | Every `:::note` bullet (across all note blocks) shares at least one keyword with the page body outside the notes, so notes cannot assert behavior the page never demonstrates. |
+| `scripts/check_program_covers_cells.py` | Every executable cell shares substantive code with the `:::program` block, so the editor reproduces what the cells teach; `standalone_cells = true` opts out visibly. |
+| `scripts/check_prose_duplication.py` | No verbatim repeated paragraphs, no cell prose copied from the intro, no duplicate note bullets. |
+| `scripts/check_inline_links.py` | Inline `[text](target)` links in prose resolve to real `/examples` or `/journeys` pages. |
+| `scripts/score_example_criteria.py` | Heuristic criterion scores per example; fails when a curated score exceeds the heuristic by more than the delta bound, so the score registry cannot inflate without the page changing. |
+| `scripts/check_quality_scores.py` | Curated scores meet the 9.0 target / 8.5 hard minimum, waivers carry future expiry dates and are dropped when stale, and the journey-section average stays above its floor. |
+| `scripts/check_no_figure_rationales.py` | No-figure opt-outs carry a reason and an unexpired `review_after` date. |
+| `scripts/check_journey_outcomes.py` | Every journey section declares learner outcomes. |
+| `scripts/audit_example_graph.py --check` | The `see_also` graph has no broken targets, self-links, over-linked pages, or orphaned examples. |
+
+The single source of truth for the registries is `docs/quality-registries.toml`. Add confusable pairs, broad tours, footguns, journey metadata, figure attachments, captions, and curated scores there, then update the owning page so the tokens appear in cells or prose.
+
+## Adding a new example end to end
+
+1. Write `src/example_sources/<slug>.md` (frontmatter, intro prose, one `:::program`, cells, notes) and add the slug to `manifest.toml`'s `order`.
+2. Add `see_also` links in both directions — `scripts/audit_example_graph.py --check` fails orphaned pages.
+3. Score the page against `docs/example-quality-rubric.md` and add the `[[example_quality_scores]]` entry to `docs/quality-registries.toml`.
+4. Attach a figure by adding `[[figure_attachments]]` and `[[example_figure_scores]]` entries to `docs/quality-registries.toml` (or add a `no_figure_rationales` entry with a review date); keep only new paint functions in `src/marginalia.py`. Review the result on `public/prototyping/marginalia-gestalt.html`, which shows the production caption under every figure.
+5. Run `make build`, `make verify-examples`, `make quality-checks`, and `scripts/format_examples.py --check`.
+6. Run the full `make verify` with a local Worker running, then commit — including the regenerated `src/example_sources_data.py`, `src/editorial_registry_data.py`, `src/asset_manifest.py`, and fingerprinted assets.
+
+## Secrets and deploy configuration
+
+Deployment uses repository secrets: `CLOUDFLARE_API_TOKEN` and `CLOUDFLARE_ACCOUNT_ID` (Preview workflow), plus optional `PBE_SMOKE_BYPASS_SECRET` so deploy smoke tests can run edited-code POSTs past the Turnstile challenge. Runtime Worker secrets (`TURNSTILE_SECRET_KEY`, `TURNSTILE_CLEARANCE_SECRET`, `PBE_SMOKE_BYPASS_SECRET`) are managed with `wrangler secret put`; see `docs/turnstile-runner-protection-spec.md`.
 
 ## Style expectations