Skip to content

web: render /crawl in the site shell - #1

Merged
ralyodio merged 1 commit into
mainfrom
worktree-crawl-page-site-layout
Sep 22, 2026
Merged

ralyodio merged 1 commit into
mainfrom
worktree-crawl-page-site-layout

Conversation

@ralyodio

Copy link
Copy Markdown
Contributor

What

https://r4ck.dev/crawl was the x402 gateway's own bare HTML page: system font, no top bar, no footer, none of the site's tokens. The one page on the site that did not look like the site.

@profullstack/x402-gateway takes a page(ctx) renderer, so /crawl now renders through the site Layout like every other route. Same words, every number still read off the gateway's context (price, days, header, networks, charged and free crawler lists, the verbatim offer). The throttled variant ("You have used up the free allowance") and the ?days=N variant render too.

How

  • The gateway hook must return a string, not a promise. The asset-version helper moves from routes/static.js to lib/assets.js and hashes synchronously (readFileSync + Bun.CryptoHasher, once per asset per process), so the Layout drops its three awaits and is a plain function.
  • views/crawl.jsx is the page; renderCrawl(ctx) throws loudly if the tree ever renders async instead of putting [object Promise] on a live page.
  • lib/gate.js passes page: renderCrawl. The page keeps noindex.

Tests

bun test: 39 pass, 2 skip (the two that need DATABASE_URL). New test/crawl.test.js covers the string render inside the shell, throttled and multi-day variants, escaping, and gateway.page(). bun run lint clean.

Also ran the gateway's real handle() with a GPTBot user agent against /crawl?days=3: 402, text/html, <title>Crawl pass · r4ck</title>, top bar present, quotes 3.00 USD for 3 × one day.

🤖 Generated with Claude Code

The x402 gateway's sales page was its own bare HTML: system font, no top
bar, no footer, none of the site's tokens. It was the one page on r4ck.dev
that did not look like r4ck.dev.

The gateway takes a `page(ctx)` renderer, so /crawl now renders through the
Layout like every other route, with the same words and every number still
read off the gateway's context. The hook must answer with a string, so the
asset-version helper moves to lib/assets.js and reads synchronously, which
lets the Layout drop its awaits. Static serving keeps using the same helper.

Tests cover the string render inside the shell, the throttled and multi-day
variants, escaping, and the gateway serving it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@ralyodio
ralyodio merged commit d6acdac into main Sep 22, 2026
3 checks passed
@ralyodio
ralyodio deleted the worktree-crawl-page-site-layout branch September 22, 2026 03:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant