Skip to content

feat(tools): add export_map_html_tool for maps in agent-built pages - #275

Merged
mattpodwysocki merged 4 commits into
mainfrom
feat/export-map-html-tool
Oct 14, 2026
Merged

mattpodwysocki merged 4 commits into
mainfrom
feat/export-map-html-tool

Conversation

@mattpodwysocki

Copy link
Copy Markdown
Contributor

Why

render_map_tool only renders inside the chat, as an MCP App. When an agent is building a site or file that needs a map (ChatGPT/Codex in the report), it had nothing to put in the page, so it wrote its own Mapbox GL JS code and asked the user for a pk. token or left YOUR_MAPBOX_ACCESS_TOKEN in the file.

What

New export_map_html_tool. It takes the same input as render_map_tool (markers, layers, payload_refs) and returns a complete standalone HTML page of about 3 KB.

  • The page carries the map data and a public token in JSON tags and loads the shared map script from jsDelivr, pinned to @mapbox/mcp-server@<version>/dist/map-app/standalone.js with a sha384 integrity hash. Keeping it small matters because the agent copies the whole page into a file write; inlining the ~60 KB script would mean re-emitting ~16k tokens.
  • Token: the account's default public token, resolved the same way the in-chat map does it, or public_token if passed (must be pk.). The result recommends a URL-restricted token for published sites.
  • Routes, isochrones and search results are still fetched by the page on load, like the in-chat map. The result says how many layers do this.
  • Server instructions point "building a web page, site or file" at the new tool, only when it's enabled.
  • Refactors, with no behavior change: the in-chat page is split into mapAppShell() and mapAppScript() (output byte-for-byte identical), and render_map_tool's ref resolution and payload merging moved to assembleMapPayload() so both tools share it.
  • The build writes dist/map-app/standalone.js, which ships in the npm package. When running from source, where the version reads 0.0.0, the tool inlines the script instead of linking to a URL that would 404.

Eval

This uses the tool-selection eval from #274, with two new site-building cases: the agent has a write_file host tool and is asked to build a page with walksheds or a route. A run passes when the agent calls export_map_html_tool and then writes the file. A separate check looks for any token ask or placeholder token. Three reps each:

Claude Opus 5.5, main Claude, this branch chat-latest, main chat-latest, this branch
Site cases pass 0/12 12/12 0/6 6/6
No token ask or placeholder 1/12 11/12 1/6 5/6
Map-requested cases pass 24/24 24/24 12/12 12/12
Other cases that switched to the export tool 0 0 0 0
Other cases pass 75/102 77/102 37/51 38/51
  • On main, every site run hand-wrote GL JS. Most left a placeholder token, which is the reported behavior.
  • On this branch, the one flagged run per provider is caused by the eval's stub, not the tool. The eval returns placeholder HTML in place of a real page, so the agent noticed and either offered to rebuild the page with a token (Claude) or wrote its own page (chat-latest). With the real tool output, neither would have happened.
  • Ordinary "show me a map" prompts still go to render_map_tool.

Testing

  • npm test: 995 pass, including new tests for:
    • the tool
    • the standalone script, run in a VM sandbox: it reads its token and endpoint from the page and calls the configured endpoint
    • the server instructions
  • The in-chat page output is byte-for-byte identical before and after the split.
  • Built package: the integrity hash of dist/map-app/standalone.js matches what the tool writes into pages, and npm pack includes the file.
  • Headless Chrome: a generated page loads the hosted script and creates the map, and a copy with a wrong hash does not run the script.

Before release

A page from version X loads the map script from @mapbox/mcp-server@X on jsDelivr. The hosted server needs to run a version that's already been published to npm, or new pages won't load their map until it is.

🤖 Generated with Claude Code

mattpodwysocki and others added 2 commits October 12, 2026 13:30
render_map_tool only renders inside the chat, so an agent building a site
had to hand-write Mapbox GL JS and ask the user for a public token. The new
tool takes render_map_tool's input and returns a ~3 KB standalone HTML page:
map data and a public token in JSON tags, plus the shared map script loaded
from jsDelivr, pinned by package version and sha384 integrity hash.

The in-chat page is split into mapAppShell() and mapAppScript() with
byte-identical output; the build writes a standalone copy of the script to
dist/map-app/standalone.js. Payload assembly moves to assembleMapPayload()
so both tools share it. Server instructions point page building at the new
tool when it is enabled.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@mattpodwysocki
mattpodwysocki requested a review from a team as a code owner October 13, 2026 16:58
mattpodwysocki added a commit that referenced this pull request Oct 13, 2026
- EVAL_PROVIDER=openai runs the same cases through the Responses API,
  with server-side tool search for deferred mode (eval:tools:openai).
- Each stubbed tool result gets its own ref, so an invented or
  hand-built render_map_tool call is caught by ref_ok.
- Two site-building cases add a write_file host tool and expect
  export_map_html_tool (#275); needs_tools lets them fail cleanly on
  servers without it.
- New no_token_ask metric flags asking the user for a Mapbox token or
  writing a placeholder token into a file.
- EVAL_MODES selects eager/deferred; rescore reads EVAL_FLOW.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
].filter(Boolean);

const structured: ExportMapHtmlOutput = {
html,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there any potential size issue:

render_map_tool respects MAX_INLINE_PAYLOAD_BYTES, but this tool puts the whole merged payload into the HTML without a limit. That includes geometry resolved from refs, such as union/buffer results and large GeoJSON. The HTML is also returned twice, once in the text content and once in structuredContent.html (ExportMapHtmlTool.ts:184 and :171).

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, both fixed in 69af1e0.

  • Map data over 200 KB now gets rejected with next steps (run big polygons/lines through simplify_tool, drop stuff the page doesn't need, or split it across maps). render_map_tool can fall back to a stored ref when it's over its limit, but this page has to carry its data and the agent copies the whole thing into a file write, so a hard cap made more sense here.
  • The HTML is only in the text content now. structuredContent just has bytes, counts, token source and script URL, so it's not sent to the model twice.

Added tests for both.

The page has to carry its data, with no server-side ref to fall back on
like render_map_tool, and the agent copies the whole page into a file
write. Reject map data over 200 KB with next steps (simplify_tool, drop
unneeded features, split across maps).

The HTML was also in both the text content and structuredContent.html,
doubling the tokens sent to the model. structuredContent now has only
the size and metadata.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
mattpodwysocki added a commit that referenced this pull request Oct 13, 2026
…274)

* feat(evals): add tool-selection eval for description routing regressions

Runs 17 prompts against Claude and grades which tools it calls against
expected trajectories, with all descriptions loaded and again behind the
API's tool search. The tool surface comes from a real McpServer read back
over MCP; tool results are canned per case, so no Mapbox calls are made.
EVAL_SERVER_ROOT compares another checkout (e.g. a PR worktree).

Runner and lite report builder are vendored from the claude-api skill's
eval scaffold. Adds @anthropic-ai/sdk and tsx as dev dependencies.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(changelog): add PR number for tool-selection eval

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(evals): add OpenAI provider, site-building cases and token check

- EVAL_PROVIDER=openai runs the same cases through the Responses API,
  with server-side tool search for deferred mode (eval:tools:openai).
- Each stubbed tool result gets its own ref, so an invented or
  hand-built render_map_tool call is caught by ref_ok.
- Two site-building cases add a write_file host tool and expect
  export_map_html_tool (#275); needs_tools lets them fail cleanly on
  servers without it.
- New no_token_ask metric flags asking the user for a Mapbox token or
  writing a placeholder token into a file.
- EVAL_MODES selects eager/deferred; rescore reads EVAL_FLOW.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
@mattpodwysocki
mattpodwysocki merged commit 1a1dea9 into main Oct 14, 2026
5 checks passed
ctufts added a commit that referenced this pull request Oct 15, 2026
… pages (#275)" (#279)

This reverts commit 1a1dea9.
The work is kept on the feat/export-map-html-tool branch to re-land later.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants