Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
7e1f3b8
docs(specs): executable approval tools for the demo graph
blove Sep 5, 2026
a0d6b98
docs(plans): executable approval tools for the demo graph
blove Sep 5, 2026
c9c4abf
feat(examples/chat): seed a demo-owned backup inventory into the grap…
blove Sep 5, 2026
3f9e6cd
feat(examples/chat): list_backups reads the inventory through Injecte…
blove Sep 5, 2026
1e8a9ac
feat(examples/chat): delete_backups interrupts for approval before it…
blove Sep 5, 2026
ab0a4d1
feat(examples/chat): bind list_backups and delete_backups and route c…
blove Sep 5, 2026
3306d03
fix(chat): parse a JSON-object string tool result into tool-view props
blove Sep 5, 2026
17a18f0
feat(examples/chat): BackupTableComponent tool view with pending, row…
blove Sep 5, 2026
a15430d
feat(examples/chat): register the list_backups view on the hero and e…
blove Sep 5, 2026
bf11556
feat(examples/chat): approval chip runs the hero scenario; hero fixtu…
blove Sep 5, 2026
40d85cd
chore(examples/chat): measure the post-approval turn live and record …
blove Sep 5, 2026
5a4531c
feat(hero): re-record the walkthrough with the executable approval tools
blove Sep 5, 2026
b172443
feat(website): re-record both hero posters against the executed appro…
blove Sep 5, 2026
8502a6c
chore(docs): regenerate api docs
github-actions[bot] Sep 5, 2026
9f997b5
Merge branch 'main' into blove/hero-executable-approval-design-23b7b3
blove Sep 5, 2026
00843bd
Merge branch 'main' into blove/hero-executable-approval-design-23b7b3
blove Sep 5, 2026
8a7b11c
Merge branch 'main' into blove/hero-executable-approval-design-23b7b3
blove Sep 5, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion apps/website/content/docs/chat/api/api-docs.json
Original file line number Diff line number Diff line change
Expand Up @@ -4969,7 +4969,7 @@
{
"name": "ChatToolViewsComponent",
"kind": "class",
"description": "Renders a frontend component for a tool call by reusing the chat\ncomposition's `views` registry. A tool call whose `name` matches a\nregistry key is bridged into a synthetic one-element render spec\n(`{ root: name, elements: { [name]: { type: name, props } } }`) and\nrendered through the existing render-spec pipeline.\n\nProps merge the live `args` (present while the call streams) with the\n`result` (on completion) and always include `status`, so a view\ncomponent can show its own loading/empty/error states. `RenderElement`\nfilters props down to the component's declared inputs, so extra keys\n(and a `status` a component chooses not to declare) are harmless.",
"description": "Renders a frontend component for a tool call by reusing the chat\ncomposition's `views` registry. A tool call whose `name` matches a\nregistry key is bridged into a synthetic one-element render spec\n(`{ root: name, elements: { [name]: { type: name, props } } }`) and\nrendered through the existing render-spec pipeline.\n\nProps merge the live `args` (present while the call streams) with the\n`result` (on completion; a JSON-object string result is parsed first,\nsince that is how a LangGraph ToolMessage carries a dict return) and\nalways include `status`, so a view\ncomponent can show its own loading/empty/error states. `RenderElement`\nfilters props down to the component's declared inputs, so extra keys\n(and a `status` a component chooses not to declare) are harmless.",
"params": [],
"examples": [],
"properties": [
Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified apps/website/public/screenshots/hero-walkthrough-poster.webp
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1,548 changes: 1,548 additions & 0 deletions docs/superpowers/plans/2026-09-05-hero-executable-approval-tools.md

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
# Executable approval tools for the demo graph

**Date:** 2026-09-04
**Status:** Approved design. Blocked on a prerequisite bug fix; the recording step is additionally gated on a live measurement.
**Surface:** `examples/chat/python/src/graph.py`, `examples/chat/angular`, and the hero walkthrough.

## 1. Why

The homepage hero's approval beat currently replays an authored narration: after the human approves, the transcript shows a compact three-step cleanup plan. Measured against the real backend, the product does not do that.

| Prompt | Pauses for approval | Post-approval turn |
|---|---|---|
| Bare request | 12/12 | 0/14 plans — clarifying questions, median 621 chars |
| Request that asks to be walked through it | 22/22 | 22/22 plans, median **5,802 chars / 75 lines**, ending in "What I need from you to proceed" |

Shipped narration: **482 characters**.

So the demo depicts behaviour the product does not have. That is invention, not curation, and re-labelling the frame would not fix it.

**The root cause is structural.** After approval the graph has nothing to execute — `request_approval` is the only relevant tool, and there is no tool that inventories or deletes anything. The model's only available move is to describe at length, or to ask. Give it tools and it acts instead, which is what should make a true recording compact.

A second problem the same change fixes: today the guardrail is a system-prompt instruction. It complies 22/22, but a prompt is not a guarantee, and "we told the model to ask first" is not an answer any team shipping irreversible actions would accept.

## 2. Decisions

| Decision | Choice |
|---|---|
| What "real" means | Genuinely executed against demo-owned state, and visible on screen |
| Where state lives | The graph's `State`, so it rides the checkpoint |
| How the inventory renders | A registered tool view, not generative UI and not markdown |
| How approval is enforced | The destructive tool interrupts itself; code, not prompt |
| Reach | Hero plus the demo's own surfaces, so a visitor can run it |

## 3. Mechanism

### State

A `backups` channel on `State`, seeded from a fixed list on first use so every thread starts identically and a recording is reproducible. Each row carries an id, location, creation date, size, and an optional `retain` tag.

Living in the checkpoint means it is per-thread isolated, and it survives the interrupt, the resume, and a page reload. One scenario then carries the interrupt claim, the durable-thread claim, and the "the pause is a checkpoint, not a modal" line the homepage already makes.

### Tools

`list_backups(older_than_days)` reads state through `InjectedState`, seeding it when empty, and returns the matching rows. This is the inventory the viewer sees.

`delete_backups(ids)` calls `interrupt()` **before doing anything**, with an approval payload naming exactly what will go. On resume it removes those ids via `Command(update=...)` and returns an audit summary. Declined, it deletes nothing and says so. It hard-refuses any id tagged `retain`, so the exclusion is enforced rather than suggested.

Everything before the `interrupt()` call must be idempotent, because LangGraph re-runs the node on resume and `interrupt()` returns the resume value on the second pass. `request_approval` already works this way and is the precedent.

The interrupt keeps the existing `{ type: 'approval_request', reason }` shape so `chat-interrupt-panel` renders unchanged.

**Verified available in this LangGraph:** `InjectedState` and `Command`.

### Why this makes the guardrail real

There is no path to deletion that skips the interrupt, because the interrupt is the first statement in the tool. A prompt cannot talk past it and neither can a jailbreak. That is the pattern teams actually need, and it is what the current demo only asserts.

`request_approval` stays for the generic case and its existing users; the hero's pause now comes from the destructive tool itself.

### Rendering

A `BackupTableComponent` registered as `views({ list_backups: BackupTableComponent })`, merged with the A2UI catalog the hero already passes. The view reads the call's status as well as its result, so it shows a pending state while the tool runs and then the rows — the tool-progress beat comes for free rather than as a separate claim.

### Reach

Tool views and a welcome-suggestion chip go into the demo shell as well as the hero, so a visitor who takes control can run the scenario rather than only watching it. The generic `request_approval`, its aimock fixture, the marketing clip and the cockpit interrupt example are untouched.

## 4. Testing

Python tests for seeding, filtering, and the retain refusal. The test that carries the whole claim: calling `delete_backups` without an approval must interrupt and delete nothing — mutation-checked, because a guardrail test that cannot fail is worse than none.

Angular specs for the tool view's three states. The hero e2e continues to run on the committed replay.

## 5. Gates

**Prerequisite, landing first as its own change.** The duplicate first-prompt submit races the interrupt into an HTTP 400 (`No tool output found for function call …`). It failed 7 of 7 live takes on the longer prompt and 5 of 9 on the shorter one, so nothing can be recorded from a real run until it is fixed. It is a production defect on the live takeover path regardless. aimock never validates function-call/output pairing, which is why replay-based CI cannot see it.

**Measurement gate before re-recording.** This spec asserts that real tools make the model's turn compact, on the reasoning that it is verbose precisely because it can only describe. That is a hypothesis. It must be measured live before committing to a re-recording, the way the two prompt hypotheses were. If the turn is still thousands of characters with tools in hand, stop and rethink rather than trimming by hand.

**Measured 2026-09-05 (10 live runs, gpt-5-mini, `examples/chat/python/scripts/measure_approval_turn.py`):** interrupt from `delete_backups` in 10/10, `request_approval` in 0/10; `list_backups` first in 10/10; the three deletable rows removed and the two retained rows kept in 10/10; post-approval turn median **196 chars / 1 line**, max 228. Gate passed; recording proceeds. One wrinkle: in 4/10 runs the model's first `delete_backups` call included a retained id and was refused before any interrupt, and it re-called with the correct set — the guardrail doing its job, at the cost of one extra tool chip before the pause. The committed take is one with a single call.

## 6. Out of scope

Making this the canonical interrupt story in the docs guide and the cockpit capability example. "Gate the destructive tool in code" is better guidance than what the interrupts guide teaches today, but that is a follow-up once the pattern has proven itself here.

Also deferred: the auto-generated thread title rendering as a stray assistant bubble with its own action row, found alongside the 400. Related smell — every assistant message across all three turns shares one id, the reused OpenAI Responses id.
28 changes: 0 additions & 28 deletions examples/chat/angular/e2e/fixtures/hero-approval.json

This file was deleted.

6 changes: 6 additions & 0 deletions examples/chat/angular/e2e/hero.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,9 @@ import { test, expect } from '@playwright/test';
import { attachBrowserHygiene } from './test-helpers';

test.describe('hero walkthrough', () => {
// The replay is a real live stream now (hundreds of token events per run),
// so one full pass runs well past Playwright's 30s default.
test.describe.configure({ timeout: 120_000 });
test('replays to the interrupt, takes over to live, and can replay again', async ({ page }) => {
const hygiene = attachBrowserHygiene(page);
await page.goto('/hero');
Expand Down Expand Up @@ -30,6 +33,9 @@ test.describe('hero walkthrough', () => {
await page.goto('/hero');
const pill = page.locator('[data-hero-pill]');
await expect(page.locator('chat-interrupt-panel')).toBeAttached({ timeout: 60_000 });
// The pause comes from delete_backups, after list_backups has rendered its
// registered tool view — the inventory the visitor is being asked about.
await expect(page.locator('app-backup-table [data-state="rows"]')).toBeAttached();
// Accept is pressed by the script; the A2UI surface from run 3 renders.
await expect(page.locator('a2ui-surface').first()).toBeAttached({ timeout: 90_000 });
await expect(pill).toContainText(/recorded LangGraph run/i);
Expand Down
37 changes: 20 additions & 17 deletions examples/chat/angular/e2e/record-hero-poster-mobile.record.ts
Original file line number Diff line number Diff line change
Expand Up @@ -13,23 +13,26 @@
* bubble behind it, and a still of a live Accept / Edit / Respond dialog
* invites taps that do nothing.
*
* The 2800ms wait is shared with the desktop recorder and is about the scripted
* cursor, not the text: at 1500ms it is still parked where it pressed Accept,
* which at phone width drops the arrowhead onto the word `retain` in step 3.
* By 2800ms it has reached the composer, which reads as the walkthrough about
* to type again rather than as a smudge on the prose. See that recorder for the
* measured timeline; PHONE WIDTH IS THE BINDING CONSTRAINT on the value, because
* the composer here starts filling at ~2970ms while the desktop capture has
* until ~3200ms. The previous 2500ms was measured against an older
* `public/hero-replay.json` and dropped the arrowhead onto "Nothing has been
* deleted yet" once that recording changed, so re-measure whenever it does.
* The wait is about the scripted cursor, not the text. Until the resumed answer
* finishes and HOLD_AFTER_ANSWER_MS elapses the cursor stays parked where it
* pressed Accept, which at phone width drops the arrowhead onto the
* `delete_backups` tool chip. It then glides to the composer for CURSOR_MOVE_MS
* and typing starts the moment it arrives, so the only clean frame is mid-glide:
* cursor en route, composer still empty. Measured against the 2026-09-05
* recording by sampling the DOM every 250ms after the panel detaches: parked
* until ~4.6s, gliding until ~5.3s, typing from ~5.4s, send at ~8.5s. 5000ms
* is late in the glide, past the answer text and above the composer. Runs drift
* by up to a second, so if a capture shows
* the arrowhead back on the chip, re-run rather than retune. (7400ms landed
* AFTER the second prompt had been sent, which the assertions below cannot
* catch: the composer has cleared again by then.) The desktop recorder keeps
* its own value because its Accept spot is empty space. Re-measure whenever
* `public/hero-replay.json` changes.
*
* The height budget is just as coupled, and to the FIXTURE rather than the
* replay: `e2e/fixtures/hero-approval.json` supplies the answer text, and its
* opening line has to fit on ONE line at 390px (about 44 characters) or the
* whole block shifts up and the first line is sliced off the top edge. A draft
* that opened "Approved. Here is the cleanup I would run once you confirm the
* backup locations:" wrapped to two lines and did exactly that.
* The height budget is just as coupled, and to the REPLAY: the opening line
* of the recorded post-approval answer in `public/hero-replay.json` has to fit
* on ONE line at 390px (about 44 characters) or the whole block shifts up and
* the first line is sliced off the top edge. Re-check it on every re-record.
*
* Geometry: 390x650 is the phone design width the reviews already use, and it
* is exactly 3:5 — the ratio `.hero-demo-stage` holds below 768px — so
Expand Down Expand Up @@ -59,7 +62,7 @@ test('capture mobile hero poster', async ({ page }) => {
const interruptPanel = page.locator('chat-interrupt-panel');
await interruptPanel.waitFor({ timeout: 60_000 });
await interruptPanel.waitFor({ state: 'detached', timeout: 60_000 });
await page.waitForTimeout(2800);
await page.waitForTimeout(5000);
// Guards the beat: `.hero__take` ships in normal flow, and a composer with
// the next prompt already typed into it means the wait has drifted late. The
// a2ui check catches a capture that drifted PAST typing into the second run,
Expand Down
14 changes: 8 additions & 6 deletions examples/chat/angular/e2e/record-hero.config.ts
Original file line number Diff line number Diff line change
@@ -1,12 +1,14 @@
/**
* Playwright config for recording the hero walkthrough fixture. Mirrors
* `record-demo.config.ts` — same aimock-backed global setup — but captures no
* video: the artifact is `public/hero-replay.json`, not a clip.
* Playwright config for the hero POSTER recorders. Mirrors
* `record-demo.config.ts` — same aimock-backed global setup, so the demo
* boots without an API key — but captures stills, not a clip.
*
* `testMatch` picks up only the hero record script, so recording never runs in
* CI and the e2e suite never records.
* `record-hero-fixture.record.ts` also matches `testMatch`, but must NOT be
* run through this config: the fixture is recorded against the real model via
* `record-hero-live.config.ts`, because the post-approval turn now executes
* real tools and aimock cannot stage that sequence.
*
* npx playwright test --config examples/chat/angular/e2e/record-hero.config.ts record-hero-fixture
* npx playwright test --config examples/chat/angular/e2e/record-hero.config.ts record-hero-poster
*/
import { defineConfig } from '@playwright/test';

Expand Down
Loading
Loading