diff --git a/cockpit/runtimes/mastra/angular/docs/wire-capture-subagents.md b/cockpit/runtimes/mastra/angular/docs/wire-capture-subagents.md new file mode 100644 index 000000000..da27af630 --- /dev/null +++ b/cockpit/runtimes/mastra/angular/docs/wire-capture-subagents.md @@ -0,0 +1,246 @@ +# Mastra sub-agents: installed-API findings and delegation wire capture + +Task 0 spike evidence for the Mastra subagent PRs. Captured 2026-09-02 against +the live `deployments/ag-ui-mastra` service (`@mastra/core@1.63.2`, +`@ag-ui/mastra@1.1.2`) with a scratch `weather_forecaster` child agent +registered on the camping-trip supervisor. The scratch changes were reverted; +this document is the only artifact. + +## Verdict summary + +| Question | Answer | +| --- | --- | +| Sub-agents API present at installed `@mastra/core@1.63.2`? | **Yes** — no version bump needed. | +| Child registration | `agents: { weather_forecaster: childAgent }` on the supervisor's `Agent` config. | +| Delegation hooks | `delegation: { onDelegationStart, onDelegationComplete }` in stream options; works via the supervisor's `defaultOptions` (verified live — both hooks fired). | +| Delegation on the AG-UI wire today | An ordinary backend tool call named `agent-`: `TOOL_CALL_START` → one `TOOL_CALL_ARGS` blob → `TOOL_CALL_END` → `TOOL_CALL_RESULT` whose `content` is JSON `{text, subAgentThreadId, subAgentResourceId, subAgentToolResults}`. | +| Does child text stream incrementally anywhere? | **In-process yes, on the wire no.** The parent `fullStream` carries every child chunk wrapped as `tool-output` (`payload: {output: , toolCallId, toolName}`) — 82 inner `text-delta` chunks in the raw tap — but `@ag-ui/mastra`'s chunk processor drops them (`case "tool-output": break`). Only the final text reaches AG-UI, inside `TOOL_CALL_RESULT`. | +| Delegation tool-call id (for `parentToolCallId`) | The LLM's tool-call id (e.g. `call_aUfV9K0RCDZdZK3NWt9dRDKx`). Identical across `TOOL_CALL_START/ARGS/END/RESULT` and both hooks' `toolCallId`. | + +## Installed-API details (`node_modules/@mastra/core/dist`) + +- **Registration** — `agent/types.d.ts:656` (AgentConfig): + `agents?: DynamicArgument>>` — + "Sub-Agents that the agent can access." A plain `Agent` instance satisfies + `SubAgent` (`agent/subagent.d.ts:42`). Give the child a `description`; it + becomes the delegation tool's description. +- **Tool naming** — compiled `agent-B8m3ps7U.js:34916/35498/35513`: each child + becomes a tool `agent-${agentName}` where `agentName` is the key in the + `agents` record. Its input schema is + `{prompt, threadId, resourceId, instructions, maxSteps, suspendedToolRunId, resumeData}`. +- **Hook config site** — `agent/agent.types.d.ts` (~line 671 in + `AgentExecutionOptionsBase`): `delegation?: DelegationConfig` is a + **per-invocation stream/generate option**, not an `Agent` constructor field. + Since the AG-UI bridge calls `agent.stream()` without it, set it via the + supervisor's `defaultOptions: { delegation: {...} }` — verified live that + the merged defaults fire both hooks. `DelegationConfig` + (`agent.types.d.ts:294-344`) also offers `messageFilter`, + `includeSubAgentToolResultsInModelContext`, `hookErrorStrategy`. +- **Hook signatures** (`agent.types.d.ts:61-207`): + - `onDelegationStart(ctx: DelegationStartContext)` — `primitiveId`, + `primitiveType: 'agent'|'workflow'`, `prompt`, `params{threadId, resourceId, + instructions, maxSteps}`, `iteration`, `runId` (the PARENT run id), + `threadId`, `resourceId`, `parentAgentId`, `parentAgentName`, + **`toolCallId`** (the LLM tool-call id), `messages`, `requestContext`. + May return `{proceed, rejectionReason, modifiedPrompt, modifiedInstructions, + modifiedMaxSteps}`. + - `onDelegationComplete(ctx: DelegationCompleteContext)` — same identity + fields plus `result{text, subAgentThreadId, subAgentResourceId, + finishReason, subAgentToolResults[], usage}`, `duration`, `success`, + `error?`, `bail()`. May return `{feedback, resultText}`. + - **Neither hook exposes a child run id.** The only child-run identifiers are + `subAgentThreadId`/`subAgentResourceId` in the complete-hook result — a + synthesized `subagentRunId` (`-sub` or `-sub`) is + required. +- **Parent stream chunks during delegation** (`stream/types.d.ts:313-319, 904`): + every child chunk is forwarded to the parent stream as + `{type: 'tool-output', payload: {output: , toolCallId, toolName}}` + (compiled subagent tool execute, `agent-B8m3ps7U.js:35290-35300`: + `for await (chunk of streamResult.fullStream) context.writer.write(chunk)`). + The inner `start` chunk even carries the child agent id: + `output.payload = {id: 'weather_forecaster', messageId: ...}`. + The `agent-execution-*` / `routing-agent-*` chunk types belong to the + deprecated AgentNetwork path and never appeared; do not use `.network()`. + +## Raw parent-stream tap (direct `agent.stream()`, chunk types observed) + +``` +start, step-start, +tool-call-input-streaming-start/delta/end (name agent-weather_forecaster), +tool-call toolName=agent-weather_forecaster toolCallId=call_… +tool-output x266 payload.output.type ∈ {start, step-start, text-start, + text-delta(x82), text-end, tool-call, tool-call-delta, + tool-call-input-streaming-*, tool-result, step-finish, + finish} ← the ENTIRE child stream, incrementally +tool-result toolName=agent-weather_forecaster (same toolCallId) +step-start, text-start, text-delta(x43), text-end, step-finish, finish +``` + +So incremental child text exists at the Mastra layer, keyed to the delegation +`toolCallId` — it is `@ag-ui/mastra` that discards it. + +## Live AG-UI SSE capture (scrubbed, capture-2) + +Request: `POST /agent/mastra` with +`{"threadId":"t-spike-2","runId":"r-spike-2","messages":[{"role":"user","content":"Plan a trip to Bear Lake this weekend — what will the weather be?"}], ...}`. +Delegated on the first attempt in both runs. + +``` +data: {"type":"RUN_STARTED","threadId":"t-spike-2","runId":"r-spike-2"} +data: {"type":"TOOL_CALL_START","parentMessageId":"4aca0872-…","toolCallId":"call_aUfV9K0RCDZdZK3NWt9dRDKx","toolCallName":"agent-weather_forecaster"} +data: {"type":"TOOL_CALL_ARGS","toolCallId":"call_aUfV9K0RCDZdZK3NWt9dRDKx","delta":"{\"prompt\":\"What will the weather be like at Bear Lake this weekend?\",\"threadId\":null,\"resourceId\":null,\"instructions\":null,\"maxSteps\":5,\"suspendedToolRunId\":null,\"resumeData\":null}"} +data: {"type":"TOOL_CALL_END","toolCallId":"call_aUfV9K0RCDZdZK3NWt9dRDKx"} +data: {"type":"TOOL_CALL_RESULT","toolCallId":"call_aUfV9K0RCDZdZK3NWt9dRDKx","content":"{\"text\":\"Here's the weather forecast for Bear Lake this weekend:\\n\\n- **Friday**: Mostly sunny…\",\"subAgentThreadId\":\"t-spike-2-f50a7c26-…\",\"subAgentResourceId\":\"t-spike-2-weather_forecaster\",\"subAgentToolResults\":[{\"toolName\":\"updateWorkingMemory\",\"toolCallId\":\"call_rSLW…\",\"result\":{\"success\":true},…}]}","messageId":"70b04476-…","role":"tool"} +data: {"type":"STATE_SNAPSHOT","snapshot":{}} +data: {"type":"STATE_DELTA","delta":[{"op":"add","path":"/packing_list","value":{}}]} (x3 — child inherited working memory and wrote to it) +data: {"type":"TEXT_MESSAGE_CHUNK","role":"assistant","messageId":"4aca0872-…-agui-text","delta":"The"} (x26 — the PARENT's own summary, streamed) +… +data: {"type":"STATE_SNAPSHOT","snapshot":{"packing_list":{"title":"","items":[]}}} +data: {"type":"RUN_FINISHED","threadId":"t-spike-2","runId":"r-spike-2","usage":[{…}]} +``` + +Event-type totals for the run: 1 RUN_STARTED, 1 each TOOL_CALL_START/ARGS/END/ +RESULT, 2 STATE_SNAPSHOT, 3 STATE_DELTA, 26 TEXT_MESSAGE_CHUNK, 1 RUN_FINISHED. +Notes: + +- `TOOL_CALL_ARGS` arrives as ONE blob, not deltas — the bridge buffers backend + `tool-call` chunks and flushes start+args+end together. The child's `prompt` + is therefore readable on the wire before the result. +- The child's final text is embedded in `TOOL_CALL_RESULT.content` (JSON + string) — no child TEXT_MESSAGE events of any kind today. +- Working-memory injection: with no memory of its own the child inherits the + supervisor's working-memory tooling (`injectSupervisorMemory`), so it called + `updateWorkingMemory`; those nested calls appear only in + `subAgentToolResults`, never as top-level TOOL_CALL_* events. + +## Hook payloads (stderr, capture-1 run — same shape every run) + +``` +[HOOK onDelegationStart] {"primitiveId":"weather_forecaster","primitiveType":"agent", + "prompt":"What will the weather be like at Bear Lake this weekend?", + "params":{"threadId":"t-spike-1","resourceId":"t-spike-1","maxSteps":5}, + "iteration":1,"runId":"r-spike-1","threadId":"t-spike-1","resourceId":"t-spike-1", + "parentAgentId":"mastra","parentAgentName":"mastra", + "toolCallId":"call_cmS8mH1Up9o5MxKyMmTsYCPU","messages":"<1 messages>"} +[HOOK onDelegationComplete] {"primitiveId":"weather_forecaster","primitiveType":"agent", + "result":{"text":"Here's the weather forecast…","finishReason":"stop", + "subAgentThreadId":"t-spike-1-e80e0b3d-…","subAgentResourceId":"t-spike-1-weather_forecaster", + "subAgentToolResults":[…],"usage":{"inputTokens":1066,"outputTokens":127,…}}, + "duration":4987,"success":true,"iteration":1,"runId":"r-spike-1", + "toolCallId":"call_cmS8mH1Up9o5MxKyMmTsYCPU",…} +``` + +The `toolCallId` in both hooks equals the wire tool-call id exactly. + +## Emitter wiring decision + +Inject the SUBAGENT_* events in **`deployments/ag-ui-mastra/server.mjs`'s +translation loop** — the `sub.run(input).subscribe({ next })` handler +(server.mjs:111-126) — keyed off the AG-UI events themselves, NOT off the +delegation hooks: + +1. On `TOOL_CALL_START` with `toolCallName.startsWith('agent-')`: emit + `SUBAGENT_STARTED { subagentRunId: `${toolCallId}-sub`, name: + toolCallName.slice('agent-'.length), parentToolCallId: toolCallId }` + before forwarding the frame, and remember the toolCallId → name mapping. +2. On `TOOL_CALL_RESULT` for a remembered toolCallId: parse `content` JSON, + emit an attributed `TEXT_MESSAGE_CHUNK` (single final chunk — that is all + the wire has) carrying `result.text` under the subagent identity, then + `SUBAGENT_FINISHED { subagentRunId, parentToolCallId }`, then forward (or + suppress) the TOOL_CALL_RESULT per the target contract. + +Rationale: + +- The bridge gives the server only the AG-UI Observable; the Mastra chunk + stream (where `tool-output` child deltas live) is consumed inside + `@ag-ui/mastra` and dropped there. Recovering incremental child text would + mean bypassing/forking the bridge (subscribe to `agent.stream()` ourselves + and re-implement its ~800-line chunk processor) — not worth it for Task 0's + single-final-chunk contract, but the data provably exists if a later PR + wants real streaming. +- The delegation hooks (via `defaultOptions` in `agents.mjs`) fire in a + different async context than the Observable events, with no ordering + guarantee relative to `res.write` frames — unsafe as the primary emitter + trigger. They remain available for metadata enrichment (prompt, duration, + usage, `subAgentToolResults`) if SUBAGENT_FINISHED wants a payload beyond + ids. +- Everything needed for the target contract is already deterministic on the + wire: the `agent-` name prefix, the stable `toolCallId`, and the final text + in `TOOL_CALL_RESULT.content.text`. + +## Scratch setup used (reverted) + +- Child: `new Agent({ id/name: 'weather_forecaster', description: 'Forecasts + weather for a campsite and date range…', instructions: '…3-bullet forecast + summary…', model: 'openai/gpt-4o-mini' })`. +- Supervisor additions: `agents: { weather_forecaster }`, + `defaultOptions: { delegation: { onDelegationStart, onDelegationComplete } }` + (hooks logging to stderr), and one instruction line changed to "For questions + about weather forecasts you MUST delegate to the weather_forecaster agent" + (the stock instructions force `check_conditions` for weather, which would + have fought delegation). + +## After the emitter + +Live smoke against the committed emitter (`deployments/ag-ui-mastra/subagent-emitter.mjs` +wired into `server.mjs`'s translation loop), 2026-09-02. Request: +`POST /agent/mastra` with `{"threadId":"t-post-emitter-1","runId":"r-post-emitter-1", +"messages":[{"role":"user","content":"Plan a trip to Bear Lake this weekend — what +will the weather be?"}], ...}`. Delegated on the first attempt; the injected +sequence appeared exactly once, ids consistent throughout (scrubbed): + +``` +data: {"type":"RUN_STARTED","threadId":"t-post-emitter-1","runId":"r-post-emitter-1"} +data: {"type":"TOOL_CALL_START","parentMessageId":"28f84e1d-…","toolCallId":"call_1W2Rmq…","toolCallName":"agent-weather_forecaster"} +data: {"type":"SUBAGENT_STARTED","subagentRunId":"call_1W2Rmq…-sub","name":"weather_forecaster","parentToolCallId":"call_1W2Rmq…"} +data: {"type":"TOOL_CALL_ARGS","toolCallId":"call_1W2Rmq…","delta":"{\"prompt\":\"What will the weather be like at Bear Lake this weekend?\",…}"} +data: {"type":"TOOL_CALL_END","toolCallId":"call_1W2Rmq…"} +data: {"type":"TEXT_MESSAGE_START","messageId":"call_1W2Rmq…-sub-m1","role":"assistant","subagentRunId":"call_1W2Rmq…-sub"} +data: {"type":"TEXT_MESSAGE_CONTENT","messageId":"call_1W2Rmq…-sub-m1","delta":"Here's the weather forecast for Bear Lake this weekend:\n\n- **Saturday**: Mostly sunny…","subagentRunId":"call_1W2Rmq…-sub"} +data: {"type":"TEXT_MESSAGE_END","messageId":"call_1W2Rmq…-sub-m1","subagentRunId":"call_1W2Rmq…-sub"} +data: {"type":"SUBAGENT_FINISHED","subagentRunId":"call_1W2Rmq…-sub","outcome":{"type":"success"}} +data: {"type":"TOOL_CALL_RESULT","toolCallId":"call_1W2Rmq…","content":"{\"text\":\"Here's the weather forecast…\",\"subAgentThreadId\":…}","messageId":"…","role":"tool"} +data: {"type":"STATE_SNAPSHOT","snapshot":{}} +data: {"type":"STATE_DELTA","delta":[…]} (x3) +data: {"type":"TEXT_MESSAGE_CHUNK","role":"assistant","messageId":"28f84e1d-…-agui-text","delta":"…"} (x31 — the parent's own summary) +data: {"type":"STATE_SNAPSHOT","snapshot":{"packing_list":…}} +data: {"type":"RUN_FINISHED","threadId":"t-post-emitter-1","runId":"r-post-emitter-1","usage":[…]} +``` + +Event-type totals: 1 each RUN_STARTED / TOOL_CALL_START / SUBAGENT_STARTED / +TOOL_CALL_ARGS / TOOL_CALL_END / TEXT_MESSAGE_START / TEXT_MESSAGE_CONTENT / +TEXT_MESSAGE_END / SUBAGENT_FINISHED / TOOL_CALL_RESULT / RUN_FINISHED, +2 STATE_SNAPSHOT, 3 STATE_DELTA, 31 TEXT_MESSAGE_CHUNK. + +Notes: + +- SUBAGENT_STARTED is injected immediately after TOOL_CALL_START, which on + this bridge lands BEFORE the buffered TOOL_CALL_ARGS/END flush — the card + therefore exists before the delegation prompt is readable. +- Child deltas: single final chunk — bridge drops tool-output upstream + (`case "tool-output": break` in @ag-ui/mastra), so the one + TEXT_MESSAGE_CONTENT carries the child's entire final text. + +## Browser verification + +Live check 2026-09-02: real-key `deployments/ag-ui-mastra` on the topic port + +`npx nx serve cockpit-runtimes-mastra-angular`, driving "Plan a trip to Bear +Lake this weekend - what will the weather be?" in the real UI. Screenshot of +the completed, expanded card: `e2e/manual/subagent-card-live.png`. + +- The card renders from the injected events with ZERO component code: + `chat-tool-calls` groups on `parentToolCallId` and mounts + `chat-subagent-card` — header `weather_forecaster` + the delegation + toolCallId + a `complete` pill + "1 message(s)"; expanding shows the + child's full 3-bullet forecast; the parent's own summary streams below. +- Honest timing note: the card does NOT visibly pass through a + running/empty-body phase on this runtime. A timestamped SSE probe through + the dev proxy shows the bridge withholds `TOOL_CALL_START` until the + delegation resolves — RUN_STARTED at t=0, then a ~5s silent gap while the + child runs, then TOOL_CALL_START → SUBAGENT_STARTED → … → + SUBAGENT_FINISHED → TOOL_CALL_RESULT all inside ~40ms. STARTED→FINISHED + are ~35ms apart on the wire, so the card mounts effectively already + complete (a Playwright observer that awaited card attachment read + `data-state="done"` on first sight). The emitter is not the limiter; + @ag-ui/mastra's buffered tool-call flush is (same upstream drop/buffer + behavior documented above). diff --git a/cockpit/runtimes/mastra/angular/e2e/fixtures/mastra.json b/cockpit/runtimes/mastra/angular/e2e/fixtures/mastra.json index 40d9a3791..a06e26920 100644 --- a/cockpit/runtimes/mastra/angular/e2e/fixtures/mastra.json +++ b/cockpit/runtimes/mastra/angular/e2e/fixtures/mastra.json @@ -18,6 +18,18 @@ "content": "North Pines is reserved for 2 nights — confirmation TP-0288." } }, + { + "match": { "userMessage": "Bear Lake", "hasToolResult": true }, + "response": { + "content": "Expect a sunny, mild weekend at Bear Lake — great weather for camping." + } + }, + { + "match": { "systemMessage": "You are a weather forecaster" }, + "response": { + "content": "Here's the Bear Lake weekend forecast:\n\n- Saturday: sunny, high 24°C\n- Sunday: partly cloudy, high 22°C\n- Overall: mild with light winds" + } + }, { "match": { "userMessage": "packing list" }, "response": { @@ -60,6 +72,25 @@ } ] } + }, + { + "match": { "userMessage": "Bear Lake" }, + "response": { + "toolCalls": [ + { + "name": "agent-weather_forecaster", + "arguments": { + "prompt": "What will the weather be like at Bear Lake this weekend?", + "threadId": null, + "resourceId": null, + "instructions": null, + "maxSteps": 5, + "suspendedToolRunId": null, + "resumeData": null + } + } + ] + } } ] } diff --git a/cockpit/runtimes/mastra/angular/e2e/manual/subagent-card-live.png b/cockpit/runtimes/mastra/angular/e2e/manual/subagent-card-live.png new file mode 100644 index 000000000..fac129361 Binary files /dev/null and b/cockpit/runtimes/mastra/angular/e2e/manual/subagent-card-live.png differ diff --git a/cockpit/runtimes/mastra/angular/e2e/mastra.spec.ts b/cockpit/runtimes/mastra/angular/e2e/mastra.spec.ts index 978954613..de2316629 100644 --- a/cockpit/runtimes/mastra/angular/e2e/mastra.spec.ts +++ b/cockpit/runtimes/mastra/angular/e2e/mastra.spec.ts @@ -1,5 +1,6 @@ // SPDX-License-Identifier: MIT import { test, expect } from '@playwright/test'; +import { submitAndWaitForResponse } from '@threadplane-internal/e2e-harness'; // First cockpit e2e whose backend is neither LangGraph nor Python: the // rt-mastra topic runs against the deployments/ag-ui-mastra Node service @@ -43,4 +44,16 @@ test.describe('cockpit runtimes/mastra: camping trip planner', () => { await dialog.getByRole('button', { name: 'Approve' }).click(); await expect(page.getByText(/reserved for 2 nights/i)).toBeVisible({ timeout: 30_000 }); }); + + // Delegation: the supervisor calls the registered weather_forecaster + // sub-agent (wire tool `agent-weather_forecaster`); the server-side + // emitter (deployments/ag-ui-mastra/subagent-emitter.mjs) injects + // SUBAGENT_STARTED + attributed TEXT_MESSAGE_* + SUBAGENT_FINISHED, which + // the adapter reduces into a subagent card on the tool-call group. + test('rt-mastra: delegated forecast renders a subagent card with the final text', async ({ page }) => { + const bubble = await submitAndWaitForResponse(page, 'Plan a trip to Bear Lake this weekend — what will the weather be?'); + await expect(page.locator('chat-subagent-card')).toHaveCount(1); + await expect(page.locator('chat-subagent-card')).toContainText('weather_forecaster'); + await expect(bubble).toContainText(/forecast|weather/i); + }); }); diff --git a/cockpit/runtimes/mastra/angular/src/app/mastra.component.ts b/cockpit/runtimes/mastra/angular/src/app/mastra.component.ts index bcbde3c33..9b358a327 100644 --- a/cockpit/runtimes/mastra/angular/src/app/mastra.component.ts +++ b/cockpit/runtimes/mastra/angular/src/app/mastra.component.ts @@ -56,8 +56,11 @@ interface PackingList { * `submit({ resume })` goes out as * `forwardedProps.command = { resume, interruptEvent: { toolCallId, runId } }` * — exactly what the Mastra bridge requires to resume the suspended run. - * - NO subagents surface here: Mastra reserves ACTIVITY_* for background - * tasks, a measured red cell in the matrix. + * - Sub-agent delegation (weather_forecaster) reaches the wire as a + * `agent-` tool call; the Node service's subagent emitter turns it + * into SUBAGENT_* + attributed TEXT_MESSAGE_* frames, so the standard + * chat-subagent-card renders with zero code in this component + * (docs/wire-capture-subagents.md). */ @Component({ selector: 'app-mastra', diff --git a/deployments/ag-ui-mastra/agents.mjs b/deployments/ag-ui-mastra/agents.mjs index 2334e54b6..ef604f39a 100644 --- a/deployments/ag-ui-mastra/agents.mjs +++ b/deployments/ag-ui-mastra/agents.mjs @@ -96,12 +96,29 @@ const reserveCampsiteTool = createTool({ export function createMastra(dbUrl) { const store = (id) => new LibSQLStore({ id, url: dbUrl }); + /** + * Sub-agent (spike: wire-capture-subagents.md). Registered on the + * supervisor via `agents:`; Mastra surfaces it as a backend tool named + * `agent-weather_forecaster` whose TOOL_CALL_RESULT carries the child's + * final text — server.mjs's subagent emitter turns that into SUBAGENT_* + * frames. The `description` becomes the delegation tool's description. + */ + const weatherForecaster = new Agent({ + id: 'weather_forecaster', + name: 'weather_forecaster', + description: 'Forecasts weather for a campsite and date range. Use for any weather question.', + instructions: + 'You are a weather forecaster. Given a campsite and dates, give a 3-bullet forecast summary. Be concise.', + model: MODEL, + }); + const tripAgent = new Agent({ id: 'mastra', name: 'mastra', instructions: `You are a terse camping trip planner. The packing list in working memory is the user's shared state: whenever the user adds, removes, or changes items (or starts a list), update working memory to match. 'items' is an array of {name, qty}. Never mention memory or the list mechanics. -For questions about weather or trail conditions you MUST call check_conditions. +For questions about trail conditions you MUST call check_conditions. +For questions about weather forecasts you MUST delegate to the weather_forecaster agent. When the user asks to reserve or book a campsite you MUST call reserve_campsite; after it resumes, confirm the outcome. Always answer in one short sentence.`, model: MODEL, @@ -109,6 +126,7 @@ Always answer in one short sentence.`, check_conditions: checkConditionsTool, reserve_campsite: reserveCampsiteTool, }, + agents: { weather_forecaster: weatherForecaster }, memory: new Memory({ storage: store('mastra-topic-memory'), options: { diff --git a/deployments/ag-ui-mastra/server.mjs b/deployments/ag-ui-mastra/server.mjs index a3caa4dd3..8987db394 100644 --- a/deployments/ag-ui-mastra/server.mjs +++ b/deployments/ag-ui-mastra/server.mjs @@ -22,6 +22,7 @@ import { pathToFileURL } from 'node:url'; import { dirname, resolve } from 'node:path'; import { MastraAgent } from '@ag-ui/mastra'; import { createMastra } from './agents.mjs'; +import { createSubagentInjector } from './subagent-emitter.mjs'; const AG_UI_INTERNAL_TOKEN = process.env.AG_UI_INTERNAL_TOKEN; if (!AG_UI_INTERNAL_TOKEN) { @@ -108,16 +109,18 @@ export function createAgUiServer() { resourceId: input.threadId, }); + // One injector per run: turns delegation tool calls (`agent-`) + // into SUBAGENT_* frames around the events the bridge already emits. + const injector = createSubagentInjector(); const sub = bridge.run(input).subscribe({ next: (event) => { - res.write(sseFrame(event)); + for (const e of injector.eventsFor(event)) res.write(sseFrame(e)); }, error: (err) => { // Map failures into the protocol instead of killing the socket: // the client finalizes the run as an error rather than hanging. - res.write( - sseFrame({ type: 'RUN_ERROR', message: String(err?.message ?? err) }), - ); + const runError = { type: 'RUN_ERROR', message: String(err?.message ?? err) }; + for (const e of injector.eventsFor(runError)) res.write(sseFrame(e)); res.end(); }, complete: () => { diff --git a/deployments/ag-ui-mastra/subagent-emitter.mjs b/deployments/ag-ui-mastra/subagent-emitter.mjs new file mode 100644 index 000000000..24ab9fd9c --- /dev/null +++ b/deployments/ag-ui-mastra/subagent-emitter.mjs @@ -0,0 +1,125 @@ +// SPDX-License-Identifier: MIT +// SUBAGENT_* injection for Mastra delegation tool calls. +// +// Mastra surfaces a registered sub-agent as an ordinary backend tool named +// `agent-`: TOOL_CALL_START → TOOL_CALL_ARGS → TOOL_CALL_END → +// TOOL_CALL_RESULT whose `content` is JSON `{text, subAgentThreadId, ...}` +// (measured: cockpit/runtimes/mastra/angular/docs/wire-capture-subagents.md). +// The upstream @ag-ui/mastra bridge drops the in-process child deltas +// (`case "tool-output": break`), so the honest wire contract here is a +// single final text chunk per delegation. +// +// This module is a pure transform over the outbound AG-UI event stream — +// keyed off the events themselves, NOT the Mastra delegation hooks, which +// fire in a different async context with no ordering guarantee relative to +// the Observable frames. +// +// Injected sequence per delegation tool call (child key = tool name +// minus the `agent-` prefix, subagentRunId = `-sub`): +// - AFTER TOOL_CALL_START: SUBAGENT_STARTED {subagentRunId, name, +// parentToolCallId} +// - BEFORE TOOL_CALL_RESULT (success): TEXT_MESSAGE_START/CONTENT/END +// carrying the child's final text under the subagent identity, then +// SUBAGENT_FINISHED {outcome:{type:'success'}} +// - BEFORE TOOL_CALL_RESULT (failure — parsed content says success:false +// or finishReason:'error'): SUBAGENT_ERROR {subagentRunId, message} +// - Terminal cleanup: a RUN_ERROR or RUN_FINISHED arriving while +// delegations are still pending (no TOOL_CALL_RESULT seen — e.g. the +// Observable errored mid-delegation) closes each pending card with +// SUBAGENT_ERROR before the terminal frame, so no card is left spinning. +// In the measured captures the RESULT always precedes the terminal frame, +// so this path is defensive only. + +const AGENT_TOOL_PREFIX = 'agent-'; + +/** + * Create a per-run injector. + * + * @returns {{ eventsFor(event: object): object[] }} — for each outbound + * AG-UI event, the ordered list of frames to write (injections plus the + * original event). Non-delegation events pass through as `[event]`. + */ +export function createSubagentInjector() { + /** @type {Map} pending delegations by toolCallId */ + const pending = new Map(); + + return { + eventsFor(event) { + switch (event.type) { + case 'TOOL_CALL_START': { + const name = event.toolCallName ?? ''; + if (!name.startsWith(AGENT_TOOL_PREFIX)) return [event]; + const entry = { + subagentRunId: `${event.toolCallId}-sub`, + name: name.slice(AGENT_TOOL_PREFIX.length), + }; + pending.set(event.toolCallId, entry); + return [ + event, + { + type: 'SUBAGENT_STARTED', + subagentRunId: entry.subagentRunId, + name: entry.name, + parentToolCallId: event.toolCallId, + }, + ]; + } + + case 'TOOL_CALL_RESULT': { + const entry = pending.get(event.toolCallId); + if (!entry) return [event]; // not a delegation (or unmatched) — pass through + pending.delete(event.toolCallId); + const { subagentRunId } = entry; + + const raw = typeof event.content === 'string' ? event.content : JSON.stringify(event.content); + let parsed; + try { + parsed = JSON.parse(raw); + } catch { + parsed = undefined; + } + const failed = + parsed !== undefined && + typeof parsed === 'object' && + parsed !== null && + (parsed.success === false || parsed.finishReason === 'error'); + if (failed) { + return [ + { + type: 'SUBAGENT_ERROR', + subagentRunId, + message: String(parsed.error ?? parsed.text ?? 'sub-agent delegation failed'), + }, + event, + ]; + } + + const text = typeof parsed?.text === 'string' ? parsed.text : raw; + const messageId = `${event.toolCallId}-sub-m1`; + return [ + { type: 'TEXT_MESSAGE_START', messageId, role: 'assistant', subagentRunId }, + { type: 'TEXT_MESSAGE_CONTENT', messageId, delta: text, subagentRunId }, + { type: 'TEXT_MESSAGE_END', messageId, subagentRunId }, + { type: 'SUBAGENT_FINISHED', subagentRunId, outcome: { type: 'success' } }, + event, + ]; + } + + case 'RUN_ERROR': + case 'RUN_FINISHED': { + if (pending.size === 0) return [event]; + const cleanup = [...pending.values()].map(({ subagentRunId }) => ({ + type: 'SUBAGENT_ERROR', + subagentRunId, + message: 'delegation did not complete before the run terminated', + })); + pending.clear(); + return [...cleanup, event]; + } + + default: + return [event]; + } + }, + }; +} diff --git a/deployments/ag-ui-mastra/test/subagent-emitter.test.mjs b/deployments/ag-ui-mastra/test/subagent-emitter.test.mjs new file mode 100644 index 000000000..5cedb98fd --- /dev/null +++ b/deployments/ag-ui-mastra/test/subagent-emitter.test.mjs @@ -0,0 +1,157 @@ +// SPDX-License-Identifier: MIT +// Unit tests for the SUBAGENT_* injector — synthetic AG-UI event sequences, +// exact injected ordering asserted field-for-field against the contract in +// cockpit/runtimes/mastra/angular/docs/wire-capture-subagents.md. +import { test } from 'node:test'; +import assert from 'node:assert/strict'; +import { createSubagentInjector } from '../subagent-emitter.mjs'; + +const TID = 'call_aUfV9K0RCDZdZK3NWt9dRDKx'; + +function delegationStart(tid = TID) { + return { + type: 'TOOL_CALL_START', + parentMessageId: 'pm-1', + toolCallId: tid, + toolCallName: 'agent-weather_forecaster', + }; +} + +function delegationResult(content, tid = TID) { + return { + type: 'TOOL_CALL_RESULT', + toolCallId: tid, + content, + messageId: 'tm-1', + role: 'tool', + }; +} + +/** Run a whole sequence through one injector, collecting the output frames. */ +function transform(events) { + const injector = createSubagentInjector(); + return events.flatMap((e) => injector.eventsFor(e)); +} + +test('success path: exact injected sequence, field-for-field', () => { + const resultContent = JSON.stringify({ + text: 'Here is the forecast:\n- Fri sunny\n- Sat cloudy\n- Sun rain', + subAgentThreadId: 't-1-abc', + subAgentResourceId: 't-1-weather_forecaster', + subAgentToolResults: [], + }); + const start = delegationStart(); + const args = { type: 'TOOL_CALL_ARGS', toolCallId: TID, delta: '{"prompt":"weather?"}' }; + const end = { type: 'TOOL_CALL_END', toolCallId: TID }; + const result = delegationResult(resultContent); + const finished = { type: 'RUN_FINISHED', threadId: 't-1', runId: 'r-1' }; + + const out = transform([start, args, end, result, finished]); + + assert.deepEqual(out, [ + start, + { + type: 'SUBAGENT_STARTED', + subagentRunId: `${TID}-sub`, + name: 'weather_forecaster', + parentToolCallId: TID, + }, + args, + end, + { + type: 'TEXT_MESSAGE_START', + messageId: `${TID}-sub-m1`, + role: 'assistant', + subagentRunId: `${TID}-sub`, + }, + { + type: 'TEXT_MESSAGE_CONTENT', + messageId: `${TID}-sub-m1`, + delta: 'Here is the forecast:\n- Fri sunny\n- Sat cloudy\n- Sun rain', + subagentRunId: `${TID}-sub`, + }, + { + type: 'TEXT_MESSAGE_END', + messageId: `${TID}-sub-m1`, + subagentRunId: `${TID}-sub`, + }, + { + type: 'SUBAGENT_FINISHED', + subagentRunId: `${TID}-sub`, + outcome: { type: 'success' }, + }, + result, + finished, + ]); +}); + +test('error result (success:false) → SUBAGENT_ERROR instead of text+finished', () => { + const result = delegationResult(JSON.stringify({ success: false, error: 'model refused' })); + const out = transform([delegationStart(), result]); + assert.deepEqual(out.slice(2), [ + { type: 'SUBAGENT_ERROR', subagentRunId: `${TID}-sub`, message: 'model refused' }, + result, + ]); + assert.ok(!out.some((e) => e.type === 'SUBAGENT_FINISHED')); + assert.ok(!out.some((e) => e.type.startsWith('TEXT_MESSAGE'))); +}); + +test("error result (finishReason:'error') → SUBAGENT_ERROR", () => { + const result = delegationResult(JSON.stringify({ text: 'partial', finishReason: 'error' })); + const out = transform([delegationStart(), result]); + assert.deepEqual(out.slice(2), [ + { type: 'SUBAGENT_ERROR', subagentRunId: `${TID}-sub`, message: 'partial' }, + result, + ]); +}); + +test('non-JSON result content falls back to the raw string as the delta', () => { + const result = delegationResult('plain text answer'); + const out = transform([delegationStart(), result]); + const content = out.find((e) => e.type === 'TEXT_MESSAGE_CONTENT'); + assert.equal(content.delta, 'plain text answer'); + assert.ok(out.some((e) => e.type === 'SUBAGENT_FINISHED')); +}); + +test('non-delegation tool calls pass through untouched', () => { + const events = [ + { type: 'RUN_STARTED', threadId: 't', runId: 'r' }, + { type: 'TOOL_CALL_START', toolCallId: 'c1', toolCallName: 'check_conditions' }, + { type: 'TOOL_CALL_ARGS', toolCallId: 'c1', delta: '{}' }, + { type: 'TOOL_CALL_END', toolCallId: 'c1' }, + { type: 'TOOL_CALL_RESULT', toolCallId: 'c1', content: '{"forecast":"clear"}' }, + { type: 'TEXT_MESSAGE_CHUNK', messageId: 'm', delta: 'ok' }, + { type: 'RUN_FINISHED', threadId: 't', runId: 'r' }, + ]; + assert.deepEqual(transform(events), events); +}); + +test('unmatched TOOL_CALL_RESULT (no prior delegation start) is ignored', () => { + const result = delegationResult('{"text":"orphan"}', 'call_never_started'); + assert.deepEqual(transform([result]), [result]); +}); + +test('pending delegation at RUN_FINISHED → SUBAGENT_ERROR cleanup before the terminal frame', () => { + const finished = { type: 'RUN_FINISHED', threadId: 't', runId: 'r' }; + const out = transform([delegationStart(), finished]); + assert.deepEqual(out.slice(2), [ + { + type: 'SUBAGENT_ERROR', + subagentRunId: `${TID}-sub`, + message: 'delegation did not complete before the run terminated', + }, + finished, + ]); +}); + +test('pending delegation at RUN_ERROR → SUBAGENT_ERROR cleanup, then no double-cleanup', () => { + const injector = createSubagentInjector(); + const out = [ + ...injector.eventsFor(delegationStart()), + ...injector.eventsFor({ type: 'RUN_ERROR', message: 'boom' }), + ]; + assert.equal(out.filter((e) => e.type === 'SUBAGENT_ERROR').length, 1); + assert.equal(out.at(-1).type, 'RUN_ERROR'); + // A second terminal frame injects nothing more. + assert.deepEqual(injector.eventsFor({ type: 'RUN_FINISHED' }), [{ type: 'RUN_FINISHED' }]); +});