Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
61 changes: 61 additions & 0 deletions .github/workflows/mastra-readme-tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
name: Mastra README contract

on:
pull_request:
branches: [main, dev]
paths:
- 'typescript/packages/traceai_mastra/**'
- '.github/workflows/mastra-readme-tests.yml'
push:
branches: [dev]
paths:
- 'typescript/packages/traceai_mastra/**'
- '.github/workflows/mastra-readme-tests.yml'
workflow_dispatch:

permissions:
contents: read

jobs:
readme:
name: Mastra README (${{ matrix.name }})
runs-on: ubuntu-24.04
timeout-minutes: 10
strategy:
fail-fast: false
matrix:
include:
- name: peer-floor
core: '1.16.0'
observability: '1.0.0'
exporter: '1.0.0'
- name: current
core: '1.81.0'
observability: '1.23.0'
exporter: '1.5.0'
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version: '22.23.4'
- name: Install published package and pinned peers
env:
MASTRA_CORE: ${{ matrix.core }}
MASTRA_OBSERVABILITY: ${{ matrix.observability }}
MASTRA_EXPORTER: ${{ matrix.exporter }}
run: |
npm install --prefix "$RUNNER_TEMP/mastra-readme" --ignore-scripts --no-audit --no-fund \
@traceai/mastra@0.2.0 "@mastra/core@$MASTRA_CORE" \
"@mastra/observability@$MASTRA_OBSERVABILITY" "@mastra/otel-exporter@$MASTRA_EXPORTER" \
@opentelemetry/api@1.9.1 @opentelemetry/exporter-trace-otlp-proto@0.201.1 \
typescript@5.9.3 @types/node@22.15.29
- name: Compile README and check OTLP with localhost-only networking
run: |
sudo unshare --net sh -c '
ip link set lo up
exec "$1" "$2" "$3"
' sh "$(command -v node)" \
"$GITHUB_WORKSPACE/typescript/packages/traceai_mastra/contract/readme.test.mjs" \
"$RUNNER_TEMP/mastra-readme"
134 changes: 65 additions & 69 deletions typescript/packages/traceai_mastra/README.md
Original file line number Diff line number Diff line change
@@ -1,92 +1,88 @@
# TraceAI Mastra
## Installation

Send Mastra agent, model, tool, and workflow traces to Future AGI using Mastra's native observability exporter interface. You do not need to patch your agents or add a second tracer provider.

## Install

For Mastra v1, install the integration with these matching observability packages. Upgrade the three Mastra packages together rather than mixing newer observability packages with an older core.

```shell
npm install --save @traceai/mastra
npm install @traceai/mastra @mastra/core@1.81.0 \
@mastra/observability@1.23.0 @mastra/otel-exporter@1.5.0 \
@opentelemetry/api@~1.9.0 @opentelemetry/exporter-trace-otlp-proto@~0.201.1
```

## Usage
Keep the OpenTelemetry versions above compatible with `@traceai/mastra`'s peer dependencies rather than installing the latest exporter independently.

## Set credentials

Set your Future AGI credentials in the application environment:

```shell
export FI_API_KEY="your-futureagi-api-key"
export FI_SECRET_KEY="your-futureagi-secret-key"
```

The package reads `process.env`. If you store these values in an environment file, load it with your application tooling before creating observability.

## Configure tracing

Add `createFIObservability` to your existing Mastra configuration. Use `observability`, not the removed Mastra v0 `telemetry` option.

```typescript
import { Mastra } from "@mastra/core";
import {
FiOTLPTraceExporter,
isFiSpan,
} from "@traceai/mastra";
import { createFIObservability } from "@traceai/mastra";

export const observability = createFIObservability({
serviceName: "support-agent",
projectName: "customer-support",
projectType: "observe",
});

export const mastra = new Mastra({
// ... other config
telemetry: {
serviceName: "traceai-mastra-agent", // you can rename this to whatever you want to appear in the Phoenix UI
enabled: true,
export: {
type: "custom",
exporter: new FiOTLPTraceExporter({
url: process.env.PHOENIX_COLLECTOR_ENDPOINT,
headers: {
Authorization: `Bearer ${process.env.PHOENIX_API_KEY}`,
},
// optional: filter out http, and other node service specific spans
// they will still be exported to Mastra, but not to the target of
// this exporter
spanFilter: isFiSpan,
}),
},
},
// Keep your existing agents, workflows, storage, and other configuration here.
observability,
});
```

## Examples
Run your registered agents or workflows as usual. In Future AGI, open **Observe** and select the `customer-support` project to inspect the trace and its child spans.

### Weather Agent
The exporter sends OTLP/HTTP protobuf **traces only** to `https://api.futureagi.com/tracer/v1/traces`, using `x-api-key` and `x-secret-key` headers. It sets the `project_name` and `project_type` resource attributes for project routing. An explicit `projectName` takes precedence over `FI_PROJECT_NAME`. If neither is set, `createFIObservability` uses `serviceName`.

To setup the canonical Mastra weather agent example, and then ingest the spans into TraceAI (or any other OpenInference-compatible platform), follow the steps below.
Available input and output are included in traces, which can contain prompts, responses, and tool arguments or results. Use Mastra's native `tracingOptions.hideInput` and `hideOutput` for runs that should not capture input/output. Those options do not redact every metadata field, attribute, or error; configure Mastra's [SensitiveDataFilter](https://mastra.ai/reference/observability/tracing/processors/sensitive-data-filter) for field-based redaction.

- Create a new Mastra project
## Shut down cleanly

- Add the FiOTLPTraceExporter to your Mastra project
After all agent or workflow runs finish, await shutdown before a short-lived script exits:

```typescript
// chosen-project-name/src/index.ts
import { Mastra } from "@mastra/core/mastra";
import { createLogger } from "@mastra/core/logger";
import { LibSQLStore } from "@mastra/libsql";
import {
isFiSpan,
FiOTLPTraceExporter,
} from "@traceai/mastra";
await observability.shutdown();
```

import { weatherAgent } from "./agents";
For a long-running application, call this from your application shutdown lifecycle, not after every request. Shutdown closes the exporter after flushing buffered spans.

export const mastra = new Mastra({
agents: { weatherAgent },
storage: new LibSQLStore({
url: ":memory:",
}),
logger: createLogger({
name: "Mastra",
level: "info",
}),
telemetry: {
enabled: true,
serviceName: "weather-agent",
export: {
type: "custom",
exporter: new FiOTLPTraceExporter({
url: "https://api.futureagi.com/tracer/v1/traces",
headers: {
"x-api-key": process.env.FI_API_KEY,
"x-secret-key": process.env.FI_SECRET_KEY,
},
spanFilter: isFiSpan,
}),
},
},
});
```
## Configuration

- Run the agent
`createFIObservability(options)` accepts these settings:

```shell
npm run dev
```
| Option | Default | Purpose |
| --- | --- | --- |
| `serviceName` | `"mastra-app"` | OpenTelemetry service name and project fallback when neither `projectName` nor `FI_PROJECT_NAME` is set. |
| `projectName` | `FI_PROJECT_NAME` or `serviceName` | Future AGI project to receive traces; an explicit option overrides the environment. |
| `projectType` | `"observe"` | Project type: `"observe"` or `"experiment"`. |
| `apiKey`, `secretKey` | `FI_API_KEY`, `FI_SECRET_KEY` | Future AGI authentication. |
| `baseUrl` | `FI_BASE_URL` or `https://api.futureagi.com` | Collector base URL; `/tracer/v1/traces` is appended. |
| `endpoint` | Derived from `baseUrl` | Full traces endpoint; overrides `baseUrl`. |
| `resourceAttributes` | — | Additional OpenTelemetry resource attributes. |
| `headers` | — | Additional request headers. |
| `excludeSpanTypes` | `[SpanType.MODEL_CHUNK]` | Span types to omit; pass `[]` to include streaming chunks. Use the package set above; `@mastra/observability` 1.0 does not apply this filter. |
| `timeout` | Exporter default | Export request timeout in milliseconds. |
| `batchSize` | Processor default | Maximum spans per export batch. |

If you already manage a Mastra `Observability` instance, add `createFIMastraExporter(options)` to its `exporters` array instead of creating a second observability instance. Pass `projectName` explicitly to this lower-level factory (or set `FI_PROJECT_NAME`); it does not default the project name to the service name.

## Mastra v0

The deprecated `FITraceExporter` and `isFISpan` exports remain available from `@traceai/mastra/legacy` for existing Mastra v0 applications. They are not root exports and do not work with Mastra v1's removed `telemetry` configuration. For new applications, use the v1 setup above.

See the [Mastra integration guide](https://docs.futureagi.com/docs/integrations/traceai/mastra) for more details.
172 changes: 172 additions & 0 deletions typescript/packages/traceai_mastra/contract/readme.test.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,172 @@
// Run against an isolated install of the published package, not workspace mocks:
// node contract/readme.test.mjs /absolute/path/to/install [README.md]
import assert from "node:assert/strict";
import { once } from "node:events";
import { mkdtemp, readFile, rm, writeFile } from "node:fs/promises";
import { createServer } from "node:http";
import { createRequire } from "node:module";
import { dirname, join, resolve } from "node:path";
import { test } from "node:test";
import { fileURLToPath, pathToFileURL } from "node:url";

const install = resolve(process.argv[2]);
const readme = resolve(process.argv[3] ?? fileURLToPath(new URL("../README.md", import.meta.url)));
const requireInstalled = createRequire(join(install, "package.json"));
const ts = requireInstalled("typescript");
// Decode the actual OTLP/protobuf request with OpenTelemetry's generated schema.
const requireExporter = createRequire(requireInstalled.resolve("@opentelemetry/exporter-trace-otlp-proto"));
const transformer = dirname(requireExporter.resolve("@opentelemetry/otlp-transformer"));
const { opentelemetry } = requireInstalled(join(transformer, "generated/root.js"));
const ExportTraceServiceRequest = opentelemetry.proto.collector.trace.v1.ExportTraceServiceRequest;

function attributes(values) {
return Object.fromEntries(values.map(({ key, value }) => [key, value.stringValue]));
}

test("README setup compiles and exports native Mastra traces to the configured FI project", { timeout: 30_000 }, async (t) => {
const source = await readFile(readme, "utf8");
assert.match(source, /An explicit `projectName` takes precedence over `FI_PROJECT_NAME`/);
assert.match(source, /\| `projectName` \| `FI_PROJECT_NAME` or `serviceName` \|/);
const snippets = [...source.matchAll(/```typescript\n([\s\S]*?)```/g)].map((match) => match[1]);
assert.equal(snippets.length, 2, "Keep setup and awaited shutdown executable");
const temp = await mkdtemp(join(install, "readme-contract-"));
t.after(() => rm(temp, { recursive: true, force: true }));
const tsFile = join(temp, "readme.mts");
await writeFile(tsFile, snippets.join("\n"));
const program = ts.createProgram([tsFile], {
target: ts.ScriptTarget.ES2022,
module: ts.ModuleKind.NodeNext,
moduleResolution: ts.ModuleResolutionKind.NodeNext,
strict: true,
skipLibCheck: true,
noEmit: true,
types: ["node"],
typeRoots: [join(install, "node_modules/@types")],
});
const diagnostics = ts.getPreEmitDiagnostics(program);
assert.equal(diagnostics.length, 0, ts.formatDiagnosticsWithColorAndContext(diagnostics, {
getCurrentDirectory: () => install,
getCanonicalFileName: (name) => name,
getNewLine: () => "\n",
}));

const requests = [];
const server = createServer(async (request, response) => {
const chunks = [];
for await (const chunk of request) chunks.push(chunk);
const body = Buffer.concat(chunks);
requests.push({
url: request.url,
headers: request.headers,
body: ExportTraceServiceRequest.decode(body),
});
response.writeHead(200, { "content-type": "application/x-protobuf" });
response.end();
});
server.listen(0, "127.0.0.1");
await once(server, "listening");
t.after(() => new Promise((done) => server.close(done)));
const env = {
FI_API_KEY: "synthetic-readme-api-key",
FI_SECRET_KEY: "synthetic-readme-secret-key",
FI_BASE_URL: `http://127.0.0.1:${server.address().port}`,
};
const previousEnv = Object.fromEntries(Object.keys(env).map((key) => [key, process.env[key]]));
Object.assign(process.env, env);
t.after(() => {
for (const [key, value] of Object.entries(previousEnv)) {
if (value === undefined) delete process.env[key];
else process.env[key] = value;
}
});
const jsFile = join(temp, "readme.mjs");
const setup = ts.transpileModule(snippets[0], {
compilerOptions: { target: ts.ScriptTarget.ES2022, module: ts.ModuleKind.ESNext },
}).outputText;
await writeFile(jsFile, setup);
const { observability, mastra } = await import(pathToFileURL(jsFile).href);
t.after(() => observability.shutdown());
assert.ok(mastra);
const instance = observability.getDefaultInstance();
const exporters = instance.getExporters();
assert.equal(exporters.length, 1, "Do not install a duplicate exporter");
assert.equal(exporters[0].name, "future-agi");

// Observe completion of the real exporter-bus promises, without sleeps or
// replacing conversion/transport. Mastra span.end() itself returns void.
const pending = [];
const exporter = exporters[0];
const exportEvent = exporter.exportTracingEvent.bind(exporter);
exporter.exportTracingEvent = (event) => {
const work = exportEvent(event);
pending.push(work);
return work;
};
// SpanType is a runtime enum. The values below are obtained from this install.
const { SpanType } = requireInstalled("@mastra/core/observability");
const root = instance.startSpan({
name: "support-request",
type: SpanType.AGENT_RUN,
input: "Where is my order?",
});
const child = instance.startSpan({
name: "lookup-order",
type: SpanType.TOOL_CALL,
parent: root,
input: { orderId: "synthetic-123" },
});
child.end({ output: { status: "shipped" } });
root.end({ output: "Your order has shipped." });
const hidden = instance.startSpan({
name: "private-run",
type: SpanType.AGENT_RUN,
input: "PRIVATE_INPUT_SENTINEL",
tracingOptions: { hideInput: true, hideOutput: true },
});
hidden.end({ output: "PRIVATE_OUTPUT_SENTINEL" });
const chunk = instance.startSpan({ name: "stream-chunk", type: SpanType.MODEL_CHUNK });
chunk.end();
assert.ok(pending.length > 0, "Native spans must reach the exporter bus");
await Promise.all(pending);
// Execute the README's shutdown instruction after exports have been queued.
const shutdownFile = join(temp, "shutdown.mjs");
await writeFile(shutdownFile, `import { observability } from './readme.mjs';\n${snippets[1]}`);
await import(pathToFileURL(shutdownFile).href);

assert.ok(requests.length > 0, "Shutdown must flush buffered spans");
const spans = [];
for (const request of requests) {
assert.equal(request.url, "/tracer/v1/traces");
assert.equal(request.headers["content-type"], "application/x-protobuf");
assert.equal(request.headers["x-api-key"], env.FI_API_KEY);
assert.equal(request.headers["x-secret-key"], env.FI_SECRET_KEY);
for (const resource of request.body.resourceSpans) {
const resourceAttrs = attributes(resource.resource.attributes);
assert.equal(resourceAttrs["service.name"], "support-agent");
assert.equal(resourceAttrs.project_name, "customer-support");
assert.equal(resourceAttrs.project_type, "observe");
for (const scope of resource.scopeSpans) spans.push(...scope.spans);
}
}
// The declared 1.0.0 peer floor predates Mastra's excludeSpanTypes option.
// The README pins a current package set for this option; retain legacy coverage
// without pretending older Mastra versions suppress streaming chunks.
const { version } = requireInstalled("@mastra/observability/package.json");
assert.equal(spans.length, version === "1.0.0" ? 4 : 3, "Export ended spans once, applying supported chunk filtering");
const byId = new Map(spans.map((span) => [Buffer.from(span.spanId).toString("hex"), span]));
const parentWire = byId.get(root.id);
const childWire = byId.get(child.id);
assert.ok(parentWire);
assert.ok(childWire);
assert.deepEqual(childWire.traceId, parentWire.traceId);
assert.deepEqual(childWire.parentSpanId, parentWire.spanId);
assert.equal(attributes(parentWire.attributes)["gen_ai.span.kind"], "AGENT");
assert.equal(attributes(childWire.attributes)["gen_ai.span.kind"], "TOOL");
assert.equal(attributes(parentWire.attributes)["input.value"], "Where is my order?");
assert.equal(attributes(parentWire.attributes)["output.value"], "Your order has shipped.");
assert.deepEqual(JSON.parse(attributes(childWire.attributes)["output.value"]), { status: "shipped" });
const payload = JSON.stringify(requests.map((request) => request.body));
for (const secret of ["PRIVATE_INPUT_SENTINEL", "PRIVATE_OUTPUT_SENTINEL", env.FI_API_KEY, env.FI_SECRET_KEY]) {
assert.ok(!payload.includes(secret), "Hidden I/O and auth values must not enter spans");
}
});
Loading