From 7966287f041e8c4180ae104ad864cf3a79ba12c0 Mon Sep 17 00:00:00 2001 From: Nikhil Pareek Date: Fri, 16 Oct 2026 18:24:34 -0700 Subject: [PATCH 1/5] docs(integrations): Exa tracing (TH-8324) Co-Authored-By: Claude Sonnet 5.5 --- src/pages/docs/integrations/traceai/exa.mdx | 112 ++++++++++++++++++++ 1 file changed, 112 insertions(+) create mode 100644 src/pages/docs/integrations/traceai/exa.mdx diff --git a/src/pages/docs/integrations/traceai/exa.mdx b/src/pages/docs/integrations/traceai/exa.mdx new file mode 100644 index 000000000..ff3fb14bf --- /dev/null +++ b/src/pages/docs/integrations/traceai/exa.mdx @@ -0,0 +1,112 @@ +--- +title: "Exa Integration with Future AGI for Search Tracing" +description: "Trace Exa search, contents and answer calls in Future AGI with the traceAI-exa instrumentor for the exa-py client." +--- + +## About + +`traceAI-exa` wraps the official [`exa-py`](https://github.com/exa-labs/exa-py) client so every `search`, `get_contents` and `answer` call becomes a span in Future AGI. It is a client wrapper, not an Exa exporter: Exa does not publish an OpenTelemetry exporter, and the package does not patch the HTTP layer. + +The package is Alpha. It is released from [future-agi/traceAI](https://github.com/future-agi/traceAI) (`python/frameworks/exa`). Until a release is on PyPI, install it from that directory of the repository. + +## 1. Installation + +```bash +pip install traceAI-exa +``` + +Tested with `exa-py` 2.25.0 (the latest release when this page was written, 2026-10-16) on Python 3.10, 3.12 and 3.13. The package accepts `exa-py>=2.25.0,<3`. + +## 2. Set environment variables + +```python +import os + +os.environ["EXA_API_KEY"] = "your-exa-api-key" +os.environ["FI_API_KEY"] = "your-futureagi-api-key" +os.environ["FI_SECRET_KEY"] = "your-futureagi-secret-key" +``` + +## 3. Register and instrument + +```python +import os +from exa_py import Exa +from fi_instrumentation import register +from fi_instrumentation.fi_types import ProjectType +from traceai_exa import ExaInstrumentor + +trace_provider = register( + project_type=ProjectType.OBSERVE, + project_name="exa-search", +) +ExaInstrumentor().instrument(tracer_provider=trace_provider) + +exa = Exa(api_key=os.environ["EXA_API_KEY"]) +exa.search("recent retrieval research", num_results=5) +``` + +Call `register()` first so the instrumentor has a tracer provider. A script that only calls `register()` exports the Exa span as a root trace. Under an active span (for example an agent span) the Exa span is its child. + +## What is captured + +`Exa` and `AsyncExa` calls each produce one span. `search_and_contents` (deprecated upstream) is traced as `exa.search`. `stream_search` and `stream_answer` produce one span that ends when the stream is read to the end or closed. + +| Call | Span name | Span kind | +|---|---|---| +| `search`, `search_and_contents`, `stream_search` | `exa.search` | `RETRIEVER` | +| `answer`, `stream_answer` | `exa.answer` | `RETRIEVER` | +| `get_contents` | `exa.get_contents` | `TOOL` | + +| Attribute | Value | +|---|---| +| `gen_ai.retrieval.query`, `input.value` | The query, cut to 1 KB. | +| `exa.result_count` | Number of results (citations for `answer`). Absent when unknown. | +| `exa.url_count` | Number of URLs passed to `get_contents`. | +| `exa.urls` | Only with `instrument(capture_urls=True)`: at most 20 URLs. | +| `exa.cancelled` | `true` when a call or stream was cancelled or closed early. | + +An Exa error (for example an invalid key, HTTP 401) ends the span with status `ERROR` and the message `ValueError: Request failed with status code 401: ...`; your code still receives the exception unchanged. + +Not captured: result titles, page text, highlights, answer text, model name, token counts and cost (Exa does not return model tokens for search). The Exa API key is never recorded, and no stack trace or local path is exported. + +## Privacy + +traceAI instrumentors follow the standard `TraceConfig`: the query is recorded by default. To turn it off, set `FI_HIDE_INPUTS=true` or pass a config: + +```python +from fi_instrumentation import TraceConfig + +ExaInstrumentor().instrument( + tracer_provider=trace_provider, config=TraceConfig(hide_inputs=True) +) +``` + +With hidden inputs, `input.value` is `__REDACTED__`, the query and URL attributes are dropped, and the query or URLs are removed from error text. Counts stay. `pii_redaction` and `using_session`, `using_user`, `using_metadata` and `using_tags` also apply. + +## Tested versions + +| Component | Version | +|---|---| +| `exa-py` | 2.25.0 | +| `traceAI-exa` | 0.1.0 | +| Python | 3.10, 3.12, 3.13 | + +Verified against a local Future AGI deployment with the real `exa-py` client calling a local stand-in for the Exa API (`api.exa.ai` was not called). The hierarchy, span kinds, an `ERROR` span and the privacy switch were checked in the stored spans. + +## Troubleshooting + +- **No spans appear.** Check `FI_API_KEY` and `FI_SECRET_KEY`, and that `instrument()` runs before the first Exa call. +- **Duplicate spans.** Do not combine this package with another instrumentor that wraps the same Exa client methods. +- **`ValueError` when creating `Exa()`.** The client raises when no API key is set, before any traced call, so no span is recorded. +- **Stream span stays open.** Read the stream to the end or call `close()`. + +## Limitations + +- Only the Python client. `exa-js`, websets, monitors and `beta.agent` are not traced. +- Only verbatim copies of the key, query or URLs are removed from error text; escaped or truncated copies are not. + +## Difference from related integrations + +- [MCP](/docs/integrations/traceai/mcp) traces Model Context Protocol servers and clients. `traceAI-exa` traces direct calls to the Exa API, whether or not an MCP server is involved. +- Exa as a tool inside an agent framework (LangChain, OpenAI Agents and others) is traced by that framework's instrumentor as a tool call. `traceAI-exa` adds the span for the Exa request itself, nested under it. From 9debd91b88a9bfb2053a377f9a382e92f453890c Mon Sep 17 00:00:00 2001 From: Nikhil Pareek Date: Sat, 17 Oct 2026 03:36:59 -0700 Subject: [PATCH 2/5] docs(exa): live vendor verification (2026-10-17), drop stand-in wording Co-Authored-By: Claude Sonnet 5.5 --- src/pages/docs/integrations/traceai/exa.mdx | 18 ++++++++++++++---- 1 file changed, 14 insertions(+), 4 deletions(-) diff --git a/src/pages/docs/integrations/traceai/exa.mdx b/src/pages/docs/integrations/traceai/exa.mdx index ff3fb14bf..ba04837da 100644 --- a/src/pages/docs/integrations/traceai/exa.mdx +++ b/src/pages/docs/integrations/traceai/exa.mdx @@ -7,7 +7,7 @@ description: "Trace Exa search, contents and answer calls in Future AGI with the `traceAI-exa` wraps the official [`exa-py`](https://github.com/exa-labs/exa-py) client so every `search`, `get_contents` and `answer` call becomes a span in Future AGI. It is a client wrapper, not an Exa exporter: Exa does not publish an OpenTelemetry exporter, and the package does not patch the HTTP layer. -The package is Alpha. It is released from [future-agi/traceAI](https://github.com/future-agi/traceAI) (`python/frameworks/exa`). Until a release is on PyPI, install it from that directory of the repository. +The package is released from [future-agi/traceAI](https://github.com/future-agi/traceAI) (`python/frameworks/exa`). Until a release is on PyPI, install it from that directory of the repository. ## 1. Installation @@ -84,15 +84,24 @@ ExaInstrumentor().instrument( With hidden inputs, `input.value` is `__REDACTED__`, the query and URL attributes are dropped, and the query or URLs are removed from error text. Counts stay. `pii_redaction` and `using_session`, `using_user`, `using_metadata` and `using_tags` also apply. +## Live vendor verification (2026-10-17) + +Run against the real Exa API (`api.exa.ai`) with `exa-py` 2.25.0 and `fi-instrumentation-otel` 1.1.0 on Python 3.12, exactly as the steps above describe, exporting to a local Future AGI deployment (project `th8103-rick-exa-live`). + +- Trace `fa0ce113-0c73-48a7-bbd8-6852d6062ddb`: `[agent] research-agent` with children `[retriever] exa.search`, `[tool] exa.get_contents`, `[retriever] exa.answer`, `[retriever] exa.search` (from `stream_search`) and `[retriever] exa.answer` (from `stream_answer`). `exa.result_count` was 1, 1 (`url_count` 1), 8, 0 and 8. +- Trace `edad9ed7-1183-4554-9b15-9aa2e3b20a38`: `AsyncExa.search`. +- Trace `14aa18fd-a5e5-4a5f-a019-3ba6faf7cd7b`: a call with an invalid key ends `ERROR` with `ValueError: Request failed with status code 401` and the server's `INVALID_API_KEY` message; the span has an exception event without a stack trace. +- `fi_verify` passed (hierarchy, span kinds, the `ERROR` span). Exa returns no model or token usage, so none is recorded. +- Privacy: with defaults the query appears in the stored spans. With `FI_HIDE_INPUTS=true` (traces `967cfabf-72c6-4813-86bc-9b13de89bfc2`, `2fc3dcb4-bb86-4a8d-8bb4-ec8c50f247dc`) the query marker appears in none of the stored columns and `input.value` is `__REDACTED__`. The Exa key, local paths, the user name and stack traces appear in no row in either mode. +- Observed: `stream_search` with `num_results=1` returned no chunks from the live API, so its span has a result count of 0; `stream_answer` yielded text chunks and a final citations chunk, and the span counted 8 citations. + ## Tested versions | Component | Version | |---|---| | `exa-py` | 2.25.0 | | `traceAI-exa` | 0.1.0 | -| Python | 3.10, 3.12, 3.13 | - -Verified against a local Future AGI deployment with the real `exa-py` client calling a local stand-in for the Exa API (`api.exa.ai` was not called). The hierarchy, span kinds, an `ERROR` span and the privacy switch were checked in the stored spans. +| Python | 3.10, 3.12, 3.13 (unit tests); 3.12 (live run) | ## Troubleshooting @@ -103,6 +112,7 @@ Verified against a local Future AGI deployment with the real `exa-py` client cal ## Limitations +- `find_similar` and `find_similar_and_contents` are not traced. - Only the Python client. `exa-js`, websets, monitors and `beta.agent` are not traced. - Only verbatim copies of the key, query or URLs are removed from error text; escaped or truncated copies are not. From 2026e7357c904363055988ce20fd7f1894734148 Mon Sep 17 00:00:00 2001 From: Nikhil Pareek Date: Wed, 21 Oct 2026 16:27:22 -0700 Subject: [PATCH 3/5] docs(exa): drop the pre-release install line; note exa-py 2.26.0 and aclose() (TH-8324) Removes 'until a release is on PyPI, install from the repository', which would be stale once the page is live. exa-py 2.26.0 (2026-10-19) adds its own async aclose(); the package suite passes on it (traceAI#205), so the tested-versions text says so, and the stream troubleshooting entry names aclose() for AsyncExa streams. Co-Authored-By: Claude Opus 5.5 --- src/pages/docs/integrations/traceai/exa.mdx | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/src/pages/docs/integrations/traceai/exa.mdx b/src/pages/docs/integrations/traceai/exa.mdx index ba04837da..cba6f6672 100644 --- a/src/pages/docs/integrations/traceai/exa.mdx +++ b/src/pages/docs/integrations/traceai/exa.mdx @@ -7,7 +7,7 @@ description: "Trace Exa search, contents and answer calls in Future AGI with the `traceAI-exa` wraps the official [`exa-py`](https://github.com/exa-labs/exa-py) client so every `search`, `get_contents` and `answer` call becomes a span in Future AGI. It is a client wrapper, not an Exa exporter: Exa does not publish an OpenTelemetry exporter, and the package does not patch the HTTP layer. -The package is released from [future-agi/traceAI](https://github.com/future-agi/traceAI) (`python/frameworks/exa`). Until a release is on PyPI, install it from that directory of the repository. +The package is released from [future-agi/traceAI](https://github.com/future-agi/traceAI) (`python/frameworks/exa`). ## 1. Installation @@ -15,7 +15,7 @@ The package is released from [future-agi/traceAI](https://github.com/future-agi/ pip install traceAI-exa ``` -Tested with `exa-py` 2.25.0 (the latest release when this page was written, 2026-10-16) on Python 3.10, 3.12 and 3.13. The package accepts `exa-py>=2.25.0,<3`. +Tested with `exa-py` 2.25.0 (the latest release when this page was written, 2026-10-16) on Python 3.10, 3.12 and 3.13. The package's test suite also passes on `exa-py` 2.26.0 (released 2026-10-19). The package accepts `exa-py>=2.25.0,<3`. ## 2. Set environment variables @@ -99,7 +99,7 @@ Run against the real Exa API (`api.exa.ai`) with `exa-py` 2.25.0 and `fi-instrum | Component | Version | |---|---| -| `exa-py` | 2.25.0 | +| `exa-py` | 2.25.0 (unit tests and live run); 2.26.0 (unit tests) | | `traceAI-exa` | 0.1.0 | | Python | 3.10, 3.12, 3.13 (unit tests); 3.12 (live run) | @@ -108,7 +108,7 @@ Run against the real Exa API (`api.exa.ai`) with `exa-py` 2.25.0 and `fi-instrum - **No spans appear.** Check `FI_API_KEY` and `FI_SECRET_KEY`, and that `instrument()` runs before the first Exa call. - **Duplicate spans.** Do not combine this package with another instrumentor that wraps the same Exa client methods. - **`ValueError` when creating `Exa()`.** The client raises when no API key is set, before any traced call, so no span is recorded. -- **Stream span stays open.** Read the stream to the end or call `close()`. +- **Stream span stays open.** Read the stream to the end or close it: `close()` on an `Exa` stream, `await stream.aclose()` on an `AsyncExa` stream. ## Limitations From bc899cbe5d6cec79daac92a8b2023ed9eb1eea9e Mon Sep 17 00:00:00 2001 From: Nikhil Pareek Date: Thu, 5 Nov 2026 21:14:18 -0800 Subject: [PATCH 4/5] docs(exa): publish a customer-ready Exa guide traceAI-exa 0.1.0 is released, so the guide no longer needs the dated live-run section, trace IDs, internal project name or tested-versions table that read like a test diary. The page now states the Python and exa-py ranges from the package metadata, flushes spans at the end of the example so a short script exports them, adds a short view-the-trace step and explains the get_contents URL opt-in. It registers the route in the shared integrations catalog, which is how docs navigation and discovery now work, and adds a page contract test so the diary does not come back. Co-Authored-By: Claude Opus 5.5 --- scripts/exa-page.test.mjs | 24 +++++++++++ scripts/integrations.test.mjs | 1 + src/lib/integrations.mjs | 1 + src/pages/docs/integrations/traceai/exa.mdx | 45 ++++++++------------- 4 files changed, 42 insertions(+), 29 deletions(-) create mode 100644 scripts/exa-page.test.mjs diff --git a/scripts/exa-page.test.mjs b/scripts/exa-page.test.mjs new file mode 100644 index 000000000..baca64c49 --- /dev/null +++ b/scripts/exa-page.test.mjs @@ -0,0 +1,24 @@ +import test from 'node:test'; +import assert from 'node:assert/strict'; +import { readFileSync } from 'node:fs'; +import { fileURLToPath } from 'node:url'; +import { integrations } from '../src/lib/integrations.mjs'; + +const page = readFileSync(fileURLToPath(new URL('../src/pages/docs/integrations/traceai/exa.mdx', import.meta.url)), 'utf8'); + +test('Exa guide is discoverable and installs the released client wrapper', () => { + const entry = integrations.find(({ id }) => id === 'exa'); + assert.equal(entry?.href, '/docs/integrations/traceai/exa'); + assert.equal(entry?.method, 'traceAI SDK'); + for (const text of [ + 'pip install traceAI-exa', 'exa-py>=2.25.0,<3', 'fi-instrumentation-otel>=1.1.0', + 'ExaInstrumentor().instrument(tracer_provider=trace_provider)', 'trace_provider.force_flush()', + ]) assert.ok(page.includes(text), text); +}); + +test('Exa guide keeps privacy controls and no test diary', () => { + for (const text of ['FI_HIDE_INPUTS=true', 'TraceConfig(hide_inputs=True)', 'capture_urls=True', '[redacted]']) { + assert.ok(page.includes(text), text); + } + assert.doesNotMatch(page, /Live vendor verification|verified live|Tested versions|live run|fi_verify|trace `?[0-9a-f]{8}-/i); +}); diff --git a/scripts/integrations.test.mjs b/scripts/integrations.test.mjs index 30ff909ed..caadcbfa1 100644 --- a/scripts/integrations.test.mjs +++ b/scripts/integrations.test.mjs @@ -1,6 +1,7 @@ import './ollama-java.test.mjs'; import './azure-openai-java.test.mjs'; import './langsmith-page.test.mjs'; +import './exa-page.test.mjs'; import './vercel-platform-page.test.mjs'; import './scalekit-page.test.mjs'; import test from 'node:test'; diff --git a/src/lib/integrations.mjs b/src/lib/integrations.mjs index df0f390a2..075a95865 100644 --- a/src/lib/integrations.mjs +++ b/src/lib/integrations.mjs @@ -87,6 +87,7 @@ export const integrations = [ { id: 'instructor', name: 'Instructor', href: '/docs/integrations/traceai/instructor', category: 'frameworks', summary: 'Trace structured-output requests and validation.', method: 'traceAI SDK', logo: 'instructor' }, { id: 'promptflow', name: 'Prompt Flow', href: '/docs/integrations/traceai/promptflow', category: 'frameworks', summary: 'Trace OpenAI calls inside Prompt Flow pipelines.', method: 'OpenAI client', logo: 'promptflow', aliases: ['PromptFlow'] }, { id: 'guardrails', name: 'Guardrails AI', href: '/docs/integrations/traceai/guardrails', category: 'frameworks', summary: 'Capture validation and guard execution spans.', method: 'traceAI SDK', logo: 'guardrails' }, + { id: 'exa', name: 'Exa', href: '/docs/integrations/traceai/exa', category: 'frameworks', summary: 'Trace search, contents and answer calls from the Python client.', method: 'traceAI SDK', logo: 'generic-framework', aliases: ['Exa search', 'exa-py', 'traceAI-exa'] }, { id: 'fastmcp-python', name: 'FastMCP Python', href: '/docs/integrations/frameworks/fastmcp', category: 'frameworks', summary: 'Export native Python MCP client and server spans.', method: 'Native OTLP', logo: 'mcp' }, { id: 'mcp-python', name: 'MCP for Python', href: '/docs/integrations/traceai/mcp-python', category: 'frameworks', summary: 'Export native MCP 2.x client requests and server tool spans.', method: 'Native OTLP', logo: 'mcp', aliases: ['Model Context Protocol Python'] }, { id: 'mcp', name: 'Model Context Protocol', href: '/docs/integrations/traceai/mcp', category: 'frameworks', summary: 'Propagate context with TypeScript and existing Python MCP integrations.', method: 'traceAI SDK', logo: 'mcp', aliases: ['MCP'] }, diff --git a/src/pages/docs/integrations/traceai/exa.mdx b/src/pages/docs/integrations/traceai/exa.mdx index cba6f6672..7406a47dd 100644 --- a/src/pages/docs/integrations/traceai/exa.mdx +++ b/src/pages/docs/integrations/traceai/exa.mdx @@ -3,19 +3,17 @@ title: "Exa Integration with Future AGI for Search Tracing" description: "Trace Exa search, contents and answer calls in Future AGI with the traceAI-exa instrumentor for the exa-py client." --- -## About - `traceAI-exa` wraps the official [`exa-py`](https://github.com/exa-labs/exa-py) client so every `search`, `get_contents` and `answer` call becomes a span in Future AGI. It is a client wrapper, not an Exa exporter: Exa does not publish an OpenTelemetry exporter, and the package does not patch the HTTP layer. -The package is released from [future-agi/traceAI](https://github.com/future-agi/traceAI) (`python/frameworks/exa`). - ## 1. Installation +Use Python 3.10 through 3.13. + ```bash pip install traceAI-exa ``` -Tested with `exa-py` 2.25.0 (the latest release when this page was written, 2026-10-16) on Python 3.10, 3.12 and 3.13. The package's test suite also passes on `exa-py` 2.26.0 (released 2026-10-19). The package accepts `exa-py>=2.25.0,<3`. +It accepts `exa-py>=2.25.0,<3` and `fi-instrumentation-otel>=1.1.0`. ## 2. Set environment variables @@ -44,9 +42,15 @@ ExaInstrumentor().instrument(tracer_provider=trace_provider) exa = Exa(api_key=os.environ["EXA_API_KEY"]) exa.search("recent retrieval research", num_results=5) + +trace_provider.force_flush() ``` -Call `register()` first so the instrumentor has a tracer provider. A script that only calls `register()` exports the Exa span as a root trace. Under an active span (for example an agent span) the Exa span is its child. +Call `register()` before `instrument()` so the instrumentor has a tracer provider. Each Exa call is a root span on its own, or a child of the active span (for example an agent span) when there is one. `force_flush()` sends buffered spans before a short script exits. + +## 4. View the trace + +Open **Observe** in Future AGI and select the `exa-search` project. Each call appears as an `exa.search`, `exa.answer` or `exa.get_contents` span. ## What is captured @@ -66,9 +70,9 @@ Call `register()` first so the instrumentor has a tracer provider. A script that | `exa.urls` | Only with `instrument(capture_urls=True)`: at most 20 URLs. | | `exa.cancelled` | `true` when a call or stream was cancelled or closed early. | -An Exa error (for example an invalid key, HTTP 401) ends the span with status `ERROR` and the message `ValueError: Request failed with status code 401: ...`; your code still receives the exception unchanged. +An Exa error, such as an invalid API key, ends the span with status `ERROR` and records the exception type and message. Your code still receives the exception unchanged. -Not captured: result titles, page text, highlights, answer text, model name, token counts and cost (Exa does not return model tokens for search). The Exa API key is never recorded, and no stack trace or local path is exported. +Not captured: result titles, page text, highlights, answer text, model name, token counts and cost. The Exa API key is never recorded: wherever it appears verbatim in a recorded query, URL or error message, it is replaced by `[redacted]`. No stack trace or local path is exported. ## Privacy @@ -84,28 +88,11 @@ ExaInstrumentor().instrument( With hidden inputs, `input.value` is `__REDACTED__`, the query and URL attributes are dropped, and the query or URLs are removed from error text. Counts stay. `pii_redaction` and `using_session`, `using_user`, `using_metadata` and `using_tags` also apply. -## Live vendor verification (2026-10-17) - -Run against the real Exa API (`api.exa.ai`) with `exa-py` 2.25.0 and `fi-instrumentation-otel` 1.1.0 on Python 3.12, exactly as the steps above describe, exporting to a local Future AGI deployment (project `th8103-rick-exa-live`). - -- Trace `fa0ce113-0c73-48a7-bbd8-6852d6062ddb`: `[agent] research-agent` with children `[retriever] exa.search`, `[tool] exa.get_contents`, `[retriever] exa.answer`, `[retriever] exa.search` (from `stream_search`) and `[retriever] exa.answer` (from `stream_answer`). `exa.result_count` was 1, 1 (`url_count` 1), 8, 0 and 8. -- Trace `edad9ed7-1183-4554-9b15-9aa2e3b20a38`: `AsyncExa.search`. -- Trace `14aa18fd-a5e5-4a5f-a019-3ba6faf7cd7b`: a call with an invalid key ends `ERROR` with `ValueError: Request failed with status code 401` and the server's `INVALID_API_KEY` message; the span has an exception event without a stack trace. -- `fi_verify` passed (hierarchy, span kinds, the `ERROR` span). Exa returns no model or token usage, so none is recorded. -- Privacy: with defaults the query appears in the stored spans. With `FI_HIDE_INPUTS=true` (traces `967cfabf-72c6-4813-86bc-9b13de89bfc2`, `2fc3dcb4-bb86-4a8d-8bb4-ec8c50f247dc`) the query marker appears in none of the stored columns and `input.value` is `__REDACTED__`. The Exa key, local paths, the user name and stack traces appear in no row in either mode. -- Observed: `stream_search` with `num_results=1` returned no chunks from the live API, so its span has a result count of 0; `stream_answer` yielded text chunks and a final citations chunk, and the span counted 8 citations. - -## Tested versions - -| Component | Version | -|---|---| -| `exa-py` | 2.25.0 (unit tests and live run); 2.26.0 (unit tests) | -| `traceAI-exa` | 0.1.0 | -| Python | 3.10, 3.12, 3.13 (unit tests); 3.12 (live run) | +`get_contents` records only the number of requested URLs by default, because URLs can carry tokens or personal data. Pass `capture_urls=True` to `instrument()` to record up to 20 of them; hidden inputs turn URL capture off. ## Troubleshooting -- **No spans appear.** Check `FI_API_KEY` and `FI_SECRET_KEY`, and that `instrument()` runs before the first Exa call. +- **No spans appear.** Check `FI_API_KEY` and `FI_SECRET_KEY`, that `instrument()` runs before the first Exa call, and that a short script calls `trace_provider.force_flush()` before it exits. - **Duplicate spans.** Do not combine this package with another instrumentor that wraps the same Exa client methods. - **`ValueError` when creating `Exa()`.** The client raises when no API key is set, before any traced call, so no span is recorded. - **Stream span stays open.** Read the stream to the end or close it: `close()` on an `Exa` stream, `await stream.aclose()` on an `AsyncExa` stream. @@ -116,7 +103,7 @@ Run against the real Exa API (`api.exa.ai`) with `exa-py` 2.25.0 and `fi-instrum - Only the Python client. `exa-js`, websets, monitors and `beta.agent` are not traced. - Only verbatim copies of the key, query or URLs are removed from error text; escaped or truncated copies are not. -## Difference from related integrations +## Related integrations - [MCP](/docs/integrations/traceai/mcp) traces Model Context Protocol servers and clients. `traceAI-exa` traces direct calls to the Exa API, whether or not an MCP server is involved. -- Exa as a tool inside an agent framework (LangChain, OpenAI Agents and others) is traced by that framework's instrumentor as a tool call. `traceAI-exa` adds the span for the Exa request itself, nested under it. +- When Exa runs as a tool inside an agent framework (LangChain, OpenAI Agents and others), that framework's instrumentor traces the tool call. `traceAI-exa` adds the span for the Exa request itself, nested under it. From 84930d0e6028da3e97c5c3936683e7ddcf32721a Mon Sep 17 00:00:00 2001 From: Nikhil Pareek Date: Thu, 5 Nov 2026 22:24:18 -0800 Subject: [PATCH 5/5] docs(exa): apply privacy settings before instrumentation Explain that the privacy-configured call replaces the initial instrumentation in step 3, and that FI_HIDE_INPUTS must be set before the first call. Repeated instrument calls and later environment changes do not update the released instrumentor's existing wrappers. Add page-contract coverage for the setup ordering and replacement guidance. Co-Authored-By: Claude Code --- scripts/exa-page.test.mjs | 12 ++++++++++++ src/pages/docs/integrations/traceai/exa.mdx | 6 +++++- 2 files changed, 17 insertions(+), 1 deletion(-) diff --git a/scripts/exa-page.test.mjs b/scripts/exa-page.test.mjs index baca64c49..bf650cd46 100644 --- a/scripts/exa-page.test.mjs +++ b/scripts/exa-page.test.mjs @@ -16,6 +16,18 @@ test('Exa guide is discoverable and installs the released client wrapper', () => ]) assert.ok(page.includes(text), text); }); +test('Exa privacy settings apply before the first instrumentation, not as a second call', () => { + const setup = page.split('## 3. Register and instrument')[1].split('## 4. View the trace')[0]; + const privacy = page.split('## Privacy')[1].split('## Troubleshooting')[0]; + + assert.match(setup, /\[privacy settings\]\(#privacy\) before the first `instrument\(\)` call/); + assert.ok(setup.indexOf('[privacy settings](#privacy)') < setup.indexOf('ExaInstrumentor().instrument(')); + assert.match(privacy, /replace the `ExaInstrumentor\(\)\.instrument\(\.\.\.\)` call in step 3 with the following, before making any Exa requests/); + assert.match(privacy, /`FI_HIDE_INPUTS=true` before the first `instrument\(\)` call/); + assert.match(privacy, /Calling `instrument\(\)` again or changing the environment afterward does not reconfigure existing wrappers/); + assert.match(privacy, /restart your application with the new privacy settings/); +}); + test('Exa guide keeps privacy controls and no test diary', () => { for (const text of ['FI_HIDE_INPUTS=true', 'TraceConfig(hide_inputs=True)', 'capture_urls=True', '[redacted]']) { assert.ok(page.includes(text), text); diff --git a/src/pages/docs/integrations/traceai/exa.mdx b/src/pages/docs/integrations/traceai/exa.mdx index 7406a47dd..7ebd51c8d 100644 --- a/src/pages/docs/integrations/traceai/exa.mdx +++ b/src/pages/docs/integrations/traceai/exa.mdx @@ -27,6 +27,8 @@ os.environ["FI_SECRET_KEY"] = "your-futureagi-secret-key" ## 3. Register and instrument +The example below records search queries. To hide them, apply the [privacy settings](#privacy) before the first `instrument()` call. + ```python import os from exa_py import Exa @@ -76,7 +78,7 @@ Not captured: result titles, page text, highlights, answer text, model name, tok ## Privacy -traceAI instrumentors follow the standard `TraceConfig`: the query is recorded by default. To turn it off, set `FI_HIDE_INPUTS=true` or pass a config: +The query is recorded by default. To hide inputs, replace the `ExaInstrumentor().instrument(...)` call in step 3 with the following, before making any Exa requests: ```python from fi_instrumentation import TraceConfig @@ -86,6 +88,8 @@ ExaInstrumentor().instrument( ) ``` +Alternatively, set `FI_HIDE_INPUTS=true` before the first `instrument()` call. Calling `instrument()` again or changing the environment afterward does not reconfigure existing wrappers. If you have already instrumented the client, restart your application with the new privacy settings. + With hidden inputs, `input.value` is `__REDACTED__`, the query and URL attributes are dropped, and the query or URLs are removed from error text. Counts stay. `pii_redaction` and `using_session`, `using_user`, `using_metadata` and `using_tags` also apply. `get_contents` records only the number of requested URLs by default, because URLs can carry tokens or personal data. Pass `capture_urls=True` to `instrument()` to record up to 20 of them; hidden inputs turn URL capture off.