Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
5936e09
test(harness): add shared OTLP harness for TH-8103 contract tests
nik13 Oct 4, 2026
d3134a9
feat(exa): add traceAI-exa instrumentor for TH-8324
nik13 Oct 4, 2026
32f5c9d
test(harness): accept the collector /tracer/v1/traces path
nik13 Oct 4, 2026
3eaadc8
fix(harness): decode chunked OTLP bodies and record request metadata
nik13 Oct 4, 2026
5969514
Merge harness 3eaadc8 (chunked bodies, Receiver.requests()) into feat…
nik13 Oct 4, 2026
4610564
test(exa): collector contract through the shared harness
nik13 Oct 4, 2026
292daab
fix(exa): redact the Exa key read from the client headers
nik13 Oct 5, 2026
1d3e822
fix(exa): cap the query at 1 KB of UTF-8 on a character boundary
nik13 Oct 5, 2026
f41a29f
feat(exa): set input.value to the recorded query
nik13 Oct 5, 2026
7d23275
feat(exa): record a get_contents URL count; URLs only with capture_urls
nik13 Oct 5, 2026
5051471
fix(exa): count answer citations and accumulated stream citations
nik13 Oct 5, 2026
28fd0da
fix(exa): make the Exa span current; keep the vendor stream type
nik13 Oct 5, 2026
875ab0b
feat(exa): mark cancelled calls and streams with exa.cancelled
nik13 Oct 5, 2026
a93881f
fix(exa): end async stream spans on close() and release on aclose()
nik13 Oct 5, 2026
48a92b3
test(exa): real-SDK coverage for AsyncExa, streams and uninstrument
nik13 Oct 5, 2026
04182a9
build(exa): accept exa-py >=2.25.0,<3 instead of an exact pin
nik13 Oct 5, 2026
633dc91
build(exa): require fi-instrumentation-otel >=1.1.0
nik13 Oct 5, 2026
8cc8979
test(exa): run examples/search.py through harness.run
nik13 Oct 5, 2026
fdd936d
test(exa): require STATUS_CODE_OK on successful contract spans
nik13 Oct 5, 2026
f131f6d
build(exa): list only the tested Python versions in classifiers
nik13 Oct 5, 2026
81841c4
docs(exa): document attributes, URL capture, key redaction, cancellation
nik13 Oct 5, 2026
ac068a8
docs: list traceAI-exa in the README package tables
nik13 Oct 5, 2026
6b8d324
fix(exa): honour TraceConfig/FI_HIDE_INPUTS via FITracer; key-safe, c…
nik13 Oct 6, 2026
c35945b
build(exa): require Python >=3.10, the lowest tested version
nik13 Oct 6, 2026
db12b3a
docs(exa): trace config, error text, aclose and project_type
nik13 Oct 6, 2026
997f9f5
fix(exa): end stream spans still open at interpreter exit
nik13 Oct 6, 2026
fd1c109
fix(exa): end the span when str(error) fails; hide get_contents URLs …
nik13 Oct 6, 2026
338560d
docs(exa): name URLs and normalised echoes in the error-text limit
nik13 Oct 6, 2026
9a47981
chore(exa): declare Python 3.12 and add the package changelog
nik13 Oct 7, 2026
6024014
fix(exa): align span keys with the tool tracer family and harden wrap…
nik13 Oct 12, 2026
c80662c
docs: move the README catalog row to the TH-8103 catalog PR
nik13 Oct 12, 2026
b44e450
fix(exa): do not export exception stack traces (local paths)
nik13 Oct 17, 2026
13bb6b0
docs(exa): record live vendor verification (2026-10-17), note find_si…
nik13 Oct 17, 2026
9afda66
Merge dev into feat/th-8324-exa
nik13 Oct 21, 2026
c5ea631
fix(exa): exa-py 2.26.0 adds its own async aclose(); keep the test an…
nik13 Oct 21, 2026
27d9bb1
docs(exa): say the suite also passes on exa-py 2.26.0 (TH-8324)
nik13 Oct 21, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 26 additions & 0 deletions python/frameworks/exa/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# Changelog

All notable changes to `traceAI-exa` will be documented here.

## [0.1.0] - Unreleased

### Added
- `ExaInstrumentor` for the `exa-py` client (`exa-py>=2.25.0,<3`): `search`, `get_contents`, `answer`,
the deprecated `search_and_contents` alias, their async twins, and `stream_search` / `stream_answer`.
- Spans record the redacted query (capped at 1024 UTF-8 bytes) and document or citation counts; response
bodies and the API key are never recorded.
- `get_contents` records `exa.url_count`; `instrument(capture_urls=True)` adds up to 20 redacted URLs in `exa.urls`.
- `TraceConfig` / `FI_*` environment support (`FI_HIDE_INPUTS`, PII redaction, `using_session` and friends).
- Key-safe error text capped at 1 KB, cancelled-call handling, and export of streams still open at
interpreter exit.
- Spans use the repo's keys: `gen_ai.span.kind` (`RETRIEVER` for search, answer and streams; `TOOL` for
`get_contents`), the query in `gen_ai.retrieval.query`, and counts in `exa.result_count` / `exa.url_count`.
- Only methods defined on each client class are wrapped, so `AsyncExa` can never wrap an inherited `Exa`
method twice; a tracer that fails to start a span leaves the call untraced instead of raising.
- The exception event has `exception.type` and `exception.message` only; no stack trace (local paths) is exported.
- `pii_redaction` also applies to the error status and the exception event.

### Verified
- 2026-10-17: live run against api.exa.ai with exa-py 2.25.0; response shapes matched, no code change needed.
- 2026-10-21: the test suite also passes on exa-py 2.26.0, which adds its own async `aclose()`; a traced
stream's `aclose()` delegates to it (no live run on 2.26.0).
143 changes: 143 additions & 0 deletions python/frameworks/exa/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,143 @@
# traceAI-exa

OpenTelemetry instrumentation for the [`exa-py`](https://github.com/exa-labs/exa-py) client.

This package wraps Exa client calls; it is not an Exa exporter. Exa does not publish an exporter.

## Installation

```bash
pip install traceAI-exa
```

It accepts `exa-py>=2.25.0,<3` and is tested with `exa-py` 2.25.0 on Python
3.10, 3.11, 3.12 and 3.13. The test suite also passes on `exa-py` 2.26.0.

Live vendor verification (2026-10-17): a real run against `api.exa.ai` with
`exa-py` 2.25.0 (`search`, `get_contents`, `answer`, both streams, `AsyncExa.search`
and an invalid-key 401 recorded as ERROR) exported the expected spans to Future
AGI, and `FI_HIDE_INPUTS=true` kept the query out of every stored column.

## Usage

```python
from exa_py import Exa
from fi_instrumentation import register
from fi_instrumentation.fi_types import ProjectType
from traceai_exa import ExaInstrumentor

tracer_provider = register(project_name="exa-search", project_type=ProjectType.OBSERVE)
ExaInstrumentor().instrument(tracer_provider=tracer_provider)

client = Exa(api_key="your-exa-api-key")
client.search("recent retrieval research", num_results=5)
```

## Instrumented calls

`Exa` and `AsyncExa` calls to `search`, `get_contents`, `answer`, and the
deprecated `search_and_contents` alias (traced as `exa.search`) each produce
one span. `stream_search` and `stream_answer` produce one span that stays open
until the stream is fully read, closed, or garbage-collected; the returned
object is still an instance of the vendor's stream class. A stream still open
when the interpreter exits is ended as cancelled before `register()`'s exit
hook shuts the exporter down (when `instrument()` runs after `register()`).

`gen_ai.span.kind` is `RETRIEVER` for `search`, `answer` and the two streams,
and `TOOL` for `get_contents`, which fetches the pages you name rather than
searching. Each span is a child of the active span
when there is one, and a root span otherwise. The span is current while
exa-py sends the request, so a span from an HTTP client instrumentation nests
under it.

## Span attributes

| Attribute | Calls | Value |
|---|---|---|
| `gen_ai.retrieval.query`, `input.value` | `search`, `answer`, streams | The query, with the client's Exa API key replaced by `[redacted]`, cut to 1 KB of UTF-8 on a character boundary. |
| `exa.url_count` | `get_contents` | Number of requested URLs (a single URL, a list of URLs, or a list of results). |
| `exa.urls` | `get_contents` with `capture_urls=True` | See [get_contents URLs](#get_contents-urls). |
| `exa.result_count` | successful calls | `results` for `search` and `get_contents`, `citations` for `answer`, and the citations summed over every chunk of a fully read stream. Absent when the count is unknown (an error or a cancellation), never 0 for unknown. |
| `exa.cancelled` | cancelled calls and streams | `true`; see [Errors and cancellation](#errors-and-cancellation). |

The spans never carry response content: no result titles, text, highlights or
summaries, no answer or stream text, no cost, token count or model name. There
is no setting that adds them. The Exa API key is never recorded; it is
replaced by `[redacted]` wherever it appears verbatim in a recorded query, URL
or error text.

### get_contents URLs

`get_contents` spans record the number of requested URLs, not the URLs,
because URLs can carry tokens or personal data. To record the URLs as well,
opt in when instrumenting:

```python
ExaInstrumentor().instrument(tracer_provider=tracer_provider, capture_urls=True)
```

With `capture_urls=True`, `exa.urls` holds at most the first 20
requested URLs, each with the Exa API key replaced by `[redacted]` and cut to
1 KB. Query strings are otherwise kept as given. Hidden inputs (below) turn
URL capture off.

## Trace config and context

Spans are created through `fi_instrumentation`'s `FITracer`, as in the other
traceAI instrumentors. Pass a `TraceConfig` to `instrument()`, or set the
matching `FI_*` environment variables; without `config`, the environment is
read when `instrument()` runs.

```python
from fi_instrumentation import TraceConfig

ExaInstrumentor().instrument(
tracer_provider=tracer_provider, config=TraceConfig(hide_inputs=True)
)
```

With `hide_inputs=True` (or `FI_HIDE_INPUTS=true`), `input.value` is
`__REDACTED__`, `gen_ai.retrieval.query` and `exa.urls` are not recorded,
and the query or the requested URLs are replaced by `__REDACTED__` in error
text. The URL and result counts stay. `pii_redaction` applies to the recorded
attributes and to the error text described below, and
`using_session`, `using_user`, `using_metadata` and `using_tags` attributes are
added to the Exa spans.

## Errors and cancellation

An error raised by exa-py (for example `ValueError` for an HTTP 4xx/5xx
response) sets the span status to ERROR and records an exception event. The
exception is re-raised unchanged.

exa-py puts the server's response body in its error message, and a body can
repeat the request. On the span, the status description
(`<ExceptionType>: <message>`) and the event's `exception.message` therefore
have the Exa key replaced by `[redacted]` (and, with hidden inputs, the query
or the requested URLs by `__REDACTED__`; with `pii_redaction`, PII by tokens such
as `<EMAIL_ADDRESS>`) and the message cut to 1 KB of UTF-8.
If the exception's own `str()` fails, the span records only its type. The event carries
`exception.type` and `exception.message` only: no `exception.stacktrace` is
exported, because frames hold local file paths, user names and host names.
Chained exceptions' messages are not copied. Only verbatim
copies are found: an escaped, normalised or truncated echo of the key, query or
URLs (for example a JSON-escaped `\/` or a percent-encoded URL) is not.

Closing a stream before its last chunk (`close()` or `aclose()`), dropping it
mid-iteration, or cancelling an `AsyncExa` call or stream ends the span with
status ERROR, description `cancelled`, and `exa.cancelled` = `true`.

A traced `AsyncExa` stream always has `aclose()`, which ends the span and closes
the HTTP response. exa-py's own async streams have `aclose()` from 2.26.0 only;
2.25.0's have none. On both versions their `close()` raises `RuntimeError`
(traced or not; the span is ended first). Code that must also run on exa-py
2.25.0 without this instrumentor can call `aclose()` only when it exists, for
example `aclose = getattr(stream, "aclose", None)`.

A missing API key raises `ValueError` in `Exa()` or `AsyncExa()` itself,
before any traced method runs, so no span is recorded for it.

`find_similar` and `find_similar_and_contents` are not traced.

Do not use this package together with another instrumentor that wraps the same
Exa client methods, or duplicate spans may result.
30 changes: 30 additions & 0 deletions python/frameworks/exa/examples/search.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
"""Run one traced Exa search against the configured Exa endpoint."""

import os

from exa_py import Exa
from fi_instrumentation import register
from fi_instrumentation.fi_types import ProjectType
from traceai_exa import ExaInstrumentor


def main() -> None:
tracer_provider = register(
project_name="exa-example",
project_type=ProjectType.OBSERVE,
batch=False,
verbose=False,
)
ExaInstrumentor().instrument(tracer_provider=tracer_provider)

client_options = {"api_key": "exa-dummy-key"}
base_url = os.getenv("EXA_BASE_URL")
if base_url:
client_options["base_url"] = base_url
client = Exa(**client_options)
client.search("TraceAI Exa instrumentation example", num_results=1)
tracer_provider.force_flush()


if __name__ == "__main__":
main()
34 changes: 34 additions & 0 deletions python/frameworks/exa/pyproject.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
[tool.poetry]
name = "traceAI-exa"
version = "0.1.0"
description = "OpenTelemetry instrumentation for Exa"
authors = ["Future AGI <no-reply@futureagi.com>"]
readme = "README.md"
packages = [
{ include = "traceai_exa" }
]
classifiers = [
"Development Status :: 4 - Beta",
"Intended Audience :: Developers",
"License :: OSI Approved :: Apache Software License",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
"Programming Language :: Python :: 3.13",
"Topic :: Software Development :: Libraries :: Python Modules",
]

[tool.poetry.dependencies]
python = ">=3.10,<3.14"
fi-instrumentation-otel = ">=1.1.0"
exa-py = ">=2.25.0,<3"
wrapt = ">=1.0.0"

[tool.poetry.group.dev.dependencies]
pytest = ">=7.0.0"
pytest-asyncio = ">=0.21.0"

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"
Loading
Loading