Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
5936e09
test(harness): add shared OTLP harness for TH-8103 contract tests
nik13 Oct 4, 2026
32f5c9d
test(harness): accept the collector /tracer/v1/traces path
nik13 Oct 4, 2026
3eaadc8
fix(harness): decode chunked OTLP bodies and record request metadata
nik13 Oct 4, 2026
9afeaef
test(voyage): real-SDK and contract tests for traceAI-voyage
nik13 Oct 5, 2026
8565287
test(voyage): cover in-process local models; format tests
nik13 Oct 5, 2026
01c923e
feat(voyage): add traceAI-voyage for voyageai embed and rerank
nik13 Oct 5, 2026
7ead208
docs(voyage): README with attributes, privacy, hosts and limits
nik13 Oct 5, 2026
5c01dc0
docs: list traceAI-voyage in the README package tables
nik13 Oct 5, 2026
b2140ac
build(voyage): declare Python 3.12; the suite runs on 3.10 to 3.13
nik13 Oct 5, 2026
72c6eaf
feat(voyage): record the rerank query and scores by default (PRD J2/A…
nik13 Oct 5, 2026
998d61a
fix(voyage): cap recorded rerank scores at 64 results
nik13 Oct 5, 2026
58c6bd2
fix(voyage): apply pii_redaction to the error status and exception event
nik13 Oct 5, 2026
27094c4
docs(voyage): add CHANGELOG.md with the 0.1.0 entry
nik13 Oct 5, 2026
1645425
test(voyage): pin pii_redaction on server.address, as the README states
nik13 Oct 5, 2026
7a30179
docs: move the README catalog row to the TH-8103 catalog PR
nik13 Oct 12, 2026
9b0adff
fix(voyage): do not export exception.stacktrace (local paths, user name)
nik13 Oct 17, 2026
0078243
docs(voyage): record live vendor verification (2026-10-17)
nik13 Oct 17, 2026
eb0b600
Merge dev into feat/th-8321-voyage-ai
nik13 Oct 21, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions python/frameworks/voyage/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
## [0.1.0] - 2026-10-05
### Feature
- First release. OpenTelemetry instrumentation for the official Voyage AI
Python client (`voyageai>=0.3.7,<1`, `fi-instrumentation-otel>=1.1.0`),
tested on Python 3.10 to 3.13.
- `VoyageInstrumentor` wraps `Client` and `AsyncClient` `embed` (one
`voyage.embed` EMBEDDING span per call) and `rerank` (one `voyage.rerank`
RERANKER span per call). The client's own retries stay inside that span.
- Spans record the model, text and document counts, the requested and returned
embedding dimensions, the host as `server.address`, and the client's
`total_tokens` on `gen_ai.usage.input_tokens` and `gen_ai.usage.total_tokens`.
- The rerank span records the query and the relevance scores by default (PRD
J2 / AC-03). Embed texts and rerank documents are recorded only with
`instrument(capture_content=True)`. TraceConfig `hide_inputs` and
`hide_outputs` drop them. Recorded content is capped at 64 items and 2 KB per
string. Embedding vectors are never recorded.
- The Voyage API key is never exported. `pii_redaction` also covers the error
status and the exception event.
- Errors set status ERROR and record one exception event. A cancelled async
call or a Ctrl-C ends the span as `cancelled`.
- `multimodal_embed` and `contextualized_embed` are not traced in this release.
197 changes: 197 additions & 0 deletions python/frameworks/voyage/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,197 @@
# traceAI-voyage

OpenTelemetry instrumentation for the official
[Voyage AI Python client](https://github.com/voyage-ai/voyageai-python)
(`voyageai`).

`traceAI-openai` does not trace Voyage: it wraps the OpenAI client only and
never sees `voyageai.Client`. Use this package for Voyage embeddings and
reranking.

## Installation

```bash
pip install traceAI-voyage
```

It accepts `voyageai>=0.3.7,<1` (0.3.7 is the first release that routes Atlas
keys) and `fi-instrumentation-otel>=1.1.0`. The suite runs on Python 3.10,
3.11, 3.12 and 3.13 against `voyageai` 0.3.7 and 0.5.0.

## Usage

```python
import voyageai
from fi_instrumentation import register
from fi_instrumentation.fi_types import ProjectType
from traceai_voyage import VoyageInstrumentor

trace_provider = register(project_type=ProjectType.OBSERVE, project_name="my-rag-app")
VoyageInstrumentor().instrument(tracer_provider=trace_provider)

client = voyageai.Client() # reads VOYAGE_API_KEY
client.embed(["What is the capital of France?"], model="voyage-3.5", input_type="query")
client.rerank("capital of France", ["Paris is...", "Berlin is..."], model="rerank-2.5", top_k=1)
```

`examples/embed_and_rerank.py` is a runnable version of this.

The methods are patched on the client classes, so a client built before
`instrument()` is traced too; only calls made while instrumented are traced.
`uninstrument()` restores the original methods.

## Hosts and `base_url`

The instrumentor never sets or changes `base_url`. The Voyage client picks the
host from the API key, and a `base_url` you pass to the client yourself wins.
In `voyageai` 0.3.7 to 0.5.0, keys that start with `al-` (created in MongoDB
Atlas) go to `https://ai.mongodb.com/v1` and other keys (created on the Voyage
platform) to `https://api.voyageai.com/v1`. MongoDB also documents geography
hosts such as `eu.ai.mongodb.com` and `us.ai.mongodb.com`; to use one, pass it
as `base_url` when you construct the client.

The span records only the host the client is using, as `server.address`. A
key sent to the wrong host fails in the client; the span records that error
and nothing retries against another host.

## What is traced

| Call | Span name | `gen_ai.span.kind` |
|---|---|---|
| `Client.embed`, `AsyncClient.embed` | `voyage.embed` | `EMBEDDING` |
| `Client.rerank`, `AsyncClient.rerank` | `voyage.rerank` | `RERANKER` |

One span per call: a list of texts is one span, and the client's own retries
(`max_retries`) happen inside that one span. The span is a child of the active
span when there is one, and it is current while the SDK sends the request, so
a span from an HTTP client instrumentation nests under it. Local models that
`embed` runs in process (`voyageai` 0.5+, for example `voyage-4-nano`) are
traced the same way, without `server.address`; their token count is the one the
client computes locally.

Not traced (no span, not an error): `multimodal_embed`,
`contextualized_embed`, the deprecated module functions
`voyageai.get_embedding(s)` / `aget_embedding(s)`, and the TypeScript client.

## Span attributes

Always recorded when known (never content):

| Attribute | Calls | Value |
|---|---|---|
| `gen_ai.provider.name` | all | `voyage` |
| `gen_ai.operation.name` | all | `embeddings` or `rerank` |
| `gen_ai.request.model` | all | The model; for `embed` without `model`, the SDK default it falls back to. |
| `embedding.model_name` / `reranker.model_name` | embed / rerank | Same model. |
| `server.address` | all | Host from the client's base URL (no path, query or credentials). |
| `voyage.embedding.count` | embed | Number of texts in the call (a bare string is 1). |
| `voyage.input_type`, `voyage.output_dtype`, `voyage.output_dimension` | embed | As passed; absent when not passed. |
| `voyage.embedding.dimension`, `gen_ai.embeddings.dimension.count` | embed, success | Length of the first returned vector; times 8 for the packed `binary` / `ubinary` dtypes. |
| `voyage.rerank.document_count` | rerank | Number of documents sent. |
| `reranker.top_k` | rerank | As passed; absent when not passed. |
| `voyage.rerank.result_count` | rerank, success | Number of results returned. |
| `gen_ai.usage.input_tokens`, `gen_ai.usage.total_tokens` | success | The client's `total_tokens`. Voyage reports one total and every embedding or rerank token is input, so both keys carry it. Never estimated. No cost is set. |
| `voyage.cancelled` | cancelled calls | `true`. |

Usage is written only on the Voyage span, once, on the `gen_ai.usage.*` keys
the Future AGI collector reads. If another instrumentation in your app (for
example a framework that wraps the same embedding call) also records usage for
it, the trace counts those tokens twice.

## Live vendor verification (2026-10-17)

Run against the real Voyage API (`api.voyageai.com`, default host, no `base_url`) with `voyageai` 0.5.0, `voyage-3.5-lite` embeddings and `rerank-2.5-lite`, exported to Future AGI: an `AGENT` span with `voyage.embed` (EMBEDDING, tokens recorded), `voyage.rerank` (RERANKER, tokens recorded) and an invalid-key `voyage.embed` as an `ERROR` span (`AuthenticationError`). Default, `capture_content=True` and `FI_HIDE_*` privacy modes were checked with a marker scan. Not yet run live: the Atlas host (`ai.mongodb.com`), geography hosts and local in-process models.

## Privacy and content capture

By default the rerank span records the query and the relevance scores, as PRD
J2 / AC-03 require, so this default is not a deviation from the spec. Embed
texts and rerank documents are recorded only with `capture_content=True`.
Embedding vectors are never recorded.

| Attribute | Calls | Value | Recorded | Dropped by |
|---|---|---|---|---|
| `reranker.query` | rerank | The query. | By default | `hide_inputs`, `hide_input_text` |
| `input.value` (`input.mime_type` = `text/plain`) | rerank | The query. | By default | `hide_inputs`, `hide_input_text` |
| `output.value` (`output.mime_type` = `application/json`) | rerank | JSON list of `{"index", "relevance_score"}` in the client's order. | By default | `hide_outputs` |
| `input.value` (`application/json`) | rerank | JSON `{"query": ..., "documents": [...]}`, in place of the plain query. | Only with `capture_content=True` | `hide_inputs`, `hide_input_text` |
| `input.value` (`application/json`) | embed | JSON list of the texts. | Only with `capture_content=True` | `hide_inputs`, `hide_input_text` |

Hiding inputs keeps the scores (they are numbers, not input text), and hiding
outputs keeps the query. The counts (`voyage.embedding.count`,
`voyage.rerank.document_count`, `voyage.rerank.result_count`) stay whatever is
hidden. `TraceConfig` reads `FI_HIDE_INPUTS` / `FI_HIDE_OUTPUTS` from the
environment, so `FI_HIDE_INPUTS=true FI_HIDE_OUTPUTS=true` records no content
at all.

To record embed texts and rerank documents too, opt in:

```python
from fi_instrumentation import TraceConfig

VoyageInstrumentor().instrument(
tracer_provider=trace_provider,
capture_content=True,
config=TraceConfig(), # optional; honours hide_inputs / hide_outputs / pii_redaction
)
```

At most 64 texts, 64 documents and 64 scores (the first 64 results, in the
client's order) are recorded, and the query and each text or document are
cut to 2 KB of UTF-8 on a character boundary; `voyage.embedding.count`,
`voyage.rerank.document_count` and `voyage.rerank.result_count` stay exact.

`TraceConfig(pii_redaction=True)` (or `FI_PII_REDACTION=true`) applies
`fi_instrumentation`'s regex PII redaction, which replaces email addresses,
SSNs, card numbers, `sk-`/`pk-` live/test/prod API keys, IPv4 addresses and
phone numbers with tokens such as `<EMAIL_ADDRESS>`. It covers:

- every string span attribute: the query, captured texts and documents, the
scores JSON, and the other attributes above (an IP `server.address` becomes
`<IP_ADDRESS>`);
- the error status description and the `exception` event's
`exception.message`, after the API key is
removed from them.

It does not change span names, event names or `exception.type`. The patterns
match digits, not meaning: a relevance score with 10 or more digits after the
decimal point is rewritten too (for example `0.<CREDIT_CARD>`), which leaves
that `output.value` as invalid JSON.

Embedding vectors are never recorded, with or without capture, and
`hide_embedding_vectors` has nothing to hide here.

The Voyage API key is never recorded. The instrumentor does not read it from
the environment itself; it takes the key the client holds (`client.api_key`,
`client._params["api_key"]`) and a module-level `voyageai.api_key`, and
replaces any occurrence in the recorded query, captured content, error
messages and stack traces with `[redacted]`.

## Errors and cancellation

An error raised by the client (for example `voyageai.error.AuthenticationError`
for HTTP 401, or `voyageai.error.Timeout`) sets the span status to ERROR with
`<ExceptionType>: <message>` and records one `exception` event; the key is
redacted from both, and with `pii_redaction=True` so is PII (see above). The
exception reaches your code unchanged.

Cancelling an `AsyncClient` call (`asyncio.CancelledError`) or interrupting a
blocking call (`KeyboardInterrupt`) ends the span with status ERROR,
description `cancelled` and `voyage.cancelled` = `true`, with no exception
event. Result attributes and usage are absent on error and cancellation, never
written as 0.

The instrumentation never changes what a call returns or raises: a failure
while building attributes or starting the span is logged at debug level and the
Voyage call runs as if uninstrumented.

## Configuration errors

`instrument(config=...)` must be a `fi_instrumentation.TraceConfig` and
`capture_content` a `bool`; anything else raises `TypeError` and wraps
nothing. On an installed `voyageai` outside `>=0.3.7,<1`, `instrument()` logs
an error and wraps nothing (OpenTelemetry's dependency check).

## License

Apache-2.0.
50 changes: 50 additions & 0 deletions python/frameworks/voyage/examples/embed_and_rerank.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
"""Trace one Voyage embed call and one rerank call.

Set FI_API_KEY, FI_SECRET_KEY and VOYAGE_API_KEY, then run this file. The
Voyage client picks its host from the key (Atlas keys go to ai.mongodb.com,
Voyage platform keys to api.voyageai.com). VOYAGE_BASE_URL is only read so a
test can point the client at a local fake; leave it unset otherwise.
"""

import os

import voyageai
from fi_instrumentation import register
from fi_instrumentation.fi_types import ProjectType
from traceai_voyage import VoyageInstrumentor

DOCUMENTS = [
"EXAMPLE-DOC: Voyage embeddings turn text into vectors.",
"EXAMPLE-DOC: A reranker orders documents by relevance to a query.",
]
QUERY = "EXAMPLE-QUERY: which document explains reranking?"


def main() -> None:
tracer_provider = register(
project_name="voyage-example",
project_type=ProjectType.OBSERVE,
batch=False,
verbose=False,
)
VoyageInstrumentor().instrument(tracer_provider=tracer_provider)

# The client reads VOYAGE_API_KEY. base_url=None keeps the client's own
# host choice; the instrumentor never sets one.
client = voyageai.Client(base_url=os.getenv("VOYAGE_BASE_URL"))

embeddings = client.embed(DOCUMENTS, model="voyage-3.5", input_type="document")
print(
"embedded {0} documents, {1} tokens".format(
len(embeddings.embeddings), embeddings.total_tokens
)
)

reranking = client.rerank(QUERY, DOCUMENTS, model="rerank-2.5", top_k=2)
print("top document index: {0}".format(reranking.results[0].index))

tracer_provider.force_flush()


if __name__ == "__main__":
main()
34 changes: 34 additions & 0 deletions python/frameworks/voyage/pyproject.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
[tool.poetry]
name = "traceAI-voyage"
version = "0.1.0"
description = "OpenTelemetry instrumentation for the Voyage AI Python client"
authors = ["Future AGI <no-reply@futureagi.com>"]
license = "Apache-2.0"
readme = "README.md"
packages = [
{ include = "traceai_voyage" }
]
classifiers = [
"Development Status :: 4 - Beta",
"Intended Audience :: Developers",
"License :: OSI Approved :: Apache Software License",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
"Programming Language :: Python :: 3.13",
"Topic :: Software Development :: Libraries :: Python Modules",
]

[tool.poetry.dependencies]
python = ">=3.10,<3.14"
fi-instrumentation-otel = ">=1.1.0"
voyageai = ">=0.3.7,<1"
wrapt = ">=1.15.0"

[tool.poetry.group.dev.dependencies]
pytest = ">=7.0.0"

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"
Loading
Loading