Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
57 changes: 57 additions & 0 deletions .github/workflows/mistralai-python-tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
name: Mistral Python SDK tests

on:
pull_request:
branches: [dev, main]
paths:
- 'python/frameworks/mistralai/**'
- 'python/tests/test_framework_mistralai.py'
- '.github/workflows/mistralai-python-tests.yml'
push:
branches: [dev]
paths:
- 'python/frameworks/mistralai/**'
- '.github/workflows/mistralai-python-tests.yml'
workflow_dispatch:

permissions:
contents: read

jobs:
sdk:
name: Mistral ${{ matrix.sdk }} / Python ${{ matrix.python }}
runs-on: ubuntu-latest
timeout-minutes: 15
strategy:
fail-fast: false
matrix:
include:
- {python: '3.10', sdk: '1.1.0'}
- {python: '3.12', sdk: '1.12.4'}
- {python: '3.10', sdk: '3.6.0'}
- {python: '3.13', sdk: '3.7.0'}
env:
FI_BASE_URL: http://127.0.0.1:9
FI_GRPC_URL: http://127.0.0.1:9
steps:
- uses: actions/checkout@v4
with:
persist-credentials: false
- uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python }}
- name: Build and install the distribution
env:
SDK_VERSION: ${{ matrix.sdk }}
run: |
python -m venv .venv
.venv/bin/pip install build poetry-core pytest==8.4.2 pytest-asyncio==0.24.0 pytest-mock==3.14.0
.venv/bin/python -m build python/frameworks/mistralai --outdir dist
.venv/bin/pip install dist/*.whl "mistralai==$SDK_VERSION"
.venv/bin/pip check
- name: Run SDK, README and OTLP contracts without outbound network
run: |
sudo unshare --net sh -c 'ip link set lo up; exec "$@"' sh \
"$PWD/.venv/bin/python" -I -m pytest --noconftest \
python/frameworks/mistralai/tests python/tests/test_framework_mistralai.py \
-q -o asyncio_default_fixture_loop_scope=function
101 changes: 60 additions & 41 deletions python/frameworks/mistralai/README.md
Original file line number Diff line number Diff line change
@@ -1,67 +1,86 @@
# MistralAI OpenTelemetry Integration
# Mistral AI tracing

## Overview
This integration provides support for using OpenTelemetry with the MistralAI framework. It enables tracing and monitoring of applications built with MistralAI.
Trace Mistral chat completions, embeddings, and conversations in Future AGI, including model names, messages, token usage, tool calls, and errors.

## Installation
## Install

1. **Install traceAI MistralAI**
Use Python 3.10–3.13 and the current Mistral SDK:

```bash
pip install traceAI-mistralai
```sh
pip install traceAI-mistralai 'mistralai>=3.6,<4'
```

Set `FI_API_KEY`, `FI_SECRET_KEY`, and `MISTRAL_API_KEY` in your environment. Use your own Mistral API key for model requests.

### Set Environment Variables
Set up your environment variables to authenticate with FutureAGI
## Trace a completion

```python
import os

os.environ["FI_API_KEY"] = FI_API_KEY
os.environ["FI_SECRET_KEY"] = FI_SECRET_KEY
os.environ["MISTRAL_API_KEY"] = MISTRAL_API_KEY
```

## Quickstart

### Register Tracer Provider
Set up the trace provider to establish the observability pipeline. The trace provider:

```python
from fi_instrumentation import register
from fi_instrumentation.fi_types import ProjectType
from mistralai.client import Mistral
from traceai_mistralai import MistralAIInstrumentor

trace_provider = register(
provider = register(
project_type=ProjectType.OBSERVE,
project_name="mistralai_app"
project_name="mistral-app",
)
MistralAIInstrumentor().instrument(tracer_provider=provider)

try:
with Mistral(api_key=os.environ["MISTRAL_API_KEY"]) as client:
response = client.chat.complete(
model="mistral-small-latest",
messages=[{"role": "user", "content": "Suggest three names for a neighborhood bookstore."}],
)
print(response.choices[0].message.content)
finally:
provider.shutdown()
```

### Configure MistralAI Instrumentation
Instrument the MistralAI client to enable telemetry collection. This step ensures that all interactions with the MistralAI SDK are tracked and monitored.
Open **Observe** in Future AGI and select `mistral-app`. The completion appears as an LLM span with the model and returned usage.

```python
from traceai_mistralai import MistralAIInstrumentor
The integration routes the SDK's native OpenTelemetry spans through your provider. Do not add another Mistral instrumentor or configure a second Mistral exporter. Clients created before instrumentation are supported too. Configure tracing once at startup; finish active requests and close streams before uninstrumenting.

## Stream a response

Use the stream's context manager so the connection and span close even when you stop reading early:

MistralAIInstrumentor().instrument(tracer_provider=trace_provider)
```python
with client.chat.stream(
model="mistral-small-latest",
messages=[{"role": "user", "content": "Describe a cozy bookstore."}],
) as stream:
for event in stream:
print(event.data.choices[0].delta.content or "", end="")
```

### Create MistralAI Components
Set up your MistralAI client with built-in observability.
For async requests, use `await client.chat.complete_async(...)` or `async with await client.chat.stream_async(...)`. Keep the client open until the stream is closed. Token usage is available when the service returns it; an early-closed stream may not receive usage.

## Control captured content

Messages and tool arguments can contain sensitive data. To keep model names, token counts, timing, and error status without recording input or output content, replace the instrumentor call above with:

```python
from mistralai import Mistral

client = Mistral(api_key=os.environ["MISTRAL_API_KEY"])

if __name__ == "__main__":
response = client.agents.complete(
agent_id="agent_id",
messages=[
{"role": "user", "content": "plan a vacation for me in Tbilisi"},
],
)
print(response)
from fi_instrumentation import TraceConfig

MistralAIInstrumentor().instrument(
tracer_provider=provider,
config=TraceConfig(hide_inputs=True, hide_outputs=True),
)
```

This also hides error details by default. Embedding vectors are not captured.

## Supported APIs

- **SDK 3.6–3.x:** native sync/async chat and agent completions, streaming, embeddings, and `beta.conversations.start`, `append`, and `restart`, including their async and streaming variants. Conversation streaming errors and final usage are captured on the native span.
- **SDK 1.1–1.x:** the existing `MistralAIInstrumentor` entry point continues to support chat and agent completions and streams, plus embeddings. Keep `from mistralai import Mistral` for this SDK line. Conversation tracing requires SDK 3.6 or later.
- SDK 1.0, 2.x, and 3.0–3.5 are outside the supported range. Pin one of the ranges above rather than overriding pip's dependency check.

## Troubleshooting

- **`cannot import name Mistral`**: use `from mistralai.client import Mistral` with SDK 3.6 or later. The top-level import belongs to SDK 1.x.
- **Missing streaming spans**: fully consume or explicitly close the stream before flushing or shutting down your tracer provider.
- **Duplicate spans**: remove other Mistral SDK instrumentors. This integration adapts native telemetry rather than adding a second completion span.
2 changes: 1 addition & 1 deletion python/frameworks/mistralai/examples/agent_completions.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
EvalTagType,
ProjectType,
)
from mistralai import Mistral
from mistralai.client import Mistral
from traceai_mistralai import MistralAIInstrumentor

# Configure trace provider with custom evaluation tags
Expand Down
2 changes: 1 addition & 1 deletion python/frameworks/mistralai/examples/chat_completions.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
EvalTagType,
ProjectType,
)
from mistralai import Mistral
from mistralai.client import Mistral
from traceai_mistralai import MistralAIInstrumentor

# Configure trace provider with custom evaluation tags
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
EvalTagType,
ProjectType,
)
from mistralai import Mistral
from mistralai.client import Mistral
from traceai_mistralai import MistralAIInstrumentor

# Configure trace provider with custom evaluation tags
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
EvalTagType,
ProjectType,
)
from mistralai import Mistral
from mistralai.client import Mistral
from traceai_mistralai import MistralAIInstrumentor

# Configure trace provider with custom evaluation tags
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
EvalTagType,
ProjectType,
)
from mistralai import Mistral
from mistralai.client import Mistral
from traceai_mistralai import MistralAIInstrumentor

# Configure trace provider with custom evaluation tags
Expand Down
2 changes: 1 addition & 1 deletion python/frameworks/mistralai/examples/new-1.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

from fi_instrumentation import register
from fi_instrumentation.fi_types import ProjectType
from mistralai import Mistral
from mistralai.client import Mistral
from traceai_mistralai import MistralAIInstrumentor

trace_provider = register(
Expand Down
3 changes: 2 additions & 1 deletion python/frameworks/mistralai/examples/requirements.txt
Original file line number Diff line number Diff line change
@@ -1 +1,2 @@
mistralai
traceAI-mistralai
mistralai>=3.6,<4
Loading
Loading