Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
84 changes: 84 additions & 0 deletions .github/workflows/ollama-provider-tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
name: OpenAI provider migrations

on:
pull_request:
branches: [main, dev]
paths:
- 'python/frameworks/openai/**'
- 'python/frameworks/vllm/**'
- 'python/fi_instrumentation/**'
- 'python/pyproject.toml'
- '.github/workflows/ollama-provider-tests.yml'
push:
branches: [dev]
paths:
- 'python/frameworks/openai/**'
- 'python/frameworks/vllm/**'
- 'python/fi_instrumentation/**'
- 'python/pyproject.toml'
- '.github/workflows/ollama-provider-tests.yml'
workflow_dispatch:

permissions:
contents: read

jobs:
provider:
name: Ollama and vLLM (OpenAI ${{ matrix.openai }})
runs-on: ubuntu-24.04
timeout-minutes: 10
strategy:
fail-fast: false
matrix:
openai: ['1.69.0', '3.42.0']
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
- uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
with:
python-version: '3.11'
- name: Install SDK and test dependencies
env:
OPENAI_VERSION: ${{ matrix.openai }}
run: >-
python -m pip install -e python -e python/frameworks/openai -e python/frameworks/vllm
"openai==$OPENAI_VERSION" 'pytest==8.4.2' 'pytest-asyncio==0.24.0' 'pytest-socket==0.7.0'
- name: Test attribution over localhost OTLP
env:
FI_BASE_URL: http://127.0.0.1:9
FI_GRPC_URL: http://127.0.0.1:9
run: |
sudo unshare --net -- bash -e -c '
ip link set lo up
"$1" -m pytest "$2/python/frameworks/openai/tests/test_ollama_provider.py" \
"$2/python/frameworks/openai/tests/test_vllm_migration.py" \
-q -o addopts="" -p no:cacheprovider --disable-socket \
--allow-hosts=127.0.0.1 --allow-unix-socket
"$1" -m pytest "$2/python/frameworks/vllm/tests" \
-q -o addopts="" -p no:cacheprovider --disable-socket --allow-unix-socket
' _ "$(command -v python)" "$PWD"

legacy:
name: vLLM legacy (OpenAI 1.0.0)
runs-on: ubuntu-24.04
timeout-minutes: 10
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
- uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
with:
python-version: '3.10'
- name: Install legacy package without the shared instrumentor
run: >-
python -m pip install -e python -e python/frameworks/vllm
'openai==1.0.0' 'httpx==0.27.2' 'pytest==8.4.2'
'pytest-asyncio==0.24.0' 'pytest-socket==0.7.0'
- name: Test legacy SDK with outbound network disabled
run: |
sudo unshare --net -- bash -e -c '
ip link set lo up
"$1" -m pytest "$2/python/frameworks/vllm/tests" \
-q -o addopts="" -p no:cacheprovider --disable-socket --allow-unix-socket
' _ "$(command -v python)" "$PWD"
52 changes: 46 additions & 6 deletions python/frameworks/openai/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,42 @@ export FI_SECRET_KEY="your-future-agi-secret-key"
export OPENAI_API_KEY="your-openai-api-key"
```

## Ollama and self-hosted OpenAI-compatible clients

Keep using this package when your application calls Ollama through the OpenAI
SDK. The official `https://ollama.com/v1` endpoint is attributed to `ollama`
automatically. A local address can host many providers, so set the provider
explicitly for each local or self-hosted client:

```python
from openai import OpenAI
from traceai_openai import OpenAIInstrumentor, set_client_provider

# Use the trace provider configured in the quickstart below.
OpenAIInstrumentor().instrument(tracer_provider=trace_provider)

client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama")
set_client_provider(client, provider="ollama")
```

The override controls `gen_ai.provider.name` on sync and async requests,
including streams, embeddings, Responses and failures. It changes tracing
metadata only; it adds no spans and sends no provider setting to the server.
Use a lowercase provider identifier (1–64 characters, starting with a letter;
letters, digits, `.`, `_` and `-` are accepted), not a URL or secret.

Set the override before making requests. It takes precedence over host detection;
`set_client_provider(client, provider=None)` restores host detection. Other
clients are unaffected. A client returned by `copy()` or `with_options()` is a
separate instance: set its override explicitly too, especially if its endpoint
changes. Unconfigured loopback, proxy and unknown hosts retain the historical
`openai` label; the instrumentor never assumes a local server is Ollama.

For applications using the native `ollama` Python client instead, use
`traceai-ollama`. Do not install both instrumentors for the same API call.
See [Ollama's OpenAI compatibility guide](https://docs.ollama.com/api/openai-compatibility)
for supported endpoints and models.

## Quickstart

### Register Tracer Provider
Expand Down Expand Up @@ -51,12 +87,16 @@ Create the client after calling `OpenAIInstrumentor().instrument(...)`.
```python
from openai import OpenAI

client = OpenAI()
response = client.chat.completions.create(
model="gpt-4o-mini",
messages=[{"role": "user", "content": "Tell me a short joke."}],
)
print(response.choices[0].message.content)
# OpenAI reads OPENAI_API_KEY from your environment.
with OpenAI() as client:
response = client.chat.completions.create(
model="gpt-4o-mini",
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Can you tell me a joke?"},
],
)
print((response.choices[0].message.content or "").strip())
trace_provider.force_flush() # Flush before a short-lived script exits.
```

Expand Down
Loading
Loading