Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
73 changes: 73 additions & 0 deletions .github/workflows/litellm-python-tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
name: LiteLLM Python compatibility

on:
pull_request:
paths:
- 'python/frameworks/litellm/**'
- 'python/tests/test_framework_litellm.py'
- 'python/tests/harness/**'
- 'python/fi_instrumentation/**'
- '.github/workflows/litellm-python-tests.yml'
push:
branches: [dev]
paths:
- 'python/frameworks/litellm/**'
- 'python/tests/test_framework_litellm.py'
- 'python/tests/harness/**'
- 'python/fi_instrumentation/**'
- '.github/workflows/litellm-python-tests.yml'

permissions:
contents: read

jobs:
contract:
name: LiteLLM (${{ matrix.sdk }}, Python ${{ matrix.python }})
runs-on: ubuntu-24.04
timeout-minutes: 15
strategy:
fail-fast: false
matrix:
include:
- sdk: floor
python: '3.10'
- sdk: latest
python: '3.12'
- sdk: latest
python: '3.13'
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
- uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
with:
python-version: ${{ matrix.python }}
- name: Install integration and selected SDK
env:
SDK: ${{ matrix.sdk }}
run: |
python -m venv .venv
if [ "$SDK" = floor ]; then
# PyPI no longer serves 1.43.0; use its immutable upstream release commit.
.venv/bin/pip install -e python/frameworks/litellm \
'litellm @ git+https://github.com/BerriAI/litellm.git@4603eedcc13bfab2da5a82aed209e6ef496c384d' \
'fi-instrumentation-otel==1.2.2' 'openai==1.40.0' 'httpx==0.27.2' \
pytest pytest-asyncio
else
# Use the same Responses runtime documented for users, not an
# undeclared runner-only MCP dependency.
.venv/bin/pip install -e 'python/frameworks/litellm[responses]' \
--upgrade litellm pytest pytest-asyncio
fi
.venv/bin/pip check
- name: Run with outbound network disabled
run: |
sudo unshare --net -- bash -e -c '
ip link set lo up
export LITELLM_LOCAL_MODEL_COST_MAP=True
export HOME="$2/.test-home"
mkdir -p "$HOME"
"$1" -I -m pytest --noconftest -q -o addopts="" \
"$2/python/frameworks/litellm/tests" \
"$2/python/tests/test_framework_litellm.py"
' _ "$PWD/.venv/bin/python" "$PWD"
18 changes: 18 additions & 0 deletions python/fi_instrumentation/instrumentation/_chat_input.py
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,24 @@ def _content(value: Any, config: "TraceConfig") -> Any:
if detail is not None:
reference["detail"] = detail
result.append({"type": kind, "image_url": reference})
elif kind == "input_audio":
audio = part.get("input_audio")
if not _object(audio):
raise ValueError("invalid audio content")
data, fmt = audio.get("data"), audio.get("format")
if type(data) is not str or type(fmt) is not str:
raise ValueError("invalid audio content")
# Speech carries the same semantic content as text; the binary blob
# is masked when input text is private, not under the image flag.
result.append(
{
"type": kind,
"input_audio": {
"data": _REDACTED if config.hide_input_text else data,
"format": fmt,
},
}
)
else:
# Never trust a caller-supplied modality label we do not understand.
result.append(_REDACTED)
Expand Down
30 changes: 22 additions & 8 deletions python/frameworks/litellm/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,18 +8,26 @@ This integration provides support for using OpenTelemetry with the Litellm frame
1. **Install traceAI Litellm**

```bash
pip install pip install traceAI-litellm
pip install traceAI-litellm
```

For `responses` / `aresponses` on Python 3.10+, install the Responses runtime extra:

### Set Environment Variables
Set up your environment variables to authenticate with FutureAGI
```bash
pip install 'traceAI-litellm[responses]'
```

```python
import os
Current LiteLLM versions import `mcp` on the Responses path, even without MCP tools. The extra supplies that upstream runtime dependency; the base installation remains compatible with older LiteLLM versions without Responses support.


### Set environment variables

os.environ["FI_API_KEY"] = FI_API_KEY
os.environ["FI_SECRET_KEY"] = FI_SECRET_KEY
Set your Future AGI credentials for telemetry and your own OpenAI API key for the examples below. Future AGI credentials do not authenticate model inference; OpenAI bills inference to your OpenAI account. When you use another provider through LiteLLM, configure that provider's credentials instead.

```bash
export FI_API_KEY="your-future-agi-api-key"
export FI_SECRET_KEY="your-future-agi-secret-key"
export OPENAI_API_KEY="your-openai-api-key"
```

## Quickstart
Expand All @@ -38,14 +46,20 @@ trace_provider = register(
```

### Configure Litellm Instrumentation
Instrument the Litellm client to enable telemetry collection. This step ensures that all interactions with the Litellm SDK are tracked and monitored.
Instrument the LiteLLM client to trace `completion` / `acompletion`, `text_completion` / `atext_completion`, `completion_with_retries`, `embedding` / `aembedding`, and `image_generation` / `aimage_generation`. On LiteLLM versions that expose them, `responses` / `aresponses` are also traced (install the Responses extra above). Chat, text completion and Responses streams are traced as you consume them; other SDK endpoints are not covered.

Reported chat/text completion finish reasons are exported in `gen_ai.response.finish_reasons`, ordered by choice index, and on each indexed output message. No finish reason is invented for an unread or partially consumed stream. Tool telemetry excludes MCP authorization, headers and server URLs; provider requests are unchanged. Function schemas are preserved; other built-in tools currently export only their type.

Exhaust streams or explicitly call `close()` / `await aclose()` to finish telemetry and release provider resources promptly. Discarding a stream finalizes its span on garbage collection, but does not guarantee immediate transport cleanup.

```python
from traceai_litellm import LiteLLMInstrumentor

LiteLLMInstrumentor().instrument(tracer_provider=trace_provider)
```

Use one tracing integration for each LiteLLM call. Do not also enable LiteLLM's native OpenTelemetry callback for calls traced by this instrumentor.

### Create Litellm Components
Set up your Litellm client with built-in observability.

Expand Down
Loading
Loading