Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
66 changes: 66 additions & 0 deletions .github/workflows/fireworks-python-tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
name: Fireworks Python compatibility

on:
pull_request:
paths:
- 'python/frameworks/fireworks/**'
- 'python/frameworks/openai/**'
- 'python/fi_instrumentation/**'
- 'python/tests/harness/**'
- '.github/workflows/fireworks-python-tests.yml'
push:
branches: [dev]
paths:
- 'python/frameworks/fireworks/**'
- 'python/frameworks/openai/**'
- 'python/fi_instrumentation/**'
- 'python/tests/harness/**'
- '.github/workflows/fireworks-python-tests.yml'

permissions:
contents: read

jobs:
contract:
name: Fireworks (openai ${{ matrix.sdk }})
runs-on: ubuntu-24.04
timeout-minutes: 15
strategy:
fail-fast: false
matrix:
include:
- sdk: '1.0.0'
python: '3.10'
- sdk: '1.69.0'
python: '3.10'
- sdk: '3.42.0'
python: '3.12'
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
persist-credentials: false
- uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
with:
python-version: ${{ matrix.python }}
- name: Install source integrations and the selected SDK
env:
SDK: ${{ matrix.sdk }}
run: |
python -m venv .venv
.venv/bin/pip install -e python -e python/frameworks/fireworks "openai==$SDK" 'httpx<0.28' pytest pytest-asyncio
if [ "$SDK" != '1.0.0' ]; then
.venv/bin/pip install -e python/frameworks/openai
fi
- name: Run with outbound network disabled
env:
SDK: ${{ matrix.sdk }}
run: |
sudo unshare --net -- bash -e -c '
ip link set lo up
"$1" -I -m pytest "$2/python/frameworks/fireworks/tests" -q -o addopts=""
if [ "$3" = "1.0.0" ]; then
"$1" -I -m pytest "$2/python/frameworks/openai/tests/test_fireworks_route.py" -q -o addopts=""
else
"$1" -I -m pytest "$2/python/frameworks/openai/tests" -q -o addopts=""
fi
' _ "$PWD/.venv/bin/python" "$PWD" "$SDK"
101 changes: 56 additions & 45 deletions python/frameworks/fireworks/README.md
Original file line number Diff line number Diff line change
@@ -1,71 +1,82 @@
# TraceAI Fireworks AI Instrumentation
# Trace Fireworks with the OpenAI integration

OpenTelemetry instrumentation for Fireworks AI, enabling comprehensive observability for LLM API calls.
Use `traceAI-openai` to trace Fireworks' OpenAI-compatible Python API. You do not need a separate Fireworks instrumentor for new applications.

## Installation
## Install

Use Python 3.10–3.13 and OpenAI SDK 1.69.0 or later.

```bash
pip install traceai-fireworks
pip install --upgrade traceAI-openai 'fi-instrumentation-otel>=1.2.2' openai
```

## Quick Start
Set `FIREWORKS_API_KEY`, `FIREWORKS_MODEL`, `FI_API_KEY` and `FI_SECRET_KEY` in your application environment. Use a Fireworks model ID available to your account.

## Trace a request

```python
import os

from fi_instrumentation import register
from fi_instrumentation.fi_types import ProjectType
from traceai_fireworks import FireworksInstrumentor
from openai import OpenAI
from traceai_openai import OpenAIInstrumentor

# Setup TraceAI
trace_provider = register(
provider = register(
project_type=ProjectType.OBSERVE,
project_name="my-fireworks-app",
project_name="fireworks-app",
)
OpenAIInstrumentor().instrument(tracer_provider=provider)

try:
with OpenAI(
api_key=os.environ["FIREWORKS_API_KEY"],
base_url="https://api.fireworks.ai/inference/v1",
) as client:
response = client.chat.completions.create(
model=os.environ["FIREWORKS_MODEL"],
messages=[{"role": "user", "content": "Suggest two names for a weather app."}],
)
print(response.choices[0].message.content)
finally:
provider.shutdown()
```

# Instrument Fireworks
FireworksInstrumentor().instrument(tracer_provider=trace_provider)
In a long-running application, shut down the provider only when the application stops. Use `AsyncOpenAI` for asynchronous requests; the same instrumentor handles both clients and streaming responses.

# Now use OpenAI client with Fireworks
from openai import OpenAI
The span includes the Fireworks provider, model, elapsed time, messages, tool-call arguments and token usage when returned by the API. Tool-call arguments belong to the model span; this does not trace the execution of your tool functions. This route covers the OpenAI-compatible API, not the separate `fireworks-ai` SDK or its training and management APIs.

client = OpenAI(
api_key="your-fireworks-api-key",
base_url="https://api.fireworks.ai/inference/v1",
)
## Protect request and response content

Tracing captures content by default. Pass a `TraceConfig` when enabling the instrumentor to hide sensitive content while retaining model and usage metadata:

response = client.chat.completions.create(
model="accounts/fireworks/models/llama-v3p1-8b-instruct",
messages=[{"role": "user", "content": "Hello!"}]
```python
from fi_instrumentation import TraceConfig

OpenAIInstrumentor().instrument(
tracer_provider=provider,
config=TraceConfig(
hide_inputs=True,
hide_outputs=True,
hide_llm_invocation_parameters=True,
hide_error_details=True,
),
)
```

## Features

- Automatic tracing of Fireworks AI API calls
- Support for both synchronous and asynchronous operations
- Streaming response support
- Token usage tracking
- Request/response attribute capture
- OpenTelemetry semantic conventions for GenAI
Use this call **instead of** the default instrumentation call above, not after it. These settings change telemetry, not the request sent to Fireworks.

## Captured Attributes
## Migrate from `traceAI-fireworks`

- `gen_ai.request.model` - Model name
- `gen_ai.request.max_tokens` - Maximum tokens
- `gen_ai.request.temperature` - Temperature setting
- `gen_ai.usage.input_tokens` - Input token count
- `gen_ai.usage.output_tokens` - Output token count
- `gen_ai.prompt.{n}.role` - Message roles
- `gen_ai.prompt.{n}.content` - Message contents
- `gen_ai.completion.{n}.content` - Response content
`FireworksInstrumentor` is deprecated but its import, `instrument()` and `uninstrument()` entry points remain available. Existing applications using the older OpenAI SDK floor of 1.0.0 can continue using it while upgrading.

## Requirements
1. Upgrade to OpenAI SDK 1.69.0 or later and install `traceAI-openai`.
2. Replace the `FireworksInstrumentor` import and setup with `OpenAIInstrumentor` as above. Move your existing `tracer_provider` and `TraceConfig` arguments to that call.
3. Remove the old setup and uninstall `traceAI-fireworks` after checking that nothing else in your application imports it.

- Python >= 3.9
- openai >= 1.0.0
- fi-instrumentation >= 0.1.0
- opentelemetry-api >= 1.0.0
- opentelemetry-sdk >= 1.0.0
If both updated instrumentors are enabled during migration, `OpenAIInstrumentor` owns the request span. Its provider and privacy configuration apply; the deprecated instrumentor does not add a second span. Uninstrumenting either route does not disable the other route.

## License
## References

Apache-2.0
- [Fireworks OpenAI compatibility](https://docs.fireworks.ai/tools-sdks/openai-compatibility)
- [traceAI OpenAI integration](https://github.com/future-agi/traceAI/tree/dev/python/frameworks/openai)
9 changes: 9 additions & 0 deletions python/frameworks/fireworks/traceai_fireworks/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@
"""

import logging
import warnings
from importlib import import_module
from typing import Any, Collection

Expand Down Expand Up @@ -71,6 +72,14 @@ def instrumentation_dependencies(self) -> Collection[str]:
return _instruments

def _instrument(self, **kwargs: Any) -> None:
warnings.warn(
"FireworksInstrumentor is deprecated. Use traceai_openai.OpenAIInstrumentor "
"with base_url='https://api.fireworks.ai/inference/v1' instead. "
"When both are enabled, OpenAIInstrumentor owns tracing and its "
"tracer_provider and TraceConfig apply.",
DeprecationWarning,
stacklevel=3,
)
if not (tracer_provider := kwargs.get("tracer_provider")):
tracer_provider = trace_api.get_tracer_provider()
if not (config := kwargs.get("config")):
Expand Down
46 changes: 33 additions & 13 deletions python/frameworks/fireworks/traceai_fireworks/_wrappers.py
Original file line number Diff line number Diff line change
@@ -1,9 +1,11 @@
"""Wrapper classes for Fireworks AI instrumentation."""

import logging
import sys
from abc import ABC
from contextlib import contextmanager
from typing import Any, Dict, Iterator, Tuple
from urllib.parse import urlsplit

from fi_instrumentation import safe_json_dumps
from fi_instrumentation.fi_types import FiSpanKindValues, SpanAttributes, MessageAttributes
Expand Down Expand Up @@ -52,13 +54,25 @@ def _is_fireworks_client(instance: Any) -> bool:
if hasattr(instance, "_client"):
client = instance._client
if hasattr(client, "base_url"):
base_url = str(client.base_url)
return FIREWORKS_BASE_URL in base_url
host = urlsplit(str(client.base_url)).hostname
return bool(host and host.lower().rstrip(".") == FIREWORKS_BASE_URL)
except Exception:
pass
return False


def _shared_instrumentor_enabled() -> bool:
"""Let the request-level instrumentor own spans when both routes are enabled.

Do not import the optional package here: legacy users can keep openai 1.0,
which predates the shared instrumentor's supported SDK floor. Inspect live
state on each call so either installation/uninstallation order works.
"""
module = sys.modules.get("traceai_openai")
instrumentor = getattr(module, "OpenAIInstrumentor", None)
return bool(instrumentor and instrumentor().is_instrumented_by_opentelemetry)


def _get_input_attributes(
model: str,
messages: list,
Expand All @@ -68,7 +82,6 @@ def _get_input_attributes(
attributes: Dict[str, AttributeValue] = {
SpanAttributes.GEN_AI_SPAN_KIND: FiSpanKindValues.LLM.value,
SpanAttributes.GEN_AI_PROVIDER_NAME: "fireworks",
SpanAttributes.GEN_AI_PROVIDER_NAME: "fireworks",
SpanAttributes.GEN_AI_REQUEST_MODEL: model,
}

Expand Down Expand Up @@ -175,12 +188,19 @@ def _get_stream_output_attributes(

if hasattr(last_chunk, "usage") and last_chunk.usage:
usage = last_chunk.usage
if hasattr(usage, "prompt_tokens"):
attributes[SpanAttributes.GEN_AI_USAGE_INPUT_TOKENS] = usage.prompt_tokens
if hasattr(usage, "completion_tokens"):
attributes[SpanAttributes.GEN_AI_USAGE_OUTPUT_TOKENS] = usage.completion_tokens
if hasattr(usage, "total_tokens"):
attributes[SpanAttributes.GEN_AI_USAGE_TOTAL_TOKENS] = usage.total_tokens
# OpenAI 1.0 exposes Fireworks' extra streaming usage as a dict.
for field, attribute in (
("prompt_tokens", SpanAttributes.GEN_AI_USAGE_INPUT_TOKENS),
("completion_tokens", SpanAttributes.GEN_AI_USAGE_OUTPUT_TOKENS),
("total_tokens", SpanAttributes.GEN_AI_USAGE_TOTAL_TOKENS),
):
value = (
usage.get(field)
if isinstance(usage, dict)
else getattr(usage, field, None)
)
if value is not None:
attributes[attribute] = value

# Add raw output
try:
Expand All @@ -203,8 +223,8 @@ def __call__(
args: Tuple[Any, ...],
kwargs: Dict[str, Any],
) -> Any:
# Check if this is a Fireworks client
if not _is_fireworks_client(instance):
# One request must have one owner, regardless of setup order.
if _shared_instrumentor_enabled() or not _is_fireworks_client(instance):
return wrapped(*args, **kwargs)

# Extract parameters
Expand Down Expand Up @@ -274,8 +294,8 @@ async def __call__(
args: Tuple[Any, ...],
kwargs: Dict[str, Any],
) -> Any:
# Check if this is a Fireworks client
if not _is_fireworks_client(instance):
# One request must have one owner, regardless of setup order.
if _shared_instrumentor_enabled() or not _is_fireworks_client(instance):
return await wrapped(*args, **kwargs)

# Extract parameters
Expand Down
Loading
Loading