Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
106 changes: 106 additions & 0 deletions .github/workflows/openai-compatible-recipe-tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,106 @@
name: OpenAI-compatible recipe tests

on:
pull_request:
branches: [main, dev]
paths:
- 'python/examples/**'
- 'python/fi_instrumentation/**'
- 'python/frameworks/openai/**'
- 'python/tests/harness/**'
- '.github/workflows/openai-compatible-recipe-tests.yml'
push:
branches: [dev]
paths:
- 'python/examples/**'
- 'python/fi_instrumentation/**'
- 'python/frameworks/openai/**'
- 'python/tests/harness/**'
- '.github/workflows/openai-compatible-recipe-tests.yml'
workflow_dispatch:

permissions:
contents: read

jobs:
plan:
name: Recipe plan
runs-on: ubuntu-24.04
outputs:
matrix: ${{ steps.plan.outputs.matrix }}
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
ref: ${{ github.event.pull_request.head.sha || github.sha }}
persist-credentials: false
- id: plan
run: |
python3 - <<'PY'
import json, os
from pathlib import Path
matrix = []
for recipe in ('databricks', 'lepton', 'orcarouter', 'anannas-ai', 'doubleword'):
root = Path('python/examples') / recipe
if not root.exists():
continue # Recipes may land independently.
assert (root / 'src/app.py').is_file(), f'{recipe}: missing app'
assert (root / 'requirements.txt').is_file(), f'{recipe}: missing requirements'
assert list((root / 'tests').glob('test_*.py')), f'{recipe}: missing tests'
for sdk, mode in (('3.24.0', 'source'), ('1.69.0', 'source'), ('3.24.0', 'published')):
matrix.append({'recipe': recipe, 'sdk': sdk, 'mode': mode})
assert matrix, 'No recipe tests found'
with open(os.environ['GITHUB_OUTPUT'], 'a') as out:
out.write('matrix=' + json.dumps({'include': matrix}) + '\n')
PY
recipe:
name: Recipe (${{ matrix.recipe }}, openai ${{ matrix.sdk }}, ${{ matrix.mode }})
needs: plan
runs-on: ubuntu-24.04
timeout-minutes: 15
strategy:
fail-fast: false
matrix: ${{ fromJSON(needs.plan.outputs.matrix) }}
env:
PYTHONDONTWRITEBYTECODE: '1'
FI_BASE_URL: http://127.0.0.1:9
FI_GRPC_URL: http://127.0.0.1:9
RECIPE: ${{ matrix.recipe }}
SDK_VERSION: ${{ matrix.sdk }}
PACKAGE_MODE: ${{ matrix.mode }}
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
ref: ${{ github.event.pull_request.head.sha || github.sha }}
persist-credentials: false
- uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
with:
python-version: '3.11.9'
- name: Install SDK and harness dependencies
run: |
python -m pip install "openai==$SDK_VERSION" httpx 'wrapt<2' \
opentelemetry-api opentelemetry-sdk opentelemetry-instrumentation \
opentelemetry-exporter-otlp-proto-http requests protobuf \
opentelemetry-proto pydantic jsonschema pytest
if [ "$PACKAGE_MODE" = published ]; then
python -m pip install 'traceAI-openai==0.1.10' 'fi-instrumentation-otel==1.1.0'
fi
- name: Run isolated recipe contracts
run: |
if [ "$PACKAGE_MODE" = published ]; then
export PYTHONPATH="python/examples/$RECIPE/src:python/tests"
else
export PYTHONPATH="python/examples/$RECIPE/src:python:python/frameworks/openai:python/tests"
fi
python -m pytest "python/examples/$RECIPE/tests" -q \
--noconftest -p no:cacheprovider -o addopts='' -rfEs
result:
name: OpenAI-compatible recipes
if: always()
needs: [plan, recipe]
runs-on: ubuntu-24.04
steps:
- name: Require every planned recipe contract
env:
PLAN_RESULT: ${{ needs.plan.result }}
RECIPE_RESULT: ${{ needs.recipe.result }}
run: test "$PLAN_RESULT" = success && test "$RECIPE_RESULT" = success
181 changes: 181 additions & 0 deletions python/examples/databricks/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,181 @@
# Databricks (OpenAI-compatible) with traceAI

Trace Chat Completions through the official `openai` Python SDK at your Databricks workspace's documented OpenAI-compatible base URL. This recipe uses `traceai-openai`; it needs no Databricks tracing package.

## Install

From this directory, install the pinned dependencies:

```bash
pip install -r requirements.txt
```

Or install the packages directly:

```bash
pip install traceAI-openai fi-instrumentation-otel openai
```

The requirements pin `openai==3.24.0`, `traceAI-openai==0.1.10`, and `fi-instrumentation-otel==1.1.0`. The `traceai-openai` SDK floor is `openai==1.69.0`.

## Configure

| Environment variable | Required | Purpose |
|---|---|---|
| `FI_API_KEY` | Yes | Future AGI API key; tracer only. |
| `FI_SECRET_KEY` | Yes | Future AGI secret key; tracer only. |
| `DATABRICKS_TOKEN` | Yes | Databricks token from your secret store; OpenAI client only. Never commit it. |
| `DATABRICKS_MODEL` | Yes, unless using `--model` | Your model service name or serving endpoint name. |
| `DATABRICKS_BASE_URL` | Yes | Your workspace-specific SDK base URL. There is no public default. |
| `FI_BASE_URL` | No | Future AGI collector origin; uses the tracer's default when unset. |
| `FI_PROJECT_NAME` | No | Observe project name; defaults to `databricks-openai-recipe`; tracer only. |

Choose one of these two different surfaces. Replace `<workspace-host>` with the host from your workspace URL:

| Surface | Documented base URL form | `model` example |
|---|---|---|
| Unity Gateway model services | `https://<workspace-host>/ai-gateway/mlflow/v1` | `system.ai.claude-sonnet-4-5` |
| Model Serving serving endpoints | `https://<workspace-host>/serving-endpoints` | `openai-chat-endpoint` |

Use your own model service or endpoint name. Both names above are examples. `DATABRICKS_BASE_URL` is required and replaces the base URL for either surface.

## Run

Replace all key and workspace placeholders before running. This example chooses model services; use the second URL form and your endpoint name for Model Serving.

```bash
export FI_API_KEY="placeholder-fi-api-key"
export FI_SECRET_KEY="placeholder-fi-secret-key"
export DATABRICKS_TOKEN="placeholder-databricks-key"
export DATABRICKS_BASE_URL="https://<workspace-host>/ai-gateway/mlflow/v1"
export DATABRICKS_MODEL="system.ai.claude-sonnet-4-5"
python src/app.py
python src/app.py --stream
```

You can override the model with `--model` and the question with `--prompt`. Set `FI_PROJECT_NAME` to send the traces to another Observe project; unset or empty, they go to `databricks-openai-recipe`. Missing configuration or a refused URL exits with code 2 before tracing starts. The URL check returns allowed URLs unchanged. The OpenAI SDK then adds a trailing slash to its stored `base_url` and appends `/chat/completions` for this call.

## Code

Instrument before creating the client. Validate the workspace URL with the helper in [src/app.py](src/app.py).

```python
import os
from openai import OpenAI
from fi_instrumentation import register
from fi_instrumentation.fi_types import ProjectType
from traceai_openai import OpenAIInstrumentor
from app import check_base_url
provider = register(project_name=os.environ.get("FI_PROJECT_NAME") or "databricks-openai-recipe", # as in src/app.py
project_type=ProjectType.OBSERVE, set_global_tracer_provider=False, verbose=False)
OpenAIInstrumentor().instrument(tracer_provider=provider)
client = OpenAI(api_key=os.environ["DATABRICKS_TOKEN"],
base_url=check_base_url(os.environ["DATABRICKS_BASE_URL"]))
answer = client.chat.completions.create(model=os.environ["DATABRICKS_MODEL"],
messages=[{"role": "user", "content": "What is a lakehouse?"}])
print(answer.choices[0].message.content)
provider.force_flush()
```

## What you see in Future AGI

A Chat Completions call produces one `ChatCompletion` span with `gen_ai.span.kind=LLM`. On a successful non-streamed response, `gen_ai.request.model` is the model id the provider returns in its response. The fixtures return the model service or endpoint name sent in the request.

Up to `traceai-openai` 0.1.10, the provider field says `openai` (`gen_ai.provider.name`), the shared instrumentor's label for this recipe. Later versions (future-agi/traceAI#249) say `databricks` for workspace hosts (`*.cloud.databricks.com`, `*.azuredatabricks.net`, `*.gcp.databricks.com`); other base URLs keep `openai`.

Token usage appears in `gen_ai.usage.input_tokens`, `gen_ai.usage.output_tokens`, and `gen_ai.usage.total_tokens` when the response has usage. Missing usage is omitted, never filled with zero.

Streaming accumulates the answer in `output.value` through the same instrumentor. A default stream without usage exports no usage attributes. With `stream_options={"include_usage": True}` and a final usage chunk, all three usage attributes are exported. Your endpoint must support that option to use it.

`traceai-openai` 0.1.10 and earlier omit `gen_ai.request.model` on streamed and failed calls; later versions record the requested model there. The requested model remains in the `gen_ai.request.parameters` JSON unless `FI_HIDE_LLM_INVOCATION_PARAMETERS=true`. Errors have span status ERROR and an exception event.

## Provider specifics

### Model services

For Unity Gateway model services, use `/ai-gateway/mlflow/v1` and send your model service name as `model`. The Databricks documentation's example.staging.cloud.databricks.com is a placeholder, not a real host. Replace it with your workspace host; the app refuses the sample host.

### Serving endpoints

For Model Serving serving endpoints, use `/serving-endpoints` and send your endpoint name as `model`. These two suffixes select different surfaces. Do not combine them. The token comes from `DATABRICKS_TOKEN`; the SDK sends `Authorization: Bearer <token>`. Keep it in a secret store. Databricks recommends machine-to-machine OAuth for production workloads.

`model` is the model service or endpoint name, not a frozen foundation-model id. A pricing row keyed by a foundation-model id will not match an endpoint name. An endpoint name needs a matching pricing row for cost lookup.

The app refuses workspace roots, `/api/2.0/serving-endpoints`, and `/serving-endpoints/<name>/invocations`. The last form is a REST invocation URL, not an SDK base URL. It also refuses `/ai-gateway/gemini` and `/ai-gateway/anthropic`, which are gateway native APIs for other SDKs. Databricks workspace hosts must use `https://`; plain `http://` would send the token in cleartext before any redirect. A query string or fragment on a workspace URL is refused. Placeholder checks include percent-encoded forms. Host checks ignore case and trailing dots. Customer proxies and loopback URLs are allowed unchanged.

### Embeddings, only when the endpoint task is embeddings

Use `client.embeddings.create(model="system.ai.gte-large-en", input="A short example.")` only when your selected endpoint task is embeddings. Use your own embedding service name. A chat endpoint can reject this call; that is an endpoint task mismatch.

The gateway fixture produces one `CreateEmbeddingResponse` span with `gen_ai.span.kind=EMBEDDING` and provider `openai` (`databricks` in later versions). Up to 0.1.10 it records the response model in `embedding.model_name`, not `gen_ai.request.model`; later versions record it in both and replace the vectors in `output.value` with a short JSON summary. It exports input and total token usage, with no output token usage. Input text is exported in `input.value` and `embedding.embeddings.0.embedding.text` by default. The recipe's CLI runs Chat Completions.

### Inside the workspace

This recipe constructs the stock `openai.OpenAI` client directly; `databricks-openai` is not a recipe dependency.

## Privacy

For Chat Completions, set `FI_HIDE_INPUTS=true` or `FI_HIDE_OUTPUTS=true` before tracing starts to keep prompt or response text out of the exported span. `TraceConfig(hide_inputs=True)` passed to the instrumentor is another way to hide chat inputs. The provider still receives the prompt. Future AGI masking does not change the provider's own logging or Databricks payload logs.

The embedding fixture pins exported input text separately. These chat masking tests do not establish masking of embedding-specific text attributes.

## Limits / not covered

- SQL `ai_query` and AI Functions.
- The MLflow Deployments SDK and Databricks-internal MLflow trace forwarding.
- Agent / Genie code and other agent frameworks.
- The Gemini/Anthropic gateway native APIs.
- The `DatabricksOpenAI` helper.
- Direct REST invocation calls and workspace provisioning.

The fixtures use these syntactic test hosts, not real ones, with `httpx.MockTransport`:

```text
https://dbc-00000000-0000.cloud.databricks.com/ai-gateway/mlflow/v1
https://dbc-00000000-0000.cloud.databricks.com/serving-endpoints
```

## Tests

Run from the repository root. The suite checks both surfaces, credentials, spans, usage, streaming, errors, chat masking, embeddings, URL refusal, and CLI subprocesses. Vendor requests use fixtures or a server on `127.0.0.1`. Subprocess guards refuse other hosts before DNS and record refused attempts. A negative control proves the guard is active.

Repository source:

```bash
env -u PYTHONPATH PYTHONDONTWRITEBYTECODE=1 \
PYTHONPATH="python/examples/databricks/src:python:python/frameworks/openai:python/tests" \
uv run --no-project --python 3.11 \
--with 'openai==3.24.0' --with httpx --with 'wrapt<2' \
--with opentelemetry-api --with opentelemetry-sdk --with opentelemetry-instrumentation \
--with opentelemetry-exporter-otlp-proto-http --with requests --with protobuf \
--with opentelemetry-proto --with pydantic --with jsonschema --with pytest \
pytest python/examples/databricks/tests -q -p no:cacheprovider --noconftest -o addopts= -rfEs
```

Published packages:

```bash
env -u PYTHONPATH PYTHONDONTWRITEBYTECODE=1 \
PYTHONPATH="python/examples/databricks/src:python/tests" \
uv run --no-project --python 3.11 \
--with 'openai==3.24.0' --with httpx --with 'wrapt<2' \
--with 'traceAI-openai==0.1.10' --with 'fi-instrumentation-otel==1.1.0' \
--with opentelemetry-api --with opentelemetry-sdk --with opentelemetry-instrumentation \
--with opentelemetry-exporter-otlp-proto-http --with requests --with protobuf \
--with opentelemetry-proto --with pydantic --with jsonschema --with pytest \
pytest python/examples/databricks/tests -q -p no:cacheprovider --noconftest -o addopts= -rfEs
```

Dependency variants for local checks (for the floor row, change `openai==3.24.0` to `openai==1.69.0` in the repository-source command):

| Python | `openai` | traceAI packages | Local check configuration |
|---|---|---|---|
| 3.10, 3.11, 3.12, 3.13 | 3.24.0 | `traceai-openai` and `fi_instrumentation` from this repository | Commands above |
| 3.11 | 1.69.0 (the `traceai-openai` floor) | from this repository | Commands above |
| 3.11 | 3.24.0 | published `traceAI-openai==0.1.10` and `fi-instrumentation-otel==1.1.0` from PyPI | Commands above |

## Official API references

- [Query model services](https://docs.databricks.com/aws/en/ai-gateway/query-model-services)
- [Migrate to Unity Gateway](https://docs.databricks.com/aws/en/ai-gateway/migrate-to-unity-gateway)
- [Foundation Model REST API](https://docs.databricks.com/aws/en/machine-learning/foundation-model-apis/api-reference)
4 changes: 4 additions & 0 deletions python/examples/databricks/requirements.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
# OpenAI SDK and published traceAI package pins.
openai==3.24.0
traceAI-openai==0.1.10
fi-instrumentation-otel==1.1.0
Loading
Loading