Skip to content

Route Databricks-hosted models to gateway auth in relayed sessions (hybrid OSS + relayed) - #565

Open
masonc08 wants to merge 3 commits into
mainfrom
masonc08/hybrid-oss-relayed-routing
Open

Route Databricks-hosted models to gateway auth in relayed sessions (hybrid OSS + relayed)#565
masonc08 wants to merge 3 commits into
mainfrom
masonc08/hybrid-oss-relayed-routing

Conversation

@masonc08

@masonc08 masonc08 commented Sep 11, 2026

Copy link
Copy Markdown
Collaborator

What

Lets a relayed (subscription-relay) Claude Code session also reach Databricks-hosted (system.ai / OSS) models in the same session. The loopback proxy now routes per request by the requested model.

A relayed session authenticates with the caller's subscription OAuth in Authorization; Databricks-hosted models authenticate with a Databricks token in Authorization. Claude Code sends only one Authorization per session, so today a session is one or the other. This makes a single session serve both.

How

All modes already hit the same upstream ({workspace}/ai-gateway/anthropic/); they differ only in auth + the provider-service header. So the proxy can multiplex per request:

  • Databricks-hosted model (namespace-qualified id — system.ai.*, catalog.schema.model, databricks-*) → gateway auth: the Databricks token goes in Authorization, and the swap + Databricks-Model-Provider-Service headers are dropped so the gateway serves the model directly. Byte-identical to the normal non-relayed path.
  • Relayed subscription model (bare Anthropic id — claude-opus-4-1, …) → today's behavior: OAuth passed through untouched, Databricks token injected as the X-Databricks-AI-Gateway-Token swap header.

Gated by start_proxy(hybrid_oss_routing=...), enabled only for the relayed launch. A pure-relay session only sends bare Anthropic ids, so its behavior is unchanged (the classifier relays anything without a dot). Users surface Databricks-hosted models through Claude Code's existing modelPicker / --model; the proxy makes those selections route correctly.

Testing

Screenshot 2026-09-11 at 12 09 01

Automated (hermetic, test_gateway_proxy.py):

  • TestHybridOssRouting drives the handler end to end and asserts the forwarded headers per model: a Databricks-hosted id gets the gateway token in Authorization with the swap + provider-service headers dropped; a bare Anthropic id keeps the OAuth passthrough + swap header; with routing off, even a qualified id relays (pure-relay sessions unchanged).
  • is_databricks_routed_model (qualified vs bare vs missing id), _request_model (body parsing), and the forwarded_request_headers header shapes. Full hermetic suite green.

Live backend validation: confirmed against a staging workspace that the gateway serves both a Databricks-hosted Claude model and an OSS model when a Messages request carries a Databricks bearer in Authorization and no provider-service header — the exact request shape the proxy's Databricks route emits. So both halves are covered: the per-model header logic by the unit tests, and the gateway serving those requests by the live check. The proxy's route diagnostic (UCODE_RELAYED_PROXY_DIAGNOSTICS=1) distinguishes relay vs databricks per request.

masonc08 and others added 2 commits September 11, 2026 14:14
…ions

Lets a relayed (subscription-relay) Claude Code session also reach Databricks-hosted
OSS / system.ai models. The loopback proxy now inspects each request's `model`: a
namespace-qualified Databricks id is re-routed to gateway auth (Databricks token in
`Authorization`, MPS + swap headers dropped), while a bare Anthropic subscription id
keeps today's OAuth-passthrough relay. So one session can switch between the relayed
Enterprise subscription and Databricks models via `/model`.

Gated by start_proxy(hybrid_oss_routing=...), on only for the relayed launch; a
pure-relay session sends only bare Anthropic ids, so behavior is unchanged. Users
surface Databricks models through Claude Code's existing modelPicker / --model.

Toward AIGTWY-4490.

Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
@masonc08
masonc08 force-pushed the masonc08/hybrid-oss-relayed-routing branch from 6be9d4d to 33f2209 Compare September 11, 2026 14:15
@masonc08
masonc08 marked this pull request as ready for review September 11, 2026 16:12
@masonc08
masonc08 requested a review from lilly-luo September 11, 2026 16:12
Co-authored-by: Isaac <no-reply@databricks.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant