From 1e0eb54a08ab8f8d974fde42642659bb1b810547 Mon Sep 17 00:00:00 2001 From: Jim Schaff Date: Wed, 12 Aug 2026 14:37:17 -0400 Subject: [PATCH] feat(k8s): make LiteLLM's URL prefix configurable via SERVER_ROOT_PATH The proxy is served at /kartik on dev, but LiteLLM built its URLs from its own root: /kartik/ui redirected to an absolute http:///ui/, escaping the prefix and landing on the Next.js frontend (then Auth0, then a 400). Setting SERVER_ROOT_PATH makes it prefix-aware, so the admin UI is reachable. Set per-environment in config/*/litellm.env rather than base/, since the prefix is a property of how each environment exposes the proxy: /kartik on dev, unset (root) for prod and local. It flows through the existing litellm-config ConfigMap, which the deployment already consumes via envFrom -- and because configMapGenerator hashes the name, the change rolls the pod automatically. Safe for in-cluster callers: SERVER_ROOT_PATH is passed to FastAPI as root_path (proxy_server.py:1347), which only affects generated URLs, not route matching, and matches the ingress rewrite-target that already strips the prefix. The backend keeps calling http://litellm:4000/v1/... and /user/* unprefixed. The one behavioral change is in user_api_key_auth.py:630, where unprefixed requests no longer match the pass-through-route special case -- unused here. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_0174W6CHp7FMt7c9sKdBhbp1 --- kustomize/config/vcell-ai-local/litellm.env | 3 +++ kustomize/config/vcell-ai-rke-dev/litellm.env | 9 +++++++++ kustomize/config/vcell-ai-rke/litellm.env | 4 ++++ 3 files changed, 16 insertions(+) diff --git a/kustomize/config/vcell-ai-local/litellm.env b/kustomize/config/vcell-ai-local/litellm.env index 98ccfc1..1d511aa 100644 --- a/kustomize/config/vcell-ai-local/litellm.env +++ b/kustomize/config/vcell-ai-local/litellm.env @@ -14,3 +14,6 @@ LOCAL_LLM_API_BASE=http://ollama:11434 # Langfuse tracing callback (host only; keys are secret) LANGFUSE_HOST=https://cloud.langfuse.com + +# SERVER_ROOT_PATH: set this only if the proxy is exposed under a URL prefix +# (dev serves it at /kartik). Local reaches it directly, so leave it unset. diff --git a/kustomize/config/vcell-ai-rke-dev/litellm.env b/kustomize/config/vcell-ai-rke-dev/litellm.env index 7353177..0fbb546 100644 --- a/kustomize/config/vcell-ai-rke-dev/litellm.env +++ b/kustomize/config/vcell-ai-rke-dev/litellm.env @@ -14,3 +14,12 @@ LOCAL_LLM_API_BASE=http://ollama:11434 # Langfuse tracing callback (host only; keys are secret) LANGFUSE_HOST=https://cloud.langfuse.com + +# This proxy is exposed through the shared host at /kartik (see the dev +# ingress). Tell LiteLLM so it builds prefix-aware URLs -- without this its +# admin UI redirects to an absolute /ui/, which escapes the prefix and lands on +# the Next.js frontend instead. Passed to FastAPI as root_path, which is the +# "reverse proxy strips the prefix" contract, so it matches the ingress +# rewrite-target and does NOT change route matching: the backend's in-cluster +# calls to http://litellm:4000/v1/... keep working unprefixed. +SERVER_ROOT_PATH=/kartik diff --git a/kustomize/config/vcell-ai-rke/litellm.env b/kustomize/config/vcell-ai-rke/litellm.env index 62eff79..ed878ef 100644 --- a/kustomize/config/vcell-ai-rke/litellm.env +++ b/kustomize/config/vcell-ai-rke/litellm.env @@ -14,3 +14,7 @@ LOCAL_LLM_API_BASE=http://ollama:11434 # Langfuse tracing callback (host only; keys are secret) LANGFUSE_HOST=https://cloud.langfuse.com + +# SERVER_ROOT_PATH: set this only if the proxy is exposed under a URL prefix +# (dev serves it at /kartik). Prod does not expose LiteLLM through the ingress, +# so leave it unset -- it defaults to "" and the UI/redirects stay at the root.