feat(k8s): make LiteLLM's URL prefix configurable via SERVER_ROOT_PATH - #95
Merged
Conversation
The proxy is served at /kartik on dev, but LiteLLM built its URLs from its own root: /kartik/ui redirected to an absolute http://<host>/ui/, escaping the prefix and landing on the Next.js frontend (then Auth0, then a 400). Setting SERVER_ROOT_PATH makes it prefix-aware, so the admin UI is reachable. Set per-environment in config/*/litellm.env rather than base/, since the prefix is a property of how each environment exposes the proxy: /kartik on dev, unset (root) for prod and local. It flows through the existing litellm-config ConfigMap, which the deployment already consumes via envFrom -- and because configMapGenerator hashes the name, the change rolls the pod automatically. Safe for in-cluster callers: SERVER_ROOT_PATH is passed to FastAPI as root_path (proxy_server.py:1347), which only affects generated URLs, not route matching, and matches the ingress rewrite-target that already strips the prefix. The backend keeps calling http://litellm:4000/v1/... and /user/* unprefixed. The one behavioral change is in user_api_key_auth.py:630, where unprefixed requests no longer match the pass-through-route special case -- unused here. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0174W6CHp7FMt7c9sKdBhbp1
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #93 / #94. The proxy is served at
/kartikon dev, but LiteLLM built its URLs from its own root, so/kartik/uiredirected to an absolutehttp://<host>/ui/— escaping the prefix, landing on the Next.js frontend, and dead-ending in Auth0.SERVER_ROOT_PATHmakes it prefix-aware.Where it lives.
config/*/litellm.env, notbase/— the prefix is a property of how each environment exposes the proxy (/kartikon dev; unset, i.e. root, for prod and local). It flows through the existinglitellm-configConfigMap that the deployment already consumes viaenvFrom, and sinceconfigMapGeneratorhashes the ConfigMap name, the change rolls the pod on its own with no base edit.Why this doesn't break in-cluster callers.
SERVER_ROOT_PATHis handed to FastAPI asroot_path(proxy_server.py:1347) — the "reverse proxy strips the prefix" contract, which matches the ingressrewrite-target: /$2we already have.root_pathaffects generated URLs and docs, not route matching, so the backend keeps callinghttp://litellm:4000/v1/...and/user/*unprefixed (backend/app/core/config.py:44).The one real behavioral change:
normalize_route_for_root_path(user_api_key_auth.py:630) returnsNonefor unprefixed requests once a root path is set, so those no longer match the pass-through route special case. We don't use LiteLLM's pass-through routes, so this is inert here.Caveat on the UI: at startup LiteLLM rewrites its bundled UI assets in place to embed the prefix, and skips that with a warning if the directory isn't writable (
proxy_server.py:1885-1895). Our container has noreadOnlyRootFilesystem, so it should apply — worth confirming/kartik/uiactually renders after rollout.Config-only, no image bump.
🤖 Generated with Claude Code
https://claude.ai/code/session_0174W6CHp7FMt7c9sKdBhbp1