fix(k8s): stop stripping the /kartik prefix before LiteLLM - #96
Merged
Conversation
The dashboard 404'd at /kartik/ui/ even with SERVER_ROOT_PATH set. LiteLLM expects to RECEIVE the prefix and strip it itself -- its root_path is not the usual "reverse proxy already stripped it" contract, contrary to what #95 assumed. With the prefix stripped by nginx AND claimed by root_path, Starlette's mounted StaticFiles computes a child root_path of /kartik/ui against an actual path of /ui/, fails the startswith check, resolves the wrong file and serves 404.html. Confirmed empirically: /kartik/kartik/ui/ returned 200 with the real dashboard (nginx strips one prefix, LiteLLM gets the other) while /kartik/ui/ returned 404. The API was unaffected either way, which is why this only showed up now. rewrite-target is ingress-wide, so /kartik cannot share backend-ingress, which needs the opposite behaviour for /api. Split it into its own Ingress with no rewrite, carrying the streaming timeouts and the same proxy-buffer-size bump frontend-ingress needed for its session cookie. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0174W6CHp7FMt7c9sKdBhbp1
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #95, which set
SERVER_ROOT_PATH=/kartikbut left the dashboard 404ing at/kartik/ui/.What #95 got wrong. It assumed LiteLLM's
root_pathfollows the usual FastAPI contract where the reverse proxy has already stripped the prefix. It doesn't — LiteLLM expects to receive/kartik/...and strip it itself (seenormalize_route_for_root_path, which returnsNonefor routes not under the root path).With the prefix stripped by nginx and claimed by
root_path, Starlette's mountedStaticFilesderives a childroot_pathof/kartik/uiwhile the actual request path is/ui/. Thestartswithcheck fails, it resolves the wrong file, and serves404.html.Confirmed empirically rather than by reading alone:
/kartik/ui//ui/404.htmlfallback)/kartik/kartik/ui//kartik/ui/<title>LiteLLM Dashboard</title>The double-prefix trick makes nginx strip one and hand LiteLLM the other — which is exactly the shape it wants.
The API worked under either arrangement (
/v1/*matches with or without the prefix), which is why this only surfaced once we looked at the UI.The fix.
rewrite-targetis an ingress-wide annotation, so/kartikcan't sharebackend-ingress—/apineeds the opposite behaviour. Split into alitellm-ingresswith no rewrite, carrying the streaming timeouts and the sameproxy-buffer-size: 16kthatfrontend-ingressneeded for its session cookie (the dashboard login sets a JWT cookie).Config-only, no image bump.
🤖 Generated with Claude Code
https://claude.ai/code/session_0174W6CHp7FMt7c9sKdBhbp1