Repository navigation
feat(router): publish model capabilities on /v1/models and /v1/models/{id} - #30
Merged
Merged
Conversation
…/{id}
Clients can see per model whether a prompt or hotwords have any effect, whether
word timestamps and confidence are returned, and the longest audio and largest
upload accepted, instead of guessing from the model name. The values come from
the router configuration (defaults match the bundled servers, override with
<NAME>_CAPABILITIES or router.capabilities in the chart); what a backend claims
about itself is never passed through. The field is additive.
Possible relctl commands for this Pull RequestVersion scheme: calver. The version of the release is the date of the release ( Need more help? Have a look at my repo This message was created by relctl and can be disabled by the env variable |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Every model entry on
GET /v1/modelsgets an additivecapabilitiesobject, andGET /v1/models/{id}is added (OpenAI shape, 404 object for unknown ids).{"tasks":["transcribe"],"audio":{"prompt":false,"hotwords":false,"word_times":true,"segment_times":true,"confidence":true,"auto_language":true,"force_language":false,"max_seconds":120,"max_bytes":26214400}}Why
The two transcription backends differ: one takes a prompt and hotwords, the other accepts and ignores them; limits on audio length and upload size differ as well. A client had to guess from the model name. With the field it can send hints only where they help, ask for word timestamps only where they exist, and cut audio to the accepted length.
How
capabilitiesfield is dropped; tested).<NAME>_CAPABILITIES(JSON) orrouter.capabilities.<backend>in the chart. A malformed value stops the router at start.Tests
Config defaults, override and malformed input; models list carries capabilities (upstream OK and static fallback);
/v1/models/{id}incl. ids with a slash and the 404; no field without configuration.go test -race ./...andhelm templatewith and withoutrouter.capabilities.