Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions config/config.example.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -380,6 +380,12 @@ providers:
# models:
# - id: "accounts/fireworks/models/gpt-oss-120b"

chutes:
type: chutes
api_key: "${CHUTES_API_KEY}"
# base_url defaults to "https://llm.chutes.ai/v1".
# Set base_url when using a different compatible endpoint.

Comment thread
coderabbitai[bot] marked this conversation as resolved.
meta:
type: meta
api_key: "..."
Expand Down
4 changes: 3 additions & 1 deletion docs/advanced/configuration.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -276,14 +276,15 @@ Set these to automatically register providers. No YAML configuration required.
| `ZAI_API_KEY` | Z.ai |
| `XAI_API_KEY` | xAI (Grok) |
| `GROQ_API_KEY` | Groq |
| `CHUTES_API_KEY` | Chutes AI (`CHUTES_BASE_URL` optional) |
| `AZURE_API_KEY` | Azure OpenAI (`AZURE_BASE_URL` also required) |
| `ORACLE_API_KEY` | Oracle GenAI (`ORACLE_BASE_URL` also required) |
| `OLLAMA_BASE_URL` | Ollama (no API key needed) |
| `SGLANG_BASE_URL` | SGLang (no API key needed unless upstream requires) |
| `VLLM_BASE_URL` | vLLM (no API key needed unless upstream requires) |
| `LLMD_BASE_URL` | llm-d Router/EPP (no API key needed unless its Gateway requires one) |

Most providers can use a custom base URL via `<PROVIDER>_BASE_URL` (for example `OPENAI_BASE_URL`). DeepSeek defaults to `https://api.deepseek.com`; set `DEEPSEEK_BASE_URL` only for a compatible proxy or alternate DeepSeek endpoint. OpenRouter defaults to `https://openrouter.ai/api/v1` and can be overridden with `OPENROUTER_BASE_URL`. Kilo AI defaults to `https://api.kilo.ai/api/gateway` and can be overridden with `KILO_BASE_URL`. Z.ai defaults to `https://api.z.ai/api/paas/v4`; set `ZAI_BASE_URL=https://api.z.ai/api/coding/paas/v4` for the GLM Coding Plan endpoint. SGLang defaults to `http://localhost:30000/v1` when `SGLANG_API_KEY` is set, but keyless deployments should set `SGLANG_BASE_URL` explicitly to register the provider. vLLM follows the same pattern at `http://localhost:8000/v1`. llm-d has no universal endpoint, so `LLMD_BASE_URL` is always required; `LLMD_API_KEY` is optional. Azure uses `AZURE_BASE_URL` for its deployment base URL and accepts an optional `AZURE_API_VERSION` override; otherwise it defaults to `2024-10-21`. Oracle requires `ORACLE_BASE_URL` because its OpenAI-compatible endpoint is region-specific.
Most providers can use a custom base URL via `<PROVIDER>_BASE_URL` (for example `OPENAI_BASE_URL`). Chutes AI defaults to `https://llm.chutes.ai/v1` and can be overridden with `CHUTES_BASE_URL`. DeepSeek defaults to `https://api.deepseek.com`; set `DEEPSEEK_BASE_URL` only for a compatible proxy or alternate DeepSeek endpoint. OpenRouter defaults to `https://openrouter.ai/api/v1` and can be overridden with `OPENROUTER_BASE_URL`. Kilo AI defaults to `https://api.kilo.ai/api/gateway` and can be overridden with `KILO_BASE_URL`. Z.ai defaults to `https://api.z.ai/api/paas/v4`; set `ZAI_BASE_URL=https://api.z.ai/api/coding/paas/v4` for the GLM Coding Plan endpoint. SGLang defaults to `http://localhost:30000/v1` when `SGLANG_API_KEY` is set, but keyless deployments should set `SGLANG_BASE_URL` explicitly to register the provider. vLLM follows the same pattern at `http://localhost:8000/v1`. llm-d has no universal endpoint, so `LLMD_BASE_URL` is always required; `LLMD_API_KEY` is optional. Azure uses `AZURE_BASE_URL` for its deployment base URL and accepts an optional `AZURE_API_VERSION` override; otherwise it defaults to `2024-10-21`. Oracle requires `ORACLE_BASE_URL` because its OpenAI-compatible endpoint is region-specific.

Every provider type also accepts a comma-separated configured model list via
`<PROVIDER>_MODELS`, for example `OPENROUTER_MODELS`, `ORACLE_MODELS`,
Expand Down Expand Up @@ -409,6 +410,7 @@ export GEMINI_API_KEY="..." # Registers "gemini" provider
export DEEPSEEK_API_KEY="..." # Registers "deepseek" provider
export XAI_API_KEY="..." # Registers "xai" provider
export GROQ_API_KEY="gsk_..." # Registers "groq" provider
export CHUTES_API_KEY="cpk_..." # Registers "chutes" provider
Comment thread
coderabbitai[bot] marked this conversation as resolved.
export OPENROUTER_API_KEY="sk-or-..." # Registers "openrouter" provider
export KILO_API_KEY="..." # Registers "kilo" provider
export ZAI_API_KEY="..." # Registers "zai" provider
Expand Down
4 changes: 4 additions & 0 deletions docs/features/passthrough-api.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -133,6 +133,10 @@ Passthrough is intentionally narrow while the API is in beta.

- `openai`, `anthropic`, `openrouter`, `kilo`, `zai`, `sglang`, `vllm`, `llmd`, and `deepseek` are enabled by
default.
- Chutes supports passthrough but requires explicit operator opt-in because
passthrough can forward provider-native routes that do not identify a model.
Add `chutes` to `ENABLED_PASSTHROUGH_PROVIDERS` only when you intend to expose
that surface.
- GoModel does not translate passthrough request bodies or response bodies.
- Provider-native error bodies and status codes are proxied instead of converted
into OpenAI-compatible responses.
Expand Down
7 changes: 7 additions & 0 deletions docs/providers/overview.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,7 @@ support, not every individual model capability exposed by an upstream provider.
| DeepSeek | `DEEPSEEK_API_KEY` | `deepseek-v4-pro` | ✅ | ✅ | ❌ | ❌ | ❌ | ✅ | [DeepSeek](/providers/deepseek) |
| Groq | `GROQ_API_KEY` | `llama-3.3-70b-versatile` | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | — |
| Fireworks AI | `FIREWORKS_API_KEY` (`FIREWORKS_BASE_URL` optional) | `accounts/fireworks/models/gpt-oss-120b` | ✅ | ✅ | ✅ | ❌ | ❌ | ✅ | — |
| Chutes AI | `CHUTES_API_KEY` (`CHUTES_BASE_URL` optional) | `Qwen/Qwen3-32B-TEE` | ✅ | ✅ | ❌ | ❌ | ❌ | ✅ | — |
| Meta (Muse Spark) | `META_API_KEY` (`META_BASE_URL` optional) | `muse-spark-1.1` | ✅ | ✅ | ❌ | ❌ | ❌ | ✅ | — |
| OpenRouter | `OPENROUTER_API_KEY` | `google/gemini-2.5-flash` | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | — |
| Kilo AI | `KILO_API_KEY` (`KILO_BASE_URL` optional) | `anthropic/claude-sonnet-4.5` | ✅ | ✅ | ❌ | ❌ | ❌ | ✅ | — |
Expand Down Expand Up @@ -90,6 +91,12 @@ support, not every individual model capability exposed by an upstream provider.
- **Fireworks AI** — model IDs are account-scoped paths such as
`accounts/fireworks/models/gpt-oss-120b`; use them verbatim in requests and
in `FIREWORKS_MODELS`.
- **Chutes AI** — defaults to `https://llm.chutes.ai/v1` and discovers its
current model IDs, context limits, capabilities, and pricing from the live
catalog. GoModel translates `/v1/responses` requests to chat completions;
Chutes' shared LLM endpoint does not expose embeddings. Passthrough support
requires explicit operator opt-in by adding `chutes` to
`ENABLED_PASSTHROUGH_PROVIDERS`.
- **Meta (Muse Spark)** — the Meta Model API is OpenAI-compatible; set
`META_API_KEY` and route to `muse-spark-1.1`. Muse Spark models are not in
the upstream model catalog yet, so declare `context_window` and `pricing`
Expand Down
99 changes: 99 additions & 0 deletions internal/providers/chutes/chutes.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
// Package chutes provides Chutes AI integration for the LLM gateway.
package chutes

import (
"context"
"io"
"net/http"

"github.com/enterpilot/gomodel/internal/core"
"github.com/enterpilot/gomodel/internal/llmclient"
"github.com/enterpilot/gomodel/internal/providers"
"github.com/enterpilot/gomodel/internal/providers/openai"
)

const defaultBaseURL = "https://llm.chutes.ai/v1"

// Registration provides factory registration for the Chutes AI provider.
var Registration = providers.Registration{
Type: "chutes",
New: New,
Discovery: providers.DiscoveryConfig{
DefaultBaseURL: defaultBaseURL,
},
}

// Provider implements Chutes' OpenAI-compatible chat surface. Chutes does not
// expose native Responses or embeddings endpoints, so Responses requests are
// translated through chat completions and embeddings fail locally.
type Provider struct {
compat *openai.CompatibleProvider
}

var _ core.Provider = (*Provider)(nil)
var _ core.PassthroughProvider = (*Provider)(nil)

// New creates a new Chutes AI provider.
func New(cfg providers.ProviderConfig, opts providers.ProviderOptions) core.Provider {
return &Provider{compat: openai.NewCompatibleProvider(cfg.APIKey, opts, compatibleConfig(
providers.ResolveBaseURL(cfg.BaseURL, defaultBaseURL),
))}
}

// NewWithHTTPClient creates a new Chutes AI provider with a custom HTTP client.
// If httpClient is nil, http.DefaultClient is used.
func NewWithHTTPClient(apiKey string, baseURL string, httpClient *http.Client, hooks llmclient.Hooks) *Provider {
return &Provider{compat: openai.NewCompatibleProviderWithHTTPClient(apiKey, httpClient, hooks, compatibleConfig(
providers.ResolveBaseURL(baseURL, defaultBaseURL),
))}
}

// compatibleConfig returns the shared OpenAI-compatible transport settings for Chutes.
func compatibleConfig(baseURL string) openai.CompatibleProviderConfig {
return openai.CompatibleProviderConfig{
ProviderName: "chutes",
BaseURL: baseURL,
SetHeaders: setHeaders,
}
}

// setHeaders applies Chutes' bearer-token authentication.
func setHeaders(req *http.Request, apiKey string) {
providers.SetAuthHeaders(req, apiKey, providers.AuthHeaderConfig{AuthScheme: "Bearer "})
}

// SetBaseURL changes the Chutes API base URL.
func (p *Provider) SetBaseURL(baseURL string) {
p.compat.SetBaseURL(baseURL)
}

// ChatCompletion sends a chat completion request to Chutes.
func (p *Provider) ChatCompletion(ctx context.Context, req *core.ChatRequest) (*core.ChatResponse, error) {
return p.compat.ChatCompletion(ctx, req)
}

// StreamChatCompletion sends a streaming chat completion request to Chutes.
func (p *Provider) StreamChatCompletion(ctx context.Context, req *core.ChatRequest) (io.ReadCloser, error) {
return p.compat.StreamChatCompletion(ctx, req)
}

// Responses translates an OpenAI Responses request through Chutes chat completions.
func (p *Provider) Responses(ctx context.Context, req *core.ResponsesRequest) (*core.ResponsesResponse, error) {
return providers.ResponsesViaChat(ctx, p, req)
}

// StreamResponses translates a streaming Responses request through Chutes chat completions.
func (p *Provider) StreamResponses(ctx context.Context, req *core.ResponsesRequest) (io.ReadCloser, error) {
return providers.StreamResponsesViaChat(ctx, p, req, "chutes")
}

// Embeddings returns an error because the shared Chutes LLM endpoint does not
// expose an OpenAI-compatible embeddings route.
func (p *Provider) Embeddings(_ context.Context, _ *core.EmbeddingRequest) (*core.EmbeddingResponse, error) {
return nil, core.NewInvalidRequestError("chutes does not support embeddings", nil)
}

// Passthrough forwards an opaque request to Chutes.
func (p *Provider) Passthrough(ctx context.Context, req *core.PassthroughRequest) (*core.PassthroughResponse, error) {
return p.compat.Passthrough(ctx, req)
}
Loading