Skip to content

feat(providers): add Hetzner experimental inference provider - #14

Closed
weselben wants to merge 9 commits into
mainfrom
feat/hetzner-provider
Closed

feat(providers): add Hetzner experimental inference provider#14
weselben wants to merge 9 commits into
mainfrom
feat/hetzner-provider

Conversation

@weselben

@weselben weselben commented Aug 17, 2026

Copy link
Copy Markdown
Owner

TL;DR

Hetzner runs an experimental OpenAI-compatible inference API at https://inference.hetzner.com/api/v1. GoModel had no provider for it. This PR adds the hetzner provider. It wraps the shared OpenAI chat adapter with zero overrides. The upstream API was confirmed to be plain vLLM OpenAI-compatible before implementation: standard SSE chunks, standard max_tokens, standard image_url vision parts, Bearer auth.

Files to review (9, +533 / -1):

File Why
internal/providers/hetzner/hetzner.go (start here) Provider implementation. Mirrors the kimicode pattern field-for-field. Zero adapter overrides.
internal/providers/hetzner/hetzner_test.go (new) 12 unit tests. 100% statement coverage. No golden JSON. No live calls. Includes a regression-proof embeddings test and a Responses-via-chat smoke test.
run/providers.go Factory registration. One import, one factory.Add.
run/lifecycle_test.go TestMain_HetznerProviderRegistration. Mirrors the kimicode test.
run/providers_test.go Adds hetzner to the dashboard-types lockstep list.
internal/providers/config_test.go Parser fixture entry plus a focused base-URL resolution test.
docs/providers/hetzner.mdx (new) Provider guide. Leads with the experimental warning. Includes model-ID provenance and passthrough caveats.
docs/providers/overview.mdx Table row plus a provider note covering model-ID source and passthrough-as-adapter-capability.
docs/docs.json Navigation entry after providers/kimicode.

Reviewer notes

  • No adapter overrides. Confirmed against the upstream Hetzner docs: standard SSE chunks, standard max_tokens, and the Bearer default. No SetHeaders, no AdaptChatRequest, no RequestMutator.
  • Embeddings returns a typed "not supported" error. The embedded adapter would handle /v1/embeddings; Hetzner doesn't expose one. The override matches the kilo pattern. The unit test asserts zero upstream requests via httptest, so a regression that forwards embeddings upstream fails deterministically instead of hitting the network.
  • Responses-via-chat is exercised by a test. TestResponses_TranslatesToChatCompletions proves the doc claim that /v1/responses is served through chat translation.
  • No hardcoded model list or rate-limit table. The live catalogue changed during the experiment. The guide directs readers to /v1/models and the official docs. The example model ID in the overview row carries a provenance note in the provider note.
  • Experimental status leads every surface. Guide top, overview row label, provider note.

Deliberately out of scope

These are conscious omissions while Hetzner's API is experimental. Any reviewer raising them should defer to the rationale below.

  • .env.template and config.example.yaml entries. No HETZNER_API_KEY, HETZNER_BASE_URL, HETZNER_MODELS rows. Operators set the env var directly; applyProviderEnvVars discovers the provider via HETZNER_API_KEY (asserted by the config test). Entries land when the API leaves experimental status.
  • Contract test fixture and recorded golden JSON. No tests/contract/hetzner_test.go, no testdata/hetzner/. Live fixture recording is out of scope while the API is experimental and changes. The provider package instead ships 12 Go unit tests with 100% statement coverage.
  • Embeddings support. Hetzner documents no /v1/embeddings endpoint. The provider overrides Embeddings with a typed error instead of forwarding.
  • Pricing and cost-table sync. Free while experimental. cost load-balancing cannot rank hetzner by price until upstream publishes pricing.
  • Dashboard UI changes, Helm chart changes. Out of scope by intent; provider is reachable via the factory and the existing dashboard surface.

Tests

go test ./internal/providers/hetzner/ -cover reports 100.0% statement coverage. go test ./... is green (80 packages, 0 FAIL). go build ./... is clean. gofmt -l clean on the touched files.

Links


This PR description was generated with AI assistance.

@coderabbitai

coderabbitai Bot commented Aug 17, 2026

Copy link
Copy Markdown

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 1e0a5f4f-17f1-4cd3-bada-51a1f5f74a6f

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@weselben weselben left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review loop findings (4: 0 red, 1 yellow, 2 blue, 1 question).

.env.template:L607 — 🟡 risk: no HETZNER_API_KEY / HETZNER_BASE_URL / HETZNER_MODELS entries; every other registered provider type has a documented row. Add the trio for discoverability parity.

internal/providers/hetzner/hetzner_test.go:L266 — 🔵 nit: file missing trailing newline at EOF.

tests/contract/: 🔵 nit: no contract replay test or testdata/hetzner/ fixture; kimicode has one. Optional, but mirror the precedent if live recording is feasible.

internal/providers/hetzner/hetzner.go:L34 — ❓ q: Embeddings forwards /v1/embeddings and surfaces the raw upstream 404 to callers. Docs and TestEmbeddings_ForwardsToUpstreamWhichReturnsError already flag the footgun. Worth overriding Embeddings to return a clear "not supported" error so users get a typed signal instead of leaking upstream body text.

Comment thread internal/providers/hetzner/hetzner_test.go Outdated
Comment thread internal/providers/hetzner/hetzner.go
weselben added a commit that referenced this pull request Aug 17, 2026
Review findings from PR #14:
- override Embeddings to return a typed "not supported" error instead
  of forwarding to the absent upstream /v1/embeddings (kilo precedent)
- add missing trailing newline to hetzner_test.go
- update hetzner.mdx to document the typed error
@weselben

weselben commented Aug 17, 2026

Copy link
Copy Markdown
Owner Author

pr-resolve summary

Review findings triaged. 2 resolved, 2 skipped, 0 unanswered.

Resolved in d6bf585:

  • 🔵 nit (hetzner_test.go trailing newline) — added the newline.
  • ❓ q (hetzner.go embeddings) — Embeddings now overrides the adapter and returns a typed hetzner does not support embeddings error. No upstream call is made. Kilo precedent. TestEmbeddings_ReturnsUnsupportedError asserts it. Guide updated.

Skipped with reason:

  • 🟡 risk (.env.template entries) — deliberate. Map Add Hetzner OpenAI-compatible provider #7 lists env-template and config.example.yaml entries as out of scope while Hetzner's API is experimental. Entries land when the API stabilizes.
  • 🔵 nit (contract test fixture) — deliberate. Map Add Hetzner OpenAI-compatible provider #7 lists live fixture recording as out of scope while the API is experimental. Golden JSON deferred for the same reason.

Validation: go test ./... green (79 packages, 0 FAIL). go test -cover ./internal/providers/hetzner/ reports 100.0% statement coverage.

Mirror the kimicode pattern: wrap the shared openai.ChatCompatible adapter
behind a thin Registration/New/NewWithHTTPClient surface. Hetzner exposes
chat completions, model listing, and passthrough via OpenAI-compat at
https://inference.hetzner.com/api/v1. No embeddings endpoint is documented
upstream; the embedded adapter advertises the capability, but embedding
requests will fail at the provider.
Wire hetzner.Registration into defaultProviderFactory and assert it is
registered and instantiable. Add hetzner to the expected provider type
list kept in lockstep with the dashboard's Add Provider selector.
Add hetzner entry to testDiscoveryConfigs and a focused test that
applyProviderEnvVars discovers the type and resolves its default base
URL. .env.template and config.example.yaml are out of scope while
Hetzner's API is experimental.
New hetzner.mdx leads with the experimental warning and documents
configuration, runtime model discovery, rate limits (429, windows change
during the experiment), and free-while-experimental pricing. Overview
table gains a hetzner row and a provider note; docs.json gets the nav
entry. No model list or limit table is hardcoded — both moved during the
experimental period.
Mirror the kilo test depth (test-to-impl ratio ~4x) with no golden JSON
and no live API calls. Cover registration shape, both constructors (nil
HTTP client + zero hooks paths), Bearer auth on chat and stream, model
ID passthrough, /v1/models list, embeddings upstream-failure path, and
the optional interface guard matching kilo.
Review findings from PR #14:
- override Embeddings to return a typed "not supported" error instead
  of forwarding to the absent upstream /v1/embeddings (kilo precedent)
- add missing trailing newline to hetzner_test.go
- update hetzner.mdx to document the typed error
@weselben
weselben force-pushed the feat/hetzner-provider branch from d6bf585 to 30e5052 Compare August 17, 2026 21:19
Address feedback from the second pr-review loop:
- run/providers_test.go: restore tabs lost during rebase conflict resolution
  (gofmt violation caught by the pre-commit hook)
- hetzner_test.go: harden TestEmbeddings_ReturnsUnsupportedError to assert
  zero upstream requests via httptest; a regression that forwards embeddings
  upstream fails deterministically instead of hitting the network
- hetzner_test.go: add TestResponses_TranslatesToChatCompletions so the
  "serves /v1/responses via chat" doc claim is exercised by a test
- hetzner.mdx: add a Note that the example model ID comes from the official
  Hetzner docs and may differ at read time (experimental catalogue)

@weselben weselben left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review findings (round 2, post-rebase). Reviewed at head 30e5052. Five findings. Three resolved immediately in 695f1db; two answered in thread.

Resolved in 695f1db:

  • 🔴 run/providers_test.go:179 — line lost its two leading tabs when hetzner was inserted during rebase conflict resolution. gofmt violation. Restored the tabs.
  • 🟡 internal/providers/hetzner/hetzner_test.go:227TestEmbeddings_ReturnsUnsupportedError used an empty base URL and a nil HTTP client. A regression that forwards embeddings upstream would hit the network instead of failing the test. Rewrote the test to point at an httptest server and assert zero requests received.
  • 🔵 docs/providers/hetzner.mdx:17 — doc claims /v1/responses is translated via chat completions, but no hetzner test exercised the path. Added TestResponses_TranslatesToChatCompletions.

Answered in thread (no commit):

  • 🟡 docs/providers/hetzner.mdx:38 — example model ID Qwen/Qwen3.6-35B-A3B-FP8 comes from the official Hetzner docs (checked 2026-08-17). The catalogue is experimental and moves; the guide now carries a Note naming the source and directing readers to /v1/models.
  • docs/providers/overview.mdx:63 — passthrough marked ✅. All models are documented as OpenAI-compatible; passthrough is a generic forwarder. The row label (experimental) carries the caveat.

Validation: go test ./... green (80 packages, 0 FAIL). go test -cover ./internal/providers/hetzner/ reports 100.0% statement coverage. gofmt -l clean on the fixed files.

… caveat

Move the two thread-answered round-2 findings into the docs so downstream
review bots on the upstream mirror do not re-raise them:
- overview.mdx provider note: name the example model ID's source (official
  Hetzner docs, 2026-08-17) and mark the passthrough check as adapter
  capability with unverified upstream tolerance
- hetzner.mdx: add a passthrough Note stating the forwarder is generic and
  arbitrary paths may 404/405 while the API is experimental
@weselben

Copy link
Copy Markdown
Owner Author

Round-2 thread answers are now documented in the source, not only in the review. Commit 7bde193:

  • docs/providers/overview.mdx — Hetzner provider note now names the example model ID's source (official Hetzner docs, 2026-08-17) and marks the passthrough ✅ as adapter capability with unverified upstream tolerance.
  • docs/providers/hetzner.mdx — added a passthrough Note: the forwarder is generic; arbitrary paths may return 404/405 while the API is experimental.

A review bot on the upstream mirror should no longer re-raise either point.

@weselben

Copy link
Copy Markdown
Owner Author

PR body updated to document the two round-1 deliberate skips in source, not just in review threads. New "Deliberately out of scope (map #7)" section covers:

  • .env.template / config.example.yaml entries — deferred until Hetzner leaves experimental status.
  • Contract tests and golden JSON — deferred for the same reason.
  • Embeddings, pricing/cost-table, dashboard UI, Helm — also listed so the omission is visible to upstream review bots.

Body also refreshed: 12 tests (not 11), current +533 / -1 stat, fresh Responses-via-chat smoke test mentioned, focus area updated, stale research-branch link replaced with ticket #8 resolution comment.

Three findings from upstream PR review bots, addressed in source:

- docs/providers/hetzner.mdx: drop the undocumented request-count limit
  claim (CodeRabbit verified only token-based limits are documented);
  state the actual 3M/60k per 60s and 500M/5M per 24h windows
- internal/providers/hetzner/hetzner_test.go: harden
  TestEmbeddings_ReturnsUnsupportedError with errors.As against
  *core.GatewayError so a plain error with the same text would fail
  the typed-contract assertion (CodeRabbit)
- internal/server/passthrough_support.go: add hetzner to the default
  ENABLED_PASSTHROUGH_PROVIDERS allowlist (greptile P1: provider matrix
  marked \xE2\x9C\x85 but default-configured gateway returned 400 on /p/hetzner/...)
- .env.template + docs/providers/overview.mdx + docs/providers/hetzner.mdx:
  document the default-allowlist inclusion
- internal/server/handlers_test.go: update the rejection-message
  expectation to include hetzner in the sorted allowlist
- internal/server/passthrough_support_test.go: add an assertion that
  the default allowlist contains hetzner (regression guard)
@weselben

Copy link
Copy Markdown
Owner Author

merged on upstream ENTERPILOT#701

@weselben weselben closed this Aug 17, 2026
weselben added a commit that referenced this pull request Aug 17, 2026
…LOT#701)

* feat(providers): add hetzner experimental provider

Mirror the kimicode pattern: wrap the shared openai.ChatCompatible adapter
behind a thin Registration/New/NewWithHTTPClient surface. Hetzner exposes
chat completions, model listing, and passthrough via OpenAI-compat at
https://inference.hetzner.com/api/v1. No embeddings endpoint is documented
upstream; the embedded adapter advertises the capability, but embedding
requests will fail at the provider.

* feat(run): register hetzner provider in factory

Wire hetzner.Registration into defaultProviderFactory and assert it is
registered and instantiable. Add hetzner to the expected provider type
list kept in lockstep with the dashboard's Add Provider selector.

* test(providers): add hetzner to config parser test fixtures

Add hetzner entry to testDiscoveryConfigs and a focused test that
applyProviderEnvVars discovers the type and resolves its default base
URL. .env.template and config.example.yaml are out of scope while
Hetzner's API is experimental.

* docs(providers): add hetzner provider guide

New hetzner.mdx leads with the experimental warning and documents
configuration, runtime model discovery, rate limits (429, windows change
during the experiment), and free-while-experimental pricing. Overview
table gains a hetzner row and a provider note; docs.json gets the nav
entry. No model list or limit table is hardcoded — both moved during the
experimental period.

* test(providers): hetzner unit tests, 100% statement coverage

Mirror the kilo test depth (test-to-impl ratio ~4x) with no golden JSON
and no live API calls. Cover registration shape, both constructors (nil
HTTP client + zero hooks paths), Bearer auth on chat and stream, model
ID passthrough, /v1/models list, embeddings upstream-failure path, and
the optional interface guard matching kilo.

* fix(providers): hetzner embeddings typed error, test newline

Review findings from PR #14:
- override Embeddings to return a typed "not supported" error instead
  of forwarding to the absent upstream /v1/embeddings (kilo precedent)
- add missing trailing newline to hetzner_test.go
- update hetzner.mdx to document the typed error

* fix(providers): review findings round 2

Address feedback from the second pr-review loop:
- run/providers_test.go: restore tabs lost during rebase conflict resolution
  (gofmt violation caught by the pre-commit hook)
- hetzner_test.go: harden TestEmbeddings_ReturnsUnsupportedError to assert
  zero upstream requests via httptest; a regression that forwards embeddings
  upstream fails deterministically instead of hitting the network
- hetzner_test.go: add TestResponses_TranslatesToChatCompletions so the
  "serves /v1/responses via chat" doc claim is exercised by a test
- hetzner.mdx: add a Note that the example model ID comes from the official
  Hetzner docs and may differ at read time (experimental catalogue)

* docs(providers): document hetzner model-ID provenance and passthrough caveat

Move the two thread-answered round-2 findings into the docs so downstream
review bots on the upstream mirror do not re-raise them:
- overview.mdx provider note: name the example model ID's source (official
  Hetzner docs, 2026-08-17) and mark the passthrough check as adapter
  capability with unverified upstream tolerance
- hetzner.mdx: add a passthrough Note stating the forwarder is generic and
  arbitrary paths may 404/405 while the API is experimental

* fix(providers): review-bot findings on PR ENTERPILOT#701

Three findings from upstream PR review bots, addressed in source:

- docs/providers/hetzner.mdx: drop the undocumented request-count limit
  claim (CodeRabbit verified only token-based limits are documented);
  state the actual 3M/60k per 60s and 500M/5M per 24h windows
- internal/providers/hetzner/hetzner_test.go: harden
  TestEmbeddings_ReturnsUnsupportedError with errors.As against
  *core.GatewayError so a plain error with the same text would fail
  the typed-contract assertion (CodeRabbit)
- internal/server/passthrough_support.go: add hetzner to the default
  ENABLED_PASSTHROUGH_PROVIDERS allowlist (greptile P1: provider matrix
  marked \xE2\x9C\x85 but default-configured gateway returned 400 on /p/hetzner/...)
- .env.template + docs/providers/overview.mdx + docs/providers/hetzner.mdx:
  document the default-allowlist inclusion
- internal/server/handlers_test.go: update the rejection-message
  expectation to include hetzner in the sorted allowlist
- internal/server/passthrough_support_test.go: add an assertion that
  the default allowlist contains hetzner (regression guard)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant