Skip to content

feat(tts): Chatterbox voice with Kokoro fallback; choose the voice per request - #85

Merged
DowLucas merged 3 commits into
mainfrom
feat/tts-fallback
Oct 7, 2026
Merged

DowLucas merged 3 commits into
mainfrom
feat/tts-fallback

Conversation

@DowLucas

@DowLucas DowLucas commented Oct 7, 2026 •

Copy link
Copy Markdown
Owner

Summary

The voice moves to Chatterbox Turbo (350M) on the homelab's Intel Arc B580. It won the bake-off (3.4–3.9× real time, first piece of an explanation in 1.7 s, 3.6 GB of GPU memory) and runs as a small OpenAI-compatible server in the deployment stack. Kokoro stays, as the fallback and as a voice users can choose.

Named services, a voice per request

  • TTS_SERVICES=chatterbox=http://chatterbox:8880,kokoro=http://kokoro:8880: the services in order. The first is the default voice, the rest are fallbacks in order.
  • /api/ai/voice-audio takes an optional model (a service name). That service is tried first and the others after it, which is how the plugin's /devscope:voice model kokoro works. An unknown name is ignored.
  • GET /api/ai/voice-models → {models: ["chatterbox", "kokoro"]}, default first.
  • TTS_URL (+ TTS_FALLBACK_URL) still work as the unnamed form, so the current .env keeps working.

Why fallbacks have a short leash

The B580 is shared with Ollama. While Ollama's 9B model runs, Chatterbox slows about 8× (a 15 s summary took ~40 s today, while another job hammered the 9B). So every service but the last gets TTS_PRIMARY_TIMEOUT_MS (default 10 s). The Chatterbox server also answers 503 at once when busy (measured: 2 s), so Kokoro speaks instead of you waiting.

Deploy: merge → watchtower → set TTS_SERVICES=chatterbox=http://chatterbox:8880,kokoro=http://kokoro:8880 in the stack's .env and recreate the backend.

Test plan

  • bun test tts + routes/voice + voiceSummary: 36 pass (new: parsing, default and requested order, unknown name ignored, requested voice falls back, primary timeout, both fail → null, route passes model, malformed name 400, /voice-models).
  • tsc: 34 errors both before and after this change.
  • Chatterbox server against the B580: a 235-char summary in 7.2 s with Ollama's 9B busy; a concurrent request answered 503 in 2 s; WAV header carries the real length.
  • After the env switch: default request → Chatterbox log; model: "kokoro" → Kokoro; Chatterbox stopped → Kokoro answers.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • New Features
    • Voice generation now supports selecting from configured text-to-speech models, and an API endpoint provides the available model names.
  • Bug Fixes
    • If a voice service is unavailable, times out, or fails to return audio, the system tries other configured services in order. A successful response from the first service avoids unnecessary retries.
  • Documentation
    • Updated API and server setup guides to explain model selection, service configuration, fallback behavior, and relevant error responses.

TTS_FALLBACK_URL names a second OpenAI-compatible speech service (Kokoro)
that speaks when the voice (TTS_URL, now Chatterbox Turbo on the Arc B580)
fails or does not answer within TTS_PRIMARY_TIMEOUT_MS (default 10 s; the
full 15 s applies without a fallback). The B580 is shared with Ollama, and
Chatterbox slows ~8x while Ollama's 9B runs; without this the plugin would
fall back to the Mac's local voice instead. Fallback settings are read per
call. Nothing else changes when TTS_FALLBACK_URL is unset.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Note

Currently processing new changes in this PR. This may take a few minutes, please wait...

⚙️ Run configuration
  • Configuration used: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 0ccf663c-9630-4ee9-8ba5-4935b98ee6ad
📥 Commits

Reviewing files that changed from the base of the PR and between 53a57f5 and ffa253b.

📒 Files selected for processing (6)
  • CLAUDE.md
  • packages/backend/src/ai/__tests__/tts.test.ts
  • packages/backend/src/ai/tts.ts
  • packages/backend/src/routes/__tests__/voice.test.ts
  • packages/backend/src/routes/voice.ts
  • packages/backend/src/services/voiceSummary.ts
 _________________________
< Loose code sinks ships. >
 -------------------------
  \
   \   \
        \ /\
        ( )
      .( o ).
✨ Finishing Touches
📝 Generate docstrings
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

TTS_SERVICES=chatterbox=http://chatterbox:8880,kokoro=http://kokoro:8880
names the services in order: the first is the default, the rest are
fallbacks. /api/ai/voice-audio takes an optional `model` (a service name)
that is tried first, so the plugin's `/devscope:voice model kokoro` keeps
Kokoro for one user while Chatterbox stays the default. Every service but
the last gets TTS_PRIMARY_TIMEOUT_MS. GET /api/ai/voice-models lists them.
TTS_URL/TTS_FALLBACK_URL still work as the unnamed form.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@DowLucas DowLucas changed the title feat(tts): a fallback speech service with a short leash on the first feat(tts): Chatterbox voice with Kokoro fallback; choose the voice per request Oct 7, 2026
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@DowLucas
DowLucas merged commit 94b489e into main Oct 7, 2026
5 of 6 checks passed
@DowLucas
DowLucas deleted the feat/tts-fallback branch October 7, 2026 12:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant