Make the transcription model configurable; default to gpt-transcribe - #6
sadasant-ghost[bot] wants to merge 1 commit into
Conversation
whisper-1 was hardcoded. It does not fail silently -- it fills silence with fluent invention: a Korean news sign-off and a YouTube outro from pauses in voice notes, and 'billing' heard as 'building' in a message for executives. Measured against the live API on one second of a 440 Hz sine tone, gpt-transcribe returned an empty string and whisper-1 returned 'Oh'. gpt-transcribe is also cheaper -- /bin/zsh.0045 against /bin/zsh.006 -- and billed by duration, which this endpoint needs, since it reports a per-minute cost and a token-priced model would make that a guess. CONDUCTOR_TRANSCRIBE_MODEL overrides it. An unknown or token-billed model reports cost: null rather than a plausible wrong number, and the response names the model that produced the text. 32 tests.
There was a problem hiding this comment.
2 issues found across 3 files
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="conductor_api.py">
<violation number="1" location="conductor_api.py:181">
P3: `transcribe_cost`'s docstring says None is returned only "when the model is token-billed", but the implementation returns None for any unknown model not present in `USD_PER_MIN` (the added test even asserts this for "something-new"). Widen the docstring to cover the unknown-model case so it matches the actual contract.</violation>
<violation number="2" location="conductor_api.py:189">
P3: `WHISPER_USD_PER_MIN` is defined on the added line with the comment "kept: referenced elsewhere", but nothing in the repository references it anymore — the only consumers were the lines this change replaced with `transcribe_cost(secs)`. It is dead code and the comment is misleading. Remove the line.</violation>
</file>
Reply with feedback, questions, or to request a fix.
Fix all with cubic | Re-trigger cubic
| def transcribe_cost(seconds, model=None): | ||
| """Dollars for `seconds` of audio, or None when the model is token-billed. | ||
|
|
||
| None is the honest answer for a token-priced model: the caller knows the |
There was a problem hiding this comment.
P3: transcribe_cost's docstring says None is returned only "when the model is token-billed", but the implementation returns None for any unknown model not present in USD_PER_MIN (the added test even asserts this for "something-new"). Widen the docstring to cover the unknown-model case so it matches the actual contract.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At conductor_api.py, line 181:
<comment>`transcribe_cost`'s docstring says None is returned only "when the model is token-billed", but the implementation returns None for any unknown model not present in `USD_PER_MIN` (the added test even asserts this for "something-new"). Widen the docstring to cover the unknown-model case so it matches the actual contract.</comment>
<file context>
@@ -146,7 +146,47 @@ def translate_command(description, cwd):
+def transcribe_cost(seconds, model=None):
+ """Dollars for `seconds` of audio, or None when the model is token-billed.
+
+ None is the honest answer for a token-priced model: the caller knows the
+ duration and nothing about the token count, and a number invented here
+ would be indistinguishable from a real one.
</file context>
| return None if rate is None else round(seconds / 60 * rate, 4) | ||
|
|
||
|
|
||
| WHISPER_USD_PER_MIN = USD_PER_MIN["whisper-1"] # kept: referenced elsewhere |
There was a problem hiding this comment.
P3: WHISPER_USD_PER_MIN is defined on the added line with the comment "kept: referenced elsewhere", but nothing in the repository references it anymore — the only consumers were the lines this change replaced with transcribe_cost(secs). It is dead code and the comment is misleading. Remove the line.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At conductor_api.py, line 189:
<comment>`WHISPER_USD_PER_MIN` is defined on the added line with the comment "kept: referenced elsewhere", but nothing in the repository references it anymore — the only consumers were the lines this change replaced with `transcribe_cost(secs)`. It is dead code and the comment is misleading. Remove the line.</comment>
<file context>
@@ -146,7 +146,47 @@ def translate_command(description, cwd):
+ return None if rate is None else round(seconds / 60 * rate, 4)
+
+
+WHISPER_USD_PER_MIN = USD_PER_MIN["whisper-1"] # kept: referenced elsewhere
# Voice input reaches an agent as ordinary text, and speech-to-text mishears in ways
</file context>
Stacked on #4.
whisper-1was hardcoded inconductor_api.py. It is the cheapest thing to get wrong and the most expensive to trust, because it does not fail silently — it fills silence with fluent invention.Over one week of Daniel's voice notes it produced a Korean news sign-off, a YouTube outro, and turned "billing" into "building" in a message he was drafting for executives.
Measured, not assumed
One second of a 440 Hz sine tone, against the live API:
gpt-transcribe""gpt-4o-mini-transcribe""whisper-1"Oh"Whisper invented a word from a pure tone, on demand, which is the failure mode above reproduced in one request.
The change
Default is now
gpt-transcribe: quieter on non-speech, cheaper ($0.0045 vs $0.006), and billed by duration — which this endpoint needs, because/api/transcribereports a per-minute cost and a token-priced model would make that number a guess.CONDUCTOR_TRANSCRIBE_MODELoverrides it, matching how every other knob in this file already works.Two smaller things that follow from being honest about cost:
cost: nullrather than a plausible wrong number.gpt-4o-mini-transcribeis better than Whisper but token-billed, and this endpoint cannot know the token count.Verified live
Deployed and restarted before opening this:
32 tests, including one asserting the default is cheaper than what it replaced and one asserting a token-billed model reports no cost rather than fiction.
Summary by cubic
Makes the transcription model configurable and defaults to
gpt-transcribeinstead of hardcodedwhisper-1, which hallucinated speech on silence. The endpoint now reports cost based on the model's per-minute rate, returningnullwhen the model is token-billed, and includes the model name in responses.Bug Fixes
whisper-1inventing words on pure tones is fixed by switching the default togpt-transcribe.cost: nullinstead of a guessed number.CONDUCTOR_TRANSCRIBE_MODELto override the model.modelto the/api/transcriberesponse so output can be attributed.whisper-1price constant for compatibility.Written for commit 7646944. Summary will update on new commits.