Expose max_new_tokens run option - #74
Open
vladkalinichencko wants to merge 1 commit into
Open
Conversation
Contributor
|
Thanks I will take a look tomorrow |
vladkalinichencko
force-pushed
the
expose-max-new-tokens
branch
from
July 30, 2026 12:32
1d7e660 to
b14d32e
Compare
Contributor
|
Sorry, it's taking me a bit longer to review this since it's a header change. I will want to take a closer look, which will just take some time |
Author
|
No worries, thanks. I have rebased this onto the current main / 0.2.0 API, regenerated every binding and the Swift ABI hash, and rerun the native C API smoke test plus the Rust, TypeScript, Python, and Swift checks. The companion Handy PR cjpais/Handy#1633 is also rebased and conflict-free. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This exposes a per-run
max_new_tokensoption through the C API and all generated language bindings.Some autoregressive ASR families can accept the audio input but still hit the fixed generation budget before emitting EOS. That returns
OUTPUT_TRUNCATEDwith a partial transcript, but callers previously had no way to request a larger output budget.The default behavior stays unchanged: non-positive values use the existing family defaults. Qwen3-ASR and Canary honor a positive value for single and batch runs, bounded by the decoder context.
Rebased onto the current
main/ 0.2.0 API, preserving the newer diarization fields and generated ABI.Validated with:
cargo fmt --all --checkcargo check -p transcribe-cpptranscribe_api_smoketest