Skip to content

Expose max_new_tokens run option - #74

Open
vladkalinichencko wants to merge 1 commit into
handy-computer:mainfrom
vladkalinichencko:expose-max-new-tokens
Open

Expose max_new_tokens run option#74
vladkalinichencko wants to merge 1 commit into
handy-computer:mainfrom
vladkalinichencko:expose-max-new-tokens

Conversation

@vladkalinichencko

@vladkalinichencko vladkalinichencko commented Jul 8, 2026

Copy link
Copy Markdown

This exposes a per-run max_new_tokens option through the C API and all generated language bindings.

Some autoregressive ASR families can accept the audio input but still hit the fixed generation budget before emitting EOS. That returns OUTPUT_TRUNCATED with a partial transcript, but callers previously had no way to request a larger output budget.

The default behavior stays unchanged: non-positive values use the existing family defaults. Qwen3-ASR and Canary honor a positive value for single and batch runs, bounded by the decoder context.

Rebased onto the current main / 0.2.0 API, preserving the newer diarization fields and generated ABI.

Validated with:

  • Python and Rust binding-generator checks
  • Swift ABI hash check
  • cargo fmt --all --check
  • cargo check -p transcribe-cpp
  • TypeScript typecheck
  • native CMake build and transcribe_api_smoke test

@cjpais

cjpais commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Thanks I will take a look tomorrow

@cjpais

cjpais commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Sorry, it's taking me a bit longer to review this since it's a header change. I will want to take a closer look, which will just take some time

@vladkalinichencko

Copy link
Copy Markdown
Author

No worries, thanks. I have rebased this onto the current main / 0.2.0 API, regenerated every binding and the Swift ABI hash, and rerun the native C API smoke test plus the Rust, TypeScript, Python, and Swift checks. The companion Handy PR cjpais/Handy#1633 is also rebased and conflict-free.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants