Hotwords/keywords support for Moss and Granite - #120
Open
moribm wants to merge 6 commits into
Open
Conversation
Introduce an optional keyword/hotword biasing hint on the public run params, shared by every family that opts in. - New transcribe_run_params::hotwords field and the TRANSCRIBE_FEATURE_HOTWORDS capability bit. - Shared run_params_hotwords() accessor that is struct_size / NULL / empty-string aware, so old callers that predate the field are safe. - abihash and the Python, TypeScript, Rust, and Swift bindings regenerated/updated to match. No family honors the field yet; this commit is the ABI surface only, kept atomic so the abihash drift gate stays green. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
- Advertise TRANSCRIBE_FEATURE_HOTWORDS for moss-transcribe-diarize. - Compute the baked prompt's body->close split at load (first eos in the suffix) and insert the caller's hotword tokens there at run time, leaving the prefix, audio-pad positions, and closing tokens untouched. - An empty/omitted list is byte-identical to the unbiased prompt. - Add a pure host-side prompt-builder unit test covering split insertion, the clamp fallback, and the run_params_hotwords guard. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
- Advertise TRANSCRIBE_FEATURE_HOTWORDS on the granite AR variants. - Append IBM's trained " Keywords: <list>" clause to the task instruction so decoding biases toward the caller's terms. The surface form is load-bearing, so the list is passed through verbatim. - Warn when combined with word timestamps or diarization, which IBM does not document for keyword biasing. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Parse --hotwords "w1, w2, w3" and wire it into both run-param sites. Families that do not advertise TRANSCRIBE_FEATURE_HOTWORDS ignore it. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Keep the reference harness aligned with the runtime: the moss dump/WER runners append the Chinese hotword clause, and the granite dump/WER runners append " Keywords: <list>". Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Document the --hotwords flag, the transcribe_run_params::hotwords field, the load-bearing surface-form caveats (silent fallback on a paraphrase), and term-count guidance across the moss and granite model pages. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Added hotwords/keywords as a string that can be provided to models that support it. I've integrated it with Moss-Transcribe-Diarize and Granite models. Likely other models support it as well but I haven't reviewed all of them (maybe in a future PR if this one gets accepted?).
I've separated it into 6 commits to make it easier to review, please let me know what changes are necessary. I tried to keep it stylistically as close to the existing project as possible.
Areas with most changes:
AI Assistance
Copilot with Claude Opus 4.8 was used to generate the changes. I have reviewed all the output and it looks reasonable, however, I have not written any C++ code in 15+ years. I'm mostly familiar with Java, Kotlin, TypeScript, and Python.
Code also reviewed with Fable 5.
I reviewed all the documentation and some of the papers for the models as well and provided it as context for generating the code.
There are some minor variations in prompting depending on the features and model that are enabled for Granite models. This is difficult to verify exactly on my part and I'm not sure if the documentation for Granite is completely accurate here. Specifically the plus model has a leading whitespace for the prompt in the documentation.
https://huggingface.co/ibm-granite/granite-speech-4.1-2b-plus
https://huggingface.co/ibm-granite/granite-speech-4.1-2b
Validation
I tested this with my local meeting notes and it works well for both Moss and Granite (granite-speech-4.1-2b-plus) which are the two models that I use.
I did not test WER or any stats.