feat(android/voice): online/offline engine adapters, safe model switching and import completion (#149) by Slacker-LLC · Pull Request #150 · Slacker-LLC/openIME

Slacker-LLC · 2026-10-20T02:53:26Z

Issue #149：Android 语音模型与识别运行时解耦

本 PR 仍只推送到 feat/voice-engine-adapters-149，目标分支为 main，未合并。

已实现

统一 VoiceEngineAdapter / VoiceEngineRegistry：根据经过校验的 manifest 选择已编译引擎；未知引擎或格式/语音模式拒绝并解释原因。
统一 VoiceModelRuntime / VoiceModelSession，保留原 sherpa-onnx 流式接口及 PCM 录音管线，支持纯最终结果的离线推理（不伪装 partial）。
两种 sherpa 运行时：OnlineRecognizer 与基于 sherpa-onnx 1.13.6 真实 API 的 OfflineRecognizer。离线模型需明确声明 engineId=sherpa-onnx-offline、recognitionMode=offline 及 family/file roles；目前支持离线 Paraformer、Transducer、Zipformer CTC、NeMo CTC 的适配定义。这是在线/离线两种识别模式，不是声称适配了第二种独立推理框架。
向后兼容的 manifest 字段：engineId、recognitionMode、schemaVersion、modelFormat、sampleRate、channelCount、quantization、licenseId、sourceUrl，新增字段纳入信任指纹，旧模型指纹不变。
模型适配能力决定设置页标点、热词、流式模式和导入时提示；录音不再与具体解码器类型耦合。
引擎/模型切换时通过已选模型 identity 重新初始化识别运行时，并等待旧会话释放，防止旧回调释放新 recognizer。
导入采用暂存、校验、预加载与旧版本备份/失败回滚；启用模型在工作线程先做内存准入和 native 初始化，失败时不切换偏好；模型包不能执行自带 native 库或脚本。
导入成功显示“导入完成 → 完成”；点击完成立即重新初始化输入法的语音组件，保留当前输入框、候选与布局，避免直接杀 IME 进程。
单元测试与独立架构文档 docs/VOICE_ENGINE_ADAPTERS.md。

验证与安全边界

新增 API 34 Android instrumentation：利用仓库自带 voice-beta4.wav 录音，调用真实内置 Paraformer 解码并检查最终文本；该测试必须以 CI 通过为准。\n- GitHub Actions Android CI：./gradlew :app:testDebugUnitTest :app:lintDebug :app:assembleDebug :app:assembleDebugAndroidTest，通过后继续 API 26/29/31/34 模拟器仪表测试；以 CI 实际结论为准，未通过前不宣称完成验证。
仍需可用离线权重完成真实语音端到端识别与性能测试；模型清单可校验不等于手机上已经确认精度/稳定性。
无 INTERNET 权限、无录音落盘、禁用任意 native 插件装载。
新的独立后端（MNN/ncnn/LiteRT 等）属于 [架构] Android 语音模型/推理框架解耦：自动适配、可扩展引擎与模型即插即用 #149 的持续扩展能力与后续独立工作，不在未验证前宣称兼容。
[功能建议] 完善中文流式语音模型兼容性，建立模型仓库与 App 内下载目录 #147 仅维护官方链接与具体模型家族适配，不建立额外模型仓库。

Related #149、#147。该 PR 不自动关闭 #149；需先审阅构建、仪表测试及实际 ASR 验证。

本轮继续补充

独立的 VoiceModelInstallTransaction：可回滚 staged→installed 替换；恢复崩溃留下的备份，不覆盖已装的可用模型。
VoiceModelRepository 跨实例安装互斥与持久化信任状态恢复，避免预热/安装竞争导致异常回滚。
VoiceModelInstallTransactionTest 六项验证：成功、校验失败、原生加载失败回滚、两种中断恢复和危险路径拒绝。
Android 14 增加 VoiceOfflineEngineInstrumentedTest：CI 临时下载 sherpa 官方 NeMo CTC 英文小模型，ADB 放入调试应用私有目录，以真正的 sherpa OfflineRecognizer 完成最终结果解码，验证离线模式不输出流式 partial；其他 API 不下载模型，不影响正式 APK 体积。
完整验收以当前功能分支最新 CI 为准；该测试的成功结果不代表所有第三方离线模型均受支持。
录音期模型操作准入：记录当前语音手势 generation，禁止导入、切换、删除；迟到的上一会话 final 不会解除当前手势保护，并添加对应 JUnit 测试。

…ontract

…data

…pters

…uage parsing

…expression

…ity negotiation

…tibility limits

…milies

…fline decoding

…ss ASR modes

… families

…ntization and provenance

…-flight ASR sessions

…a model

…model

…ollback

…le manager

…ping active editor

…ecycle on confirmation

…del fields

…on and Done restart

… changes

…ded PCM on API 34

…system transaction

…ate backup races

…placement

…l recovery

…only model

…fline ASR test

…uring capture

… service teardown

…recording

…pters-149

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

## 范围语音丢字、空格滑动光标分隔符、九键拼音栏锁争用三处修复。 **语音句尾丢字** - 松手后再录 200ms，由采集线程自己停止录音器，并用 `READ_NON_BLOCKING` 读空录音器内部剩余的音频；外部 stop 只作看门狗（再等 400ms）。 - Paraformer 收尾静音由 0.3 秒加到 0.66 秒，与 sherpa-onnx 官方 `python-api-examples/online-decode-files.py:414` 一致。 - 语音松手后等转写期间，空格滑动不再触发 `prepareForManualInput`（它会取消整段识别）。 **空格滑动光标** - 预编辑里的 `'` 和空格不再占一格（`ni|'hao` 与 `ni'|hao` 是同一位置），逻辑在 `PreeditCursorPolicy`。 **九键拼音栏** - `NineKeyLocalDecoder` 的只读查询（trie 与各表在 init 后不再写入）去掉 `@Synchronized`，主线程拼音栏查询不再排在候选线程 `resolve()` 后面；`readingOptions` 加 16 项 LRU。 ## 验证 - `:app:testDebugUnitTest`、`:app:lintDebug`、`:app:assembleDebug`、`:app:assembleDebugAndroidTest` 在与 #150、#152 合并后的分支上本地通过。 - 新增测试：`PreeditCursorPolicyTest`、`NineKeyDecoderThreadingTest`、`LocalAudioVoiceBackendTest` 的残余排空用例。 - CI：Build and verify 与 Compatibility API 26 / 29 / 31 / 34。未验证（需真机）：200ms 宽限与残余排空对句尾丢字的实际改善量（`stop()` 之后 `read()` 能否取出残留音频因机型而异，可对照日志 `captureFinished … capturedSamples` 与 `captureResidual samples=`）；0.66 秒静音对流式 Paraformer 末尾字的影响；九键在主线程跑 `resolveNineKey` 的耗时。 🤖 Generated with [Claude Code](https://claude.com/claude-code)

…pters-149 Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

从 main（含 #152、#151、#150）出发，只改 `VERSION` 与 `CHANGELOG.md`，代码与已合并的三个 PR 完全一致。1.2.0 之后新增功能与面板，按 docs/RELEASE.md 升 MINOR。 - 界面：新增“更多 → 编辑工具栏”；表情与符号面板共用底部分类栏；回车后清除工具栏焦点框。 - 输入：退格按完整表情删除；空格滑动光标不再被预编辑分隔符占格；九键拼音栏不再与候选线程争锁。 - 语音：减少句尾丢字；新增 sherpa-onnx 离线模型适配；模型安装先暂存、校验再提交；导入完成后无需重启输入法。验证：`:app:testDebugUnitTest`、`:app:lintDebug`、`:app:assembleDebug`、`release_check.py check` 通过；三个功能 PR 的 Build and verify 与 Compatibility API 26 / 29 / 31 / 34 均通过；API 31 模拟器完整仪表测试 222 项通过；`typing_stress_e2e.py` 在模拟器上 14 项全部通过（`nine-rail` 在仪表测试后的残留数据上红过一次，清数据重跑通过，v1.2.0 基线同样通过）。已知限制：没有真机验证句尾丢字的实际改善量，也没有在手机上验证真实离线权重的精度与性能。 🤖 Generated with [Claude Code](https://claude.com/claude-code)

Slacker-LLC added 30 commits October 20, 2026 10:46

feat(voice): introduce engine adapter registry and model capability c…

fe78eba

…ontract

feat(voice): add backward-compatible engine and recognition mode meta…

5314f3d

…data

refactor(voice): route runtime creation through registered engine ada…

7997f9c

…pters

fix(voice): validate mode before adapter selection and normalize lang…

3eddd16

…uage parsing

feat(voice): expose verified adapter capabilities during model import

9e2dbb3

feat(voice): show matched engine and real model capabilities in settings

5f90002

fix(voice): import engine identifier extension and clarify UI string …

661f335

…expression

test(voice): cover adapter routing, legacy trust metadata and capabil…

08536f6

…ity negotiation

docs(voice): describe adapter boundary, model manifest and real compa…

27f6472

…tibility limits

docs(voice): link adapter architecture from docs index

fa8f92c

docs(voice): document engine registry and mode support

388f81a

fix(voice): keep public import DTO independent of internal engine enum

e349910

fix(voice): keep public import DTO independent of internal engine enum

34af4f5

refactor(voice): leave model-family-specific validation to each engine

d4f4215

refactor(voice): make non-sherpa adapters free to define their own fa…

3ff5551

…milies

refactor(voice): generalize ASR session and runtime for online and of…

4b39ebd

…fline decoding

refactor(voice): share capture, preloading and session ownership acro…

fd52bf8

…ss ASR modes

refactor(voice): share capture, preloading and session ownership acro…

035e82a

…ss ASR modes

feat(voice): implement real sherpa offline ASR adapter for four model…

c2667ed

… families

feat(voice): register offline decoder alongside online sherpa adapter

64ade31

feat(voice): extend model manifest with optional v2 input format, qua…

14e8dd1

…ntization and provenance

feat(voice): preflight audio format and model data format per adapter

7462544

feat(voice): preflight audio format and model data format per adapter

04b0492

fix(voice): pin candidate runtimes for safe activation and protect in…

d932b97

…-flight ASR sessions

fix(voice): pin candidate runtimes for safe activation and protect in…

6af83e1

…-flight ASR sessions

feat(voice): support pinned runtime initialization before activating …

8ad1ed1

…a model

feat(voice): isolate offline model preflight from currently selected …

47b4ad3

…model

feat(voice): stage models transactionally with native preflight and r…

8c83000

…ollback

feat(voice): make model identity and idle checks available to lifecyc…

8d42685

…le manager

feat(voice): guard offline session ownership across engine changes

8bbe781

Slacker-LLC added 6 commits October 20, 2026 11:12

feat(ime): reinitialize voice runtime after model import without drop…

57c878e

…ping active editor

feat(voice): show Done after import and immediately restart voice lif…

aee2dc8

…ecycle on confirmation

fix(voice): let sherpa offline config infer decoder from populated mo…

f385fba

…del fields

docs(voice): document online/offline adapters, transactional activati…

027b57f

…on and Done restart

fix(voice): roll back user-trust approval when model replacement fails

1746e09

test(voice): cover offline adapter and v2 manifest compatibility

67692ac

Slacker-LLC changed the title ~~refactor(android/voice): introduce engine adapters and model capability negotiation (#149)~~ feat(android/voice): online/offline engine adapters, safe model switching and import completion (#149) Oct 20, 2026

Slacker-LLC added 14 commits October 20, 2026 11:21

fix(voice): invalidate loaded recognizer when engine or mode metadata…

fc95794

… changes

test(android): exercise real bundled Paraformer inference using recor…

6caae4e

…ded PCM on API 34

feat(voice): extract recoverable model installation transaction

98a78db

fix(voice): let package copier create staging directory

3c16242

fix(voice): recover interrupted installs and use rollback-tested file…

f8f38cb

…system transaction

fix(voice): serialize selection, recovery and model install to elimin…

4c24a17

…ate backup races

fix(voice): recover previous user approval after interrupted model re…

4afb13a

…placement

test(voice): cover model replacement, rollback and interrupted-instal…

f8fe0d5

…l recovery

test(voice): run genuine offline NeMo CTC inference with official CI-…

c9ab21c

…only model

ci(voice): download official NeMo CTC weights only for real API 34 of…

cb4ee96

…fline ASR test

feat(voice): track active recording generation to prevent switching d…

7487422

…uring capture

feat(voice): mark recording ownership through final, cancellation and…

0d1ba9e

… service teardown

fix(voice): block import, delete, and model activation during active …

299052e

…recording

test(voice): assert stale final cannot unlock newer recording

c7ddd36

Slacker-LLC mentioned this pull request Oct 22, 2026

fix(android/ui): unify emoji/symbol navigation, grapheme delete, toolbar editor #152

Merged

Slacker-LLC and others added 2 commits October 23, 2026 07:42

Merge remote-tracking branch 'origin/main' into feat/voice-engine-ada…

5fdc1b6

…pters-149

docs: add the voice engine adapter changes to Unreleased

ed0d2b5

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

Merge remote-tracking branch 'origin/main' into feat/voice-engine-ada…

882cbdb

…pters-149 Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

Slacker-LLC merged commit 58f8c41 into main Oct 23, 2026
7 checks passed

Slacker-LLC deleted the feat/voice-engine-adapters-149 branch October 23, 2026 00:22

Slacker-LLC mentioned this pull request Oct 23, 2026

feat: 发布 1.3.0，工具栏编辑、表情退格、语音离线适配与句尾丢字修复 #153

Merged

Navigation Menu

Repository navigation

feat(android/voice): online/offline engine adapters, safe model switching and import completion (#149) - #150

feat(android/voice): online/offline engine adapters, safe model switching and import completion (#149)#150
Slacker-LLC merged 57 commits into
mainfrom
feat/voice-engine-adapters-149

Slacker-LLC commented Oct 20, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

Slacker-LLC commented Oct 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Issue #149：Android 语音模型与识别运行时解耦

已实现

验证与安全边界

本轮继续补充

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Slacker-LLC commented Oct 20, 2026 •

edited

Loading