Skip to content

fix(runtime): handle blank and incomplete CMVN files - #3842

Merged
LauraGPT merged 1 commit into
modelscope:mainfrom
BillVan1945:fix/runtime-cmvn-readers
Nov 5, 2026
Merged

LauraGPT merged 1 commit into
modelscope:mainfrom
BillVan1945:fix/runtime-cmvn-readers

Conversation

@BillVan1945

Copy link
Copy Markdown
Contributor

Summary

The standalone funasr-onnx and funasr-torch CMVN readers raise IndexError on blank lines or truncated section headers. Files with no recognized statistics can also return empty arrays and fail later during inference.

Apply the guards from #3760 to both runtime readers: skip blank lines, bound the next-line lookup, and raise a ValueError naming the file when either set of statistics is missing. Valid statistics and their float64 dtype are preserved.

Related to #3759. Thanks to @Lesereingrape for the original report and main-package fix in #3760.

Type of change

  • Bug fix
  • Documentation
  • Example or demo
  • Runtime or deployment
  • Benchmark or evaluation
  • Model/training change

Validation

Tested on Linux with Python 3.10.12, NumPy 1.26.4 and kaldi-native-fbank 1.22.5, using CPU only.

Run the regression tests from the repository root with NumPy, kaldi-native-fbank and pytest installed:

python -m pytest -q tests/test_runtime_cmvn.py
  • Before the fix (main at 04e6fb63): 26 failed, 2 passed.
  • After the fix: 28 passed.
  • Tests cover both frontend constructors, blank-line placement, missing statistics and truncated sections.
  • The shipped am.mvn produces identical statistics and filterbank/LFR/CMVN features after adding blank lines. No model downloads are needed.

Related package checks:

python -m pytest -q tests/test_runtime_cmvn.py \
  tests/test_funasr_torch_package.py tests/test_funasr_onnx_release.py

Result: 50 passed, 4 subtests passed. Both runtime wheel/sdist builds and twine check also passed.

  • python -m compileall funasr examples tests
  • Docs or links checked
  • Runtime/deployment command tested (CPU frontend checks above)

User impact

Runtime users can load valid CMVN files containing blank lines and receive a file-specific error during frontend construction when statistics are missing.

Notes for reviewers

This extends #3760 to the two standalone Python runtimes. Compact single-line CMVN files now fail early with a ValueError; support for parsing that format is unchanged.

@LauraGPT LauraGPT left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed all three changed files at b04804c96dcfc9a4cb945f3d108a3fb8908d1205. Both standalone runtime readers now skip blank lines, guard a truncated section tag, and fail at construction with the exact CMVN path when either statistics block is absent. Valid float64 statistics and the existing parsing format are preserved.

Independent validation on current main 7fdb20978d235c0f9ff27c42cab4b1f5912b78ba, candidate tree d5ef784092736d2f4815d36a533f53ac21c8a837:

  • The exact baseline reproduced 26 failures / 2 passes; the candidate passed all 50 selected parser and package tests plus 4 subtests and py_compile.
  • Tests cover both runtime copies, blank/truncated/missing/compact forms, file-specific errors, and equality of statistics plus real filterbank/LFR/CMVN output for the shipped am.mvn fixture.
  • Exact-head Actions artifacts and logs were downloaded and verified: ONNX 36 tests on Python 3.11 and 3.12, torch package 14 tests on 3.10 and 3.12, and runtime evaluation 52 tests on 3.10 and 3.12; all passed. Artifact byte sizes, SHA-256, ZIP integrity, JUnit and source-commit markers were checked.
  • This is CPU parser/package behavior only: no model weights or inference engines are changed, and no ASR-quality, GPU, Windows, NPU or full-suite claim is made.

GitHub's displayed potential merge remains cached against the old base and was not used. The accepted tree was derived from the fresh current main plus the three exact-head blobs, with no overlap. This is a COMMENTED review, not an approval or protection bypass.

@LauraGPT
LauraGPT merged commit 2172ec5 into modelscope:main Nov 5, 2026
6 checks passed
LauraGPT added a commit that referenced this pull request Nov 5, 2026
Accept versioned standalone package bytes and reused exact-source #3842 CPU/parser evidence.

Signed-off-by: LauraGPT <zhifu.gzf@alibaba-inc.com>
LauraGPT added a commit that referenced this pull request Nov 5, 2026
Accept versioned standalone package bytes and reused exact-source #3842 CPU/parser evidence.

Signed-off-by: LauraGPT <zhifu.gzf@alibaba-inc.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants