Repository navigation
fix(runtime): handle blank and incomplete CMVN files - #3842
Conversation
LauraGPT
left a comment
There was a problem hiding this comment.
Reviewed all three changed files at b04804c96dcfc9a4cb945f3d108a3fb8908d1205. Both standalone runtime readers now skip blank lines, guard a truncated section tag, and fail at construction with the exact CMVN path when either statistics block is absent. Valid float64 statistics and the existing parsing format are preserved.
Independent validation on current main 7fdb20978d235c0f9ff27c42cab4b1f5912b78ba, candidate tree d5ef784092736d2f4815d36a533f53ac21c8a837:
- The exact baseline reproduced 26 failures / 2 passes; the candidate passed all 50 selected parser and package tests plus 4 subtests and
py_compile. - Tests cover both runtime copies, blank/truncated/missing/compact forms, file-specific errors, and equality of statistics plus real filterbank/LFR/CMVN output for the shipped
am.mvnfixture. - Exact-head Actions artifacts and logs were downloaded and verified: ONNX 36 tests on Python 3.11 and 3.12, torch package 14 tests on 3.10 and 3.12, and runtime evaluation 52 tests on 3.10 and 3.12; all passed. Artifact byte sizes, SHA-256, ZIP integrity, JUnit and source-commit markers were checked.
- This is CPU parser/package behavior only: no model weights or inference engines are changed, and no ASR-quality, GPU, Windows, NPU or full-suite claim is made.
GitHub's displayed potential merge remains cached against the old base and was not used. The accepted tree was derived from the fresh current main plus the three exact-head blobs, with no overlap. This is a COMMENTED review, not an approval or protection bypass.
Accept versioned standalone package bytes and reused exact-source #3842 CPU/parser evidence. Signed-off-by: LauraGPT <zhifu.gzf@alibaba-inc.com>
Accept versioned standalone package bytes and reused exact-source #3842 CPU/parser evidence. Signed-off-by: LauraGPT <zhifu.gzf@alibaba-inc.com>
Summary
The standalone
funasr-onnxandfunasr-torchCMVN readers raiseIndexErroron blank lines or truncated section headers. Files with no recognized statistics can also return empty arrays and fail later during inference.Apply the guards from #3760 to both runtime readers: skip blank lines, bound the next-line lookup, and raise a
ValueErrornaming the file when either set of statistics is missing. Valid statistics and their float64 dtype are preserved.Related to #3759. Thanks to @Lesereingrape for the original report and main-package fix in #3760.
Type of change
Validation
Tested on Linux with Python 3.10.12, NumPy 1.26.4 and kaldi-native-fbank 1.22.5, using CPU only.
Run the regression tests from the repository root with NumPy, kaldi-native-fbank and pytest installed:
mainat04e6fb63): 26 failed, 2 passed.am.mvnproduces identical statistics and filterbank/LFR/CMVN features after adding blank lines. No model downloads are needed.Related package checks:
Result: 50 passed, 4 subtests passed. Both runtime wheel/sdist builds and
twine checkalso passed.python -m compileall funasr examples testsUser impact
Runtime users can load valid CMVN files containing blank lines and receive a file-specific error during frontend construction when statistics are missing.
Notes for reviewers
This extends #3760 to the two standalone Python runtimes. Compact single-line CMVN files now fail early with a
ValueError; support for parsing that format is unchanged.