Skip to content

fix(normalize): correct variance with reproducible CPU/CUDA regressions - #3856

Merged
LauraGPT merged 3 commits into
mainfrom
codex/pr3853-precision-regressions-20261107
Nov 7, 2026
Merged

LauraGPT merged 3 commits into
mainfrom
codex/pr3853-precision-regressions-20261107

Conversation

@LauraGPT

@LauraGPT LauraGPT commented Nov 7, 2026

Copy link
Copy Markdown
Collaborator

Summary

Integrates #3853 while preserving the original contributor commit and authorship. The appended signed commit synchronizes retained main content to the exact tested integration tree without rewriting history. UtteranceMVN's both-enabled branch excludes centered padding from variance and divides by standard deviation once. Other normalization branches retain their existing behavior.

Adds the submitted regression file to NumPy CI. The audio integration test now explicitly uses IEEE FP32 rather than environment-dependent TF32, restoring both backend settings afterward. AST comparison verifies every original assertion, tolerance, reference and test body is unchanged.

Maintainer validation

  • On unchanged main: 21 of 55 submitted CPU/CUDA cases fail, including a preexisting TF32-sensitive mean-only encoder comparison. Candidate: 104 pass and 2 CUDA audio comparisons fail out of 106 selected cases; that failure is retained.
  • Falsifiable precision check: with TF32 disabled only in the test process, unchanged main still fails the variance-normalized audio case (max valid-feature difference 2.441162); the candidate passes both CUDA cases at unchanged tolerances.
  • The added fixture passes all 4 CPU/CUDA audio cases, 12 real inference calls, 629 CUDA kernels. The 102 unaffected selected cases are retained against unchanged source, giving 106 unique accepted selected cases. No full-suite claim.
  • Real L20Z checks cover FP64/FP32/FP16/BF16 normalization, padding/order/batch invariance, epsilon, gradients, unchanged switches, local generated checkpoint round-trip, and repository audio through a small randomly initialized Transformer. These are not pretrained recognition-quality results.
  • Private HOME/cache and kernel network isolation; no new pretrained weights or shared/production environment changes. Exact new-head hosted CI is required before normal merge; hosted CPU CUDA skips do not replace L20Z evidence.

Compatibility

Both norm_means=True and norm_vars=True now produce corrected feature values. Existing checkpoints trained with both enabled require recognition-quality reevaluation. norm_vars defaults to False; this PR does not claim pretrained quality is unchanged. No release, Nano work or production modification is included.

BillVan1945 and others added 3 commits November 8, 2026 00:56
Preserve the original PR #3853 commit. Synchronize retained main content to the exact tested integration tree without rewriting history. Use IEEE FP32 only inside audio integration tests, restore backend settings, and keep all assertions and tolerances unchanged. Add the regression file to the existing NumPy matrix.

Signed-off-by: LauraGPT <zhifu.gzf@alibaba-inc.com>
Resolve the confirmed companion/main conflict by retaining the accepted tokenizer changes and both CI selections. All 58 combined CPU/CUDA regressions passed on the exact combined source. No history rewrite or protection bypass.

Signed-off-by: LauraGPT <zhifu.gzf@alibaba-inc.com>
@LauraGPT
LauraGPT merged commit 1d8979d into main Nov 7, 2026
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants