Fix Variance ensemble metric ignoring the dim argument - #2004
Open
Nicholas022400701 wants to merge 3 commits into
Open
Nicholas022400701 wants to merge 3 commits into
Nicholas022400701 wants to merge 3 commits into
Conversation
Variance.__call__ took the sample count from the leading dimension and subtracted a mean that only broadcast along the leading dimension, so a dim other than 0 either raised a broadcast error or returned a wrong variance without warning. _update_var had the same broadcast problem for a batch_dim other than 0. Signed-off-by: 区梓灏 <116372750+Nicholas022400701@users.noreply.github.com>
The existing test_means_var only covers dim 0 and is skipped without CUDA, so the broken dim handling never ran in CI. This test runs on CPU and covers a shape that used to raise a broadcast error and a square shape that used to return a wrong variance without an error. Signed-off-by: 区梓灏 <116372750+Nicholas022400701@users.noreply.github.com>
Signed-off-by: 区梓灏 <116372750+Nicholas022400701@users.noreply.github.com>
Nicholas022400701
requested review from
NickGeneva and
dallasfoster
as code owners
September 18, 2026 13:40
Contributor
CODEOWNERS review mapCurrent for commit ⏳ @dallasfoster — 1 file(s)
⏳ @NickGeneva — 1 file(s)
No CODEOWNER
Comment |
Contributor
|
The PR appears safe to merge; the dimension-aware variance calculations and regression tests are internally consistent. SummaryThis PR fixes variance calculations that previously ignored or incorrectly broadcast non-leading ensemble dimensions.
Reviews (1) · Last reviewed commit: "Apply ruff format to the new test" |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PhysicsNeMo Pull Request
Description
closes #2003
Variance.__call__inphysicsnemo/metrics/general/ensemble_metrics.pyaccepts adimargument but still assumed the ensemble dimension is the leading one in two places. It took the sample count frominputs.shape[0]whileMean.__call__right above it usesinputs.shape[dim], and it computed the centered sum of squares astorch.sum((inputs - self.sum / self.n) ** 2, dim=dim)withself.sumalready reduced, which only broadcasts whendimis 0. For any otherdimthe call either raised a broadcastingRuntimeError, or, when the size ofdimhappened to match the size of the leading dimension, it ran through and returned a wrong variance without any warning._update_varhad the same broadcasting problem for abatch_dimother than 0.Changes:
Variance.__call__takes the sample count frominputs.shape[dim]and unsqueezes the reduced sum back alongdimbefore subtracting it, in both the distributed branch and the plain branch._update_varunsqueezestemp_sumalongbatch_dimthe same way.test/metrics/test_ensemble_metrics_dim.pycoversdim1, 2 and -1 for a shape that used to raise and for a square shape that used to return a wrong value, and checks_update_varwith the samebatch_dimagainsttorch.varon the concatenated input. The existingtest_means_varonly coversdim=0and is skipped without CUDA, so this bug never showed up in CI.Nothing else in the file changed.
Variance.updatestill only supports the leading dimension, as it did before, because it has nodimargument.Verification on a CPU GitHub Actions runner with torch CPU wheels and Python 3.12:
ruff format --check,ruff checkandinterrogateon the touched files: https://github.com/Nicholas022400701/oss-ci/actions/runs/35351315842main, all 6 cases fail, 5 with the broadcastingRuntimeErrorand the square shape with a wrong value: https://github.com/Nicholas022400701/oss-ci/actions/runs/35350614463I did not touch CHANGELOG.md in this PR. I can add a line under Fixed in the 2.3.0 section if you want it here.
AI disclosure: I used an AI coding agent to help write this patch, the tests and this description. I have read the change and the tests myself and I will answer review comments personally.
Checklist
dimargument now does what the docstring already says.Dependencies
None.
Review Process
All PRs are reviewed by the PhysicsNeMo team before merging.
Depending on which files are changed, GitHub may automatically assign a maintainer for review.
We are also testing AI-based code review tools (e.g., Greptile), which may add automated comments with a confidence score.
This score reflects the AI’s assessment of merge readiness and is not a qualitative judgment of your work, nor is
it an indication that the PR will be accepted / rejected.
AI-generated feedback should be reviewed critically for usefulness.
You are not required to respond to every AI comment, but they are intended to help both authors and reviewers.
Please react to Greptile comments with 👍 or 👎 to provide feedback on their accuracy.