Fix comp_used staleness and NaN under sharing - #243
Conversation
Review findings addressedThe reviewer ran the revert matrix per change. Unlike #241, this PR's tests do go red on revert — but it found a subtler version of the same blind spot, plus a real gap I introduced. 1. Two tests skipped rather than failed under the bug. Their guards ( Fixed by forcing the merge deterministically ( 2. Added an explicit 3. The freeze comment overclaimed. It said unused columns are frozen at their last finite value, but Confirmed, not changed
Two low-confidence notes I am leaving as-is: a |
Closes #240.
Root cause
identify_shared_componentsrebuiltcomp_used = np.ones(...)on every call. The merge loop skips any pair wherek1 == k2, so oncecomp_listis already merged, no merge fires and the mask comes back all-True while half the columns are dead.The unmasked mixture update then divided 0/0 for every merged-away column:
and the fit returned normally.
Fix
comp_usedis now derived from the finalcomp_list— exactly the set of referenced columns — matchingAMICATorchNG.comp_used, which is a property derived the same way and so cannot go stale.The mixture update now skips unused columns and freezes them at their last finite value, as the torch backend already did. Fortran carries NaN there harmlessly behind its own
comp_usedmask; keeping them finite means a fit cannot report success while holding NaN parameters.np.errstateguards the discarded branch of eachnp.where, so the 0/0 that is computed and thrown away no longer warns every iteration.Parity
Default (disjoint
comp_list) is bit-identical for single- and multi-model — verified before/after on real sample EEG. Every change is behind theusedmask, which is all-True without a merge.Tests
pamica/tests/test_numpy_share_comps.py, 5 tests. Verified by revertingcomp_usedtonp.ones(...):test_comp_used_matches_the_columns_comp_list_referencesandtest_sharing_leaves_finite_mixture_parametersboth fail.What is deliberately not here
#242 — the shared-column A-update, where numpy takes one step per contributing model instead of Fortran's single weighted application — is not fixed in this PR.
I implemented it (a one-line change to use the
dAkalready computed above, identical to the torch backend), then wrote three tests for it: a magnitude comparison against private columns, a cross-backend comparison againstAMICATorchNG, and a finiteness check. All three passed equally with the per-model loop and with the single weighted application — none distinguished the two. The magnitude test's premise was simply wrong: a merged column legitimately moves ~3x a private one, because the averaged direction need not be small.Rather than ship a numerical change to the A-update with tests that give false confidence — the exact failure caught in #241's review — the change is reverted and #242 stays open until it has a test that fails on revert. The comment at the loop records why.