Fix ECE confidence binary crash - #1215
Conversation
The function indexed prob[:,0]/label[:,0], requiring 2D arrays, but its only caller (binary_metrics_fn) passes 1D positive-class probs and 1D 0/1 labels, so ECE/ECE_adapt always raised IndexError. Use the positive class (class 1) as confidence and the 0/1 label as target, tolerating 1D and 2D inputs. Also corrects the class-0 vs class-1 indexing.
|
I think this still crashes with PyHealth's normal binary model output. |
fixed! |
Issue
ece_confidence_binary indexed prob[:,0] and label[:,0] (calibration.py:150), requiring 2D arrays. Its only real caller, binary_metrics_fn (binary.py:91), passes 1D positive-class probabilities and 1D 0/1 labels, so ECE and ECE_adapt always raised IndexError for binary tasks via the documented API. As a secondary defect, even for 2D input the code used column 0 (class 0) despite the docstring stating the metric is computed on class 1.
Fix
Compute confidence from the positive class (prob[:,1] when 2D, else the 1D array) and use the 0/1 label as the accuracy target (label[:,1] when 2D one-hot, else the 1D array). This makes the function work with the 1D shapes the caller actually uses and corrects the class-0/class-1 indexing. ece_classwise is unaffected (it calls a separate internal helper, not this function).
Notes
Added new test file tests/core/test_calibration_binary_ece.py with two tests. No prior tests existed for this function.