Add support for edge cases

[ysims]

These are bugs/issues that came up while using the code on a variety of real datasets.

• Stop NaNs propogating through
• Improved check for binary tasks
• Warnings on automatically created spatial blocks that end up being small. It is currently hard for the user to see that the spatial blocks may be undersized. Small blocks negatively affect metrics and changes interpretation of metrics.

[Copilot]

🟡 Changes recommended

The updated binary/macro selection logic can still mis-handle multiclass folds that happen to contain labels {0,1}, producing incorrect metrics rather than just avoiding errors.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

This PR addresses edge cases observed on real datasets by hardening evaluation and preprocessing utilities against NaN propagation, mis-detected classification task types in CV folds, and undersized spatial CV blocks.

Changes:

• Refines binary-vs-multiclass handling in compute_classification_metrics to avoid invalid metric computation in certain folds.
• Makes feature-importance aggregation NaN-aware so a single NaN run/fold doesn’t wipe out averages and standard deviations.
• Adds warnings when KMeans-generated spatial blocks are very small, since they can destabilize fold metrics/interpretation.

File summaries

File	Description
ecosci/evaluation/metrics.py	Adjusts how average mode is selected for precision/recall/F1 to better handle edge cases in CV folds.
ecosci/evaluation/feature_importance.py	Switches seed/fold aggregation to nanmean/nanstd to prevent NaNs from propagating through summary importances.
ecosci/data/spatial.py	Warns when spatial blocks are extremely small, which can cause unstable or undefined fold-level evaluation in some settings.

Review details

Suppressed comments (1)

ecosci/evaluation/feature_importance.py:281

• Similarly here, the comment attributes NaNs to R^2 being undefined for <2-sample test sets, but that only applies when using R^2 scoring (regression). Qualifying this avoids confusion since classification permutation importance uses accuracy and won’t hit this specific NaN path.

            # Average importances across all folds (nan-aware: a fold whose test
            # set has fewer than 2 samples makes permutation importance's R^2
            # scorer undefined for that fold, and a plain mean would let that
            # one NaN-only fold wipe out every feature's average)

• Files reviewed: 3/3 changed files
• Comments generated: 3
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.