Questions on balanced data

Thank you for the nice project. I have a few questions about label distributions.

Does the AUCMLoss or CompositionalAUCLoss work for balanced training data? Suppose both train and test are (almost) balanced binary classification tasks.
What if the training data is balanced while the test data is imbalanced?

Additionally, I have noticed that AUCMLoss converges very slowly on balanced training data. Do you have any suggestions on the training parameters?

Thanks again!

Thanks for your interest!

Yes, I think they work for balanced training data as long as you optimize AUC. Nevertheless, it is unclear if it can deliver better performance on standard cross-entropy loss.
It is a good question. If the test data is imbalanced, you should definite consider AUC maximization or even AUPRC maximization (SOAP) or partial AUC maximization (SOPA-s).

There are many factors could affect the convergence, including the nature of your data, the model structure and also the optimizer.

Navigation Menu