Class-attribute Priors: Adapting Optimization to Heterogeneity and Equity Goal. (arXiv:2401.14343v1 [cs.LG])


Trendy classification issues exhibit heterogeneities throughout particular person
courses: Every class might have distinctive attributes, resembling pattern measurement, label
high quality, or predictability (simple vs troublesome), and variable significance at
test-time. With out care, these heterogeneities impede the training course of,
most notably, when optimizing equity targets. Confirming this, beneath a
gaussian combination setting, we present that the optimum SVM classifier for balanced
accuracy must be adaptive to the category attributes. This motivates us to
suggest CAP: An efficient and basic technique that generates a class-specific
studying technique (e.g. hyperparameter) based mostly on the attributes of that class.
This manner, optimization course of higher adapts to heterogeneities. CAP results in
substantial enhancements over the naive strategy of assigning separate
hyperparameters to every class. We instantiate CAP for loss perform design and
post-hoc logit adjustment, with emphasis on label-imbalanced issues. We present
that CAP is aggressive with prior artwork and its flexibility unlocks clear
advantages for equity targets past balanced accuracy. Lastly, we consider
CAP on issues with label noise in addition to weighted take a look at targets to
showcase how CAP can collectively adapt to completely different heterogeneities.

Supply hyperlink


Please enter your comment!
Please enter your name here