[2306.02775] Enter-gradient area particle inference for neural community ensembles


Obtain a PDF of the paper titled Enter-gradient area particle inference for neural community ensembles, by Trung Trinh and three different authors

Obtain PDF
HTML (experimental)

Summary:Deep Ensembles (DEs) reveal improved accuracy, calibration and robustness to perturbations over single neural networks partly resulting from their practical range. Particle-based variational inference (ParVI) strategies improve range by formalizing a repulsion time period based mostly on a community similarity kernel. Nevertheless, weight-space repulsion is inefficient resulting from over-parameterization, whereas direct function-space repulsion has been discovered to supply little enchancment over DEs. To sidestep these difficulties, we suggest First-order Repulsive Deep Ensemble (FoRDE), an ensemble studying methodology based mostly on ParVI, which performs repulsion within the area of first-order enter gradients. As enter gradients uniquely characterize a perform as much as translation and are a lot smaller in dimension than the weights, this methodology ensures that ensemble members are functionally totally different. Intuitively, diversifying the enter gradients encourages every community to study totally different options, which is predicted to enhance the robustness of an ensemble. Experiments on picture classification datasets and switch studying duties present that FoRDE considerably outperforms the gold-standard DEs and different ensemble strategies in accuracy and calibration below covariate shift resulting from enter perturbations.

Submission historical past

From: Trung Trinh [view email]
Mon, 5 Jun 2023 11:00:11 UTC (122 KB)
Wed, 14 Feb 2024 13:24:27 UTC (761 KB)
Tue, 5 Mar 2024 16:44:43 UTC (761 KB)

Supply hyperlink


Please enter your comment!
Please enter your name here