Misspecification uncertainties in near-deterministic regression


The anticipated loss is an higher certain to the mannequin generalization error which admits sturdy PAC-Bayes bounds for studying. Nonetheless, loss minimization is thought to disregard misspecification, the place fashions can’t precisely reproduce observations. This results in important underestimates of parameter uncertainties within the massive information, or underparameterized, restrict. We analyze the generalization error of near-deterministic, misspecified and underparametrized surrogate fashions, a regime of broad relevance in science and engineering. We present posterior distributions should cowl each coaching level to keep away from a divergent generalization error and derive an ensemble {ansatz} that respects this constraint, which for linear fashions incurs minimal overhead. The environment friendly strategy is demonstrated on mannequin issues earlier than utility to excessive dimensional datasets in atomistic machine studying. Parameter uncertainties from misspecification survive within the underparametrized restrict, giving correct prediction and bounding of take a look at errors.

Supply hyperlink


Please enter your comment!
Please enter your name here