[2310.20708] Sudden Enhancements to Anticipated Enchancment for Bayesian Optimization

Obtain a PDF of the paper titled Sudden Enhancements to Anticipated Enchancment for Bayesian Optimization, by Sebastian Ament and 4 different authors

Obtain PDF
HTML (experimental)

Summary:Anticipated Enchancment (EI) is arguably the preferred acquisition operate in Bayesian optimization and has discovered numerous profitable purposes, however its efficiency is commonly exceeded by that of newer strategies. Notably, EI and its variants, together with for the parallel and multi-objective settings, are difficult to optimize as a result of their acquisition values vanish numerically in lots of areas. This issue usually will increase because the variety of observations, dimensionality of the search house, or the variety of constraints develop, leading to efficiency that’s inconsistent throughout the literature and most frequently sub-optimal. Herein, we suggest LogEI, a brand new household of acquisition features whose members both have an identical or roughly equal optima as their canonical counterparts, however are considerably simpler to optimize numerically. We display that numerical pathologies manifest themselves in “basic” analytic EI, Anticipated Hypervolume Enchancment (EHVI), in addition to their constrained, noisy, and parallel variants, and suggest corresponding reformulations that treatment these pathologies. Our empirical outcomes present that members of the LogEI household of acquisition features considerably enhance on the optimization efficiency of their canonical counterparts and surprisingly, are on par with or exceed the efficiency of latest state-of-the-art acquisition features, highlighting the understated position of numerical optimization within the literature.