Ought to We Stick the Touchdown?


Obtain a PDF of the paper titled Linear Convergence of Black-Field Variational Inference: Ought to We Stick the Touchdown?, by Kyurae Kim and a pair of different authors

Obtain PDF

Summary:We show that black-box variational inference (BBVI) with management variates, notably the sticking-the-landing (STL) estimator, converges at a geometrical (historically known as “linear”) price beneath excellent variational household specification. Specifically, we show a quadratic certain on the gradient variance of the STL estimator, one which encompasses misspecified variational households. Mixed with earlier works on the quadratic variance situation, this straight implies convergence of BBVI with the usage of projected stochastic gradient descent. For the projection operator, we think about a site with triangular scale matrices, which the projection onto is computable in $Theta(d)$ time, the place $d$ is the dimensionality of the goal posterior. We additionally enhance current evaluation on the common closed-form entropy gradient estimators, which permits comparability in opposition to the STL estimator, offering specific non-asymptotic complexity ensures for each.

Submission historical past

From: Kyurae Kim [view email]
Thu, 27 Jul 2023 06:32:43 UTC (263 KB)
Mon, 23 Oct 2023 19:19:11 UTC (298 KB)
Wed, 21 Feb 2024 23:03:37 UTC (13,054 KB)

Supply hyperlink


Please enter your comment!
Please enter your name here