Abstract
Variational inference is a powerful framework, used to approximate intractable posteriors through variational distributions. The de facto standard is to rely on Gaussian variational families, which come with numerous advantages: they are easy to sample from, simple to parametrize, and many expectations are known in closed-form or readily computed by quadrature. In this paper, we view the Gaussian variational approximation problem through the lens of gradient flows. We introduce a flexible and efficient algorithm based on a linear flow leading to a particle-based approximation. We prove that, with a sufficient number of particles, our algorithm converges linearly to the exact solution for Gaussian targets, and a low-rank approximation otherwise. In addition to the theoretical analysis, we show, on a set of synthetic and real-world high-dimensional problems, that our algorithm outperforms existing methods with Gaussian targets while performing on a par with non-Gaussian targets.
Highlights
Variational inference is a powerful framework, used to approximate intractable posteriors through variational distributions
We introduce Gaussian Particle Flow (GPF) and Gaussian Flow (GF), two computationally tractable approaches, to obtain a Variational Gaussian Approximation (VGA)
We investigate the behavior of our algorithm with non-Gaussian target distributions
Summary
Representing uncertainty is a ubiquitous problem in machine learning. Reliable uncertainties are key for decision making, especially in contexts where the trade-off between exploitation and exploration plays a central role, such as Bayesian optimization [1], active learning [2], and reinforcement learning [3]. The Gaussian family is by far the most popular variational approximation used in VI [6,7]. Gaussian variational families are easy to sample from, reparametrize, and marginalize. They are amenable to diagonal covariance approximations, making them scalable to high dimensions. Flow (GPF), a framework to approximate a Gaussian variational distribution with particles. GPF is derived from a continuous-time flow, where the necessary expectations over the evolving densities are approximated by particles. We compare our approach with other VGA algorithms, both in fully controlled synthetic settings and on a set of real-world problems
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
