Improving the Mean-Field Approximation in Belief Networks Using Bahadur's Reparameterisation of the Multivariate Binary Distribution

Abstract

We develop a new extension to the Mean-Field approximation for inference in graphical models which has advantages over other approximation schemes which have been proposed. The method is economical in its use of variational parameters and the approximating conditional distribution can be specified with direct reference to the dependence structure of the variables in the graphical model. We apply the method to sigmoid belief networks.