Improved variational inference with dynamic routing flow

Abstract

How to transform a family of simple distributions to approximate an intractable posterior distribution in a scalable manner is a key problem in variational inference. Recent researches have been studied to generate a flexible approximate posterior distribution by utilizing a flow-based model with a long flow structure. However, when the dimension of the data increases, a long flow structure brings the problem of computational complexity and large variance. Therefore, we propose a variational inference with dynamic routing flow (DRF), which ensures the multiformity of the flows with shorter flow structure in this paper. The proposed model consists of a series of iterative sub-modules transformations, and each sub-module is enabled with a greater expression power by routing-by-agreement to achieve a group of weighted mixture of invertible transformations. These sub-modules route can be computed parallelly and they share the same group of invertible functions, which makes the inference more efficiency. The experimental results show that the proposed DRF model achieves significant performance on the posterior distribution estimation both in accuracy and precision.