Pseudo Independent Conditional Approximation for Training the Mixtures of Gaussian Processes

Abstract

The mixture of Gaussian processes (MGP) is a powerful probabilistic model for regression and classification. However, how to effectively infer the posteriors and learn the parameters in the model is still a very challenging problem due to the exponential complexity of computation. Although several approximation schemes have been utilized to reduce the computational cost, they usually do not provide reasonable interpretations. In this paper, we first propose a specific variational approximation mechanism for the MGP model in which the joint distribution of latent indicators and latent functions can be factorized as the product of two independent variational distributions. The resulted inference procedure can be well interpreted under the framework of information theory. Inspired by this perspective, we then propose a new approximation method called pseudo independent conditional approximation (PIC) for training the MGP model. It is demonstrated by the experimental results that our proposed training method is more effective than the other existing methods.