A unifying view for the mixture model of sparse Gaussian processes

Abstract

The mixture of Gaussian processes (MGP) is generally accurate but slow in multimodal probabilistic regression. To reduce the computational burden of MGP, a mixture of sparse Gaussian processes (MSGP) with a fully independent training conditional (FITC) approximation was designed. However, MSGP with FITC is not always accurate. In this paper, we propose MSGP with a variational inference (VI) approximation to overcome this problem and establish a unifying view of MSGPs with FITC and VI based on their marginal likelihoods to explore the relationship between two MSGPs. Experiments on synthetic and real-world datasets show that MSGP with VI is more favorable than MSGP with FITC in certain behaviors, such as lower heteroscedasticity of noise, less clumping behavior of inducing inputs, more effective recovery of the full MGP and higher probabilistic regression accuracy at similar efficiency.