Sparse Gaussian processes with manifold-preserving graph reduction

Abstract

Gaussian processes are a popular and effective Bayesian method for classification and regression. Due to the cube time complexity with respect to the size of the training set, time consumption will be too high for Gaussian processes to deal with big data sets. Based on the manifold assumption, manifold-preserving graph reduction (MPGR) is a simple and efficient graph sparsification method originally proposed for sparse semi-supervised learning. High representative points are selected, while outliers and noisy points are excluded by MPGR with low time consumption. In this paper, we apply MPGR to sparse supervised learning, and utilize MPGR to get a sparse representation of Gaussian processes. A fatal shortcoming of MPGR is that it does not effectively consider the influence of outputs to Gaussian processes. We proceed to exploit outputs for sparse Gaussian processes by embedding a method that considers outputs in the MPGR framework, and propose manifold-preserving graph reduction with outputs (MPGRO) for sparse Gaussian processes. In this paper, we utilize informative vector machine (IVM) for MPGRO. Experimental results on seven real data sets show that MPGR is better than IVM for sparse Gaussian processes, and MPGRO is better than MPGR and IVM for sparse Gaussian processes and is comparable on error rates to the method utilizing full training sets.