Sequential training of semi-supervised classification based on sparse Gaussian process regression

Abstract

Gaussian process regression (GPR) is a very important Bayesian approach in machine learning applications. It has been extensively used in semi-supervised learning tasks. In this paper, we propose a sequential training method for solving semi-supervised binary classification problem. It assigns targets to test inputs sequentially making use of sparse Gaussian process regression models. The proposed approach deals with only one part of the whole data set at a time. Firstly, the IVM produces a sparse approximation to a Gaussian process (GP) by combining assumed density filtering (ADF) with a heuristic for choosing points based on minimizing posterior entropy, and then a sparse GPR classifier is learnt on part of the whole data set. Secondly, the representative points selected in the first step including part of remainder examples are used to train another sparse GPR classifier. Repeat the two steps sequentially until all unlabeled examples are deal with. The proposed approach is simple and easy to implement. The hyperparameters are estimated easily by maximizing the marginal likelihood without resorting to expensive cross-validation technique. The evaluations of the proposed method on several real world data sets reveal promising results.