A new approach to probabilistic classification based on Gaussian process and support vector machine

Abstract

This paper proposes three algorithms that combine the Support Vector Machine (SVM) and Gaussian Process (GP) in a unified framework to classify large datasets efficiently and obtain probabilistic information on the classification results. The first algorithm works in two steps. In Step 1, the algorithm judiciously selects a sample of size m from the training dataset of size n, where m ≪ n. Step 2 then runs a Gaussian Process Classification (GPC) on the selected sample. The second algorithm is based on the first one, where instead of a standard GPC, a sparse GPC (a low-cost variant of standard GPC) is run in Step 2. The third algorithm is based on manipulating the Gaussian Process Regression (GPR) to classify the data. Unlike GPC, GPR uses an exact inference method that greatly reduces the computational complexity. We have experimented with seven datasets to compare the performance of the proposed algorithms with the existing state-of-the-art methodologies. In addition to providing probabilistic information, the proposed algorithms have proved to be computationally efficient, especially in the training phase, where they consistently deliver faster results than the existing algorithms. The accuracies provided by the proposed algorithms are satisfactory, and at times even surpass the accuracy obtained by the existing algorithms.