The Group Latent Variable Approach to Probit Binary Classifications

Abstract

This paper considers the binary classification with the probit model under the expectation–maximization (EM) algorithm. Usually, in the Bayesian approach of the probit model, the latent variables are introduced to handle with the intractable problem. For each training sample, there is a corresponding latent variable. However, the EM algorithm requires matrix inversions which demand for the expansive computational cost when the number of training samples is large. To overcome this problem, we employ the group latent-variable approach where for each of the training samples there are corresponding multiple latent variables instead of just one. The major advantage of this approach, which is originated from Bayesian backfitting, is that there are no requirements for matrix inversions in the EM algorithm for the probit model. In this paper, to obtain sparse classifiers the Laplacian prior is employed and the method to control the degree of sparseness is presented. Although the sparsity of the classifier is not determined in a full automatic way, it can be controlled by specifying just one parameter. In other words, we are free to choose the degree of sparseness. The proposed method is compared with support vector machine, relevance vector machine, and generalized LASSO.