Doubly Sparse Bayesian Kernel Logistic Regression

Abstract

When input patterns have redundant features in regression analysis or pattern recognition, the prediction accuracy is likely to be lowered. For a kernel regression in a reproducing kernel Hilbert space, as the number of observed input patterns increases, the dimension of parameters increases since a kernel regression model using the kernel method is represented by the linear sum of kernel functions corresponding to input patterns. This can yield overfitting. This paper proposes a method for simultaneously selecting features and model coefficients. To express a sparsity of the features and the weight coefficients, we generate binary vectors, where all the elements are 0 or 1 sampled from the beta process. The proposed method can select effective features and estimate sparse weight coefficients by introducing the binary vectors into the kernel regression model. Numerical examples support the efficacy of the proposed method.