Optimal Sparse Kernel Learning in the Empirical Kernel Feature Space for Hyperspectral Classification

Abstract

In this paper, a novel framework for optimal sparse kernel learning for support vector machine (SVM) classifier in a finite-dimensional space called the empirical kernel feature space (EKFS) is presented. In conventional sparse kernel learning techniques, feature selection algorithms are optimal up to linear kernel because the contributions of individual features in the input space to the margin of the classifier can be determined explicitly for a linear kernel. But the use of nonlinear kernels leads to high dimensional, possibly infinite dimensional, Reproducing Kernel Hilbert Spaces (RKHS). Here, feature selection problem is highly combinatorial, and is NP-hard to solve because the number of all the possible combinations of the input features mapped from the input space into the RKHS is often prohibitively large to determine the contributions of subsets of features. To tackle this issue, in the proposed work, feature selection is explicitly and optimally performed in the EKFS instead of in the corresponding RKHS. Unlike the RKHS, the EKFS associated with any positive definite kernel including Gaussian RBF kernel can explicitly be built by using empirical kernel mapping. The feature selection in the EKFS has the same effect as the feature selection in the RKHS since both the EKFS and RKHS associated with same kernel have the same geometrical structure. The features in the EKFS are a kernel representation of each input vector with respect to all the training samples available. Thus, they represent nonlinear similarity measure of each data point with respect to reference samples with known labels. The proposed sparse kernel learning can optimally select multiple subsets of newly mapped features in the EKFS in order to improve the generalization performance of the classifier. The sparse kernel-based learning is tested on several hyperspectral datasets and a performance comparison among different feature selection techniques is presented.