G-Optimal Feature Selection with Laplacian regularization

Abstract

Feature selection is an important preprocessing step in many applications where the data points are of high dimension. It is designed to find the most informative feature subset to facilitate data visualization, clustering, classification, and ranking. In this paper, we consider the feature selection problem in unsupervised scenarios. Typical unsupervised feature selection algorithms include Q-α and Laplacian Score. Both of them select the most informative features by discovering the clustering or geometrical structure in the data. However, they fail to consider the performance of some specific learning task, e.g. regression, by using the selected features. Based on Laplacian Regularized Least Squares (LapRLS) which incorporates the manifold structure into the regression model, we propose a novel feature selection approach called Laplacian G-Optimal Feature Selection (LapGOFS). It minimizes the maximum variance of the predicted value of the regression model. By using techniques from manifold learning and optimal experimental design, our proposed approach can select the most informative features which can improve the learning performance the most. Extensive experimental results over various real data sets have demonstrated the effectiveness of the proposed algorithm.