Two-Stage Discriminative Feature Selection

Abstract

Recently, a filter supervised feature selection method, namely discriminative feature selection (DFS), was proposed, which combines linear discriminant analysis (LDA) and sparsity regularization effectively. However, DFS is computationally expensive due to the use of eigenvalue decomposition on large matrix. In this paper, we propose a two-stage DFS method, namely TSDFS, to improve the efficiency and keep the accuracy of classification. A direct LDA based feature selection is firstly performed to achieve dimension reduction preprocessing of the data. Then, a DFS procedure is performed efficiently on the reduced data in the second stage. The high efficiency of the whole TSDFS is credited with the high efficiency of dimension reduction preprocessing. In addition, we employ nested cross-validation technology to achieve automatic parameter selection for TSDFS. Extensive experimental results based on several benchmark data sets validate the effectiveness of TSDFS.