Discriminative Feature Selection via Joint Trace Ratio Criterion and l2,1-norm Regularization

Abstract

Dealing with high-dimensional dataset has always been an important problem and feature selection is one of useful tools. In this paper, we develop a new supervised feature selection method by combining Trace Ratio Criterion of Linear Discriminant Analysis (TRC-LDA) and group sparsity regularization. The TRC-LDA is a widely-used criterion for supervised learning that can well preserve discriminative information of dataset. It is good for handling classification problems but cannot be directly used for feature selection. On the other hand, imposing the l 2,0 -norm to the projection matrix of TRC-LDA will force some rows in it to be zero while keep other rows nonzero making the index of nonzero rows to be the selected features, however, l 2,0 -nom minimizing problem is NP-hard and intractable. To solve the above problem, in this paper, we impose l 2,1 -norm, i.e. an approximation of l 2,0 -norm, to projection matrix W of TRC-LDA to achieve feature selection. As a result, the proposed method can both achieve feature selection as well as capture the discriminative structure of data. Extensive simulations based on several real-world datasets verify the effectiveness of the proposed methods.