Abstract
Linear discriminant analysis (LDA) has attracted many attentions as a classical tool for both classification and dimensionality reduction. Classical LDA performs quite well in simple and low dimensional setting while it is not suitable for small sample size data (SSS). Feature selection is an effective way to solve this problem. As a variant of LDA, sparse optimal scoring (SOS) with $\ell _{0}$ -norm regularization is considered in this paper. By using a new continuous nonconvex nonsmooth function to approximate $\ell _{0}$ -norm, we propose a novel difference of convex functions algorithm (DCA) for sparse optimal scoring. The most favorable property of the proposed DCA is its subproblem admits an analytical solution. The effectiveness of the proposed method is validated via theoretical analysis as well as some illustrative numerical experiments.
Highlights
Linear discriminant analysis is a classical method for classification and dimensionality reduction in many applications, because of its simplicity, robustness, and predictive accuracy [1]
An alternating scheme based on difference of convex functions algorithm (DCA) is proposed by using a nonconvex continuous function to approximate 0-norm and making suitable DC decomposition
One of the advantages of the new DCA is that its subproblems are smooth and have closed form solution at every iteration
Summary
Linear discriminant analysis is a classical method for classification and dimensionality reduction in many applications, because of its simplicity, robustness, and predictive accuracy [1]. Motivated by [15], a new DC algorithm (DCA) will be proposed to solve sparse optimal scoring in this paper by using a suitable approximation of 0-norm. Some nonconvex continuous approximation functions have been proposed for 0-norm, such as Capped- 1 approximation and piecewise exponential concave approximation These two renowned nonconvex approximations have been successfully applied to SOS problem in [15], where the corresponding subproblem can be written as min w∈Rp. Obviously, problem (15) is nonsmooth and it is difficult and time consuming to solve this subproblem. We can conclude that our algorithm is theoretically faster than the methods in [16]
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
