Abstract
This paper studies high-dimensional linear discriminant analysis (LDA). First, we review the $\ell_{1}$ penalized least square LDA proposed in [10], which could circumvent estimation of the annoying high-dimensional covariance matrix. Then detailed theoretical analyses of this sparse LDA are established. To be specific, we prove that the penalized estimator is $\ell_{2}$ consistent in high-dimensional regime and the misclassification error rate of the penalized LDA is asymptotically optimal under a set of reasonably standard regularity conditions. The theoretical results are complementary to the results to [10], together with which we have more understanding of the $\ell_{1}$ penalized least square LDA (or called Lassoed LDA).
Highlights
Classification problem is important in a lot of fields such as pattern recognition, bioinformatics etc
A lot of researchers noticed that in high-dimensional classification problems, it is critical to have a “good” estimation of the covariance matrix. [2] showed that one could use a diagonal matrix instead, which used the idea of Naive Bayes and assumed that features are independent. [7] pointed out that even if a “Naive Bayes” rule is used, if there are too many features in the model, the performance of linear discriminant analysis (LDA) is still poor
Let Σ be the shared covariance matrix and δ be the difference between the mean vectors. [6] proposed a sparse LDA by directly assuming that Σ−1δ is sparse and their method circumvents estimation of the inverse covariance matrix directly
Summary
Classification problem is important in a lot of fields such as pattern recognition, bioinformatics etc. Using Lasso to solve sparse LDA has already been proposed in [10] [10] showed that under irrepresentable condition ([14]) and some other regularity conditions, the Lassoed discriminant analysis could consistently identify the important features (or predictors) in high-dimensional regime. 1. Under the restricted eigenvalue condition on the covariance matrix and some other regularity conditions, the Lassoed LDA estimator (defined later) is proved to be 2 consistent in high-dimensional regime.
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
