Efficient optimally regularized discriminant analysis

Abstract

Regularized discriminant analysis (RDA) and its special case uncorrelated linear discriminant analysis (ULDA) are important subspace learning methods proposed recently to handle the small sample size (SSS) problem of linear discriminant analysis (LDA). One important unsolved issue of RDA is how to automatically determine an appropriate regularization parameter without resorting to unscalable procedures like cross-validation (CV). In this paper, we develop a novel efficient algorithm to automatically estimate the regularization parameter based on a geometric interpretation of RDA. We further provide a formal analysis of the proposed method, and show that it is robust to the perturbation in the feature space of the training data. The extensive experiments on various benchmark datasets verify the scalability and effectiveness of our approach, compared with the state-of-the-art algorithms.