Linear Discriminant Analysis with Adherent Regularization

Abstract

Sparse modeling have become one of the standard approaches for latent variable analysis in the literature of statistics, machine learning and signal processing. This paper considers a supervised dimension reduction, which is a fundamental problem in data science. Particularly, the problem of linear discriminant analysis is considered. Extending the previous attempt to impose sparsity invoking regularization for Fisher's discriminant model, the proposed method bridges two different formulations of linear discriminant analysis, namely, the Fisher's discriminant model and the normal model, via a particular form of regularization. The proposed discriminant problem is efficiently solved by using the proximal point algorithm. The proposed method is shown to work well through experiments using both artificial and real-world datasets.