Large-Dimensional Characterization of Robust Linear Discriminant Analysis

Abstract

In standard discriminant analysis, data are commonly assumed to follow a Gaussian distribution, a condition which is often violated in practice. In this work, to account for potential spurious or mislabeled observations in the training data, we consider a robust version of regularized linear discriminant analysis (LDA) classifiers. Essential to such robust version of LDA is the design of a robust discriminant rule which relies on a robust estimate of the covariance matrix of the training data. We propose to use a regularized version of M-estimators of covariance matrices belonging to Maronna's class of estimators. In the regime where both the number of variables and the number of training samples are large, building upon recent results from random matrix theory, we show that when the training data are free from outliers, each classifier within the class of proposed robust classifiers is asymptotically equivalent to traditional, non-robust classifiers. Rather surprisingly, this entails that the use of robust estimators does not degrade the performance of LDA, up to a transformation of the regularization parameter that we precisely characterize. We also demonstrate that the proposed robust classifiers lead to a better classification accuracy when the data are corrupted by outliers or random noise. Furthermore, through simulations on the popular MNIST data set and considering different classification tasks, we show that the worse the classification error of traditional methods is, the further gain is to be expected with the use of our proposed method.