Covariance pooling and stabilization for classification

Abstract

It is well known that the quadratic discriminant rule (QD) is optimal for large, normal training sets in the sense of minimizing the overall misclassification rate. However, when the size of the training set is small compared to the number of variables, the performance of QD is degraded because it uses unstable sample mean vectors and covariance matrices. Friedman [8] and Greene and Rayens [9] recently proposed different methods for addressing the problem of unstable covariance matrices. This article details a critical comparison of these two methods, finding important strengths and weaknesses in both. In addition, a third discriminant rule which combines ideas from both of the aforementioned methods is developed and included in the comparisons.