Classification Error Rate Estimators Evaluated by Unconditional Mean Squared Error

Abstract

In this article the criterion of unconditional mean squared error is used to compare four commonly used estimators of error rates in discriminant analysis. The leave-one-out estimator, which has relatively small bias, is found to perform well relative to the other estimators when a large number of explanatory variables are used in the discriminant function. With a small number of explanatory variables, the large variance of this estimator results in poor performance. We also find the estimators that assume normally distributed explanatory variables to be nonrobust when the parent distributions are skewed or have large tails.