The Relative Importance of Bias and Variability in the Estimation of the Variance of a Statistic

Abstract

The concept of mean squared error, while useful in the comparison of location-type estimators, can be misleading for variance estimators, since it does not address the relative importance of bias and variability, and the differing effects of negative bias and positive bias, on test size and confidence interval coverage. A simple model is presented here to quantify these effects. It is shown that bias (particularly negative bias) can be a severe problem in this regard, and a less (negatively) biased, but more variable, variance estimator would be preferred.