Tail Behavior of Regression Estimators and their Breakdown Points

Abstract

Following Jureckova (1981) we introduce a finite-sample measure of performance of regression estimators based on tail behavior. The least squares estimator is studied in detail, and we find that it achieves good tail performance under strictly Gaussian conditions. However, the tail performance of the least-squares estimator is found to be extremely poor in the case of heavy-tailed error distributions. Further analysis of the least-squares estimator with light-tailed errors indicates the strong influence of the design matrix in determining tail performance. Turning to the tail behavior of various robust estimators of the parameters of the linear model, we focus on tail performance under heavy (algebraic) tailed errors. The 11-estimator is seen to be a leading case: we find a simple characterization of its tail behavior in terms of the design configuration and show that a broad class of M-estimators have the same performance. Perhaps most significantly, it is shown that our finite-sample measure of tail performance is, for heavy tailed error distributions, essentially the same as the finite sample concept of breakdown point introduced by Donoho and Huber (1983). This finding provides an important probabilistic interpretation of the breakdown point and clarifies the role of tail behavior as a quantitative measure of robustness. This link is further explored for high-breakdown regression estimators including Rousseeuw's (1984) leastmedian-of-squares estimator.