A theoretical study of Stein's covariance estimator

Abstract

The traditional covariance estimator, the sample covariance matrix (which is also the MLE), is known to be a poor estimator, unless the sample size is much larger than the dimension of the covariance matrix. Stein's estimator has often been regarded as a much better alternative to the MLE in small sample sizes and is traditionally used with an isotonizing algorithm, the purpose of which is to retain positivity and the original order of the sample eigenvalues. Despite the superior performance of Stein's isotonized estimator in numerical investigations, its theoretical properties have not been explored in detail, and important questions still remain unanswered. One particular question of interest is to identify the regimes under which Stein's estimator is guaranteed to perform well. A second goal is to determine the extent to which the performance of Stein's estimator depends on the isotonizing algorithm. The presence of the ad hoc isotonizing algorithm, however, renders a theoretical analysis rather difficult, and consequently risk functions are not easily quantifiable for comparison purposes. Hence formal decision theoretical results are difficult to obtain and have been elusive ever since the estimator was introduced. Despite these hurdles, in this paper we show that an analysis of Stein's covariance estimator within the unbiased estimator of risk (UBEOR) framework can nevertheless lead to important theoretical and methodological insights that are relevant for applications. Our analysis demonstrates that Stein's estimator may give only modest risk reductions when it is not isotonized, and when it is isotonized, the risk reductions can be significant. In particular, three broad regimes are identified regarding the behavior of Stein's UBEOR. The theoretical insights are then affirmed at the level of risk functions via numerical simulations.