Efficient computation of limit spectra of sample covariance matrices

Abstract

Models from random matrix theory (RMT) are increasingly used to gain insights into the behavior of statistical methods under high-dimensional asymptotics. However, the applicability of the framework is limited by numerical problems. Consider the usual model of multivariate statistics where the data is a sample from a multivariate distribution with a given covariance matrix. Under high-dimensional asymptotics, there is a deterministic map from the distribution of eigenvalues of the population covariance matrix (the population spectral distribution or PSD), to the of empirical spectral distribution (ESD). The current methods for computing this map are inefficient, and this limits the applicability of the theory. We propose a new method to compute numerically the ESD from an arbitrary input PSD. Our method, called SPECTRODE, finds the support and the density of the ESD to high precision; we prove this for finite discrete distributions. In computational experiments SPECTRODE outperforms existing methods by orders of magnitude in speed and accuracy. We apply it to compute expectations and contour integrals of the ESD, which are often central in applications. We also illustrate that SPECTRODE is directly useful in statistical problems, such as estimation and hypothesis testing for covariance matrices. Our proposal, implemented in open source software, may broaden the use of RMT in high-dimensional data analysis.