Large Sample Covariance Matrices Research Articles

Large sample covariance matrices of Gaussian observations with uniform correlation decay

We derive the Marchenko–Pastur (MP) law for sample covariance matrices of the form Vn=1nXXT, where X is a p×n data matrix and p/n→y∈(0,∞) as n,p→∞. We assume the data in X stems from a correlated joint normal distribution. In particular, the correlation acts both across rows and across columns of X, and we do not assume a specific correlation structure, such as separable dependencies. Instead, we assume that correlations converge uniformly to zero at a speed of an/n, where an may grow mildly to infinity. We employ the method of moments tightly: We identify the exact condition on the growth of an which will guarantee that the moments of the empirical spectral distributions (ESDs) converge to the MP moments. If the condition is not met, we can construct an ensemble for which all but finitely many moments of the ESDs diverge. We also investigate the operator norm of Vn under a uniform correlation bound of C/nδ, where C,δ>0 are fixed, and observe a phase transition at δ=1. In particular, convergence of the operator norm to the maximum of the support of the MP distribution can only be guaranteed if δ>1. The analysis leads to an example for which the MP law holds almost surely, but the operator norm remains stochastic in the limit, and we provide its exact limiting distribution.

Eigenvalue Distribution of a High-Dimensional Distance Covariance Matrix With Application

We introduce a new random matrix model called distance covariance matrix in this paper, whose normalized trace is equivalent to the distance covariance. We first derive a deterministic limit for the eigenvalue distribution of the distance covariance matrix when the dimensions of the vectors and the sample size tend to infinity simultaneously. This limit is valid when the vectors are independent or weakly dependent through a finite-rank perturbation. It is also universal and independent of the details of the distributions of the vectors. Furthermore, the top eigenvalues of this distance covariance matrix are shown to obey an exact phase transition when the dependence of the vectors is of finite rank. This finding enables the construction of a new detector for such weak dependence where classical methods based on large sample covariance matrices or sample canonical correlations may fail in the considered high-dimensional framework.

Properties of eigenvalues and eigenvectors of large-dimensional sample correlation matrices

This paper is to study the properties of eigenvalues and eigenvectors of high-dimensional sample correlation matrices. We first improve the result of Jiang (Sankhyā 66 (2004) 35–48), Xiao and Zhou (J. Theoret. Probab. 23 (2010) 1–20) and the Theorem 1 of El Karoui (Ann. Appl. Probab. 19 (2009) 2362–2405), both concerning the limiting spectral distribution and the extreme eigenvalues of sample correlation matrices, by allowing a more general fourth moment condition. Then, we establish a central limit theorem (CLT) for the linear statistics of the eigenvectors of large sample correlation matrices. We discover that the difference between the functional CLT of the sample covariance matrix and the sample correlation matrix is fundamentally influenced by the direction of a nonrandom projection vector. In the special case where the square root of the correlation matrix is identity, the difference will be determined by the sum of the fourth powers of the entries of the projection vector. These results also indicate that the eigenmatrix of sample correlation matrices is not asymptotically Haar if the underlying distribution is Gaussian. In other words, the normalization based on the sample variances affects the asymptotic properties of the eigenmatrix of the Wishart matrix. Furthermore, we establish a theorem concerning CLT for the linear statistics of the eigenvectors of large sample covariance matrices. This theorem improves the main results in Bai, Miao and Pan (Ann. Probab. 35 (2007) 1532–1572), which requires the assumption that the fourth moment of the underlying variable matches the one of Gaussian distribution, as well as Theorem 1.3 in Pan and Zhou (Ann. Appl. Probab. 18 (2008) 1232–1270), which relaxed the Gaussian like fourth moment requirement but assumes the maximum entries of the projection vector converge to 0 uniformly. We illustrate the usefulness of the theoretical results through an application in communications.

Limiting Spectral Distribution for Large Sample Covariance Matrices with Graph-Dependent Elements

We obtain the limiting spectral distribution for large sample covariance matrices associated with random vectors having graph-dependent entries under the assumption that the interdependence among the entries grows with the sample size n. Our results are tight. In particular, they give necessary and sufficient conditions for the Marchenko-Pastur theorem for sample covariance matrices with m-dependent orthonormal elements when m = o(n).

Likelihood ratio tests under model misspecification in high dimensions

We investigate the likelihood ratio test for a large block-diagonal covariance matrix with an increasing number of blocks under the null hypothesis. While so far the likelihood ratio statistic has only been studied for normal populations, we establish that its asymptotic behavior is invariant under a much larger class of distributions. This implies robustness against model misspecification, which is common in high-dimensional regimes. Demonstrating the flexibility of our approach, we additionally establish asymptotic normality of the log-likelihood ratio test for the equality of many large sample covariance matrices under model uncertainty. A simulation study and an analysis of a data set from psychology emphasize the usefulness of our findings.

Asymptotic independence of spiked eigenvalues and linear spectral statistics for large sample covariance matrices

We consider general high-dimensional spiked sample covariance models and show that their leading sample spiked eigenvalues and their linear spectral statistics are asymptotically independent when the sample size and dimension are proportional to each other. As a byproduct, we also establish the central limit theorem of the leading sample spiked eigenvalues by removing the block diagonal assumption on the population covariance matrix, which is commonly needed in the literature. Moreover, we propose consistent estimators of the L4 norm of the spiked population eigenvectors. Based on these results, we develop a new statistic to test the equality of two spiked population covariance matrices. Numerical studies show that the new test procedure is more powerful than some existing methods.

Limiting spectral distribution for large sample covariance matrices with graph-dependent elements

Для выборочных ковариационных матриц, отвечающих случайным векторам с граф-зависимыми элементами и размерностью, растущей пропорционально объему выборки $n$, найдены условия, при которых предельный спектр матриц имеет ту же форму, что и в случае гауссовских векторов с аналогичной ковариационной структурой. Полученные условия точны. В частности, они являются необходимыми и достаточными в теореме Марченко-Пастура для выборочных ковариационных матриц, отвечающих случайным векторам с $m$-зависимыми ортонормированными элементами при $m=o(n)$.

Precise large deviations for dependent subexponential variables

In this paper, we study precise large deviations for the partial sums of a stationary sequence with a subexponential marginal distribution. Our main focus is on distributions which either have a regularly varying or a lognormal-type tail. We apply the results to prove limit theory for the maxima of the entries large sample covariance matrices.

Central Limit Theorem for Mutual Information of Large MIMO Systems With Elliptically Correlated Channels

By considering elliptical correlations in the channel matrix of a large MIMO system, this paper investigates how the non-linear dependency affects the asymptotic behaviors of Shannon's mutual information, as the numbers of antennas at transmitter and receiver both tend to infinity at the same rate. Beyond the almost sure convergence of the information per transmit antenna, we also propose several central limit theorems for the information statistics under various scenarios. The results show that the non-linear correlation has a little impact on the almost sure convergence but has a tremendous effect on the fluctuation of the information statistics. In particular, the centralized information statistics could experience a phase transition in convergence rate when the strength of the non-linear correlation increases. As extensions, several central limit theorems are established as well for general linear spectral statistics of large sample covariance matrices in elliptical distributions.

Some sphericity tests for high dimensional data based on ratio of the traces of sample covariance matrices

In this paper, we investigate the sphericity test for high dimensional data. The test statistic is proposed based on the ratio of traces of sample covariance matrices. The asymptotic distributions of the test statistics under the null hypothesis are established by linear spectral statistics of large sample covariance matrices. We also obtain the asymptotic power functions in the case of the spiked population model as a specific alternative. Compared with some existing tests, the proposed test can handle data having unknown means and non-Gaussian population with general fourth moment. The numerical performance demonstrates that the proposed tests have satisfactory properties in terms of size and power.

Robust estimation for image noise based on eigenvalue distributions of large sample covariance matrices

In this paper, we propose a novel algorithm to estimate Gaussian noise levels for captured natural images by rigorously analyzing the limiting distributions of the eigenvalue spectrum of a large covariance matrix with Gaussian samples. In order to select a relatively homogeneous region that best represents the noise, the corresponding image patches are first rearranged to construct a high-dimensional noise covariance matrix. And then, an optimal criterion for classifying homogeneous regions is derived based on the statistical relationship between the largest and the second largest eigenvalues of a sample covariance matrix. Moreover, we further explore the reasons for the bias of the maximum likelihood estimator of the noise variance both in high-dimensional settings and finite samples. According to random matrix theory, we clarify the asymptotic properties of the trace of a sample covariance matrix to measure the error bounds of estimation and then propose a new bias-corrected estimator. To this end, an effective estimation method for the noise level is devised based on the boundness and asymptotic behavior of pure noise eigenvalues of the selected patches. The estimation performance of our method has been guaranteed both theoretically and empirically. Experimental results have demonstrated that our approach can reliably infer true noise variance and is superior to the competing methods in terms of both estimation accuracy and robustness.

Unbounded largest eigenvalue of large sample covariance matrices: Asymptotics, fluctuations and applications

Given a large sample covariance matrix SN=1nΓN1/2ZNZN⁎ΓN1/2, where ZN is a N×n matrix with i.i.d. centered entries, and ΓN is a N×N deterministic Hermitian positive semidefinite matrix, we study the location and fluctuations of λmax(SN), the largest eigenvalue of SN as N,n→∞ and Nn−1→r∈(0,∞) in the case where the empirical distribution μΓN of eigenvalues of ΓN is tight (in N) and λmax(ΓN) goes to +∞. These conditions are in particular met when μΓN weakly converges to a probability measure with unbounded support on R+.We prove that asymptotically λmax(SN)∼λmax(ΓN). Moreover when the ΓN's are block-diagonal, and the following spectral gap condition is assumed:limsupN→∞λ2(ΓN)λmax(ΓN)<1, where λ2(ΓN) is the second largest eigenvalue of ΓN, we prove Gaussian fluctuations for λmax(SN)/λmax(ΓN) at the scale n.In the particular case where ZN has i.i.d. Gaussian entries and ΓN is the N×N autocovariance matrix of a long memory Gaussian stationary process (Xt)t∈Z, the columns of ΓN1/2ZN can be considered as n i.i.d. samples of the random vector (X1,…,XN)⊤. We then prove that ΓN is similar to a diagonal matrix which satisfies all the required assumptions of our theorems, hence our results apply to this case.

Random Matrix Theory for Heavy-Tailed Time Series

This paper is a review of recent results for large random matrices with heavy-tailed entries. First, we outline the development of and some classical results in random matrix theory. We focus on large sample covariance matrices, their limiting spectral distributions, and the asymptotic behavior of their largest and smallest eigenvalues and their eigenvectors. The limits significantly depend on the finite or infiniteness of the fourth moment of the entries of the random matrix. We compare the results for these two regimes which give rise to completely different asymptotic theories. Finally, the limits of the extreme eigenvalues of sample correlation matrices are examined.

Joint Central Limit Theorem for Eigenvalue Statistics from Several Dependent Large Dimensional Sample Covariance Matrices with Application

AbstractLet Xn = (xij) be a k×n data matrix with complex‐valued, independent and standardized entries satisfying a Lindeberg‐type moment condition. We consider simultaneously R sample covariance matrices , where the Qr's are non‐random real matrices with common dimensions p×k(k≥p). Assuming that both the dimension p and the sample size n grow to infinity, the limiting distributions of the eigenvalues of the matrices {Bnr} are identified, and as the main result of the paper, we establish a joint central limit theorem (CLT) for linear spectral statistics of the R matrices {Bnr}. Next, this new CLT is applied to the problem of testing a high‐dimensional white noise in time series modelling. In experiments, the derived test has a controlled size and is significantly faster than the classical permutation test, although it does have lower power. This application highlights the necessity of such joint CLT in the presence of several dependent sample covariance matrices. In contrast, all the existing works on CLT for linear spectral statistics of large sample covariance matrices deal with a single sample covariance matrix (R = 1).

The limiting spectral distribution for large sample covariance matrices with unbounded m-dependent entries

ABSTRACTIn this note, the limiting spectral distribution for large sample covariance matrices with unbounded m-dependent structure is obtained under the third moment for the entries. This partially extends the results of Hui and Pan (Comm. Statist. Theory and Methods, 2010, 39: 935–941).

Comparison between two types of large sample covariance matrices

Soit $\{X_{ij}\}$, $i,j=1,2,\ldots$, un tableau à double entrées, les $X_{ij}$ étant des variables aléatoires réelles indépendantes et identiquement distribuées (i.i.d.) et où $\mathbf{E} X_{11}=\mu$, $\mathbf{E} \vert X_{11}-\mu\vert ^{2}=1$ et $\mathbf{E} |X_{11}|^{4}<\infty$. Considérons les matrices de covariances empiriques suivantes (avec/sans centrage empirique): $\mathcal{S}=\frac{1}{n}\sum^{n}_{j=1}(\mathbf{s} _{j}-\bar{ \mathbf {s}})(\mathbf{s} _{j}-\bar{ \mathbf {s}})^{T}$ et $\mathbf{S}=\frac{1}{n}\sum^{n}_{j=1}\mathbf{s} _{j}\mathbf{s} _{j}^{T}$, avec $\bar{ \mathbf {s}}=\frac{1}{n}\sum^{n}_{j=1}\mathbf{s} _{j}$ et $\mathbf{s} _{j}=\mathbf{T}^{1/2}_{n}(X_{1j},\ldots,X_{pj})^{T}$, où $(\mathbf{T}^{1/2}_{n})^{2}=\mathbf{T}_{n}$ est une matrice déterministe définie positive. Nous démontrons que, sous le régime asymptotique $n\rightarrow\infty$ et $p/n$ converge vers une constante positive, le théorème central limite pour la statistique $\mathcal{S}$ est différent de celui concernant la statistique $\mathbf{S}$. En outre, nous montrons que cette différence de comportement n’est pas observée pour le comportement moyen des vecteurs propres.

Blind Pilot Decontamination

A subspace projection to improve channel estimation in massive multi-antenna systems is proposed and analyzed. Together with power-controlled hand-off, it can mitigate the pilot contamination problem without the need for coordination among cells. The proposed method is blind in the sense that it does not require pilot data to find the appropriate subspace. It is based on the theory of large random matrices that predicts that the eigenvalue spectra of large sample covariance matrices can asymptotically decompose into disjoint bulks as the matrix size grows large. Random matrix and free probability theory are utilized to predict under which system parameters such a bulk decomposition takes place. Simulation results are provided to confirm that the proposed method outperforms conventional linear channel estimation if bulk separation occurs.

Functional CLT of eigenvectors for large sample covariance matrices

In order to investigate property of the eigenvector matrix of sample covariance matrix $$\mathbf {S}_n$$ , in this paper, we establish the central limit theorem of linear spectral statistics associated with a new form of empirical spectral distribution $$H^{\mathbf {S}_n}$$ , based on eigenvectors and eigenvalues of sample covariance matrix $$\mathbf {S}_n$$ . Using Bernstein polynomial approximations, we prove the central limit theorem for linear spectral statistics of $$H^{\mathbf {S}_n}$$ , indexed by a set of functions with continuous third order derivatives over an interval including the support of Marcenko–Pastur law. This result provides further evidences to support the conjecture that the eigenmatrix of sample covariance matrix is asymptotically Haar distributed.

Limiting Spectral Distribution of Large Sample Covariance Matrices Associated with a Class of Stationary Processes

In this paper, we derive an extension of the Marc̆enko–Pastur theorem to a large class of weak dependent sequences of real-valued random variables having only moment of order 2. Under a mild dependence condition that is easily verifiable in many situations, we derive that the limiting spectral distribution of the associated sample covariance matrix is characterized by an explicit equation for its Stieltjes transform, depending on the spectral density of the underlying process. Applications to linear processes, functions of linear processes, and ARCH models are given.

On limiting spectral distribution of large sample covariance matrices by VARMA(p,q)

We studied the limiting spectral distribution of large-dimensional sample covariance matrices of a stationary and invertible VARMA(p,q) model. Relationship of the power spectral density and limiting spectral distribution of large population dimensional covariance matrices of ARMA(p,q) is established. The equation about Stieltjes transform of large-dimensional sample covariance matrices is also derived. As applications, the classical M-P law, VAR(1) and VMA(1) can be regarded as special examples.