Eigenvalues Of Sample Covariance Matrix Research Articles

Quadratic shrinkage for large covariance matrices

This paper constructs a new estimator for large covariance matrices by drawing a bridge between the classic (Stein (1975)) estimator in finite samples and recent progress under large-dimensional asymptotics. The estimator keeps the eigenvectors of the sample covariance matrix and applies shrinkage to the inverse sample eigenvalues. The corresponding formula is quadratic: it has two shrinkage targets weighted by quadratic functions of the concentration (that is, matrix dimension divided by sample size). The first target dominates mid-level concentrations and the second one higher levels. This extra degree of freedom enables us to outperform linear shrinkage when the optimal shrinkage is not linear, which is the general case. Both of our targets are based on what we term the “Stein shrinker”, a local attraction operator that pulls sample covariance matrix eigenvalues towards their nearest neighbors, but whose force diminishes with distance (like gravitation). We prove that no cubic or higher-order nonlinearities beat quadratic with respect to Frobenius loss under large-dimensional asymptotics. Non-normality and the case where the matrix dimension exceeds the sample size are accommodated. Monte Carlo simulations confirm state-of-the-art performance in terms of accuracy, speed, and scalability.

Shrinkage estimation of large covariance matrices: Keep it simple, statistician?

Under rotation-equivariant decision theory, sample covariance matrix eigenvalues can be optimally shrunk by recombining sample eigenvectors with a (potentially nonlinear) function of the unobservable population covariance matrix. The optimal shape of this function reflects the loss/risk that is to be minimized. We solve the problem of optimal covariance matrix estimation under a variety of loss functions motivated by statistical precedent, probability theory, and differential geometry. A key ingredient of our nonlinear shrinkage methodology is a new estimator of the angle between sample and population eigenvectors, without making strong assumptions on the population eigenvalues. We also introduce a broad family of covariance matrix estimators that can handle all regular functional transformations of the population covariance matrix under large-dimensional asymptotics. In addition, we compare via Monte Carlo simulations our methodology to two simpler ones from the literature, linear shrinkage and shrinkage based on the spiked covariance model.

Estimation of the Number of Spiked Eigenvalues in a Covariance Matrix by Bulk Eigenvalue Matching Analysis

ABSTRACT The spiked covariance model has gained increasing popularity in high-dimensional data analysis. A fundamental problem is determination of the number of spiked eigenvalues, K. For estimation of K, most attention has focused on the use of top eigenvalues of sample covariance matrix, and there is little investigation into proper ways of using bulk eigenvalues to estimate K. We propose a principled approach to incorporating bulk eigenvalues in the estimation of K. Our method imposes a working model on the residual covariance matrix, which is assumed to be a diagonal matrix whose entries are drawn from a gamma distribution. Under this model, the bulk eigenvalues are asymptotically close to the quantiles of a fixed parametric distribution. This motivates us to propose a two-step method: the first step uses bulk eigenvalues to estimate parameters of this distribution, and the second step leverages these parameters to assist the estimation of K. The resulting estimator aggregates information in a large number of bulk eigenvalues. We show the consistency of under a standard spiked covariance model. We also propose a confidence interval estimate for K. Our extensive simulation studies show that the proposed method is robust and outperforms the existing methods in a range of scenarios. We apply the proposed method to analysis of a lung cancer microarray dataset and the 1000 Genomes dataset.

Sphericity and Identity Test for High-dimensional Covariance Matrix Using Random Matrix Theory

This paper addresses the issue of testing sphericity and identity of high-dimensional population covariance matrix when the data dimension exceeds the sample size. The central limit theorem of the first four moments of eigenvalues of sample covariance matrix is derived using random matrix theory for generally distributed populations. Further, some desirable asymptotic properties of the proposed test statistics are provided under the null hypothesis as data dimension and sample size both tend to infinity. Simulations show that the proposed tests have a greater power than existing methods for the spiked covariance model.

Tracy–Widom law for the largest eigenvalue of sample covariance matrix generated by VARMA

In this paper, we derive the Tracy–Widom law for the largest eigenvalue of sample covariance matrix generated by the vector autoregressive moving average model when the dimension is comparable to the sample size. This result is applied to make inference on the vector autoregressive moving average model. Simulations are conducted to demonstrate the finite sample performance of our inference.

On robust spectrum sensing using M-estimators of covariance matrix

In this paper, we consider the spectrum sensing in cognitive radio networks when the impulsive noise appears. We propose a class of blind and robust detectors using M-estimators in eigenvalue based spectrum sensing method. The conventional eigenvalue based method uses statistics derived from the eigenvalues of sample covariance matrix(SCM) as testing statistics, which are inefficient and unstable in the impulsive noise environment. Instead of SCM, we can use M-estimators, which have good performance under both impulsive and non-impulsive noise. Among those M-estimators, We recommend the Tyler's M-estimator instead, which requires no knowledge of noise distribution and have the same probability of false alarm under different complex elliptically symmetric distributions. In addition, it performs better than the detector using sample covariance matrix when the noise is highly impulsive. It should be emphasized that this detector does not require knowledge of noise power which is required by the energy detection based methods. Simulations show that it performs better than conventional detector using sample covariance matrix in a highly impulsive noise environment.

Quadratic Shrinkage for Large Covariance Matrices

This paper constructs a new estimator for large covariance matrices by drawing a bridge between the classic Stein (1975) estimator in finite samples and recent progress under large-dimensional asymptotics. The estimator keeps the eigenvectors of the sample covariance matrix and applies shrinkage to the inverse sample eigenvalues. The corresponding formula is quadratic: it has two shrinkage targets weighted by quadratic functions of the concentration (that is, matrix dimension divided by sample size). The first target dominates mid-level concentrations and the second one higher levels. This extra degree of freedom enables us to outperform linear shrinkage when optimal shrinkage is not linear (which is the general case). Both of our targets are based on what we term the “Stein shrinker”, a local attraction operator that pulls sample covariance matrix eigenvalues towards their nearest neighbors, but whose force diminishes with distance, like gravitation. We prove that no cubic or higher-order nonlinearities beat quadratic with respect to Frobenius loss under large-dimensional asymptotics. Non-normality and the case where the matrix dimension exceeds the sample size are accommodated. Monte Carlo simulations confirm state-of-the-art performance in terms of accuracy, speed, and scalability.

Information divergence‐based low probability of intercept waveform detection for multi‐antenna intercept receivers

Based on the asymptotic spectral distribution of sample covariance matrix, a new low probability of interception (LPI) signal detection method for multi-antenna intercept receiver is proposed via reforming the noise sequence into a Wishart matrix. The authors deduce the asymptotic cumulative distribution function (CDF) of eigenvalues of sample covariance matrix. Also a numerical Kullback–Leibler divergence of the empirical spectral CDF based on test samples from the deduced asymptotic CDF is established, which is treated as the test statistic. By comparing with other signal detection methods, simulation results show the new LPI signal detection method is superior and robust.

Shrinkage Estimation of Large Covariance Matrices: Keep it Simple, Statistician?

Under rotation-equivariant decision theory, sample covariance matrix eigenvalues can be optimally shrunk by recombining sample eigenvectors with a (potentially nonlinear) function of the unobservable population covariance matrix. The optimal shape of this function reflects the loss/risk that is to be minimized. We solve the problem of optimal covariance matrix estimation under a variety of loss functions motivated by statistical precedent, probability theory, and differential geometry. A key ingredient of our nonlinear shrinkage methodology is a new estimator of the angle between sample and population eigenvectors, without making strong assumptions on the population eigenvalues. We also introduce a broad family of covariance matrix estimators that can handle all regular functional transformations of the population covariance matrix under large-dimensional asymptotics. In addition, we compare via Monte Carlo simulations our methodology to two simpler ones from the literature, linear shrinkage and shrinkage based on the spiked covariance model.

Noise Power Estimators in ISM Radio Environments: Performance Comparison and Enhancement Using a Novel Samples Separation Technique

Noise power estimation is central to efficient radio resource allocation in modern wireless communication systems. In the literature, there exist many noise power estimation methods that can be classified based on underlying theoretical principle; the most common are spectral averaging, eigenvalues of sample covariance matrix, information theory, and statistical signal analysis. However, how these estimation methods compare against each other in terms of accuracy, stability, and complexity is not well studied, and the focus instead remains on the enhancement of individual methods. In this paper, we adopt a common simulation methodology to perform a detailed performance evaluation of the prominent estimation techniques. The basis of our comparison is the signal-to-noise ratio estimation in the simulated industrial, scientific and medical band transmission, while the reference noise signal is acquired from an industrial production plant using a software-defined radio platform, USRP-2932. In addition, we analyze the impact of different techniques for noise-samples’ separation on the estimation process. As a response to defects in the existing techniques, we propose a novel noise-samples’ separation algorithm based on the adaptation of rank-order filtering. Our analysis shows that the proposed solution, apart from its low complexity, has a very good root-mean-squared error of 0.5 dB and smaller than 0.1-dB resolution, thus achieving a performance comparable with the methods exploiting information theory concepts.

Large Dynamic Covariance Matrices

Second moments of asset returns are important for risk management and portfolio selection. The problem of estimating second moments can be approached from two angles: time series and the cross-section. In time series, the key is to account for conditional heteroscedasticity; a favored model is Dynamic Conditional Correlation (DCC), derived from the ARCH/GARCH family started by Engle (1982). In the cross-section, the key is to correct in-sample biases of sample covariance matrix eigenvalues; a favored model is nonlinear shrinkage, derived from Random Matrix Theory (RMT). The present article marries these two strands of literature to deliver improved estimation of large dynamic covariance matrices. Supplementary material for this article is available online.

Optimal Eigenvalue Weighting Detection for Multi-Antenna Cognitive Radio Networks

The state-of-the-art eigenvalue-based spectrum sensing methods only consider the partial information of eigenvalues, such as the maximum, minimum, and mean values to make detection, which does not make full use of the eigenvalues to catch correlation. In this paper, we focus on all the eigenvalues of sample covariance matrix in multi-antenna cognitive radio networks and propose eigenvalue weighting-based detection schemes. According to the Neyman–Pearson criterion, the globally optimal weighting solution is the likelihood ratio test (LRT). Hence, we analyze and derive the eigenvalue-based LRT (E-LRT). Utilizing the random matrix theory, a simple closed-form expression for the E-LRT is obtained, which is exactly the optimal eigenvalue weighting scheme. Although the E-LRT is optimal, it is infeasible in practice due to its dependence on the knowledge of primary users and noise powers. Hence, we further analyze suboptimal methods and design maximum likelihood estimation-based approximation weighting approach. Under the approach, both semi-blind (only the noise power is known) and totally-blind methods are correspondingly proposed. In addition, the theoretical performance analysis of these proposed methods are provided. Simulation results are presented to verify the efficiency of the proposed algorithms.

Eigenvalue-Based Multiple Antenna Spectrum Sensing: Higher Order Moments

The problem of multiple antenna spectrum sensing in cognitive radio (CR) networks is studied in this paper. We propose two new invariant constant false-alarm rate eigenvalue-based (EVB) detectors, using the higher order moments of the sample covariance matrix eigenvalues, by exploiting the separating function estimation test framework. We find closed-form expressions for the false-alarm and detection probabilities of the proposed detectors by providing moment-based approximations of their statistical distributions. The accuracy of the obtained closed-form expressions is validated by Monte Carlo simulations. In addition, we compare the performance of the proposed detectors with that of their two counterparts, i.e., John’s and the arithmetic to geometric mean (AGM) detectors, in terms of the asymptotic relative efficiency. This comparison enables us to demonstrate the superiority of our proposed detectors over those detectors within the typical range of signal-to-noise ratio in CR application. The comparative simulation results also illustrate the superiority of the proposed detectors over John’s and the AGM detectors as well as some other state-of-the-art EVB algorithms given in the literature.

Eigenvalue based double threshold spectrum sensing under noise uncertainty for cognitive radio

Spectrum sensing is a fundamental problem in cognitive radio. The conventional energy based channel sensing method is highly vulnerable under low SNR and noise uncertainty conditions. The use of double threshold usually improves the detection performance, however, under noise uncertainty its performance deteriorates. Another method based on eigenvalues of sample covariance matrix has been studied in literature with single threshold. In this paper an eigenvalue based spectrum sensing technique with double threshold is proposed. The random matrix theory (RMT) is used to quantify the ratio of eigenvalues and derives the expression for the two thresholds required for reliable spectrum sensing. The proposed scheme overcomes the noise uncertainty and low SNR problems and out performs the conventional energy based detection method. The simulation results show that the proposed double threshold method based on eigenvalues exhibits better detection performance compared to conventional energy detection methods in terms of detection probability and number of samples required for reliable sensing.

Large Dynamic Covariance Matrices

Second moments of asset returns are important for risk management and portfolio selection. The problem of estimating second moments can be approached from two angles: time series and the cross-section. In time series, the key is to account for conditional heteroskedasticity; a favored model is Dynamic Conditional Correlation (DCC), derived from the ARCH/GARCH family started by Engle (1982). In the cross-section, the key is to correct in-sample biases of sample covariance matrix eigenvalues; a favored model is nonlinear shrinkage, derived from Random Matrix Theory (RMT). The present paper aims to marry these two strands of literature in order to deliver improved estimation of large dynamic covariance matrices.

Functional CLT of eigenvectors for large sample covariance matrices

In order to investigate property of the eigenvector matrix of sample covariance matrix $$\mathbf {S}_n$$ , in this paper, we establish the central limit theorem of linear spectral statistics associated with a new form of empirical spectral distribution $$H^{\mathbf {S}_n}$$ , based on eigenvectors and eigenvalues of sample covariance matrix $$\mathbf {S}_n$$ . Using Bernstein polynomial approximations, we prove the central limit theorem for linear spectral statistics of $$H^{\mathbf {S}_n}$$ , indexed by a set of functions with continuous third order derivatives over an interval including the support of Marcenko–Pastur law. This result provides further evidences to support the conjecture that the eigenmatrix of sample covariance matrix is asymptotically Haar distributed.

NFA FOR FACTOR NUMBER DETERMINATION IN APT

In the context of quantitative analysis of the arbitrage pricing theory (APT) model, conventional factor analytic approaches such as maximum likelihood factor analysis (MLFA) cannot provide satisfactory answers to two important questions. The first one concerns the correct identification of factor number while the second one is related to the rotation indeterminacy of factor loadings. In the literature, MLFA followed by likelihood ratio (LR) test and the analysis of eigenvalues of sample covariance matrix were two popular approaches used to determine the appropriate number of factors. However, it was shown empirically that both of them suffered from different kinds of biases. We find the recently developed non-gaussian factor analysis (NFA) model by Xu [24] provides a new perspective for the determination of the appropriate factor number k in APT, with promising empirical results demonstrated.

Some hypothesis tests for the covariance matrix when the dimension is large compared to the sample size

This paper analyzes whether standard covariance matrix tests work when dimensionality is large, and in particular larger than sample size. In the latter case, the singularity of the sample covariance matrix makes likelihood ratio tests degenerate, but other tests based on quadratic forms of sample covariance matrix eigenvalues remain well-defined. We study the consistency property and limiting distribution of these tests as dimensionality and sample size go to infinity together, with their ratio converging to a finite nonzero limit. We find that the existing test for sphericity is robust against high dimensionality, but not the test for equality of the covariance matrix to a given matrix. For the latter test, we develop a new correction to the existing test statistic that makes it robust against high dimensionality.

Signal detection via spectral theory of large dimensional random matrices

Results on the spectral behavior of random matrices as the dimension increases are applied to the problem of detecting the number of sources impinging on an array of sensors. A common strategy to solve this problem is to estimate the multiplicity of the smallest eigenvalue of the spatial covariance matrix R of the sensed data. Existing approaches rely on the closeness of the noise eigenvalues of sample covariance matrix to each other and, therefore, the sample size has to be quite large when the number of sources is large in order to obtain a good estimate. The theoretical analysis presented focuses on the splitting of the spectrum of sample covariance matrix into noise and signal eigenvalues. It is shown that when the number of sensors is large the number of signals can be estimated with a sample size considerably less than that required by previous approaches. >

On the limit of the largest eigenvalue of the large dimensional sample covariance matrix

In this paper the authors show that the largest eigenvalue of the sample covariance matrix tends to a limit under certain conditions when both the number of variables and the sample size tend to infinity. The above result is proved under the mild restriction that the fourth moment of the elements of the sample sums of squares and cross products (SP) matrix exist.