Eigenvalues Of Covariance Matrices Research Articles

On the limits of distinguishing seabed types via ambient acoustic sound

This article presents a theoretical analysis of optimally distinguishing among environmental parameters from ocean ambient sound. Recent approaches to this problem either focus on parameter estimation or attempt to classify the environment into one of many known types through machine learning. This classification problem is framed as one of hypothesis testing on the received ambient sound snapshots. The resulting test depends on the Kullback-Leibler divergence (KLD) between the distributions corresponding to different environments or sediment types. Analysis of the KLD shows the dependence on the signal-to-noise ratio, the underlying signal subspace, and the distribution of eigenvalues of the respective covariance matrices. This analysis provides insights into both when and why successful hypothesis testing is possible. Experiments demonstrate that our analysis provides insight as to why certain environmental parameters are more difficult to distinguish than others. Experiments on sediment types from the Naval Oceanographic Office Bottom Sediment type database show that certain types are indistinguishable for a given array configuration. Further, the KLD can be used to provide a quantitative alternative to examining bottom loss curves to predict array processing performance.

Single image noise level estimation by artificial noise

In this paper, we propose a new method for noise level estimation from a single noisy image contaminated by additive white Gaussian noise and gamma noise, which usually appear due to different physical imaging systems. Our method is motivated by the key observation that the curves of small eigenvalues of the patch covariance matrices corresponding to the original noisy image and the artificial noisy image are nearly parallel. Based on some assumptions, we derive a proportional relation equation between eigenvalues and variances of the noisy images in the redundant dimensions. The solution of this equation gives the basic noise estimation formula. However, the assumptions are not strictly satisfied for natural images with textures and fine details. To overcome this drawback, we modified the basic noise estimation formula by introducing a correction parameter. The proposed method is also extended to multiplicative Gamma noise estimation. Numerical results demonstrate that the proposed noise level estimation method outperforms the existing state-of-the-art approaches in accuracy for various scenes. In particular, the decreasing of Root Mean Square Error (RMSE) is an average of over 10% for the testing databases. We further demonstrate that the proposed noise estimation method can be used for blind denoising.

Large deviations of extremal eigenvalues of sample covariance matrices

AbstractLarge deviations of the largest and smallest eigenvalues of $\mathbf{X}\mathbf{X}^\top/n$ are studied in this note, where $\mathbf{X}_{p\times n}$ is a $p\times n$ random matrix with independent and identically distributed (i.i.d.) sub-Gaussian entries. The assumption imposed on the dimension size p and the sample size n is $p=p(n)\rightarrow\infty$ with $p(n)={\mathrm{o}}(n)$ . This study generalizes one result obtained in [3].

Convergence rate to the Tracy–Widom laws for the largest eigenvalue of sample covariance matrices

We establish a quantitative version of the Tracy–Widom law for the largest eigenvalue of high-dimensional sample covariance matrices. To be precise, we show that the fluctuations of the largest eigenvalue of a sample covariance matrix X∗X converge to its Tracy–Widom limit at a rate nearly N−1/3, where X is an M×N random matrix whose entries are independent real or complex random variables, assuming that both M and N tend to infinity at a constant rate. This result improves the previous estimate N−2/9 obtained by Wang (2019). Our proof relies on a Green function comparison method (Adv. Math. 229 (2012) 1435–1515) using iterative cumulant expansions, the local laws for the Green function and asymptotic properties of the correlation kernel of the white Wishart ensemble.

Tracy-Widom limit for the largest eigenvalue of high-dimensional covariance matrices in elliptical distributions

Let X be an M×N random matrix consisting of independent M-variate elliptically distributed column vectors x1,…,xN with general population covariance matrix Σ. In the literature, the quantity XX∗ is referred to as the sample covariance matrix after scaling, where X∗ is the transpose of X. In this article, we prove that the limiting behavior of the scaled largest eigenvalue of XX∗ is universal for a wide class of elliptical distributions, namely, the scaled largest eigenvalue converges weakly to the same limit regardless of the distributions that x1,…,xN follow as M,N→∞ with M∕N→ϕ0>0 if the weak fourth moment of the radius of x1 exists. In particular, via comparing the Green function with that of the sample covariance matrix of multivariate normally distributed data, we conclude that the limiting distribution of the scaled largest eigenvalue is the celebrated Tracy-Widom law.

RDS free CLT for spiked eigenvalues of high-dimensional covariance matrices

In this paper, we extend the CLT for sample spiked eigenvalues in the generalized spiked covariance model proposed in Jiang and Bai (2021a) to the case where RDS is considered free, i.e., except for an upper limit of the RDS to guarantee that the spiked eigenvalue is distant, there is no limit for p/n, which is the Ratio of Dimension to sample Size (RDS). Therefore, the choice of dimensionality and sample size is more flexible in our regime.From our work, the limitation of RDS for the CLT for spiked eigenvalues of the sample covariance matrix under usual high-dimensional settings is removed, and the ultra-high dimensional regime is also covered as a special case.

Some strong convergence theorems for eigenvalues of general sample covariance matrices

The aim of this paper is to investigate the spectral properties of sample covariance matrices under a more general population. We consider a class of matrices of the form [Formula: see text], where [Formula: see text] is a [Formula: see text] nonrandom matrix and [Formula: see text] is an [Formula: see text] matrix consisting of i.i.d standard complex entries. [Formula: see text] as [Formula: see text] while m can be arbitrary but no smaller than p. We first prove that under some mild assumptions, with probability 1, for all large n, there will be no eigenvalues in any closed interval contained in an open interval which is outside the supports of the limiting distributions for all sufficiently large n. Then we get the strong convergence result for the extreme eigenvalues as an extension of Bai-Yin law.

Extremal eigenvalues of sample covariance matrices with general population

We consider the eigenvalues of sample covariance matrices of the form Q=(Σ1/2X)(Σ1/2X)∗. The sample X is an M×N rectangular random matrix with real independent entries and the population covariance matrix Σ is a positive definite diagonal matrix independent of X. Assuming that the limiting spectral density of Σ exhibits convex decay at the right edge of the spectrum, in the limit M,N→∞ with N/M→d∈(0,∞), we find a certain threshold d+ such that for d>d+ the limiting spectral distribution of Q also exhibits convex decay at the right edge of the spectrum. In this case, the largest eigenvalues of Q are determined by the order statistics of the eigenvalues of Σ, and in particular, the limiting distribution of the largest eigenvalue of Q is given by a Weibull distribution. In case d<d+, we also prove that the limiting distribution of the largest eigenvalue of Q is Gaussian if the entries of Σ are i.i.d. random variables. While Σ is considered to be random mostly, the results also hold for deterministic Σ with some additional assumptions.

Partial generalized four moment theorem revisited

This is a complementary proof of partial generalized 4 moment theorem (PG4MT) mentioned and described in “Generalized Four Moment Theorem (G4MT) and its Application to CLT for Spiked Eigenvalues of High-dimensional Covariance Matrices”. Since the G4MT proposed in that paper requires both the matrices X and Y satisfying the assumption maxt,s|uts|2E{|x11|4I(|x11|<n)−μ}→0 with the same μ which maybe restrictive in real applications, we proposed a new G4MT, called PG4MT, without proof. After the manuscript posed in ArXiv, the authors received high interests in the proof of PG4MT through private communications and find the PG4MT more general than G4MT, it is necessary to give a detailed proof of it. Moreover, it is found that the PG4MT derives a CLT of spiked eigenvalues of sample covariance matrices which covers the work in Bai and Yao (J. Multivariate Anal. 106 (2012) 167–177) as a special case.

Shrinkage priors for single-spiked covariance models

Shrinkage estimation of eigenvalues of covariance matrices for Gaussian spiked covariance models are known to be effective especially in high-dimensional problems. Various shrinkage methods have been proposed, and these typically employ nonlinear functions of eigenvalues of sample covariance matrices. Here we investigate Bayesian shrinkage methods for single-spiked covariance models. The choice of priors and the construction of the Bayes estimator are considered, and it is proved that the Bayes estimator of the present shrinkage prior dominates asymptotically that of the Jeffreys prior regarding the Kullback–Leibler risk. The Bayes estimators are obtained as the posterior mean of the covariance matrices. In numerical simulations, the Bayes methods based on the present shrinkage prior and the Jeffreys prior are compared with other non-Bayes methods.

Generalized four moment theorem and an application to CLT for spiked eigenvalues of high-dimensional covariance matrices

We consider a more generalized spiked covariance matrix, which is a general non-negative definite matrix with the spiked eigenvalues scattered into spaces of a few bulks and the largest ones allowed to tend to infinity. The study is split into two cases by whether the maximum absolute value of the eigenvector of the corresponding spikes tends to zero or not. On one hand, if it is zero, a Generalized Four Moment Theorem (G4MT) is proposed by relaxing the matching of the 3rd and the 4th moment to the tail probability decaying with certain rate, which shows the universality of the asymptotic law for the spiked eigenvalues of the generalized spiked covariance model. On the other hand, if it is not zero, the matches of the third and fourth moments in usual four moment theorem are weakened to only requiring the match of the 4th moment. Moreover, by applying the results to the Central Limit Theorem (CLT) for the spiked eigenvalues of the generalized spiked covariance model, we successively remove the restrictive condition of block wise diagonal assumption on the population covariance matrix in the previous works. This condition implies an unrealistic fact that the spiked eigenvalues and bulked eigenvalues are generated by independent variables, respectively. Thus, the new CLT will have much better application domain.

Limiting laws for divergent spiked eigenvalues and largest nonspiked eigenvalue of sample covariance matrices

We study the asymptotic distributions of the spiked eigenvalues and the largest nonspiked eigenvalue of the sample covariance matrix under a general covariance model with divergent spiked eigenvalues, while the other eigenvalues are bounded but otherwise arbitrary. The limiting normal distribution for the spiked sample eigenvalues is established. It has distinct features that the asymptotic mean relies on not only the population spikes but also the nonspikes and that the asymptotic variance in general depends on the population eigenvectors. In addition, the limiting Tracy–Widom law for the largest nonspiked sample eigenvalue is obtained. Estimation of the number of spikes and the convergence of the leading eigenvectors are also considered. The results hold even when the number of the spikes diverges. As a key technical tool, we develop a central limit theorem for a type of random quadratic forms where the random vectors and random matrices involved are dependent. This result can be of independent interest.

Estimation and Testing for High-dimensional Near Unit Root Time Series

We investigate some estimation and testing issues for a class of high-dimensional near unit root time series models. We first study the asymptotic behavior of the first k largest eigenvalues of the sample covariance matrices of the time series model. Then we propose a new estimator for the high–dimensional near unit root setting through using the largest eigenvalues of the sample covariance matrices and use it to test for near unit roots. Such an approach is theoretically novel and addresses some important estimation and testing issues in the high–dimensional near unit root setting. Simulations are also conducted to demonstrate the finite–sample performance of the proposed test statistic.

Bootstrapping spectral statistics in high dimensions

Summary Statistics derived from the eigenvalues of sample covariance matrices are called spectral statistics, and they play a central role in multivariate testing. Although bootstrap methods are an established approach to approximating the laws of spectral statistics in low-dimensional problems, such methods are relatively unexplored in the high-dimensional setting. The aim of this article is to focus on linear spectral statistics as a class of prototypes for developing a new bootstrap in high dimensions, a method we refer to as the spectral bootstrap. In essence, the proposed method originates from the parametric bootstrap and is motivated by the fact that in high dimensions it is difficult to obtain a nonparametric approximation to the full data-generating distribution. From a practical standpoint, the method is easy to use and allows the user to circumvent the difficulties of complex asymptotic formulas for linear spectral statistics. In addition to proving the consistency of the proposed method, we present encouraging empirical results in a variety of settings. Lastly, and perhaps most interestingly, we show through simulations that the method can be applied successfully to statistics outside the class of linear spectral statistics, such as the largest sample eigenvalue and others.

Compressive Subspace Learning Based Wideband Spectrum Sensing for Multiantenna Cognitive Radio

Recently, sub-Nyquist sampling (SNS) based wideband spectrum sensing has emerged as a promising approach for cognitive radios. However, most of existing SNS-based approaches cannot effectively deal with the wireless channel fading due to the lack of space diversity exploitation, which would lead to poor sensing performance. To address the problem, we propose a multiantenna system, referred to as the multiantenna generalized modulated converter (MAGMC), to realize the SNS, where spatially correlated multiple-input multiple-output (MIMO) channel is considered. Based on the multiantenna system, two compressive subspace learning (CSL) approaches (mCSL and vCSL) are proposed for signal subspace learning, where wideband sectrum sensing is realized based on the signal subspace. Both proposed CSL approaches exploit space diversity, where the mCSL utilizes an antenna averaging temporal decomposition, and the vCSL (which is formulated based on a vectorization of sample matrix in the mCSL) uses a spatial-temporal joint decomposition. We further establish analytical relationships between eigenvalues of statistical covariance matrices in statistical sense in both multiantenna and single antenna scenarios. Space diversity and superiority over the single antenna scenario for both proposed CSL approaches are analyzed based on the derived analytical relationships. Moreover, the mCSL and vCSL based wideband spectrum sensing algorithms are proposed based on the system model of MAGMC and their computational complexities are given. The proposed CSL based wideband spectrum sensing algorithms can effectively deal with the wireless channel fading and simulations show the improvement on performance of wideband spectrum sensing over related works.

Unbounded largest eigenvalue of large sample covariance matrices: Asymptotics, fluctuations and applications

Given a large sample covariance matrix SN=1nΓN1/2ZNZN⁎ΓN1/2, where ZN is a N×n matrix with i.i.d. centered entries, and ΓN is a N×N deterministic Hermitian positive semidefinite matrix, we study the location and fluctuations of λmax(SN), the largest eigenvalue of SN as N,n→∞ and Nn−1→r∈(0,∞) in the case where the empirical distribution μΓN of eigenvalues of ΓN is tight (in N) and λmax(ΓN) goes to +∞. These conditions are in particular met when μΓN weakly converges to a probability measure with unbounded support on R+.We prove that asymptotically λmax(SN)∼λmax(ΓN). Moreover when the ΓN's are block-diagonal, and the following spectral gap condition is assumed:limsupN→∞λ2(ΓN)λmax(ΓN)<1, where λ2(ΓN) is the second largest eigenvalue of ΓN, we prove Gaussian fluctuations for λmax(SN)/λmax(ΓN) at the scale n.In the particular case where ZN has i.i.d. Gaussian entries and ΓN is the N×N autocovariance matrix of a long memory Gaussian stationary process (Xt)t∈Z, the columns of ΓN1/2ZN can be considered as n i.i.d. samples of the random vector (X1,…,XN)⊤. We then prove that ΓN is similar to a diagonal matrix which satisfies all the required assumptions of our theorems, hence our results apply to this case.

Adaptive Split/Merge-Based Gaussian Mixture Model Approach for Uncertainty Propagation

This paper presents an adaptive splitting and merging scheme for dynamic selection of Gaussian kernels in a Gaussian mixture model. The Gaussian kernel in the Gaussian mixture model is split into multiple components if the Kolmogorov equation error exceeds a prescribed threshold. Two different splitting mechanisms are presented in this work. The first splitting mechanism corresponds to splitting one Gaussian kernel in all directions, whereas the second splitting mechanism corresponds to splitting in only the direction of maximum nonlinearity. The state transition matrix in conjunction with unscented transformation is used to compute the departure from linearity, and hence approximate the direction of maximum nonlinearity. The merging mechanism exploits the angle between eigenvectors corresponding to the maximum eigenvalue of covariance matrices corresponding to two different Gaussian kernels to find candidate components for merging. Finally, a sparse approximation problem is defined to provide a mechanism to trade off between the number of Gaussian kernels and the Kolmogorov equation error in a mixture model. The uncertainty propagation problem for a satellite motion in a low Earth orbit is considered to show the efficacy of the proposed ideas.

Blind estimation of transmit‐antenna number for non‐Cooperative multiple‐input multiple‐output orthogonal frequency division multiplexing systems

In this study, the authors propose a non-parametric algorithm to implement the estimation of transmit-antenna number, which is a prerequisite for blind interception process of multiple-input multiple-output orthogonal frequency division multiplexing signals in frequency selective fading. Specifically, a series of test statistics are constructed by exploiting the eigenvalues of the sample covariance matrices from each subcarrier, followed by a combination of these test statistics. As a consequence, the number of transmit antennas can be determined after a serial binary hypothesis testing. The theoretical analysis and simulation results verify the rapid convergence and high reliability of the proposed algorithm at a relatively low signal-to-noise ratio .

Almost sure convergence of the largest and smallest eigenvalues of high-dimensional sample correlation matrices

In this paper, we show that the largest and smallest eigenvalues of a sample correlation matrix stemming from n independent observations of a p-dimensional time series with iid components converge almost surely to (1+γ)2 and (1−γ)2, respectively, as n→∞, if p∕n→γ∈(0,1] and the truncated variance of the entry distribution is “almost slowly varying”, a condition we describe via moment properties of self-normalized sums. Moreover, the empirical spectral distributions of these sample correlation matrices converge weakly, with probability 1, to the Marčenko–Pastur law, which extends a result in Bai and Zhou (2008). We compare the behavior of the eigenvalues of the sample covariance and sample correlation matrices and argue that the latter seems more robust, in particular in the case of infinite fourth moment. We briefly address some practical issues for the estimation of extreme eigenvalues in a simulation study.In our proofs we use the method of moments combined with a Path-Shortening Algorithm, which efficiently uses the structure of sample correlation matrices, to calculate precise bounds for matrix norms. We believe that this new approach could be of further use in random matrix theory.

Analysis of electrocardiosignals for formation of the diagnostic features of post-traumatic myocardial dystrophy

The possibilities of high-resolution electrocardiography (HR ECG) application for diagnostics of post-traumatic myocardial dystrophy having multifactorial genesis is considered in this paper. Numerical processing and analysis of electrocardiograms that belong to patients from armed forces after explosive-driven injuries have been performed based on clinical studies. Complex method of cardiosignal analysis based on combination of wavelet analysis, eigenvector decomposition and principal component analysis is developed. This method revealed that low-amplitude deviations in ECG signal in case of post-traumatic myocardial dystrophy have low-frequency nature that is linked to slow electro-physiological processes. It is shown that these low-frequency, low-amplitude components appear at a high levels (8th and 9th) of decomposition in case of 9-level wavelet decomposition of averaged cardio cycles. Integral parameters for identification of post-traumatic myocardial dystrophy features are suggested and determined on the basis of principal component analysis. These parameters are squared sum of signal projections to eigenspaces H k and mean eigenvalues of covariance matrices of electrocardiosignals ensembles λmean.