Eigenvectors Of Covariance Matrices Research Articles

Properties of eigenvalues and eigenvectors of large-dimensional sample correlation matrices

This paper is to study the properties of eigenvalues and eigenvectors of high-dimensional sample correlation matrices. We first improve the result of Jiang (Sankhyā 66 (2004) 35–48), Xiao and Zhou (J. Theoret. Probab. 23 (2010) 1–20) and the Theorem 1 of El Karoui (Ann. Appl. Probab. 19 (2009) 2362–2405), both concerning the limiting spectral distribution and the extreme eigenvalues of sample correlation matrices, by allowing a more general fourth moment condition. Then, we establish a central limit theorem (CLT) for the linear statistics of the eigenvectors of large sample correlation matrices. We discover that the difference between the functional CLT of the sample covariance matrix and the sample correlation matrix is fundamentally influenced by the direction of a nonrandom projection vector. In the special case where the square root of the correlation matrix is identity, the difference will be determined by the sum of the fourth powers of the entries of the projection vector. These results also indicate that the eigenmatrix of sample correlation matrices is not asymptotically Haar if the underlying distribution is Gaussian. In other words, the normalization based on the sample variances affects the asymptotic properties of the eigenmatrix of the Wishart matrix. Furthermore, we establish a theorem concerning CLT for the linear statistics of the eigenvectors of large sample covariance matrices. This theorem improves the main results in Bai, Miao and Pan (Ann. Probab. 35 (2007) 1532–1572), which requires the assumption that the fourth moment of the underlying variable matches the one of Gaussian distribution, as well as Theorem 1.3 in Pan and Zhou (Ann. Appl. Probab. 18 (2008) 1232–1270), which relaxed the Gaussian like fourth moment requirement but assumes the maximum entries of the projection vector converge to 0 uniformly. We illustrate the usefulness of the theoretical results through an application in communications.

On the eigenvectors of large-dimensional sample spatial sign covariance matrices

To investigate the limiting behavior of eigenvectors of the sample spatial sign covariance matrix (SSCM), the eigenvector empirical spectral distribution (VESD) is defined with weights depending on the eigenvectors. In this paper, we first show that the VESD of a large-dimensional sample SSCM converges to a generalized Marčenko–Pastur distribution when both the dimension p of observations and the sample size n tend to infinity proportionally. Further, the central limit theorem of linear spectral statistics of VESD is established, which suggests that the eigenmatrix of sample SSCM and the classical sample covariance matrix are asymptotically the same.

Dynamic Principal Component Analysis in High Dimensions

Principal component analysis is a versatile tool to reduce dimensionality which has wide applications in statistics and machine learning. It is particularly useful for modeling data in high-dimensional scenarios where the number of variables p is comparable to, or much larger than the sample size n. Despite an extensive literature on this topic, researchers have focused on modeling static principal eigenvectors, which are not suitable for stochastic processes that are dynamic in nature. To characterize the change in the entire course of high-dimensional data collection, we propose a unified framework to directly estimate dynamic eigenvectors of covariance matrices. Specifically, we formulate an optimization problem by combining the local linear smoothing and regularization penalty together with the orthogonality constraint, which can be effectively solved by manifold optimization algorithms. We show that our method is suitable for high-dimensional data observed under both common and irregular designs, and theoretical properties of the estimators are investigated under l q ( 0 ≤ q ≤ 1 ) sparsity. Extensive experiments demonstrate the effectiveness of the proposed method in both simulated and real data examples.

Empirical estimation of the eigenmodes of macroscale cortical dynamics: Reconciling neural field eigenmodes and resting-state networks

Neural field theories successfully describe normal, awake mesoscale to macroscale brain activity using linearized partial differential equations that involve a Laplacian operator. These neural field theories therefore predict spatial brain activity patterns that are simple, smooth eigenfunctions of the Laplacian operator that satisfy the Helmholtz equation and extend across the entire cortex. However, complex, localized spatial patterns such as resting-state networks have been proposed from data analysis of brain activity measurements, primarily fMRI. Using Human Connectome Project data, the Helmholtz eigenmode prediction of neural field theory is shown to be consistent with eigenvectors of both fMRI and MEG covariance matrices. It is explained why these Helmholtz equation eigenmodes have been difficult to observe due to their similar spectra, effects of covariant input stimuli to the brain, and measurement noise. This implies that complex spatial patterns like resting-state networks may be the result of complex input to the brain but simple intrinsic brain dynamics, rather than simple stimuli and complex brain dynamics, as assumed in many existing models.

Energy Efficiency and Spectral Efficiency Tradeoff in Massive MIMO Multicast Transmission with Statistical CSI.

As the core technology of 5G mobile communication systems, massive multi-input multi-output (MIMO) can dramatically enhance the energy efficiency (EE), as well as the spectral efficiency (SE), which meets the requirements of new applications. Meanwhile, physical layer multicast technology has gradually become the focus of next-generation communication technology research due to its capacity to efficiently provide wireless transmission from point to multipoint. The availability of channel state information (CSI), to a large extent, determines the performance of massive MIMO. However, because obtaining the perfect instantaneous CSI in massive MIMO is quite challenging, it is reasonable and practical to design a massive MIMO multicast transmission strategy using statistical CSI. In this paper, in order to optimize the system resource efficiency (RE) to achieve EE-SE balance, the EE-SE trade-offs in the massive MIMO multicast transmission are investigated with statistical CSI. Firstly, we formulate the eigenvectors of the RE optimization multicast covariance matrices of different user terminals in closed form, which illustrates that in the massive MIMO downlink, optimal RE multicast precoding is supposed to be done in the beam domain. On the basis of this viewpoint, the optimal RE precoding design is simplified into a resource efficient power allocation problem. Via invoking the quadratic transform, we propose an iterative power allocation algorithm, which obtains an adjustable and reasonable EE-SE tradeoff. Numerical simulation results reveal the near-optimal performance and the effectiveness of our proposed statistical CSI-assisted RE maximization in massive MIMO.

Convergence of eigenvector empirical spectral distribution of sample covariance matrices

The eigenvector empirical spectral distribution (VESD) is a useful tool in studying the limiting behavior of eigenvalues and eigenvectors of covariance matrices. In this paper, we study the convergence rate of the VESD of sample covariance matrices to the deformed Mar\v{c}enko-Pastur (MP) distribution. Consider sample covariance matrices of the form $\Sigma^{1/2} X X^* \Sigma^{1/2}$, where $X=(x_{ij})$ is an $M\times N$ random matrix whose entries are independent random variables with mean zero and variance $N^{-1}$, and $\Sigma$ is a deterministic positive-definite matrix. We prove that the Kolmogorov distance between the expected VESD and the deformed MP distribution is bounded by $N^{-1+\epsilon}$ for any fixed $\epsilon>0$, provided that the entries $\sqrt{N}x_{ij}$ have uniformly bounded 6th moments and $|N/M-1|\ge \tau$ for some constant $\tau>0$. This result improves the previous one obtained in \cite{XYZ2013}, which gave the convergence rate $O(N^{-1/2})$ assuming $i.i.d.$ $X$ entries, bounded 10th moment, $\Sigma=I$ and $M<N$. Moreover, we also prove that under the finite $8$th moment assumption, the convergence rate of the VESD is $O(N^{-1/2+\epsilon})$ almost surely for any fixed $\epsilon>0$, which improves the previous bound $N^{-1/4+\epsilon}$ in \cite{XYZ2013}.

Status detection from spatial-temporal data in pipeline network using data transformation convolutional neural network

With the scale expansion and structural upgrading of pipeline network, the detection methods based on both ends of the pipeline pressure have appeared the limitations of judging pipeline status in the multi-mode and complex network system. To overcome the limitation of early methods, a pipeline network status detection method based on data transformation convolutional neural network (DT-CNN) is proposed in this paper. Firstly, the difference among the eigenvalue distribution of data covariance matrices is calculated to detect the pipeline status by Kullback-Leibler divergence (KLD). If the eigenvalue distribution deviates from the normal status, the KLD will exceed the given threshold. Furthermore, an improved CNN model is proposed to judge pipeline status by converting the largest eigenvectors of data covariance matrices to extract features. The effectiveness of the proposed detection method is demonstrated through the simulation results of a practical pipeline network.

Beam Domain Secure Transmission for Massive MIMO Communications

We investigate the optimality and power allocation algorithm of beam domain transmission for single-cell massive multiple-input multiple-output (MIMO) systems with a multi-antenna passive eavesdropper. Focusing on the secure massive MIMO downlink transmission with only statistical channel state information of legitimate users and the eavesdropper at base station, we introduce a lower bound on the achievable ergodic secrecy sum-rate, from which we derive the condition for eigenvectors of the optimal input covariance matrices. The result shows that beam domain transmission can achieve optimal performance in terms of secrecy sum-rate lower bound maximization. For the case of single-antenna legitimate users, we prove that it is optimal to allocate no power to the beams where the beam gains of the eavesdropper are stronger than those of legitimate users in order to maximize the secrecy sum-rate lower bound. Then, motivated by the concave-convex procedure and the large dimension random matrix theory, we develop an efficient iterative and convergent algorithm to optimize power allocation in the beam domain. Numerical simulations demonstrate the tightness of the secrecy sum-rate lower bound and the near-optimal performance of the proposed iterative algorithm.

A Multi-Frame PCA-Based Stereo Audio Coding Method

With the increasing demand for high quality audio, stereo audio coding has become more and more important. In this paper, a multi-frame coding method based on Principal Component Analysis (PCA) is proposed for the compression of audio signals, including both mono and stereo signals. The PCA-based method makes the input audio spectral coefficients into eigenvectors of covariance matrices and reduces coding bitrate by grouping such eigenvectors into fewer number of vectors. The multi-frame joint technique makes the PCA-based method more efficient and feasible. This paper also proposes a quantization method that utilizes Pyramid Vector Quantization (PVQ) to quantize the PCA matrices proposed in this paper with few bits. Parametric coding algorithms are also employed with PCA to ensure the high efficiency of the proposed audio codec. Subjective listening tests with Multiple Stimuli with Hidden Reference and Anchor (MUSHRA) have shown that the proposed PCA-based coding method is efficient at processing stereo audio.

A subspace based progressive coding method for speech compression

In this study, two novel methods, which are based on Karhunen Loeve Transform (KLT) and Independent Component Analysis (ICA), are proposed for coding of speech signals. Instead of immediately dealing with eigenvalue magnitudes, the KLT- and ICA-based methods use eigenvectors of covariance matrices (or independent components for ICA) by geometrically grouping these vectors into fewer numbers of vectors. In this way, a data representation compaction is achieved. Further compression is achieved through discarding autocovariance eigenvectors corresponding to the small eigenvalues and applying vector quantization on the remaining eigenvectors. Additionally, this study proposes an iterative error refinement process, which uses the rest of the available bandwidth in order to transmit an efficient representation of the description error for better SNR. The overall process constitutes a new approach to efficient speech coding, with ICA being used in subspace speech coding for the first time. Constant bit rate (CBR) and variable bit rate (VBR) coding algorithms are employed with the proposed methods. TIMIT speech database is used in the experimental studies. Speech signals are synthesized at 2.4 kbps, 8 kbps, 12.2 kbps, 16 kbps, 16.4kbps and 19.85 kbps rates by using various frame lengths. The qualities of synthesized speech signals are compared to those of available speech codecs, i.e., LPC (2.4 kbps), G.728 (LD-CELP, 16 kbps), G.729A (CS-CELP, 8 kbps), EVS (16.4 kbps), AMR-NB (12.2 kbps) and AMR-WB (19.85 kbps).

Universality for Eigenvalue Algorithms on Sample Covariance Matrices

We prove a universal limit theorem for the halting time, or iteration count, of the power/inverse power methods and the QR eigenvalue algorithm. Specifically, we analyze the required number of iterations to compute extreme eigenvalues of random, positive definite sample covariance matrices to within a prescribed tolerance. The universality theorem provides a complexity estimate for the algorithms which, in this random setting, holds with high probability. The method of proof relies on recent results on the statistics of the eigenvalues and eigenvectors of random sample covariance matrices (i.e., delocalization, rigidity, and edge universality).

Eigenvalues and eigenvectors of heavy-tailed sample covariance matrices with general growth rates: The iid case

In this paper we study the joint distributional convergence of the largest eigenvalues of the sample covariance matrix of a p-dimensional time series with iid entries when p converges to infinity together with the sample size n. We consider only heavy-tailed time series in the sense that the entries satisfy some regular variation condition which ensures that their fourth moment is infinite. In this case, Soshnikov (2004, 2006) and Auffinger et al. (2009) proved the weak convergence of the point processes of the normalized eigenvalues of the sample covariance matrix towards an inhomogeneous Poisson process which implies in turn that the largest eigenvalue converges in distribution to a Fréchet distributed random variable. They proved these results under the assumption that p and n are proportional to each other. In this paper we show that the aforementioned results remain valid if p grows at any polynomial rate. The proofs are different from those in Auffinger et al. (2009) and Soshnikov (2004, 2006); we employ large deviation techniques to achieve them. The proofs reveal that only the diagonal of the sample covariance matrix is relevant for the asymptotic behavior of the largest eigenvalues and the corresponding eigenvectors which are close to the canonical basis vectors. We also discuss extensions of the results to sample autocovariance matrices.

Majorization–minimization for blind source separation of sparse sources

In this paper we propose the Majorization–Minimization Blind Spare Source Separation (MM-BSSS) algorithm for solving the blind source separation (BSS) problem when the source signals are known a priori to be sparse, or can be sparsely represented in some dictionary. The algorithm capitalizes on a previous result by Chartrand (2007 [24]) that shows certain classes of nonconvex functions perform better than the convex ℓ1-norm in measuring sparsity of a signal. In this paper we propose a majorization–minimization (MM) method for minimizing such a nonconvex objective function. The method can be simplified by choosing the non-convex function to be separable. In this paper we employ a previously developed technique (Mourad and Reilly, 2010 [26]) for constructing the MM surrogate function, which reduces the sparse BSS problem to an iterative computation of the minor eigenvectors of particular covariance matrices. These features permit a computationally efficient implementation. The proposed algorithm enjoys several advantages such as robustness to noise and the ability to estimate the number of source signals. Numerical results show that the proposed algorithm outperforms other well-known algorithms that solve the same problem.

Convergence of Sample Eigenvectors of Spiked Population Model

In this paper, for the general non Gaussian spiked population model, where a few fixed eigenvalues of the population covariance matrix are separated from others, we investigate the convergence properties of the eigenvectors of sample covariance matrices corresponding to the spiked population eigenvalues and angle between the population eigenvectors and sample eigenvectors as both the sample size and population size are large.

Covariance Eigenvector Sparsity for Compression and Denoising

Sparsity in the eigenvectors of signal covariance matrices is exploited in this paper for compression and denoising. Dimensionality reduction (DR) and quantization modules present in many practical compression schemes such as transform codecs, are designed to capitalize on this form of sparsity and achieve improved reconstruction performance compared to existing sparsity-agnostic codecs. Using training data that may be noisy a novel sparsity-aware linear DR scheme is developed to fully exploit sparsity in the covariance eigenvectors and form noise-resilient estimates of the principal covariance eigenbasis. Sparsity is effected via norm-one regularization, and the associated minimization problems are solved using computationally efficient coordinate descent iterations. The resulting eigenspace estimator is shown capable of identifying a subset of the unknown support of the eigenspace basis vectors even when the observation noise covariance matrix is unknown, as long as the noise power is sufficiently low. It is proved that the sparsity-aware estimator is asymptotically normal, and the probability to correctly identify the signal subspace basis support approaches one, as the number of training data grows large. Simulations using synthetic data and images, corroborate that the proposed algorithms achieve improved reconstruction quality relative to alternatives.

Wind tunnel testing of wind pressures on a large gymnasium roof

A wind tunnel test was conducted for a large steel gymnasium structure. Simultaneous pressure measurements were made on its entire ellipsoidal roof in a simulated suburban boundary layer flow field. Special attention is paid to the characteristics of fluctuating wind pressures in different zones on the roof. Some selected results are presented: 1) correlations between fluctuating wind pressures on both roof surfaces, 2) eigenvalues and eigenvectors of covariance matrices of the fluctuating wind pressures, 3) probability distributions of the fluctuating wind pressures, and 4) statistical characteristics of peak factor. Furthermore, the applicability of the quasi-steady approach is discussed in detail. Based on the results, an empirical formula for estimating the minimum pressure coefficients, using a peak factor approach, is presented. Comparison of the minimum pressure coefficients determined by the proposed formula and those obtained from the wind tunnel tests is made to examine the applicability and accuracy of the proposed formula.

A test of the hypothesis of partial common principal components

Abstract A test of the equality of the first h eigenvectors of covariance matrices of several populations is constructed without the assumption that the sampled distributions are Gaussian. It is proved that the test statistic is asymptotically chi-square distributed. In this general setting, an explicit formula for column space of the asymptotic covariance matrix of the sample eigenvectors is derived and the rank of this matrix is computed. An essential assumption in deriving the asymptotic distribution of the presented test statistic is the existence of the finite fourth moments and the simplicity of the h largest eigenvalues of population covariance matrices, which makes possible to use the formulas for derivatives of eigenvectors of symmetric matrices.

On the Asymptotic Behavior of the Sample Estimates of Eigenvalues and Eigenvectors of Covariance Matrices

This paper analyzes the asymptotic behavior of the sample estimators of the eigenvalues and eigenvectors of covariance matrices. Rather than considering traditional large sample-size asymptotics, in this paper the focus is on limited sample size situations, whereby the number of available observations is comparable in magnitude to the observation dimension. Using tools from random matrix theory, the asymptotic behavior of the traditional sample estimates is investigated under the assumption that both the sample size and the observation dimension tend to infinity, while their quotient converges to a positive quantity. Assuming that an asymptotic eigenvalue splitting condition is fulfilled, closed form asymptotic expressions of these estimators are derived, proving inconsistency of the traditional sample estimators in these asymptotic conditions. The derived expressions are shown to provide a valuable insight into the behavior of the sample estimators in the small sample size regime.

Perils of Parsimony: Properties of Reduced-Rank Estimates of Genetic Covariance Matrices

Eigenvalues and eigenvectors of covariance matrices are important statistics for multivariate problems in many applications, including quantitative genetics. Estimates of these quantities are subject to different types of bias. This article reviews and extends the existing theory on these biases, considering a balanced one-way classification and restricted maximum-likelihood estimation. Biases are due to the spread of sample roots and arise from ignoring selected principal components when imposing constraints on the parameter space, to ensure positive semidefinite estimates or to estimate covariance matrices of chosen, reduced rank. In addition, it is shown that reduced-rank estimators that consider only the leading eigenvalues and -vectors of the "between-group" covariance matrix may be biased due to selecting the wrong subset of principal components. In a genetic context, with groups representing families, this bias is inverse proportional to the degree of genetic relationship among family members, but is independent of sample size. Theoretical results are supplemented by a simulation study, demonstrating close agreement between predicted and observed bias for large samples. It is emphasized that the rank of the genetic covariance matrix should be chosen sufficiently large to accommodate all important genetic principal components, even though, paradoxically, this may require including a number of components with negligible eigenvalues. A strategy for rank selection in practical analyses is outlined.

Central limit theorem for signal-to-interference ratio of reduced rank linear receiver

Let $\mathbf{s}_{k}=\frac{1}{\sqrt{N}}(v_{1k},\ldots,v_{Nk})^{T}$, with {vik, i, k=1, …} independent and identically distributed complex random variables. Write Sk=(s1, …, sk−1, sk+1, …, sK), Pk=diag(p1, …, pk−1, pk+1, …, pK), Rk=(SkPkSk*+σ2I) and Akm=[sk, Rksk, …, Rkm−1sk]. Define βkm=pksk*Akm(Akm*×RkAkm)−1Akm*sk, referred to as the signal-to-interference ratio (SIR) of user k under the multistage Wiener (MSW) receiver in a wireless communication system. It is proved that the output SIR under the MSW and the mutual information statistic under the matched filter (MF) are both asymptotic Gaussian when N/K→c>0. Moreover, we provide a central limit theorem for linear spectral statistics of eigenvalues and eigenvectors of sample covariance matrices, which is a supplement of Theorem 2 in Bai, Miao and Pan [Ann. Probab. 35 (2007) 1532–1572]. And we also improve Theorem 1.1 in Bai and Silverstein [Ann. Probab. 32 (2004) 553–605].