Spiked Population Model Research Articles

Spectral statistics of high dimensional sample covariance matrix with unbounded population spectral norm

In this paper, we establish some new central limit theorems for certain spectral statistics of a high-dimensional sample covariance matrix under a divergent spectral norm population model. This model covers the divergent spiked population model as a special case. Meanwhile, the number of the spiked eigenvalues can either be fixed or grow to infinity. It is seen from our theorems that the divergence of population spectral norm affects the fluctuations of the linear spectral statistics in a fickle way, depending on the divergence rate.

Asymptotic joint distribution of extreme eigenvalues and trace of large sample covariance matrix in a generalized spiked population model

This paper studies the joint limiting behavior of extreme eigenvalues and trace of large sample covariance matrix in a generalized spiked population model, where the asymptotic regime is such that the dimension and sample size grow proportionally. The form of the joint limiting distribution is applied to conduct Johnson–Graybill-type tests, a family of approaches testing for signals in a statistical model. For this, higher order correction is further made, helping alleviate the impact of finite-sample bias. The proof rests on determining the joint asymptotic behavior of two classes of spectral processes, corresponding to the extreme and linear spectral statistics, respectively.

Large scale analysis of generalization error in learning using margin based classification methods

Large-margin classifiers are popular methods for classification. We derive the asymptotic expression for the generalization error of a family of large-margin classifiers in the limit of both sample size n and dimension p going to ∞ with fixed ratio α = n/p. This family covers a broad range of commonly used classifiers including support vector machine, distance weighted discrimination, and penalized logistic regression. Our result can be used to establish the phase transition boundary for the separability of two classes. We assume that the data are generated from a single multivariate Gaussian distribution with arbitrary covariance structure. We explore two special choices for the covariance matrix: spiked population model and two layer neural networks with random first layer weights. The method we used for deriving the closed-form expression is from statistical physics known as the replica method. Our asymptotic results match simulations already when n, p are of the order of a few hundreds. For two layer neural networks, we reproduce the recently observed ‘double descent’ phenomenology for several classification models. We also discuss some statistical insights that can be drawn from these analysis.

High‐dimensional variable screening under multicollinearity

Variable screening is of fundamental importance in linear regression models when the number of predictors far exceeds the number of observations. Multicollinearity is a common phenomenon in high‐dimensional settings, in which two or more predictor variables are highly correlated, leading to the notorious difficulty for high‐dimensional variable screening. Sure independence screening (SIS) procedure can greatly reduce the dimensionality, but it may break down when the predictors are highly correlated. By combing the factor modelling with SIS, the profiled independence screening (PIS) approach was proposed. However, under a spiked population model, the profiled predictors could not be guaranteed to be uncorrelated and PIS may therefore be misleading. Instead of assuming either the predictors are uncorrelated as in SIS or the profiled predictors are uncorrelated as in PIS, a more general and challenging scenario is considered in which the predictors can be highly correlated. A so‐called preconditioned PIS (PPIS) method is proposed that produces asymptotically uncorrelated profiled predictors and thus leads to consistent model selection results under a spiked population model. Compared with PIS, the proposed method could handle the complex multicollinearity case, such as a spiked population model with a slow spectrum decay of population covariance matrix, while keeping the calculation simple. The promising performance of the proposed PPIS method will be illustrated via extensive simulation studies and two real examples.

Approximation of the power functions of Roy’s largest root test under general spiked alternatives

Roy’s largest root is a common test statistic in multivariate analysis, statistical signal processing and related fields. According to Anderson [An Introduction to Multivariate Statistical Analysis, 3rd edn. (Wiley, New York, 2003)], it is numerically known that compared with the other three tests of linear hypotheses, Roy’s largest root test has the highest power under rank-one alternatives. Therefore, it is important to study the asymptotic distribution of the largest root under rank-one alternatives to obtain an estimation of the power. To the best of our knowledge, no one had solved the problem until Johnstone and Nadler [Roy’s largest root test under rank-one alternatives, Biometrika 104(1) (2017) 181–193] presented a tractable approximation of the distribution of Roy’s largest root test where the alternative is of rank one and the variance of the noise tends to zero. It is natural to ask how Roy’s largest root test performs under other alternatives, for example, rank-finite alternatives. Therefore, we are more interested in the power estimates of Roy’s largest root test under wider alternatives, namely, whether its power is still higher than that of the other three tests of linear hypotheses. In fact, the distribution of the largest root under rank-finite alternatives can be characterized as the distribution of the largest sample eigenvalue among several spiked models. In this paper, we employ the asymptotic results of the spiked eigenvalues derived by Bai and Yao [Central limit theorems for eigenvalues in a spiked population model, Ann. Inst. Henri. Poincarè Probab. Statist. 44(3) (2008) 447–474; On sample eigenvalues in a generalized spiked population model, J. Multivar. Anal. 106 (2012) 167–177] and Wang and Yao [Extreme eigenvalues of large-dimensional spiked Fisher matrices with application, Ann. Statist. 45 (2017) 415–460] to obtain the approximate power of Roy’s largest root test under a high-dimensional setting; more importantly, our results are distribution-free and applicable to cases involving rank-finite alternatives.

Some sphericity tests for high dimensional data based on ratio of the traces of sample covariance matrices

In this paper, we investigate the sphericity test for high dimensional data. The test statistic is proposed based on the ratio of traces of sample covariance matrices. The asymptotic distributions of the test statistics under the null hypothesis are established by linear spectral statistics of large sample covariance matrices. We also obtain the asymptotic power functions in the case of the spiked population model as a specific alternative. Compared with some existing tests, the proposed test can handle data having unknown means and non-Gaussian population with general fourth moment. The numerical performance demonstrates that the proposed tests have satisfactory properties in terms of size and power.

Asymptotic properties of principal component analysis and shrinkage-bias adjustment under the generalized spiked population model

With the development of high-throughput technologies, principal component analysis (PCA) in the high-dimensional regime is of great interest. Most of the existing theoretical and methodological results for high-dimensional PCA are based on the spiked population model in which all the population eigenvalues are equal except for a few large ones. Due to the presence of local correlation among features, however, this assumption may not be satisfied in many real-world datasets. To address this issue, we investigate the asymptotic behavior of PCA under the generalized spiked population model. Based on our theoretical results, we propose a series of methods for the consistent estimation of population eigenvalues, angles between the sample and population eigenvectors, correlation coefficients between the sample and population principal component (PC) scores, and the shrinkage bias adjustment for the predicted PC scores. Using numerical experiments and real data examples from the genetics literature, we show that our methods can greatly reduce bias and improve prediction accuracy.

Mesoscopic perturbations of large random matrices

We consider the eigenvalues and eigenvectors of small rank perturbations of random [Formula: see text] matrices. We allow the rank of perturbation [Formula: see text] to increase with [Formula: see text], and the only assumption is [Formula: see text]. The spiked population model, proposed by Johnstone [On the distribution of the largest eigenvalue in principal components analysis, Ann. Statist. 29(2) (2001) 295–327], is of this kind, in which all the population eigenvalues are 1’s except for a few fixed eigenvalues. Our model is more general since we allow the number of non-unit population eigenvalues to grow with the population size. In both additive and multiplicative perturbation models, we study the nonasymptotic relation between the extreme eigenvalues of the perturbed random matrix and those of the perturbation. As [Formula: see text] goes to infinity, we derive the empirical distribution of the extreme eigenvalues of the perturbed random matrix. We also compute the appropriate projection of eigenvectors corresponding to the extreme eigenvalues of the perturbed random matrix. We prove that they are approximate eigenvectors of the perturbation. Our results can be regarded as an extension of the finite rank perturbation case to the full generality up to [Formula: see text].

Likelihood ratio test for partial sphericity in high and ultra-high dimensions

We consider, in the setting of p and n large, sample covariance matrices whose population counterparts follow a spiked population model, i.e., with the exception of the first (largest) few, all the population eigenvalues are equal. We study the asymptotic distribution of the partial maximum likelihood ratio statistic and use it to test for the dimension of the population spike subspace. Furthermore, we extend this to the ultra-high-dimensional case, i.e., p > n . A thorough study of the power of the test gives a correction that allows us to test for the dimension of the population spike subspace even for values of the limit of p / n close to 1 , a setting where other approaches have proved to be deficient.

Estimating the Intrinsic Dimension of Hyperspectral Images Using a Noise-Whitened Eigengap Approach

Linear mixture models are commonly used to represent hyperspectral datacube as a linear combinations of endmember spectra. However, determining of the number of endmembers for images embedded in noise is a crucial task. This paper proposes a fully automatic approach for estimating the number of endmembers in hyperspectral images. The estimation is based on recent results of random matrix theory related to the so-called spiked population model. More precisely, we study the gap between successive eigenvalues of the sample covariance matrix constructed from high dimensional noisy samples. The resulting estimation strategy is unsupervised and robust to correlated noise. This strategy is validated on both synthetic and real images. The experimental results are very promising and show the accuracy of this algorithm with respect to state-of-the-art algorithms.

A note on the CLT of the LSS for sample covariance matrix from a spiked population model

In this note, we establish an asymptotic expansion for the centering parameter appearing in the central limit theorems for linear spectral statistic of large-dimensional sample covariance matrices when the population has a spiked covariance structure. As an application, we provide an asymptotic power function for the corrected likelihood ratio statistic for testing the presence of spike eigenvalues in the population covariance matrix. This result generalizes an existing formula from the literature where only one simple spike exists.

Convergence of Sample Eigenvectors of Spiked Population Model

In this paper, for the general non Gaussian spiked population model, where a few fixed eigenvalues of the population covariance matrix are separated from others, we investigate the convergence properties of the eigenvectors of sample covariance matrices corresponding to the spiked population eigenvalues and angle between the population eigenvectors and sample eigenvectors as both the sample size and population size are large.

Reconstruction of a low-rank matrix in the presence of Gaussian noise

In this paper we study the problem of reconstruction of a low-rank matrix observed with additive Gaussian noise. First we show that under mild assumptions (about the prior distribution of the signal matrix) we can restrict our attention to reconstruction methods that are based on the singular value decomposition of the observed matrix and act only on its singular values (preserving the singular vectors). Then we determine the effect of noise on the SVD of low-rank matrices by building a connection between matrix reconstruction problem and spiked population model in random matrix theory. Based on this knowledge, we propose a new reconstruction method, called RMT, that is designed to reverse the effect of the noise on the singular values of the signal matrix and adjust for its effect on the singular vectors. With an extensive simulation study we show that the proposed method outperform even oracle versions of both soft and hard thresholding methods and closely matches the performance of a general oracle scheme.

On the sphericity test with large-dimensional observations

In this paper, we propose corrections to the likelihood ratio test and John’s test for sphericity in large-dimensions. New formulas for the limiting parameters in the CLT for linear spectral statistics of sample covariance matrices with general fourth moments are first established. Using these formulas, we derive the asymptotic distribution of the two proposed test statistics under the null. These asymptotics are valid for general population, i.e. not necessarily Gaussian, provided a finite fourth-moment. Extensive Monte-Carlo experiments are conducted to assess the quality of these tests with a comparison to several existing methods from the literature. Moreover, we also obtain their asymptotic power functions under the alternative of a spiked population model as a specific alternative.

Spectrum Sensing Algorithms via Finite Random Matrices

We address the Primary User (PU) detection (spectrum sensing) problem, relevant to cognitive radio, from a finite random matrix theoretical (RMT) perspective. Specifically, we employ recently-derived closed-form and exact expressions for the distribution of the standard condition number (SCN) of uncorrelated and semi-correlated random dual central Wishart matrices of finite sizes in the design Hypothesis-Testing algorithms to detect the presence of PU signals. In particular, two algorithms are designed, with basis on the SCN distribution in the absence (H_0) and in the presence (H_1) of PU signals, respectively. Due to an inherent property of the SCN's, the H_0 test requires no estimation of SNR or any other information on the PU signal, while the H_1 test requires SNR only. Further attractive advantages of the new techniques are: a) due to the accuracy of the finite SCN distributions, superior performance is achieved under a finite number of samples, compared to asymptotic RMT-based alternatives; b) since expressions to model the SCN statistics both in the absence and presence of PU signal are used, the statistics of the spectrum sensing problem in question is completely characterized; and c) as a consequence of a) and b), accurate and simple analytical expressions for the receiver operating characteristic (ROC) — both in terms of the probability of detection as a function of the probability of false alarm (P_D versus P_F) and in terms of the probability of acquisition as a function of the probability of miss detection (P_A versus P_M) — are yielded. It is also shown that the proposed finite RMT-based algorithms outperform all similar alternatives currently known in the literature, at a substantially lower complexity. In the process, several new results on the distributions of eigenvalues and SCNs of random Wishart Matrices are offered, including a closed-form of the Marchenko-Pastur's Cumulative Density Function (CDF) and extensions of the latter, as well as variations of asymptotic the distributions of extreme eigenvalues (Tracy-Widom) and their ratio (Tracy-Widom-Curtiss), which are simpler than those obtained with the "spiked population model".

ON DETERMINING THE NUMBER OF SPIKES IN A HIGH-DIMENSIONAL SPIKED POPULATION MODEL

In a spiked population model, the population covariance matrix has all its eigenvalues equal to units except for a few fixed eigenvalues (spikes). Determining the number of spikes is a fundamental problem which appears in many scientific fields, including signal processing (linear mixture model) or economics (factor model). Several recent papers studied the asymptotic behavior of the eigenvalues of the sample covariance matrix (sample eigenvalues) when the dimension of the observations and the sample size both grow to infinity so that their ratio converges to a positive constant. Using these results, we propose a new estimator based on the difference between two consecutive sample eigenvalues.

On sample eigenvalues in a generalized spiked population model

In the spiked population model introduced by Johnstone (2001) [11], the population covariance matrix has all its eigenvalues equal to unit except for a few fixed eigenvalues (spikes). The question is to quantify the effect of the perturbation caused by the spike eigenvalues. Baik and Silverstein (2006) [5] establishes the almost sure limits of the extreme sample eigenvalues associated to the spike eigenvalues when the population and the sample sizes become large. In a recent work Bai and Yao (2008) [4], we have provided the limiting distributions for these extreme sample eigenvalues. In this paper, we extend this theory to a generalized spiked population model where the base population covariance matrix is arbitrary, instead of the identity matrix as in Johnstone’s case. As the limiting spectral distribution is arbitrary here, new mathematical tools, different from those in Baik and Silverstein (2006) [5], are introduced for establishing the almost sure convergence of the sample eigenvalues generated by the spikes.

Central limit theorems for eigenvalues in a spiked population model

Dans un modèle de variances hétérogènes, les valeurs propres de la matrice de covariance des variables sont toutes égales à l’unité sauf un faible nombre d’entre elles. Ce modèle a été introduit par Johnstone comme une explication possible de la structure des valeurs propres de la matrice de covariance empirique constatée sur plusieurs ensembles de données réelles. Une question importante est de quantifier la perturbation causée par ces valeurs propres différentes de l’unité. Un travail récent de Baik et Silverstein établit la limite presque sûre des valeurs propres empiriques extrêmes lorsque le nombre de variables tend vers l’infini proportionnellement à la taille de l’échantillon. Ce travail établit un théorème limite central pour ces valeurs propres empiriques extrêmes. Il est basé sur un nouveau théorème limite central pour les formes sesquilinéaires aléatoires.

Eigenvalues of large sample covariance matrices of spiked population models

We consider a spiked population model, proposed by Johnstone, in which all the population eigenvalues are one except for a few fixed eigenvalues. The question is to determine how the sample eigenvalues depend on the non-unit population ones when both sample size and population size become large. This paper completely determines the almost sure limits of the sample eigenvalues in a spiked model for a general class of samples.