High-dimensional Principal Component Analysis Research Articles

A Consideration of the Recognition Taguchi Method Using High-Dimensional Principal Component Analysis

The Recognition-Taguchi (RT) method has been proposed for evaluating binary data such as image data. Additionally, the RT-PC method can be applied to continuous-type data when the variables have different units. The RT-PC method uses the principal component analysis (PCA); however, applying PCA to high-dimensional data causes complications in the estimation accuracy of the eigenvalue and eigenvector. Therefore, a method based on the sparse PCA is proposed in this context instead of the conventional PCA. However, previous studies have not investigated the assumption that the anomaly-detection performance of the RT-PC method is based on a typical high-dimensional PCA. In this study, we introduce the noise-reduction and cross-data (CD) matrix methods to the RT-PC method and evaluate their performance using Monte Carlo simulation. The RT-CD method uses the calculation process of the CD matrix method instead of PCA. The simulation analysis results show a better performance in the pattern of inner anomalies when compared to the RT-PC method. The RT-NR method uses the calculation process of the noise-reduction method in the RT-PC method instead of PCA. Finally, the RT-NR and RT-PC methods are observed to exhibit the same anomaly-detection performance.

Forecasting High-Dimensional Covariance Matrices Using High-Dimensional Principal Component Analysis

We modify the recently proposed forecasting model of high-dimensional covariance matrices (HDCM) of asset returns using high-dimensional principal component analysis (PCA). It is well-known that when the sample size is smaller than the dimension, eigenvalues estimated by classical PCA have a bias. In particular, a very small number of eigenvalues are extremely large and they are called spiked eigenvalues. High-dimensional PCA gives eigenvalues which correct the biases of the spiked eigenvalues. This situation also happens in the financial field, especially in situations where high-frequency and high-dimensional data are handled. The research aims to estimate the HDCM of asset returns using high-dimensional PCA for the realized covariance matrix using the Nikkei 225 data, it estimates 5- and 10-min intraday asset-returns intervals. We construct time-series models for eigenvalues which are estimated by each PCA, and forecast HDCM. Our simulation analysis shows that the high-dimensional PCA has better estimation performance than classical PCA for the estimating integrated covariance matrix. In our empirical analysis, we show that we will be able to improve the forecasting performance using the high-dimensional PCA and make a portfolio with smaller variance.

High-dimensional principal component analysis with heterogeneous missingness.

We study the problem of high-dimensional Principal Component Analysis (PCA) with missing observations. In a simple, homogeneous observation model, we show that an existing observed-proportion weighted (OPW) estimator of the leading principal components can (nearly) attain the minimax optimal rate of convergence, which exhibits an interesting phase transition. However, deeper investigation reveals that, particularly in more realistic settings where the observation probabilities are heterogeneous, the empirical performance of the OPW estimator can be unsatisfactory; moreover, in the noiseless case, it fails to provide exact recovery of the principal components. Our main contribution, then, is to introduce a new method, which we call primePCA, that is designed to cope with situations where observations may be missing in a heterogeneous manner. Starting from the OPW estimator, primePCA iteratively projects the observed entries of the data matrix onto the column space of our current estimate to impute the missing entries, and then updates our estimate by computing the leading right singular space of the imputed data matrix. We prove that the error of primePCA converges to zero at a geometric rate in the noiseless case, and when the signal strength is not too small. An important feature of our theoretical guarantees is that they depend on average, as opposed to worst-case, properties of the missingness mechanism. Our numerical studies on both simulated and real data reveal that primePCA exhibits very encouraging performance across a wide range of scenarios, including settings where the data are not Missing Completely At Random.

Expected stock returns, common idiosyncratic volatility and average idiosyncratic correlation

Motivated by Herskovic et al. (2016), we examine the role of the average idiosyncratic correlation (ICOR) in two types of markets: an emerging market and a developed market. Examining daily stock data from the Chinese stock market for the period 1995 to 2020 and from the US for the period 1926 to 2019, we adopt high-dimensional principal component analysis (PCA) and thresholding methods to re-estimate ICOR. We find that ICOR plays an important role in explaining the expected stock returns, as the common idiosyncratic volatility (CIV) does in Herskovic et al. (2016). ICOR has been neglected in the literature due to large estimation error in the idiosyncratic covariance matrix and our analysis provides evidence that ICOR is nonnegligible in both markets when we control for several common market factors. We show that the average idiosyncratic covariance, which is the numerator of ICOR, exhibits the same pattern as CIV. Furthermore, our regression analyses of expected stock returns in response to ICOR change in both markets show that, in contrast to the negative result for CIV, the stocks’ high risk exposure to ICOR change comes with a higher risk premium, perhaps because of the synchronized but disproportionate changes in the monthly idiosyncratic covariance and idiosyncratic volatility.

No Statistical-Computational Gap in Spiked Matrix Models with Generative Network Priors.

We provide a non-asymptotic analysis of the spiked Wishart and Wigner matrix models with a generative neural network prior. Spiked random matrices have the form of a rank-one signal plus noise and have been used as models for high dimensional Principal Component Analysis (PCA), community detection and synchronization over groups. Depending on the prior imposed on the spike, these models can display a statistical-computational gap between the information theoretically optimal reconstruction error that can be achieved with unbounded computational resources and the sub-optimal performances of currently known polynomial time algorithms. These gaps are believed to be fundamental, as in the emblematic case of Sparse PCA. In stark contrast to such cases, we show that there is no statistical-computational gap under a generative network prior, in which the spike lies on the range of a generative neural network. Specifically, we analyze a gradient descent method for minimizing a nonlinear least squares objective over the range of an expansive-Gaussian neural network and show that it can recover in polynomial time an estimate of the underlying spike with a rate-optimal sample complexity and dependence on the noise level.

LARGE COVARIANCE ESTIMATION THROUGH ELLIPTICAL FACTOR MODELS.

We propose a general Principal Orthogonal complEment Thresholding (POET) framework for large-scale covariance matrix estimation based on the approximate factor model. A set of high level sufficient conditions for the procedure to achieve optimal rates of convergence under different matrix norms is established to better understand how POET works. Such a framework allows us to recover existing results for sub-Gaussian data in a more transparent way that only depends on the concentration properties of the sample covariance matrix. As a new theoretical contribution, for the first time, such a framework allows us to exploit conditional sparsity covariance structure for the heavy-tailed data. In particular, for the elliptical distribution, we propose a robust estimator based on the marginal and spatial Kendall's tau to satisfy these conditions. In addition, we study conditional graphical model under the same framework. The technical tools developed in this paper are of general interest to high dimensional principal component analysis. Thorough numerical results are also provided to back up the developed theory.

Consistency of AIC and BIC in estimating the number of significant components in high-dimensional principal component analysis

In this paper, we study the problem of estimating the number of significant components in principal component analysis (PCA), which corresponds to the number of dominant eigenvalues of the covariance matrix of $p$ variables. Our purpose is to examine the consistency of the estimation criteria AIC and BIC based on the model selection criteria by Akaike [In 2nd International Symposium on Information Theory (1973) 267–281, Akademia Kiado] and Schwarz [Estimating the dimension of a model 6 (1978) 461–464] under a high-dimensional asymptotic framework. Using random matrix theory techniques, we derive sufficient conditions for the criterion to be strongly consistent for the case when the dominant population eigenvalues are bounded, and when the dominant eigenvalues tend to infinity. Moreover, the asymptotic results are obtained without normality assumption on the population distribution. Simulation studies are also conducted, and results show that the sufficient conditions in our theorems are essential.

Testing the order of a population spectral distribution for high-dimensional data

Large covariance matrices play a fundamental role in various high-dimensional statistics. Investigating the limiting behavior of the eigenvalues can reveal informative structures of large covariance matrices, which is particularly important in high-dimensional principal component analysis and covariance matrix estimation. In this paper, we propose a framework to test the number of distinct population eigenvalues for large covariance matrices, i.e. the order of a Population Spectral Distribution. The limiting distribution of our test statistic for a Population Spectral Distribution of order 2 is developed along with its (N,p) consistency, which is clearly demonstrated in our simulation study. We also apply our test to two classical microarray datasets.

Covariance Matrix Estimation for the Cryo-EM Heterogeneity Problem.

In cryo-electron microscopy (cryo-EM), a microscope generates a top view of a sample of randomly oriented copies of a molecule. The problem of single particle reconstruction (SPR) from cryo-EM is to use the resulting set of noisy two-dimensional projection images taken at unknown directions to reconstruct the three-dimensional (3D) structure of the molecule. In some situations, the molecule under examination exhibits structural variability, which poses a fundamental challenge in SPR. The heterogeneity problem is the task of mapping the space of conformational states of a molecule. It has been previously suggested that the leading eigenvectors of the covariance matrix of the 3D molecules can be used to solve the heterogeneity problem. Estimating the covariance matrix is challenging, since only projections of the molecules are observed, but not the molecules themselves. In this paper, we formulate a general problem of covariance estimation from noisy projections of samples. This problem has intimate connections with matrix completion problems and high-dimensional principal component analysis. We propose an estimator and prove its consistency. When there are finitely many heterogeneity classes, the spectrum of the estimated covariance matrix reveals the number of classes. The estimator can be found as the solution to a certain linear system. In the cryo-EM case, the linear operator to be inverted, which we term the projection covariance transform, is an important object in covariance estimation for tomographic problems involving structural variation. Inverting it involves applying a filter akin to the ramp filter in tomography. We design a basis in which this linear operator is sparse and thus can be tractably inverted despite its large size. We demonstrate via numerical experiments on synthetic datasets the robustness of our algorithm to high levels of noise.

Asymptotic Distribution of the Contribution Ratio in High-Dimensional Principal Component Analysis

In principal component analysis we use the sample cumulative contribution ratio as an estimate of the population cumulative contribution ratio which is a proportion of information condensed into the first several principal components. The main purpose of this paper is to derive an asymptotic distribution of the cumulative contribution ratio in a high-dimensional situation where both the dimension and the sample size are large. Its asymptotic distribution is derived under a spiked model in which the variances of the remainder population principal components are assumed to be equal and small. We consider its logit transformation which is useful in a situation where the population cumulative contribution ratio is high or low. The corresponding large sample results are also summarized with a generalization. Numerical simulation revealed that our new approximation is more accurate than the classical large sample approximation as the dimension increases.

Asymptotic Distribution of Studentized Contribution Ratio in High-Dimensional Principal Component Analysis

This article is concerned with consistent estimators of the asymptotic variances of the sample cumulative contribution ratio and the one of logit transformation. We deal with the case in which the covariance matrix has a spiked model in a high-dimensional case where the number of observations and the sample size are both large. Studentized statistics for the high-dimensional case are formulated. Our results are generalizations of Fujikoshi et al. (2008). Numerical simulations show that only the studentized statistic of the logit is reasonably accurate in high dimensions.

Asymptotic distribution of the LR statistic for equality of the smallest eigenvalues in high-dimensional principal component analysis

This paper deals with the distribution of the LR statistic for testing the hypothesis that the smallest eigenvalues of a covariance matrix are equal. We derive an asymptotic null distribution of the LR statistic when the dimension p and the sample size N approach infinity, while the ratio p / N converging on a finite nonzero limit c ∈ ( 0 , 1 ) . Numerical simulations revealed that our approximation is more accurate than the classical chi-square-type approximation as p increases in value.

Editorial data

Motivated by Herskovic et al. (2016), we examine the role of the average idiosyncratic correlation (ICOR) in two types of markets: an emerging market and a developed market. Examining daily stock data from the Chinese stock market for the period 1995 to 2020 and from the US for the period 1926 to 2019, we adopt high-dimensional principal component analysis (PCA) and thresholding methods to re-estimate ICOR. We find that ICOR plays an important role in explaining the expected stock returns, as the common idiosyncratic volatility (CIV) does in Herskovic et al. (2016). ICOR has been neglected in the literature due to large estimation error in the idiosyncratic covariance matrix and our analysis provides evidence that ICOR is nonnegligible in both markets when we control for several common market factors. We show that the average idiosyncratic covariance, which is the numerator of ICOR, exhibits the same pattern as CIV. Furthermore, our regression analyses of expected stock returns in response to ICOR change in both markets show that, in contrast to the negative result for CIV, the stocks’ high risk exposure to ICOR change comes with a higher risk premium, perhaps because of the synchronized but disproportionate changes in the monthly idiosyncratic covariance and idiosyncratic volatility.