High-dimensional Covariance Matrices Research Articles

Element Aggregation for Estimation of High-Dimensional Covariance Matrices

This study addresses the challenge of estimating high-dimensional covariance matrices in financial markets, where traditional sparsity assumptions often fail due to the interdependence of stock returns across sectors. We present an innovative element-aggregation method that aggregates matrix entries to estimate covariance matrices. This method is designed to be applicable to both sparse and non-sparse matrices, transcending the limitations of sparsity-based approaches. The computational simplicity of the method’s implementation ensures low complexity, making it a practical tool for real-world applications. Theoretical analysis then confirms the method’s consistency and effectiveness with its convergence rate in specific scenarios. Additionally, numerical experiments validate the method’s superior algorithmic performance compared to conventional methods, as well as the reduction in relative estimation errors. Furthermore, empirical studies in financial portfolio optimization demonstrate the method’s significant risk management benefits, particularly its ability to effectively mitigate portfolio risk even with limited sample sizes.

Adaptive Tests for Bandedness of High-dimensional Covariance Matrices

Estimation of the high-dimensional banded covariance matrix is widely used in multivariate statistical analysis. To ensure the validity of estimation, we aim to test the hypothesis that the covariance matrix is banded with a certain bandwidth under the high-dimensional framework. Though several testing methods have been proposed in the literature, the existing tests are only powerful for some alternatives with certain sparsity levels, whereas they may not be powerful for alternatives with other sparsity structures. The goal of this paper is to propose a new test for the bandedness of high-dimensional covariance matrix, which is powerful for alternatives with various sparsity levels. The proposed new test also be used for testing the banded structure of covariance matrices of error vectors in high-dimensional factor models. Based on these statistics, a consistent bandwidth estimator is also introduced for a banded high dimensional covariance matrix. Extensive simulation studies and an application to a prostate cancer dataset from protein mass spectroscopy are conducted for evaluating the effectiveness of the proposed adaptive tests blue and bandwidth estimator for the banded covariance matrix.

Use of random integration to test equality of high dimensional covariance matrices.

Testing the equality of two covariance matrices is a fundamental problem in statistics, and especially challenging when the data are high-dimensional. Through a novel use of random integration, we can test the equality of high-dimensional covariance matrices without assuming parametric distributions for the two underlying populations, even if the dimension is much larger than the sample size. The asymptotic properties of our test for arbitrary number of covariates and sample size are studied in depth under a general multivariate model. The finite-sample performance of our test is evaluated through numerical studies. The empirical results demonstrate that our test is highly competitive with existing tests in a wide range of settings. In particular, our proposed test is distinctly powerful under different settings when there exist a few large or many small diagonal disturbances between the two covariance matrices.

Projection inference for high-dimensional covariance matrices with structured shrinkage targets

Projection inference for high-dimensional covariance matrices with structured shrinkage targets

Optimal Estimation of High-Dimensional Covariance Matrices with Missing and Noisy Data

Optimal Estimation of High-Dimensional Covariance Matrices with Missing and Noisy Data

Riemannian Statistics Meets Random Matrix Theory: Toward Learning From High-Dimensional Covariance Matrices

Riemannian Gaussian distributions were initially introduced as basic building blocks for learning models which aim to capture the intrinsic structure of statistical populations of positive-definite matrices (here called covariance matrices). While the potential applications of such models have attracted significant attention, a major obstacle still stands in the way of these applications: there seems to exist no practical method of computing the normalising factors associated with Riemannian Gaussian distributions on spaces of high-dimensional covariance matrices. The present paper shows that this missing method comes from an unexpected new connection with random matrix theory. Its main contribution is to prove that Riemannian Gaussian distributions of real, complex, or quaternion covariance matrices are equivalent to orthogonal, unitary, or symplectic log-normal matrix ensembles. This equivalence yields a highly efficient approximation of the normalising factors, in terms of a rather simple analytic expression. The error due to this approximation decreases like the inverse square of dimension. Numerical experiments are conducted which demonstrate how this new approximation can unlock the difficulties which have impeded applications to real-world datasets of high-dimensional covariance matrices. The paper then turns to Riemannian Gaussian distributions of block-Toeplitz covariance matrices. These are equivalent to yet another kind of random matrix ensembles, here called “acosh-normal” ensembles. Orthogonal and unitary “acosh-normal” ensembles correspond to the cases of block-Toeplitz with Toeplitz blocks, and block-Toeplitz (with general blocks) covariance matrices, respectively.

Block-diagonal test for high-dimensional covariance matrices

Block-diagonal test for high-dimensional covariance matrices

Forecasting High-Dimensional Covariance Matrices Using High-Dimensional Principal Component Analysis

We modify the recently proposed forecasting model of high-dimensional covariance matrices (HDCM) of asset returns using high-dimensional principal component analysis (PCA). It is well-known that when the sample size is smaller than the dimension, eigenvalues estimated by classical PCA have a bias. In particular, a very small number of eigenvalues are extremely large and they are called spiked eigenvalues. High-dimensional PCA gives eigenvalues which correct the biases of the spiked eigenvalues. This situation also happens in the financial field, especially in situations where high-frequency and high-dimensional data are handled. The research aims to estimate the HDCM of asset returns using high-dimensional PCA for the realized covariance matrix using the Nikkei 225 data, it estimates 5- and 10-min intraday asset-returns intervals. We construct time-series models for eigenvalues which are estimated by each PCA, and forecast HDCM. Our simulation analysis shows that the high-dimensional PCA has better estimation performance than classical PCA for the estimating integrated covariance matrix. In our empirical analysis, we show that we will be able to improve the forecasting performance using the high-dimensional PCA and make a portfolio with smaller variance.

Testing and signal identification for two-sample high-dimensional covariances via multi-level thresholding

The paper considers testing and signal identification for covariance matrices from two populations of marginally sub-Gaussian distributed. A multi-level thresholding procedure is proposed for testing the equality of two high-dimensional covariance matrices, which is designed to detect sparse and faint differences between the covariances. A novel U-statistic composition is developed to establish the asymptotic distribution of the thresholding statistics in conjunction with the matrix blocking and the coupling techniques. It is shown that the proposed test is more powerful than the existing tests in detecting sparse and weak signals in covariances. Multiple testing procedures are constructed to discover different covariances and the sub-groups of variables with different covariance structures between the two populations. The proposed procedures are based on the multi-level thresholding test, which are able to control the false discovery proportion (FDP) with high power. Simulation experiments and a case study on the returns of the S&P 500 stocks before and after the COVID-19 pandemic are conducted to demonstrate and compare the utilities of the proposed methods.

Tracy-Widom limit for the largest eigenvalue of high-dimensional covariance matrices in elliptical distributions

Let X be an M×N random matrix consisting of independent M-variate elliptically distributed column vectors x1,…,xN with general population covariance matrix Σ. In the literature, the quantity XX∗ is referred to as the sample covariance matrix after scaling, where X∗ is the transpose of X. In this article, we prove that the limiting behavior of the scaled largest eigenvalue of XX∗ is universal for a wide class of elliptical distributions, namely, the scaled largest eigenvalue converges weakly to the same limit regardless of the distributions that x1,…,xN follow as M,N→∞ with M∕N→ϕ0>0 if the weak fourth moment of the radius of x1 exists. In particular, via comparing the Green function with that of the sample covariance matrix of multivariate normally distributed data, we conclude that the limiting distribution of the scaled largest eigenvalue is the celebrated Tracy-Widom law.

Fisher’s Combined Probability Test for High-Dimensional Covariance Matrices

Testing large covariance matrices is of fundamental importance in statistical analysis with high-dimensional data. In the past decade, three types of test statistics have been studied in the literature: quadratic form statistics, maximum form statistics, and their weighted combination. It is known that quadratic form statistics would suffer from low power against sparse alternatives and maximum form statistics would suffer from low power against dense alternatives. The weighted combination methods were introduced to enhance the power of quadratic form statistics or maximum form statistics when the weights are appropriately chosen. In this article, we provide a new perspective to exploit the full potential of quadratic form statistics and maximum form statistics for testing high-dimensional covariance matrices. We propose a scale-invariant power-enhanced test based on Fisher’s method to combine the p-values of quadratic form statistics and maximum form statistics. After carefully studying the asymptotic joint distribution of quadratic form statistics and maximum form statistics, we first prove that the proposed combination method retains the correct asymptotic size under the Gaussian assumption, and we also derive a new Lyapunov-type bound for the joint distribution and prove the correct asymptotic size of the proposed method without requiring the Gaussian assumption. Moreover, we show that the proposed method boosts the asymptotic power against more general alternatives. Finally, we demonstrate the finite-sample performance in simulation studies and a real application. Supplementary materials for this article are available online.

Testing the equality of multiple high-dimensional covariance matrices

In this paper, we develop a new test procedure for testing the homogeneity of multiple high-dimensional covariance matrices, where the populations may not be multivariate normal. Our results generalize the recent results in He and Chen (2018) which focused on two-sample test. In addition, numerical simulations demonstrate that the proposed test performs well and better in several cases than some existing multi-sample tests.

A Log-Det Heuristics for Covariance Matrix Estimation: The Analytic Setup

This paper studies a new nonconvex optimization problem aimed at recovering high-dimensional covariance matrices with a low rank plus sparse structure. The objective is composed of a smooth nonconvex loss and a nonsmooth composite penalty. A number of structural analytic properties of the new heuristics are presented and proven, thus providing the necessary framework for further investigating the statistical applications. In particular, the first and the second derivative of the smooth loss are obtained, its local convexity range is derived, and the Lipschitzianity of its gradient is shown. This opens the path to solve the described problem via a proximal gradient algorithm.

Testing homogeneity of high-dimensional covariance matrices under non-normality

In this paper, we test the homogeneity of multiple covariance matrices when the dimension may exceed the sample sizes. A test statistic is proposed which does not depend on the normality assumption. Furthermore, the asymptotic distribution of the test statistic is derived. Numerical simulations indicate that the proposed test has accurate significance level, and has a greater improvement in power than some existing tests.

A compound decision approach to covariance matrix estimation.

Covariance matrix estimation is a fundamental statistical task in many applications, but the sample covariance matrix is suboptimal when the sample size is comparable to or less than the number of features. Such high-dimensional settings are common in modern genomics, where covariance matrix estimation is frequently employed as a method for inferring gene networks. To achieve estimation accuracy in these settings, existing methods typically either assume that the population covariance matrix has some particular structure, for example, sparsity, or apply shrinkage to better estimate the population eigenvalues. In this paper, we study a new approach to estimating high-dimensional covariance matrices. We first frame covariance matrix estimation as a compound decision problem. This motivates defining a class of decision rules and using a nonparametric empirical Bayes g-modeling approach to estimate the optimal rule in the class. Simulation results and gene network inference in an RNA-seq experiment in mouse show that our approach is comparable to or can outperform a number of state-of-the-artproposals.

Initialization and Estimation of Weights and Bias using Bayesian Technique

Researches on artificial neural network models have shown that the method used to initialize and estimate weights and bias always determines the rate at which the network will converge and how efficient the model will perform. Although there are several methods that can be used to initialize and estimate network weights and bias, Bayesian approach is widely considered as a more efficient method for modelling artificial neural networks because it can easily compute the inverses of covariance matrices with high dimension that otherwise are computationally expensive. This study has developed a new filter, the First order Extended Ensemble Filter(FoEEF),that applies numerical solution in solving the inverses of high dimensional covariance matrices from I ^ to ′ s stochastic state-space dynamical models. The research applies the new FoEEF Filter to initializes and estimates the weights and bias of artificial neural network. Comparison on the performance of FoEEF filter in estimating weights and bias are done against the performance of EKF on function estimations by using two different functions, sin(x)+Q function and 3sin(x)3− 3sin(x)2 − sin(x) + 1 function, within 18 and 22 epochs. Emphasis of the performance is placed on the rate at which the models converge. This study gauges the performance by determining the minimum value of the mean square error produced from each epoch and average mean square error minimum value.The outcome from the study showed that FoEEF filter had the lowest value of mean square error and average mean square error which was achieved with the least number of epochs compared to EKF filter. This study concluded that the new FoEEF performed better than EKF and is a more suitable filter for that can be used for initialization and estimation of weights and bias in artificial neural network.

Size-and-Shape Space Gaussian Mixture Models for Structural Clustering of Molecular Dynamics Trajectories

Determining the optimal number and identity of structural clusters from an ensemble of molecular configurations continues to be a challenge. Recent structural clustering methods have focused on the use of internal coordinates due to the innate rotational and translational invariance of these features. The vast number of possible internal coordinates necessitates a feature space supervision step to make clustering tractable but yields a protocol that can be system type-specific. Particle positions offer an appealing alternative to internal coordinates but suffer from a lack of rotational and translational invariance, as well as a perceived insensitivity to regions of structural dissimilarity. Here, we present a method, denoted shape-GMM, that overcomes the shortcomings of particle positions using a weighted maximum likelihood alignment procedure. This alignment strategy is then built into an expectation maximization Gaussian mixture model (GMM) procedure to capture metastable states in the free-energy landscape. The resulting algorithm distinguishes between a variety of different structures, including those indistinguishable by root-mean-square displacement and pairwise distances, as demonstrated on several model systems. Shape-GMM results on an extensive simulation of the fast-folding HP35 Nle/Nle mutant protein support a four-state folding/unfolding mechanism, which is consistent with previous experimental results and provides kinetic details comparable to previous state-of-the art clustering approaches, as measured by the VAMP-2 score. Currently, training of shape-GMMs is recommended for systems (or subsystems) that can be represented by ≲200 particles and ≲100k configurations to estimate high-dimensional covariance matrices and balance computational expense. Once a shape-GMM is trained, it can be used to predict the cluster identities of millions of configurations.

A Comparative Study of Two-Sample Tests for High-Dimensional Covariance Matrices

The equality of covariance matrices is an essential assumption in means and discriminant analyses for high-dimensional data. The performance of tests for covariance matrices may vary substantially depending on the covariance structure, so using inappropriate methods to verify the assumption will result in worse performance. The purpose of this study is to assess and compare the performance of three tests for two-sample high-dimensional covariance matrices: Schott’s (2007), Srivastava and Yanagihara’s (2010), and Li and Chen’s (2012) under various covariance structures. A simulation study was conducted when the covariance structures were spherical, compound symmetric, block-diagonal, and first-order autoregressive with homogenous variances. The results show that Li and Chen’s test outperforms the others with a sample size of at least 10 under particular covariance structures. When the number of variables is increased with a fixed sample size, Li and Chen’s test still performs well, whereas Schott’s performance deteriorates. Some recommendations for selecting appropriate tests are also provided in this paper.

A Comparative Study of Two-Sample Tests for High-Dimensional Covariance Matrices

A Comparative Study of Two-Sample Tests for High-Dimensional Covariance Matrices

A Comparative Study of Two-Sample Tests for High-Dimensional Covariance Matrices

A Comparative Study of Two-Sample Tests for High-Dimensional Covariance Matrices