High-dimensional Covariance Matrices Research Articles

Two-sample test for high-dimensional covariance matrices: A normal-reference approach

Testing the equality of the covariance matrices of two high-dimensional samples is a fundamental inference problem in statistics. Several tests have been proposed but they are either too liberal or too conservative when the required assumptions are not satisfied which attests that they are not always applicable in real data analysis. To overcome this difficulty, a normal-reference test is proposed and studied in this paper. It is shown that under some regularity conditions and the null hypothesis, the proposed test statistic and a chi-squared-type mixture have the same limiting distribution. It is then justified to approximate the null distribution of the proposed test statistic using that of the chi-squared-type mixture. The distribution of the chi-squared-type mixture can be well approximated using a three-cumulant matched chi-squared-approximation with its approximation parameters consistently estimated from the data. The asymptotic power of the proposed test under a local alternative is also established. Simulation studies and a real data example demonstrate that the proposed test works well in general scenarios and outperforms the existing competitors substantially in terms of size control.

Block-diagonal idiosyncratic covariance estimation in high-dimensional factor models for financial time series

Estimation of high-dimensional covariance matrices in latent factor models is an important topic in many fields and especially in finance. Since the number of financial assets grows while the estimation window length remains of limited size, the often used sample estimator yields noisy estimates which are not even positive definite. Under the assumption of latent factor models, the covariance matrix is decomposed into a common low-rank component and a full-rank idiosyncratic component. In this paper we focus on the estimation of the idiosyncratic component, under the assumption of a grouped structure of the time series, which may arise due to specific factors such as industries, asset classes or countries. We propose a generalized methodology for estimation of the block-diagonal idiosyncratic component by clustering the residual series and applying shrinkage to the obtained blocks in order to ensure positive definiteness. We derive two different estimators based on different clustering methods and test their performance using simulation and historical data. The proposed methods are shown to provide reliable estimates and outperform other state-of-the-art estimators based on thresholding methods.

High-dimensional covariance matrices under dynamic volatility models: Asymptotics and shrinkage estimation

High-dimensional covariance matrices under dynamic volatility models: Asymptotics and shrinkage estimation

Doubly Sparse estimator for high-dimensional covariance matrices

The classical sample covariance estimator lacks desirable properties such as consistency and suffers from eigenvalue spreading in high-dimensional settings. Improved estimators have been proposed that shrink sample eigenvalues but retain the eigenvectors of the sample covariance estimator. In high dimensions, however, sample eigenvectors are generally strongly inconsistent, rendering eigenvalue shrinkage estimators suboptimal. A Doubly Sparse Covariance Estimator (DSCE) is developed that goes beyond mere eigenvalue shrinkage: a covariance matrix is decomposed into a signal part, where sparse eigenvectors are estimated via truncation, and an idiosyncratic part, estimated via thresholding. It is shown that accurate estimation is possible if the leading eigenvectors are sufficiently sparse affecting proportionately less than p of the variables. DSCE fills the gap for empirical applications that fall in-between fully sparse settings and conditionally sparse settings: DSCE takes advantage of conditional sparsity implied by factor models while allowing only a subset of variables to load on factors, which relaxes pervasiveness assumption of traditional factor models. An empirical application to the constituents of the S&P 1500 illustrates that DSCE-based portfolios outperform competing methods in terms of Sharpe ratio, maximum drawdown, and cumulative return for monthly and daily data.

Element Aggregation for Estimation of High-Dimensional Covariance Matrices

This study addresses the challenge of estimating high-dimensional covariance matrices in financial markets, where traditional sparsity assumptions often fail due to the interdependence of stock returns across sectors. We present an innovative element-aggregation method that aggregates matrix entries to estimate covariance matrices. This method is designed to be applicable to both sparse and non-sparse matrices, transcending the limitations of sparsity-based approaches. The computational simplicity of the method’s implementation ensures low complexity, making it a practical tool for real-world applications. Theoretical analysis then confirms the method’s consistency and effectiveness with its convergence rate in specific scenarios. Additionally, numerical experiments validate the method’s superior algorithmic performance compared to conventional methods, as well as the reduction in relative estimation errors. Furthermore, empirical studies in financial portfolio optimization demonstrate the method’s significant risk management benefits, particularly its ability to effectively mitigate portfolio risk even with limited sample sizes.

Adaptive Tests for Bandedness of High-dimensional Covariance Matrices

Estimation of the high-dimensional banded covariance matrix is widely used in multivariate statistical analysis. To ensure the validity of estimation, we aim to test the hypothesis that the covariance matrix is banded with a certain bandwidth under the high-dimensional framework. Though several testing methods have been proposed in the literature, the existing tests are only powerful for some alternatives with certain sparsity levels, whereas they may not be powerful for alternatives with other sparsity structures. The goal of this paper is to propose a new test for the bandedness of high-dimensional covariance matrix, which is powerful for alternatives with various sparsity levels. The proposed new test also be used for testing the banded structure of covariance matrices of error vectors in high-dimensional factor models. Based on these statistics, a consistent bandwidth estimator is also introduced for a banded high dimensional covariance matrix. Extensive simulation studies and an application to a prostate cancer dataset from protein mass spectroscopy are conducted for evaluating the effectiveness of the proposed adaptive tests blue and bandwidth estimator for the banded covariance matrix.

Use of random integration to test equality of high dimensional covariance matrices.

Testing the equality of two covariance matrices is a fundamental problem in statistics, and especially challenging when the data are high-dimensional. Through a novel use of random integration, we can test the equality of high-dimensional covariance matrices without assuming parametric distributions for the two underlying populations, even if the dimension is much larger than the sample size. The asymptotic properties of our test for arbitrary number of covariates and sample size are studied in depth under a general multivariate model. The finite-sample performance of our test is evaluated through numerical studies. The empirical results demonstrate that our test is highly competitive with existing tests in a wide range of settings. In particular, our proposed test is distinctly powerful under different settings when there exist a few large or many small diagonal disturbances between the two covariance matrices.

Projection inference for high-dimensional covariance matrices with structured shrinkage targets

Projection inference for high-dimensional covariance matrices with structured shrinkage targets

Optimal Estimation of High-Dimensional Covariance Matrices with Missing and Noisy Data

Optimal Estimation of High-Dimensional Covariance Matrices with Missing and Noisy Data

Riemannian Statistics Meets Random Matrix Theory: Toward Learning From High-Dimensional Covariance Matrices

Riemannian Gaussian distributions were initially introduced as basic building blocks for learning models which aim to capture the intrinsic structure of statistical populations of positive-definite matrices (here called covariance matrices). While the potential applications of such models have attracted significant attention, a major obstacle still stands in the way of these applications: there seems to exist no practical method of computing the normalising factors associated with Riemannian Gaussian distributions on spaces of high-dimensional covariance matrices. The present paper shows that this missing method comes from an unexpected new connection with random matrix theory. Its main contribution is to prove that Riemannian Gaussian distributions of real, complex, or quaternion covariance matrices are equivalent to orthogonal, unitary, or symplectic log-normal matrix ensembles. This equivalence yields a highly efficient approximation of the normalising factors, in terms of a rather simple analytic expression. The error due to this approximation decreases like the inverse square of dimension. Numerical experiments are conducted which demonstrate how this new approximation can unlock the difficulties which have impeded applications to real-world datasets of high-dimensional covariance matrices. The paper then turns to Riemannian Gaussian distributions of block-Toeplitz covariance matrices. These are equivalent to yet another kind of random matrix ensembles, here called “acosh-normal” ensembles. Orthogonal and unitary “acosh-normal” ensembles correspond to the cases of block-Toeplitz with Toeplitz blocks, and block-Toeplitz (with general blocks) covariance matrices, respectively.

Block-diagonal test for high-dimensional covariance matrices

Block-diagonal test for high-dimensional covariance matrices

Forecasting High-Dimensional Covariance Matrices Using High-Dimensional Principal Component Analysis

We modify the recently proposed forecasting model of high-dimensional covariance matrices (HDCM) of asset returns using high-dimensional principal component analysis (PCA). It is well-known that when the sample size is smaller than the dimension, eigenvalues estimated by classical PCA have a bias. In particular, a very small number of eigenvalues are extremely large and they are called spiked eigenvalues. High-dimensional PCA gives eigenvalues which correct the biases of the spiked eigenvalues. This situation also happens in the financial field, especially in situations where high-frequency and high-dimensional data are handled. The research aims to estimate the HDCM of asset returns using high-dimensional PCA for the realized covariance matrix using the Nikkei 225 data, it estimates 5- and 10-min intraday asset-returns intervals. We construct time-series models for eigenvalues which are estimated by each PCA, and forecast HDCM. Our simulation analysis shows that the high-dimensional PCA has better estimation performance than classical PCA for the estimating integrated covariance matrix. In our empirical analysis, we show that we will be able to improve the forecasting performance using the high-dimensional PCA and make a portfolio with smaller variance.

Testing and signal identification for two-sample high-dimensional covariances via multi-level thresholding

The paper considers testing and signal identification for covariance matrices from two populations of marginally sub-Gaussian distributed. A multi-level thresholding procedure is proposed for testing the equality of two high-dimensional covariance matrices, which is designed to detect sparse and faint differences between the covariances. A novel U-statistic composition is developed to establish the asymptotic distribution of the thresholding statistics in conjunction with the matrix blocking and the coupling techniques. It is shown that the proposed test is more powerful than the existing tests in detecting sparse and weak signals in covariances. Multiple testing procedures are constructed to discover different covariances and the sub-groups of variables with different covariance structures between the two populations. The proposed procedures are based on the multi-level thresholding test, which are able to control the false discovery proportion (FDP) with high power. Simulation experiments and a case study on the returns of the S&P 500 stocks before and after the COVID-19 pandemic are conducted to demonstrate and compare the utilities of the proposed methods.

Tracy-Widom limit for the largest eigenvalue of high-dimensional covariance matrices in elliptical distributions

Let X be an M×N random matrix consisting of independent M-variate elliptically distributed column vectors x1,…,xN with general population covariance matrix Σ. In the literature, the quantity XX∗ is referred to as the sample covariance matrix after scaling, where X∗ is the transpose of X. In this article, we prove that the limiting behavior of the scaled largest eigenvalue of XX∗ is universal for a wide class of elliptical distributions, namely, the scaled largest eigenvalue converges weakly to the same limit regardless of the distributions that x1,…,xN follow as M,N→∞ with M∕N→ϕ0>0 if the weak fourth moment of the radius of x1 exists. In particular, via comparing the Green function with that of the sample covariance matrix of multivariate normally distributed data, we conclude that the limiting distribution of the scaled largest eigenvalue is the celebrated Tracy-Widom law.

Fisher’s Combined Probability Test for High-Dimensional Covariance Matrices

Testing large covariance matrices is of fundamental importance in statistical analysis with high-dimensional data. In the past decade, three types of test statistics have been studied in the literature: quadratic form statistics, maximum form statistics, and their weighted combination. It is known that quadratic form statistics would suffer from low power against sparse alternatives and maximum form statistics would suffer from low power against dense alternatives. The weighted combination methods were introduced to enhance the power of quadratic form statistics or maximum form statistics when the weights are appropriately chosen. In this article, we provide a new perspective to exploit the full potential of quadratic form statistics and maximum form statistics for testing high-dimensional covariance matrices. We propose a scale-invariant power-enhanced test based on Fisher’s method to combine the p-values of quadratic form statistics and maximum form statistics. After carefully studying the asymptotic joint distribution of quadratic form statistics and maximum form statistics, we first prove that the proposed combination method retains the correct asymptotic size under the Gaussian assumption, and we also derive a new Lyapunov-type bound for the joint distribution and prove the correct asymptotic size of the proposed method without requiring the Gaussian assumption. Moreover, we show that the proposed method boosts the asymptotic power against more general alternatives. Finally, we demonstrate the finite-sample performance in simulation studies and a real application. Supplementary materials for this article are available online.

Testing the equality of multiple high-dimensional covariance matrices

In this paper, we develop a new test procedure for testing the homogeneity of multiple high-dimensional covariance matrices, where the populations may not be multivariate normal. Our results generalize the recent results in He and Chen (2018) which focused on two-sample test. In addition, numerical simulations demonstrate that the proposed test performs well and better in several cases than some existing multi-sample tests.

A Log-Det Heuristics for Covariance Matrix Estimation: The Analytic Setup

This paper studies a new nonconvex optimization problem aimed at recovering high-dimensional covariance matrices with a low rank plus sparse structure. The objective is composed of a smooth nonconvex loss and a nonsmooth composite penalty. A number of structural analytic properties of the new heuristics are presented and proven, thus providing the necessary framework for further investigating the statistical applications. In particular, the first and the second derivative of the smooth loss are obtained, its local convexity range is derived, and the Lipschitzianity of its gradient is shown. This opens the path to solve the described problem via a proximal gradient algorithm.

Testing homogeneity of high-dimensional covariance matrices under non-normality

In this paper, we test the homogeneity of multiple covariance matrices when the dimension may exceed the sample sizes. A test statistic is proposed which does not depend on the normality assumption. Furthermore, the asymptotic distribution of the test statistic is derived. Numerical simulations indicate that the proposed test has accurate significance level, and has a greater improvement in power than some existing tests.

A compound decision approach to covariance matrix estimation.

Covariance matrix estimation is a fundamental statistical task in many applications, but the sample covariance matrix is suboptimal when the sample size is comparable to or less than the number of features. Such high-dimensional settings are common in modern genomics, where covariance matrix estimation is frequently employed as a method for inferring gene networks. To achieve estimation accuracy in these settings, existing methods typically either assume that the population covariance matrix has some particular structure, for example, sparsity, or apply shrinkage to better estimate the population eigenvalues. In this paper, we study a new approach to estimating high-dimensional covariance matrices. We first frame covariance matrix estimation as a compound decision problem. This motivates defining a class of decision rules and using a nonparametric empirical Bayes g-modeling approach to estimate the optimal rule in the class. Simulation results and gene network inference in an RNA-seq experiment in mouse show that our approach is comparable to or can outperform a number of state-of-the-artproposals.

Initialization and Estimation of Weights and Bias using Bayesian Technique

Researches on artificial neural network models have shown that the method used to initialize and estimate weights and bias always determines the rate at which the network will converge and how efficient the model will perform. Although there are several methods that can be used to initialize and estimate network weights and bias, Bayesian approach is widely considered as a more efficient method for modelling artificial neural networks because it can easily compute the inverses of covariance matrices with high dimension that otherwise are computationally expensive. This study has developed a new filter, the First order Extended Ensemble Filter(FoEEF),that applies numerical solution in solving the inverses of high dimensional covariance matrices from I ^ to ′ s stochastic state-space dynamical models. The research applies the new FoEEF Filter to initializes and estimates the weights and bias of artificial neural network. Comparison on the performance of FoEEF filter in estimating weights and bias are done against the performance of EKF on function estimations by using two different functions, sin(x)+Q function and 3sin(x)3− 3sin(x)2 − sin(x) + 1 function, within 18 and 22 epochs. Emphasis of the performance is placed on the rate at which the models converge. This study gauges the performance by determining the minimum value of the mean square error produced from each epoch and average mean square error minimum value.The outcome from the study showed that FoEEF filter had the lowest value of mean square error and average mean square error which was achieved with the least number of epochs compared to EKF filter. This study concluded that the new FoEEF performed better than EKF and is a more suitable filter for that can be used for initialization and estimation of weights and bias in artificial neural network.