Model Covariance Matrices Research Articles

Block-diagonal idiosyncratic covariance estimation in high-dimensional factor models for financial time series

Estimation of high-dimensional covariance matrices in latent factor models is an important topic in many fields and especially in finance. Since the number of financial assets grows while the estimation window length remains of limited size, the often used sample estimator yields noisy estimates which are not even positive definite. Under the assumption of latent factor models, the covariance matrix is decomposed into a common low-rank component and a full-rank idiosyncratic component. In this paper we focus on the estimation of the idiosyncratic component, under the assumption of a grouped structure of the time series, which may arise due to specific factors such as industries, asset classes or countries. We propose a generalized methodology for estimation of the block-diagonal idiosyncratic component by clustering the residual series and applying shrinkage to the obtained blocks in order to ensure positive definiteness. We derive two different estimators based on different clustering methods and test their performance using simulation and historical data. The proposed methods are shown to provide reliable estimates and outperform other state-of-the-art estimators based on thresholding methods.

Handling dependent samples in meta-analytic structural equation models: A Wishart-based approach.

We present an approach to meta-analytic structural equation models that relies on hierarchical modeling of sample covariance matrices under the assumption that the matrices are Wishart. The approach handles the commonplace fixed- and random-effects meta-analytic SEMs, and solves the problem of dependent covariance matrices where more than one covariance matrix is obtained from a single study or study author. The ability of the approach to adequately recover parameters is examined via a simulation study. The approach is implemented in the bayesianmasem R package and a demonstration shows applications of the model.

A comparison of indirect and direct filter designs from data for LTI systems: the effect of unknown noise covariance matrices

Existing literature on model-based filter design for stochastic LTI systems assumes complete correspondence between the system and its model. When the system is not completely known, the standard indirect model-based (two-steps) filtering solution consists of: (i) identify a model of the system from measured input/output data; (ii) design a Kalman filter based on the estimated model. The performance of this indirect approach are limited by the model and noise covariance matrices accuracy. To overcome such limitations, this paper investigates a direct (one-step) solution to the filtering problem for SISO LTI systems in the Prediction Error Method (PEM) identification framework. Simulation results indicate the effectiveness of the direct filtering approach, especially when the noise covariance matrices are misspecified.

Data-driven estimation, tracking, and system identification of deterministic and stochastic optical spot dynamics.

Stabilization, disturbance rejection, and control of optical beams and optical spots are ubiquitous problems that are crucial for the development of optical systems for ground and space telescopes, free-space optical communication terminals, precise beam steering systems, and other types of optical systems. High-performance disturbance rejection and control of optical spots require the development of disturbance estimation and data-driven Kalman filter methods. Motivated by this, we propose a unified and experimentally verified data-driven framework for optical-spot disturbance modeling and tuning of covariance matrices of Kalman filters. Our approach is based on covariance estimation, nonlinear optimization, and subspace identification methods. Also, we use spectral factorization methods to emulate optical-spot disturbances with a desired power spectral density in an optical laboratory environment. We test the effectiveness of the proposed approaches on an experimental setup consisting of a piezo tip-tilt mirror, piezo linear actuator, and a CMOS camera.

Regularized Linear Regression via Covariance Fitting

The linear minimum mean-square error estimator (LMMSE) can be viewed as a solution to a certain regularized least-squares problem formulated using model covariance matrices. However, the appropriate parameters of the model covariance matrices are unknown in many applications. This raises the question: how should we choose them using only the data? Using data-adaptive matrices obtained via the covariance fitting SPICE-methodology, we show that the empirical LMMSE is equivalent to tuned versions of various known regularized estimators – such as ridge regression, LASSO, and regularized least absolute deviation – depending on the chosen covariance structures. These theoretical results unify several important estimators under a common umbrella. Furthermore, through a number of numerical examples we show that the regularization parameters obtained via covariance fitting are close to optimal for a range of different signal conditions.

Hierarchical Covariance Estimation Approach to Meta-Analytic Structural Equation Modeling

We present a fully Bayesian approach to meta-analytic SEM based on hierarchical modeling of sample covariance matrices. The approach allows for flexible models that would not be identified under a traditional maximum likelihood approach. The approach allows for the inclusion of moderators, produces a global fit index, and permits the investigation of local misspecification. Simulation-based calibration studies show that the Bayesian computation procedure produces valid inferences for commonplace meta-analytic SEM applications. We demonstrate the approach with diverse data analysis examples and provide accompanying R code to support adoption and additional study of the approach. Finally, we lay out proposals that have the potential to extend the approach to accommodate the wide variety of analyses and data conditions that comprise meta-analytic SEM applications.

Continuous Hyper-parameter OPtimization (CHOP) in an ensemble Kalman filter

Practical data assimilation algorithms often contain hyper-parameters, which may arise due to, for instance, the use of certain auxiliary techniques like covariance inflation and localization in an ensemble Kalman filter, the re-parameterization of certain quantities such as model and/or observation error covariance matrices, and so on. Given the richness of the established assimilation algorithms, and the abundance of the approaches through which hyper-parameters are introduced to the assimilation algorithms, one may ask whether it is possible to develop a sound and generic method to efficiently choose various types of (sometimes high-dimensional) hyper-parameters. This work aims to explore a feasible, although likely partial, answer to this question. Our main idea is built upon the notion that a data assimilation algorithm with hyper-parameters can be considered as a parametric mapping that links a set of quantities of interest (e.g., model state variables and/or parameters) to a corresponding set of predicted observations in the observation space. As such, the choice of hyper-parameters can be recast as a parameter estimation problem, in which our objective is to tune the hyper-parameters in such a way that the resulted predicted observations can match the real observations to a good extent. From this perspective, we propose a hyper-parameter estimation workflow and investigate the performance of this workflow in an ensemble Kalman filter. In a series of experiments, we observe that the proposed workflow works efficiently even in the presence of a relatively large amount (up to 103) of hyper-parameters, and exhibits reasonably good and consistent performance under various conditions.

New minimal, median, and maximal propagation models for dark matter searches with Galactic cosmic rays

Galactic charged cosmic rays (notably electrons, positrons, antiprotons and light antinuclei) are powerful probes of dark matter annihilation or decay, in particular for candidates heavier than a few MeV or tiny evaporating primordial black holes. Recent measurements by PAMELA, AMS-02, or VOYAGER on positrons and antiprotons already translate into constraints on several models over a large mass range. However, these constraints depend on Galactic transport models, in particular the diffusive halo size, subject to theoretical and statistical uncertainties. We update the so-called MIN-MED-MAX benchmark transport parameters that yield generic minimal, median and maximal dark-matter induced fluxes; this reduces the uncertainties on fluxes by a factor of about 2 for positrons and 6 for antiprotons, with respect to their former version. We also provide handy fitting formulae for the associated predicted secondary antiproton and positron background fluxes. Finally, for more refined analyses, we provide the full details of the model parameters and covariance matrices of uncertainties.

Sparse covariance estimation in logit mixture models

Summary This paper introduces a new data-driven methodology for estimating sparse covariance matrices of the random coefficients in logit mixture models. Researchers typically specify covariance matrices in logit mixture models under one of two extreme assumptions: either an unrestricted full covariance matrix (allowing correlations between all random coefficients), or a restricted diagonal matrix (allowing no correlations at all). Our objective is to find optimal subsets of correlated coefficients for which we estimate covariances. We propose a new estimator, called MISC (mixed integer sparse covariance), that uses a mixed-integer optimization (MIO) program to find an optimal block diagonal structure specification for the covariance matrix, corresponding to subsets of correlated coefficients, for any desired sparsity level using Markov Chain Monte Carlo (MCMC) posterior draws from the unrestricted full covariance matrix. The optimal sparsity level of the covariance matrix is determined using out-of-sample validation. We demonstrate the ability of MISC to correctly recover the true covariance structure from synthetic data. In an empirical illustration using a stated preference survey on modes of transportation, we use MISC to obtain a sparse covariance matrix indicating how preferences for attributes are related to one another.

Large‐sample approximations and change testing for high‐dimensional covariance matrices of multivariate linear time series and factor models

AbstractVarious statistical problems can be formulated in terms of a bilinear form of the covariance matrix. Examples are testing whether coordinates of a high‐dimensional random vector are uncorrelated, constructing confidence intervals for the risk of optimal portfolios or testing for the stability of a covariance matrix, especially for factor models. Extending previous works to a general high‐dimensional multivariate linear process framework and factor models, we establish distributional approximations for the associated bilinear form of the sample covariance matrix. These approximations hold for increasing dimension without any constraint relative to the sample size. The results are used to construct change‐point tests for the covariance structure, especially in order to check the stability of a high‐dimensional factor model. Tests based on the cumulated sum (CUSUM), self‐standardized CUSUM and the CUSUM statistic maximized over all subsamples are considered. Size and power of the proposed testing methodology are investigated by a simulation study and illustrated by analyzing the Fama and French factors for a change due to the SARS‐CoV‐2 pandemic.

A New Parametrization of Correlation Matrices

We introduce a novel parametrization of the correlation matrix. The reparametrization facilitates modeling of correlation and covariance matrices by an unrestricted vector, where positive definiteness is an innate property. This parametrization can be viewed as a generalization of Fisher's Z‐transformation to higher dimensions and has a wide range of potential applications. An algorithm for reconstructing the unique n × n correlation matrix from any vector in R n ( n − 1 ) / 2 is provided, and we derive its numerical complexity.

On properties of Toeplitz-type covariance matrices in models with nested random effects

Models that capture symmetries present in the data have been widely used in different applications, with early examples from psychometric and medical research. The aim of this article is to study a random effects model focusing on the covariance structure that is block circular symmetric. Useful results are obtained for the spectra of these structured matrices.

A Review of Innovation-Based Methods to Jointly Estimate Model and Observation Error Covariance Matrices in Ensemble Data Assimilation

AbstractData assimilation combines forecasts from a numerical model with observations. Most of the current data assimilation algorithms consider the model and observation error terms as additive Gaussian noise, specified by their covariance matrices and , respectively. These error covariances, and specifically their respective amplitudes, determine the weights given to the background (i.e., the model forecasts) and to the observations in the solution of data assimilation algorithms (i.e., the analysis). Consequently, and matrices significantly impact the accuracy of the analysis. This review aims to present and to discuss, with a unified framework, different methods to jointly estimate the and matrices using ensemble-based data assimilation techniques. Most of the methods developed to date use the innovations, defined as differences between the observations and the projection of the forecasts onto the observation space. These methods are based on two main statistical criteria: 1) the method of moments, in which the theoretical and empirical moments of the innovations are assumed to be equal, and 2) methods that use the likelihood of the observations, themselves contained in the innovations. The reviewed methods assume that innovations are Gaussian random variables, although extension to other distributions is possible for likelihood-based methods. The methods also show some differences in terms of levels of complexity and applicability to high-dimensional systems. The conclusion of the review discusses the key challenges to further develop estimation methods for and . These challenges include taking into account time-varying error covariances, using limited observational coverage, estimating additional deterministic error terms, or accounting for correlated noise.

Smooth interpolation of covariance matrices and brain network estimation: Part II.

This work focuses on the modeling of time-varying covariance matrices using the state covariance of linear systems. Following concepts from optimal mass transport, we investigate and compare three types of covariance paths which are solutions to different optimal control problems. One of the covariance paths solves the Schrödinger bridge problem (SBP). The other two types of covariance paths are based on generalizations of the Fisher-Rao metric in information geometry, which are the major contributions of this work. The general framework is an extension of the approach in [1] which focuses on linear systems without stochastic input. The performances of the three covariance paths are compared using synthetic data and a real-data example on the estimation of dynamic brain networks using functional magnetic resonance imaging.

Testing for covariance matrices in time-varying coefficient panel data models with fixed effects

In this paper, we study the tests for sphericity and identity of covariance matrices in time-varying coefficient high-dimensional panel data models with fixed effects. In order to construct the effective test statistics and avoid the influence of the unknown fixed effects, we apply the difference method to eliminate the dependence of the residual sample, and further construct test statistics using the trace estimators of the covariance matrices. For the estimators of the coefficient functions, we use the local linear dummy variable method. Under some regularity conditions, we study the asymptotic property of the estimators and establish the asymptotic distributions of our proposed test statistics without specifying an explicit relationship between the cross-sectional and the time series dimensions. We further show that the test statistics are asymptotic distribution-free. Subsequently simulation studies are carried out to evaluate our proposed methods. In order to assess the performance of our proposed test method, we compare with the existing test methods in panel data linear models with fixed effects.

Collaborative Sequential State Estimation Under Unknown Heterogeneous Noise Covariance Matrices

We study the problem of distributed sequential estimation of common states and measurement noise covariance matrices of hidden Markov models by networks of collaborating nodes. We adopt a realistic assumption that the true covariance matrices are possibly different (heterogeneous) across the network. This setting is frequent in many distributed real-world systems where the sensors (e.g., radars) are deployed in a spatially anisotropic environment, or where the networks may consist of sensors with different measuring principles (e.g., using different wavelengths). Our solution is rooted in the variational Bayesian paradigm. In order to improve the estimation performance, the measurements and the posterior estimates are communicated among adjacent neighbors within one network hop distance using the information diffusion strategy. The resulting adaptive algorithm selects neighbors with compatible information to prevent degradation of estimates.

The eigenstructure of the sample covariance matrices of high-dimensional stochastic volatility models with heavy tails

We consider a $p$-dimensional time series where the dimension $p$ increases with the sample size $n$. The resulting data matrix $X$ follows a stochastic volatility model: each entry consists of a positive random volatility term multiplied by an independent noise term. The volatility multipliers introduce dependence in each row and across the rows. We study the asymptotic behavior of the eigenvalues and eigenvectors of the sample covariance matrix $XX'$ under a regular variation assumption on the noise. In particular, we prove Poisson convergence for the point process of the centered and normalized eigenvalues and derive limit theory for functionals acting on them, such as the trace. We prove related results for stochastic volatility models with additional linear dependence structure and for stochastic volatility models where the time-varying volatility terms are extinguished with high probability when $n$ increases. We provide explicit approximations of the eigenvectors which are of a strikingly simple structure. The main tools for proving these results are large deviation theorems for heavy-tailed time series, advocating a unified approach to the study of the eigenstructure of heavy-tailed random matrices.

Covariance matrices for galaxy cluster weak lensing: from virial regime to uncorrelated large-scale structure

ABSTRACT Next-generation optical imaging surveys will revolutionize the observations of weak gravitational lensing by galaxy clusters and provide stringent constraints on growth of structure and cosmic acceleration. In these experiments, accurate modelling of covariance matrices of cluster weak lensing plays the key role in obtaining robust measurements of the mean mass of clusters and cosmological parameters. We use a combination of analytical calculations and high-resolution N-body simulations to derive accurate covariance matrices that span from the virial regime to linear scales of the cluster-matter cross-correlation. We validate this calculation using a public ray-tracing lensing simulation and provide a software package for calculating covariance matrices for a wide range of cluster and source sample choices. We discuss the relative importance of shape noise and density fluctuations, the impact of radial bin size, and the impact of off-diagonal elements. For a weak lensing source density ns = 10 arcmin−2, shape noise typically dominates the variance on comoving scales $r_{\rm p}\lesssim 5\ h^{-1} \, \rm Mpc$. However, for ns = 60 arcmin−2, potentially achievable with future weak lensing experiments, density fluctuations typically dominate the variance at $r_{\rm p}\gtrsim 1\ h^{-1} \, \rm Mpc$ and remain comparable to shape noise on smaller scales.

TGCnA: temporal gene coexpression network analysis using a low-rank plus sparse framework

ABSTRACTVarious gene network models with distinct physical nature have been widely used in biological studies. For temporal transcriptomic studies, the current dynamic models either ignore the temporal variation in the network structure or fail to scale up to a large number of genes due to severe computational bottlenecks and sample size limitation. Although the correlation-based gene networks are computationally affordable, they have limitations after being applied to gene expression time-course data. We proposed Temporal Gene Coexpression Network Analysis (TGCnA) framework for the transcriptomic time-course data. The mathematical nature of TGCnA is the joint modeling of multiple covariance matrices across time points using a ‘low-rank plus sparse’ framework, in which the network similarity across time points is explicitly modeled in the low-rank component. We demonstrated the advantage of TGCnA in covariance matrix estimation and gene module discovery using both simulation data and real transcriptomic data. The code is available at https://github.com/QiZhangStat/TGCnA.

EIGENVALUE DISTRIBUTIONS OF VARIANCE COMPONENTS ESTIMATORS IN HIGH-DIMENSIONAL RANDOM EFFECTS MODELS.

We study the spectra of MANOVA estimators for variance component covariance matrices in multivariate random effects models. When the dimensionality of the observations is large and comparable to the number of realizations of each random effect, we show that the empirical spectra of such estimators are well-approximated by deterministic laws. The Stieltjes transforms of these laws are characterized by systems of fixed-point equations, which are numerically solvable by a simple iterative procedure. Our proof uses operator-valued free probability theory, and we establish a general asymptotic freeness result for families of rectangular orthogonally-invariant random matrices, which is of independent interest. Our work is motivated in part by the estimation of components of covariance between multiple phenotypic traits in quantitative genetics, and we specialize our results to common experimental designs that arise in this application.