Covariance Matrix Estimation Research Articles

Modelling Wind Speed, Humidity, and Temperature in Butterworth and Melaka during Southwest Monsoon in 2020 with a Simultaneous Linear Functional Relationship

The extension of parameter estimation from a bivariate linear functional relationship model (LFRM) to simultaneous LFRM for linear variables using the maximum likelihood estimation (MLE) method is explored in this paper. The covariance matrix of the parameter estimates is derived through the Fisher information matrix. A simulation study was done to investigate the performance of the parameter estimation. According to the simulation study, the estimated parameters have a small bias. The beauty of simultaneous LFRM lies in developing the model to study the relationship between more than two linear variables while considering error terms for all variables. The applicability of the proposed simultaneous model is demonstrated using wind speed, humidity, and temperature data from Butterworth and Melaka during the southwest monsoon season of 2020.

Shrinkage and thresholding approaches for expected utility portfolios: An analysis in terms of predictive ability

In this paper, we estimate Expected Utility Portfolios (EUPs) in high-dimensional, low-sample settings using various covariance matrix estimation methods, including shrinkage and thresholding-based methods. We perform synthetic experiments comparing these methods, using Average Out-of-Sample Variance (AOV) for Global Minimum Variance (GMV) portfolios and Average Out-of-Sample Utility (AOU) for EUPs. Additionally, we propose a practical method for fund managers to select optimal models based on empirical data, relying on AOV and AOU performance measures. The results indicate that shrinkage-based methods outperform thresholding-based ones in high-dimensional settings, with non-linear shrinkage being particularly effective.

Multivariate realized volatility: an analysis via shrinkage methods for Brazilian market data

IntroductionRealized volatility analysis of assets in the Brazilian market within a multivariate framework is the focus of this study. Despite the success of volatility models in univariate scenarios, challenges arise due to increasing dimensionality of covariance matrices and lower asset liquidity in emerging markets.MethodsIn this study, we utilize intraday stock trading data from the Brazilian Market to compute daily covariance matrices using various specifications. To mitigate dimensionality issues in covariance matrix estimation, we implement penalization restrictions on coefficients through regressions with shrinkage techniques using Ridge, LASSO, or Elastic Net estimators. These techniques are employed to capture the dynamics of covariance matrices.ResultsComparison of covariance construction models is performed using the Model Confidence Set (MCS) algorithm, which selects the best models based on their predictive performance. The findings indicate that the method used for estimating the covariance matrix significantly impacts the selection of the best models. Additionally, it is observed that more liquid sectors demonstrate greater intra-sectoral dynamics.DiscussionWhile the results benefit from shrinkage techniques, the high correlation between assets presents challenges in capturing stock or sector idiosyncrasies. This suggests the need for further exploration and refinement of methods to better capture the complexities of volatility dynamics in emerging markets like Brazil.

Flat minima generalize for low-rank matrix recovery

Abstract Empirical evidence suggests that for a variety of overparameterized nonlinear models, most notably in neural network training, the growth of the loss around a minimizer strongly impacts its performance. Flat minima—those around which the loss grows slowly—appear to generalize well. This work takes a step towards understanding this phenomenon by focusing on the simplest class of overparameterized nonlinear models: those arising in low-rank matrix recovery. We analyse overparameterized matrix and bilinear sensing, robust principal component analysis, covariance matrix estimation and single hidden layer neural networks with quadratic activation functions. In all cases, we show that flat minima, measured by the trace of the Hessian, exactly recover the ground truth under standard statistical assumptions. For matrix completion, we establish weak recovery, although empirical evidence suggests exact recovery holds here as well. We complete the paper with synthetic experiments that illustrate our findings.

BOUNDED SUPPORT IN LINEAR RANDOM COEFFICIENT MODELS: IDENTIFICATION AND VARIABLE SELECTION

We consider linear random coefficient regression models, where the regressors are allowed to have a finite support. First, we investigate identification, and show that the means and the variances and covariances of the random coefficients are identified from the first two conditional moments of the response given the covariates if the support of the covariates, excluding the intercept, contains a Cartesian product with at least three points in each coordinate. We also discuss identification of higher-order mixed moments, as well as partial identification in the presence of a binary regressor. Next, we show the variable selection consistency of the adaptive LASSO for the variances and covariances of the random coefficients in finite and moderately high dimensions. This implies that the estimated covariance matrix will actually be positive semidefinite and hence a valid covariance matrix, in contrast to the estimate arising from a simple least squares fit. We illustrate the proposed method in a simulation study.

Covariance Matrix Estimation for High-Throughput Biomedical Data with Interconnected Communities

Estimating a covariance matrix is central to high-dimensional data analysis. Empirical analyses of high-dimensional biomedical data, including genomics, proteomics, microbiome, and neuroimaging, among others, consistently reveal strong modularity in the dependence patterns. In these analyses, intercorrelated high-dimensional biomedical features often form communities or modules that can be interconnected with others. While the interconnected community structure has been extensively studied in biomedical research (e.g., gene co-expression networks), its potential to assist in the estimation of covariance matrices remains largely unexplored. To address this gap, we propose a procedure that leverages the commonly observed interconnected community structure in high-dimensional biomedical data to estimate large covariance and precision matrices. We derive the uniformly minimum variance unbiased estimators for covariance and precision matrices in closed forms and provide theoretical results on their asymptotic properties. Our proposed method enhances the accuracy of covariance- and precision-matrix estimation and demonstrates superior performance compared to the competing methods in both simulations and real data analyses.

Real-Time Speed Estimation for an Induction Motor: An Automated Tuning of an Extended Kalman Filter Using Voltage-Current Sensors.

This paper aims at achieving real-time optimal speed estimation for an induction motor using the Extended Kalman filter (EKF). Speed estimation is essential for fault diagnosis in Motor Current Signature Analysis (MCSA). The estimation accuracy is obtained by exploring the noise covariance matrices estimation of the EKF algorithm. The noise covariance matrices are determined using a modified subspace model identification approach. In order to reach this goal, this method compares an estimated model of a deterministic system, derived from available input-output datasets (using voltage-current sensors), with the discrete-time state-space representation used in the Kalman filter equations. This comparison leads to the determination of model uncertainties, which are subsequently represented as noise covariance matrices. Based on the fifth-order nonlinear model of the induction motor, the rotor speed is estimated with the optimized EKF algorithm, and the algorithm is tested experimentally.

Difference-based covariance matrix estimation in time series nonparametric regression with application to specification tests

Summary Long-run covariance matrix estimation is the building block of time series inference. The corresponding difference-based estimator, which avoids detrending, has attracted considerable interest due to its robustness to both smooth and abrupt structural breaks and its competitive finite sample performance. However, existing methods mainly focus on estimators for the univariate process, while their direct and multivariate extensions for most linear models are asymptotically biased. We propose a novel difference-based and debiased long-run covariance matrix estimator for functional linear models with time-varying regression coefficients, allowing time series nonstationarity, long-range dependence, state heteroscedasticity and combinations thereof. We apply the new estimator to (i) the structural stability test, overcoming the notorious nonmonotonic power phenomena caused by piecewise smooth alternatives for regression coefficients, and (ii) the nonparametric residual-based tests for long memory, improving the performance via the residual-free formula of the proposed estimator. The effectiveness of the proposed method is justified theoretically and demonstrated by superior performance in simulation studies, while its usefulness is elaborated via real data analysis. Our method is implemented in the R package mlrv.

Persymmetric detection based on asymptotically optimal convex linear combination

Persymmetric structure has been utilized in space-time adaptive processing for heterogeneous environment, which leads to some detection methods based on persymmetric structure, such as persymmetric adaptive matched filter (PS-AMF). However, when the sample support is extremely limited, these methods still suffer the serious degradation in detection performance due to the large error in estimating covariance matrix. In this paper, an asymptotically optimal convex linear combination between transformed sample covariance matrix and identity matrix is considered herein to improve PS-AMF. The asymptotically optimal convex linear combination is conducive to diminishing the estimate error in estimating the transformed covariance matrix by weighting the transformed sample covariance matrix and identity matrix. Furthermore, the asymptotically optimal convex linear combination is improved by the reutilization of the convex linear combination, and the corresponding coefficients are derived. Then, the detection performance of the proposed method is approximately analyzed. At last, numerical simulations show that the proposed method performs well, compared with its counterparts, when the sample support is extremely limited.

English

english

A Nyström-based low-rank unitary MVDR beamforming scheme

A Nyström-based scheme is devised to approximate the real symmetrical covariance matrix of the popular unitary MVDR (U-MVDR) beamformer with low-rank kernel matrix, based on which a novel lightweight and robust Nyström-based U-MVDR (NU-MVDR) beamformer is developed. Subsequently, the full-dimensional Nyström covariance matrix estimator’s signal subspace can be calculated by eigenvalue decomposition (EVD) of two reduced-dimensional matrices, which will save a significant amount of computing load at the EVD stage. This scheme has obvious computational efficiency and lightweight storage overheads. Meanwhile, we derive the effect of the Nyström covariance matrix’s rank on the output signal-to-interference-plus-noise ratio (SINR). Numerical results demonstrate that the proposed beamformer performs well in less snapshots and steering vector errors cases.

Detecting outliers in the multivariate control charts for dispersion monitoring

AbstractOutlier detection is an important aspect of statistical process monitoring (SPM) because outliers affect the performance of control charts. SPM researchers study the negative impact of outliers on control charts for monitoring location parameters. However, there is little research on outlier detection in multivariate charts for monitoring process dispersion. This study aims to investigate the impact of outliers in multivariate control charts for monitoring covariance matrix of a process, and then to recommend techniques for detecting potential outliers present from Phase I samples. We propose a new multivariate dispersion chart that employs the determinant of logarithm of estimated covariance matrix as the monitoring statistic. Through Monte Carlo simulations, the results show how outliers from the first phase affect the overall performance of multivariate charts. The results also demonstrate that the minimum volume ellipsoid (MVE) estimator is effective in reducing the effect of outliers on the proposed control scheme than the other compared estimators.

Robust Hausman Pretest for Panel Data Model in the Presence of Heteroscedasticity and Influential Observations

The classical Hausman pretest (HT) is used to specified the right model between random and fixed effect panel data models. However, in the presence of heteroscedastic error variances and influential observations (IOs) in the data set, it may not correctly identify the right model. Therefore, this motivated us to proposed a new method termed Robust Hausman Test (RHTFIID) which employed residuals from weighted least square (WLS) instead of OLS in the construction of heteroscedasticity consistent covariance matrix (HCCM) estimator. The weighting method is based on an efficient High Leverage Points (HLPs) detection method called Fast Improvised Influential Distance (FIID) which down weight only vertical outliers and bad HLPs. The good HLPs were allowed in the estimation as they might contribute to the precision of the estimate. The result indicates that the new proposed RHTFIID outperformed the existing classical Hausman pretest by identifying the right model with and without heteroscedasticity and influential observations.

Field-level Lyman-α forest modeling in redshift space via augmented nonlocal Fluctuating Gunn-Peterson Approximation

Context. Devising fast and accurate methods of predicting the Lyman-α forest at the field level, avoiding the computational burden of running large-volume cosmological hydrodynamic simulations, is of fundamental importance to quickly generate the massive set of simulations needed by the state-of-the-art galaxy and Lyα forest spectroscopic surveys. Aims. We present an improved analytical model to predict the Lyα forest at the field level in redshift space from the dark matter field, expanding upon the widely used Fluctuating Gunn-Peterson Approximation (FGPA). Instead of assuming a unique universal relation over the whole considered cosmic volume, we introduce a dependence on the cosmic web environment (knots, filaments, sheets, and voids) in the model, thereby effectively accounting for nonlocal bias. Furthermore, we include a detailed treatment of velocity bias in the redshift space distortion modeling, allowing the velocity bias to be cosmic-web-dependent. Methods. We first mapped the dark matter field from real to redshift space through a particle-based relation including velocity bias, depending on the cosmic web classification of the dark matter field in real space. We then formalized an appropriate functional form for our model, building upon the traditional FGPA and including a cutoff and a boosting factor mimicking a threshold and inverse-threshold bias effect, respectively, with model parameters depending on the cosmic web classification in redshift space. Eventually, we fit the coefficients of the model via an efficient Markov chain Monte Carlo scheme. Results. We find evidence for a significant difference between the same model parameters in different environments, suggesting that for the investigated setup the simple standard FGPA is not able to adequately predict the Lyα forest in the different cosmic web regimes. We reproduce the summary statistics of the reference cosmological hydrodynamic simulation that we use for comparison, yielding an accurate mean transmitted flux, probability distribution function, 3D power spectrum, and bispectrum. In particular, we achieve maximum deviation and average deviation accuracy in the Lyα forest 3D power spectrum of ∼3% and ∼0.1% up to k ∼ 0.4 h Mpc−1, and ∼5% and ∼1.8% up to k ∼ 1.4 h Mpc−1. Conclusions. Our new model outperforms previous analytical efforts to predict the Lyα forest at the field level in all the probed summary statistics, and has the potential to become instrumental in the generation of fast accurate mocks for covariance matrices estimation in the context of current and forthcoming Lyα forest surveys.

Target selection in shrinkage estimation of covariance matrix: A structural similarity approach

The shrinkage estimator of a high-dimensional covariance matrix relies on a preassigned target matrix during data processing. This paper provides an adaptive approach for selecting the optimal Toeplitz target matrix. We discover a sufficient and necessary condition for characterizing the two kinds of target matrices with the Toeplitz structure, and we propose an adaptive selection algorithm by measuring the similarity between the data and the Toeplitz structure. Numerical simulations and an empirical study on monetary funds verify the effectiveness of the selection approach.

Efficient nonparametric estimation of Toeplitz covariance matrices

Abstract A new efficient nonparametric estimator for Toeplitz covariance matrices is proposed. This estimator is based on a data transformation that translates the problem of Toeplitz covariance matrix estimation to the problem of mean estimation in an approximate Gaussian regression. The resulting Toeplitz covariance matrix estimator is positive definite by construction, fully data driven and computationally very fast. Moreover, this estimator is shown to be minimax optimal under the spectral norm for a large class of Toeplitz matrices. These results are readily extended to estimation of inverses of Toeplitz covariance matrices. Also, an alternative version of the Whittle likelihood for the spectral density based on the discrete cosine transform is proposed.

Low-Complexity DOA Estimation Method Based on Joined Coprime Array

In this article, the elimination of ambiguity of a joined coprime array has been examined, with a focus on such of its properties as large aperture size and complete degree of freedom (DOF). The existing methods suffer from a high degree of computation complexity due to the loss constant characteristic and high peak searching. Therefore, in this paper, a DOA estimation method for a jointed coprime array, characterized by a low degree of computational complexity, is proposed. The variance of the diagonal eigenvalues of the estimated covariance matrix is designed to enhance the accuracy of the covariance matrix of the joined coprime array. Then, the Capon beamforming methods is employed for peak searching. The simulation shows that the proposed method accomplishes accurate estimation with shorter computation times and fewer operations compared to other DOA estimation methods.

Optimal reconciliation of hierarchical wind energy forecasts utilizing temporal correlation

Independent wind energy forecasts of a wind farm at different time horizons have limited accuracy, and they show disagreement despite relating to the same wind farm. The limited forecast accuracy is attributable to the insufficient information at a particular time horizon of the wind energy time series, whereas applying distinct forecasting methods to several time series of non-identical patterns at different time scales causes disagreement among forecasts. Mutual disagreement among less accurate forecasts negatively impacts the decision-making capabilities in associated power systems activities at distinct time scales. The configuration of time series expressing different time horizons at different levels of a non-overlapped hierarchically aggregated framework manifests a temporal hierarchy. Forecast combination through reconciliation of time series forecasts drawn at different hierarchical levels of temporal hierarchy using any state-of-the-art method facilitates the sharing of diverse information across the hierarchy; consequently, accuracy and mutual agreement of forecasts improve. Such benefits may be further enhanced by embedding intra- and inter-level forecast error correlations in the forecast reconciliation process. However, the forecast error covariance matrix of temporal hierarchy becomes a complex high-dimensional structure while accommodating intra- and inter-level correlations. Estimating such a matrix is challenging since the high-dimensional structure severely impedes the identifiability of model parameters. Besides, in the hierarchical forecast reconciliation process, the number of predictor variables is generally higher than the number of samples. This condition gives rise to a singular covariance matrix, making it non-invertible, and thus obstructs its parameter estimation. This work employs the MinT(shrinkage) covariance matrix estimator that considers all correlations and shrinks the non-diagonal components of the matrix toward zero to avert the complexity and, therefore, the non-identifiability. Additionally, the shrinkage parameter λ of MinT(shrinkage) conveniently obtains the invertible matrix. The case study validates that while incorporating the intra- and inter-level forecast error correlations, MinT(shrinkage) provides competitively accurate and mutually agreed forecasts over other reconciliation methods.

Regularized covariance matrix estimation in high dimensional approximate factor models

We propose a novel factor-based regularized covariance matrix estimator when the number of factors is large compared to the sample size and derive the convergence rates of our estimator. Empirical results demonstrate our proposed estimator performs well in finite samples.

Estimation of sparse covariance matrix via non-convex regularization

Estimation of high-dimensional sparse covariance matrix is one of the fundamental and important problems in multivariate analysis and has a wide range of applications in many fields. This paper presents a novel method for sparse covariance matrix estimation via solving a non-convex regularization optimization problem. We establish the asymptotic properties of the proposed estimator and develop a multi-stage convex relaxation method to find an effective estimator. The multi-stage convex relaxation method guarantees any accumulation point of the sequence generated is a first-order stationary point of the non-convex optimization. Moreover, the error bounds of the first two stage estimators of the multi-stage convex relaxation method are derived under some regularity conditions. The numerical results show that our estimator outperforms the state-of-the-art estimators and has a high degree of sparsity on the premise of its effectiveness.