Estimation Of Covariance Matrices Research Articles

Proportional Hazards Regression for the Analysis of Clustered Survival Data from Case-Cohort Studies

Case-cohort sampling is a commonly used and efficient method for studying large cohorts. Most existing methods of analysis for case-cohort data have concerned the analysis of univariate failure time data. However, clustered failure time data are commonly encountered in public health studies. For example, patients treated at the same center are unlikely to be independent. In this article, we consider methods based on estimating equations for case-cohort designs for clustered failure time data. We assume a marginal hazards model, with a common baseline hazard and common regression coefficient across clusters. The proposed estimators of the regression parameter and cumulative baseline hazard are shown to be consistent and asymptotically normal, and consistent estimators of the asymptotic covariance matrices are derived. The regression parameter estimator is easily computed using any standard Cox regression software that allows for offset terms. The proposed estimators are investigated in simulation studies, and demonstrated empirically to have increased efficiency relative to some existing methods. The proposed methods are applied to a study of mortality among Canadian dialysis patients.

Estimating m-regimes STAR-GARCH model using QMLE with parameter transformation

It is well known in the literature that obtaining the parameter estimates for the Smooth Transition Autoregressive-Generalized Autoregressive Conditional Heteroskedasticity (STAR-GARCH) can be problematic due to computational difficulties. Conventional optimization algorithms do not seem to perform well in locating the global optimum of the associated likelihood function. This makes Quasi-Maximum Likelihood Estimator (QMLE) difficult to obtain for STAR-GARCH models in practice. Curiously, there has been very little research investigating the cause of the numerical difficulties in obtaining the parameter estimates for STAR-GARCH using QMLE. The aim of the paper is to investigate the nature of the numerical difficulties using Monte Carlo Simulation. By examining the surface of the log-likelihood function based on simulated data, the results provide several insights into the difficulties in obtaining QMLE for STAR-GARCH models. Based on the findings, the paper also proposes a simple transformation on the parameters to alleviate these difficulties. Monte Carlo simulation results show promising signs for the proposed transform. The asymptotic and robust variance–covariance matrices of the original parameter estimates are derived as a function of the transformed parameter estimates, which greatly facilitates inferences on the original parameters.

Estimation of covariance matrix via the sparse Cholesky factor with lasso

In this paper, we discuss a parsimonious approach to estimation of high-dimensional covariance matrices via the modified Cholesky decomposition with lasso. Two different methods are proposed. They are the equi-angular and equi-sparse methods. We use simulation to compare the performance of the proposed methods with others available in the literature, including the sample covariance matrix, the banding method, and the L 1-penalized normal loglikelihood method. We then apply the proposed methods to a portfolio selection problem using 80 series of daily stock returns. To facilitate the use of lasso in high-dimensional time series analysis, we develop the dynamic weighted lasso (DWL) algorithm that extends the LARS-lasso algorithm. In particular, the proposed algorithm can efficiently update the lasso solution as new data become available. It can also add or remove explanatory variables. The entire solution path of the L 1-penalized normal loglikelihood method is also constructed.

GMM estimation of spatial autoregressive models with unknown heteroskedasticity

In the presence of heteroskedastic disturbances, the MLE for the SAR models without taking into account the heteroskedasticity is generally inconsistent. The 2SLS estimates can have large variances and biases for cases where regressors do not have strong effects. In contrast, GMM estimators obtained from certain moment conditions can be robust. Asymptotically valid inferences can be drawn with consistently estimated covariance matrices. Efficiency can be improved by constructing the optimal weighted estimation. The approaches are applied to the study of county teenage pregnancy rates. The empirical results show a strong spatial convergence among county teenage pregnancy rates.

Rank regression for analysis of clustered data: A natural induced smoothing approach

We consider rank regression for clustered data analysis and investigate the induced smoothing method for obtaining the asymptotic covariance matrices of the parameter estimators. We prove that the induced estimating functions are asymptotically unbiased and the resulting estimators are strongly consistent and asymptotically normal. The induced smoothing approach provides an effective way for obtaining asymptotic covariance matrices for between- and within-cluster estimators and for a combined estimator to take account of within-cluster correlations. We also carry out extensive simulation studies to assess the performance of different estimators. The proposed methodology is substantially much faster in computation and more stable in numerical results than the existing methods. We apply the proposed methodology to a dataset from a randomized clinical trial.

Robust inference in an heteroscedastic measurement error model

In this paper we deal with robust inference in heteroscedastic measurement error models. Rather than the normal distribution, we postulate a Student t distribution for the observed variables. Maximum likelihood estimates are computed numerically. Consistent estimation of the asymptotic covariance matrices of the maximum likelihood and generalized least squares estimators is also discussed. Three test statistics are proposed for testing hypotheses of interest with the asymptotic chi-square distribution which guarantees correct asymptotic significance levels. Results of simulations and an application to a real data set are also reported.

Estimating spatial covariance using penalised likelihood with weighted L 1 penalty

In spatial statistics, the estimation of covariance matrices is of great importance because of its role in spatial prediction and design. In this paper, we propose a penalised likelihood approach with weighted L 1 regularisation to estimate the covariance matrix for spatial Gaussian Markov random field models with unspecified neighbourhood structures. A new algorithm for ordering spatial points is proposed such that the corresponding precision matrix can be estimated more effectively. Furthermore, we develop an efficient algorithm to minimise the penalised likelihood via a novel usage of the regularised solution path algorithm, which does not require the use of iterative algorithms. By exploiting the sparsity structure in the precision matrix, we show that the LASSO type of approach gives improved covariance estimators measured by several criteria. Asymptotic properties of our proposed estimator are derived. Both our simulated examples and an application to the rainfall data set show that the proposed method performs competitively.

Statistical Calibration of Climate System Properties

Summary The behaviour of modern climate system simulators is controlled by numerous parameters. By matching model outputs with observed data we can perform inference on such parameters. This is a calibration problem that usually requires the ability to evaluate the computer code at any given configuration of the parameters. As the climate system simulator attempts to describe very complex physical phenomena, the task of running the model is very computationally demanding. Thus, a statistical model is required to approximate the model output. In this work, we use output from the Massachusetts Institute of Technology two-dimensional climate model (MIT2DCM), historical records and output from a three-dimensional climate model, to obtain estimates of the climate sensitivity, the effective thermal diffusivity in the deep ocean and the net aerosol forcing that control MIT2DCM. We use a Bayesian approach that allows for the use of scientifically based information on the climate parameters to be used in the calibration process. The model tackles the problem of dealing with multivariate computer model output and incorporates all estimation uncertainties into the posterior distributions of the climate parameters. Additionally we obtain estimates of the correlation structure of the unforced variability of temperature change patterns. These results are critical for understanding uncertainty in future climate change and provide an independent check that the information that is contained in recent climate change is robust to statistical treatment. These results include uncertainties in the estimation of the multivariate covariance matrices.

A Generalized Approach to Portfolio Optimization: Improving Performance by Constraining Portfolio Norms

We provide a general framework for finding portfolios that perform well out-of-sample in the presence of estimation error. This framework relies on solving the traditional minimum-variance problem but subject to the additional constraint that the norm of the portfolio-weight vector be smaller than a given threshold. We show that our framework nests as special cases the shrinkage approaches of Jagannathan and Ma (Jagannathan, R., T. Ma. 2003. Risk reduction in large portfolios: Why imposing the wrong constraints helps. J. Finance 58 1651–1684) and Ledoit and Wolf (Ledoit, O., M. Wolf. 2003. Improved estimation of the covariance matrix of stock returns with an application to portfolio selection. J. Empirical Finance 10 603–621, and Ledoit, O., M. Wolf. 2004. A well-conditioned estimator for large-dimensional covariance matrices. J. Multivariate Anal. 88 365–411) and the 1/N portfolio studied in DeMiguel et al. (DeMiguel, V., L. Garlappi, R. Uppal. 2009. Optimal versus naive diversification: How inefficient is the 1/N portfolio strategy? Rev. Financial Stud. 22 1915–1953). We also use our framework to propose several new portfolio strategies. For the proposed portfolios, we provide a moment-shrinkage interpretation and a Bayesian interpretation where the investor has a prior belief on portfolio weights rather than on moments of asset returns. Finally, we compare empirically the out-of-sample performance of the new portfolios we propose to 10 strategies in the literature across five data sets. We find that the norm-constrained portfolios often have a higher Sharpe ratio than the portfolio strategies in Jagannathan and Ma (2003), Ledoit and Wolf (2003, 2004), the 1/N portfolio, and other strategies in the literature, such as factor portfolios.

Derivative-free estimation methods: New results and performance analysis

The derivative-free nonlinear estimation methods exploiting the Stirling’s interpolation and the unscented transformation for discrete-time nonlinear stochastic systems are treated. The divided difference and unscented filters, smoothers, and predictors based on the methods are introduced in the unified framework. The new relations among the first order Stirling’s interpolation, the second order Stirling’s interpolation, and the unscented transformation are derived and their impact on the covariance matrices of the state estimates of the corresponding filters is analysed. The theoretical results are illustrated and used for the explanation of the unexpected behaviour of the sigma point Gaussian sum filters given as a mixture of the derivative-free filters.

Large System Spectral Analysis of Covariance Matrix Estimation

Eigendecomposition of estimated covariance matrices is a basic signal processing technique arising in a number of applications, including direction-of-arrival estimation, power allocation in multiple-input/multiple-output (MIMO) transmission systems, and adaptive multiuser detection. This paper uses the theory of non-crossing partitions to develop explicit asymptotic expressions for the moments of the eigenvalues of estimated covariance matrices, in the large system asymptote as the vector dimension and the dimension of signal space both increase without bound, while their ratio remains finite and nonzero. The asymptotic eigenvalue distribution is also obtained from these eigenvalue moments and the Stieltjes transform, and is extended to first-order approximation in the large sample-size limit. Numerical simulations are used to demonstrate that these asymptotic results provide good approximations for finite systems of moderate size.

Methods for Estimating State and Measurement Noise Covariance Matrices: Aspects and Comparison

The paper deals with estimation of noise covariance matrices in state and measurement equations of linear discrete-time stochastic dynamic systems. In the last decade several novel methods for noise covariance matrices estimation, which are based on state estimation techniques, have been proposed. Unfortunately, the novel methods have been compared mainly with classical methods proposed in the seventies only. The aim of the paper is to analyse identifiability of state noise parameters by means of the Bayesian approach and to summarise and compare the novel methods from both theoretical and numerical point of view.

An adaptive estimation of forecast error covariance parameters for Kalman filtering data assimilation

An adaptive estimation of forecast error covariance matrices is proposed for Kalman filtering data assimilation. A forecast error covariance matrix is initially estimated using an ensemble of perturbation forecasts. This initially estimated matrix is then adjusted with scale parameters that are adaptively estimated by minimizing −2log-likelihood of observed-minus-forecast residuals. The proposed approach could be applied to Kalman filtering data assimilation with imperfect models when the model error statistics are not known. A simple nonlinear model (Burgers’ equation model) is used to demonstrate the efficacy of the proposed approach.

Covariance regularization by thresholding

This paper considers regularizing a covariance matrix of p variables estimated from n observations, by hard thresholding. We show that the thresholded estimate is consistent in the operator norm as long as the true covariance matrix is sparse in a suitable sense, the variables are Gaussian or sub-Gaussian, and (log p)/n → 0, and obtain explicit rates. The results are uniform over families of covariance matrices which satisfy a fairly natural notion of sparsity. We discuss an intuitive resampling scheme for threshold selection and prove a general cross-validation result that justifies this approach. We also compare thresholding to other covariance estimators in simulations and on an example from climate data. 1. Introduction. Estimation of covariance matrices is important in a number of areas of statistical analysis, including dimension reduction by principal component analysis (PCA), classification by linear or quadratic discriminant analysis (LDA and QDA), establishing independence and conditional independence relations in the context of graphical models, and setting confidence intervals on linear functions of the means of the components. In recent years, many application areas where these tools are used have been dealing with very high-dimensional datasets, and sample sizes can be very small relative to dimension. Examples include genetic data, brain imaging, spectroscopic imaging, climate data and many others. It is well known by now that the empirical covariance matrix for samples of size n from a p-variate Gaussian distribution, Np(μ, � p), is not a good estimator of the population covariance if p is large. Many results in random matrix theory illustrate this, from the classical Mary law [29] to the more recent work of Johnstone and his students on the theory of the largest eigenvalues [12, 23, 30] and associated eigenvectors [24]. However, with the exception of a method for estimating the covariance spectrum [11], these probabilistic results do not offer alternatives to the sample covariance matrix. Alternative estimators for large covariance matrices have therefore attracted a lot of attention recently. Two broad classes of covariance estimators have emerged: those that rely on a natural ordering among variables, and assume that variables

A CLT for regularized sample covariance matrices

We consider the spectral properties of a class of regularized estimators of (large) empirical covariance matrices corresponding to stationary (but not necessarily Gaussian) sequences, obtained by banding. We prove a law of large numbers (similar to that proved in the Gaussian case by Bickel and Levina), which implies that the spectrum of a banded empirical covariance matrix is an efficient estimator. Our main result is a central limit theorem in the same regime, which to our knowledge is new, even in the Gaussian setup.

Evaluation of three inverse problem models to quantify skin microcirculation using diffusion-weighted MRI

Skin microcirculation plays an important role in diseases such as chronic venous insufficiency and diabetes. Magnetic resonance imaging (MRI) can provide quantitative information with a better penetration depth than other noninvasive methods, such as laser Doppler flowmetry or optical coherence tomography. Moreover, successful MRI skin studies have recently been reported. In this article, we investigate three potential inverse models to quantify skin microcirculation using diffusion-weighted MRI (DWI), also known as q-space MRI. The model parameters are estimated based on nonlinear least-squares (NLS). For each of the three models, an optimal DWI sampling scheme is proposed based on D-optimality in order to minimize the size of the confidence region of the NLS estimates and thus the effect of the experimental noise inherent to DWI. The resulting covariance matrices of the NLS estimates are predicted by asymptotic normality and compared to the ones computed by Monte-Carlo simulations. Our numerical results demonstrate the effectiveness of the proposed models and corresponding DWI sampling schemes as compared to conventional approaches.

Perils of Parsimony: Properties of Reduced-Rank Estimates of Genetic Covariance Matrices

Eigenvalues and eigenvectors of covariance matrices are important statistics for multivariate problems in many applications, including quantitative genetics. Estimates of these quantities are subject to different types of bias. This article reviews and extends the existing theory on these biases, considering a balanced one-way classification and restricted maximum-likelihood estimation. Biases are due to the spread of sample roots and arise from ignoring selected principal components when imposing constraints on the parameter space, to ensure positive semidefinite estimates or to estimate covariance matrices of chosen, reduced rank. In addition, it is shown that reduced-rank estimators that consider only the leading eigenvalues and -vectors of the "between-group" covariance matrix may be biased due to selecting the wrong subset of principal components. In a genetic context, with groups representing families, this bias is inverse proportional to the degree of genetic relationship among family members, but is independent of sample size. Theoretical results are supplemented by a simulation study, demonstrating close agreement between predicted and observed bias for large samples. It is emphasized that the rank of the genetic covariance matrix should be chosen sufficiently large to accommodate all important genetic principal components, even though, paradoxically, this may require including a number of components with negligible eigenvalues. A strategy for rank selection in practical analyses is outlined.

Estimation error in the average correlation of security returns and shrinkage estimation of covariance and correlation matrices

The correlation matrix of security returns is an important input component for mean–variance portfolio analysis. This study uses the average of sample correlations to estimate the correlation matrix and derives an expression of its estimation error in terms of sampling variance. This study then considers the impact of such estimation error on shrinkage estimation, where a weighted average is sought between the sample covariance matrix and an average correlation target, and between the sample correlation matrix and the target. An illustrative example using monthly returns of the current Dow Jones stocks is provided.

Bounds for Estimation of Covariance Matrices From Heterogeneous Samples

This correspondence derives lower bounds on the mean-square error (MSE) for the estimation of a covariance matrix \mbi Mp, using samples \mbi Zk,k=1,...,K, whose covariance matrices \mbi Mk are randomly distributed around \mbi Mp. This framework can be encountered e.g., in a radar system operating in a nonhomogeneous environment, when it is desired to estimate the covariance matrix of a range cell under test, using training samples from adjacent cells, and the noise is nonhomogeneous between the cells. We consider two different assumptions for \mbi Mp. First, we assume that \mbi Mp is a deterministic and unknown matrix, and we derive the Cramer-Rao bound for its estimation. In a second step, we assume that \mbi Mp is a random matrix, with some prior distribution, and we derive the Bayesian bound under this hypothesis.

Sparse estimation of large covariance matrices via a nested Lasso penalty

The paper proposes a new covariance estimator for large covariance matrices when the variables have a natural ordering. Using the Cholesky decomposition of the inverse, we impose a banded structure on the Cholesky factor, and select the bandwidth adaptively for each row of the Cholesky factor, using a novel penalty we call nested Lasso. This structure has more flexibility than regular banding, but, unlike regular Lasso applied to the entries of the Cholesky factor, results in a sparse estimator for the inverse of the covariance matrix. An iterative algorithm for solving the optimization problem is developed. The estimator is compared to a number of other covariance estimators and is shown to do best, both in simulations and on a real data example. Simulations show that the margin by which the estimator outperforms its competitors tends to increase with dimension.