Efficient Estimation of Covariance Matrices using Posterior Mode Multiple Shrinkage

Abstract

We propose an approach to the regularization of covariance matrices that can be applied to any model for which the likelihood is available in closed form. The approach is based on using mixtures of double exponential or normal distributions as priors for correlation parameters, and on maximizing the resulting log-posterior (or penalized likelihood) using a stochastic optimization algorithm. The mixture priors are capable of clustering the correlations in several groups, each with separate mean and variance, and can therefore capture a large variety of structures besides sparsity. We apply this approach to the normal linear multivariate regression model as well as several other models that are popular in the literature but have not been previously studied for the purpose of regularization, including multivariate t, normal and t copulas, and mixture of normal distributions. Simulation experiments show the potential for large efficiency gains in estimating the density of the observations in all these models. Sizable gains are also obtained in four real applications. Copyright The Author, 2012. Published by Oxford University Press. All rights reserved. For Permissions, please email: journals.permissions@oxfordjournals.org, Oxford University Press.