Covariance structure estimation with Laplace approximation

Abstract

The Gaussian covariance graph model is popular for revealing the underlying dependency structures among random variables. In this paper, we consider a spike and slab prior, which is a mixture of point-mass and normal distribution, on the off-diagonal entries. The spike and slab prior naturally introduces sparsity to the covariance structure so that the resulting posterior renders covariance structure learning. Under the spike and slab prior, we calculate the posterior model probabilities of covariance structures and natural Bayesian quantities for model selection using the Laplace approximation. We show that the error due to the Laplace approximation becomes asymptotically marginal at a rate that depends on the posterior convergence rate of the covariance matrix under the Frobenius norm. We propose a covariance structure estimation method based on the approximated posterior model probabilities. We also propose a block coordinate descent algorithm to determine the mode of the posterior density conditional on the structure of the covariance. The posterior mode is an estimate of the covariance matrix once the structure is chosen and the Laplace approximation is computed around it. Through a simulation study based on five numerical models, we demonstrate that the proposed method outperforms its competitors. The proposed method is applied to the breast cancer and Parkinson’s disease datasets, as well as the prediction of telephone call counts using telephone call center data, and compared with its competitors in terms of the linear discriminant analysis classification accuracy.