Numerical instability of calculating inverse of spatial covariance matrices

Abstract

Computing an inverse of a covariance matrix is a common computational component in Statistics. For example, Gaussian likelihood function includes the inverse of a covariance matrix. For the computation of the inverse of a spatial covariance matrix, numerically unstable results can arise when the observation locations are getting denser. In this paper, we investigate when computational instability in calculating the inverse of a spatial covariance matrix makes maximum likelihood estimator unreasonable for a Matérn covariance model. Also, some possible approaches to relax such computational instability are discussed.