Abstract

With modern high-dimensional data, complex statistical models are necessary, requiring computationally feasible inference schemes. We introduce Max-and-Smooth, an approximate Bayesian inference scheme for a flexible class of latent Gaussian models (LGMs) where one or more of the likelihood parameters are modeled by latent additive Gaussian processes. Our proposed inference scheme is a two-step approach. In the first step (Max), the likelihood function is approximated by a Gaussian density with mean and covariance equal to either (a) the maximum likelihood estimate and the inverse observed information, respectively, or (b) the mean and covariance of the normalized likelihood function. In the second step (Smooth), the latent parameters and hyperparameters are inferred and smoothed with the approximated likelihood function. The proposed method ensures that the uncertainty from the first step is correctly propagated to the second step. Because the prior density for the latent parameters is assumed to be Gaussian and the approximated likelihood function is Gaussian, the approximate posterior density of the latent parameters (conditional on the hyperparameters) is also Gaussian, thus facilitating efficient posterior inference in high dimensions. Furthermore, the approximate marginal posterior distribution of the hyperparameters is tractable, and as a result, the hyperparameters can be sampled independently of the latent parameters. We show that the computational cost of Max-and-Smooth is close to being insensitive to the number of independent data replicates, and that it scales well with increased dimension of the latent parameter vector provided that its Gaussian prior density is specified with a sparse precision matrix. In the case of a large number of independent data replicates, sparse precision matrices, and high-dimensional latent vectors, the speedup is substantial in comparison to an MCMC scheme that infers the posterior density from the exact likelihood function. The accuracy of the Gaussian approximation to the likelihood function increases with the number of data replicates per latent model parameter. The proposed inference scheme is demonstrated on one spatially referenced real dataset and on simulated data mimicking spatial, temporal, and spatio-temporal inference problems. Our results show that Max-and-Smooth is accurate and fast.

Highlights

  • Data are being generated today at an unprecedented rate

  • In this paper we focus on latent Gaussian models (LGMs), which form a general and very flexible class of models that has proven to be useful in a wide range of concrete applications

  • The models presented here have the same structure as the LGMs in Section 2.1, except that the assumption of the data density being in the exponential family is dropped and the vector x refers to several subsets of parameters found at the data level, each subset with its separate set of linear predictors at the latent level

Read more

Summary

Introduction

Data are being generated today at an unprecedented rate. Many datasets are large and exhibit complex marginal behaviors and dependence structures. In order to make MCMC sampling more efficient, Knorr-Held and Rue (2002) proposed a single block updating strategy for LGMs characterized by a univariate link function Their strategy reduces the cross-correlation between the hyperparameters and the latent parameters within the posterior samples. The two steps of the inference scheme are as follows: (i) In the first step (Max), we compute the maximum likelihood (ML) estimates of the latent parameters at each spatial, temporal, or spatio-temporal point (depending on the type of LGM considered), and we approximate the covariance of the Gaussian approximation using the inverse observed information evaluated at the ML estimate.

LGMs with a univariate link function
LGMs with a multivariate link function
General idea
The posterior density of extended LGMs
Max-and-Smooth: A two-step approximate inference approach
Settings
Gaussian data with spatially-varying log-variance
Predictions of meteorological variables on a lattice
Findings
Discussion

Talk to us

Join us for a 30 min session where you can share your feedback and ask us any queries you have

Schedule a call

Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.