Laplace Importance Sampling for Generalized Linear Mixed Models

Abstract

It is well known that the standard Laplace approximation of the integrated marginal likelihood function of a random effects model may be invalid if the dimension of the integral increases with the sample size and the resulting parameter estimates, especially those of the variance components, are biased towards zero. Bias-correction factors have been proposed in the literature but they are asymptotically correct only for the case of small variance components. Techniques for modifying the standard Laplace expansion have also been proposed but they are highly technical and problem-specific and hence unsuitable for routine use. Monte Carlo approximations of the marginal likelihood function typically make use of Markov Chain Monte Carlo (MCMC) sampling the convergence of which is difficult to check. We propose an importance sampling method where the the importance function is chosen with the aid of Laplace expansion. Since it is only used to suggest an appropriate importance function, the accuracy of Laplace expansion affects only the efficiency but not the unbiasedness of the resulting likelihood approximation. Moreover, the proposed method can be implemented using independent rather than MCMC sampling and so it is easier to check convergence and to assess variability. One set of simulations suffices for approximating the entire likelihood function but we do recommend iterating the importance sampling procedure a few times to obtain progressively better estimates of the likelihood function near its maximum. When applied to fit a logistic model with crossed random effects to the well known salamander mating data, the proposed method yields parameter estimates very comparable to those obtained using modified Laplace approximations. The proposed procedure also produces estimates very close to the “true” maximum likelihood estimates when applied to a set of binary data simulated from a crossed random effects model.