A comparison of Bayesian and likelihood-based methods for fitting multilevel models

Abstract

We use simulation studies, whose design is realistic for educational andmedicalresearch(aswellasotherfleldsofinquiry),tocompareBayesianand likelihood-basedmethodsforflttingvariance-components(VC)andrandom-efiects logistic regression (RELR) models. The likelihood (and approximate likelihood) approachesweexaminearebasedonthemethodsmostwidelyusedincurrentap- plied multilevel (hierarchical) analyses: maximum likelihood (ML) and restricted ML(REML)forGaussianoutcomes,andmarginalandpenalizedquasi-likelihood (MQL and PQL) for Bernoulli outcomes. Our Bayesian methods use Markov chain Monte Carlo (MCMC) estimation, with adaptive hybrid Metropolis-Gibbs sampling for RELR models, and several difiuse prior distributions (i i1 (†;†) and U(0; 1 ) priors for variance components). For evaluation criteria we consider bias of point estimates and nominal versus actual coverage of interval estimates in re- peated sampling. In two-level VC models we flnd that (a) both likelihood-based and Bayesian approaches can be made to produce approximately unbiased esti- mates, although the automatic manner in which REML accomplishes this is an advantage, but (b) both approaches had di-culty achieving nominal coverage in smallsamplesandwithsmallvaluesoftheintraclasscorrelation. Withthethree- levelRELRmodelsweexamineweflndthat(c)quasi-likelihoodmethodsforesti- mating random-efiects variances perform badly with respect to bias and coverage intheexamplewesimulated,and(d)Bayesiandifiuse-priormethodsleadtowell- calibratedpointandintervalRELRestimates. Whileitistruethatthelikelihood- based methods we study are considerably faster computationally than MCMC, (i) steady improvements in recent years in both hardware speed and e-ciency of MonteCarloalgorithmsand(ii)thelackofcalibrationoflikelihood-basedmethods insomecommonhierarchicalsettingscombinetomakeMCMC-basedBayesianflt- tingofmultilevelmodelsanattractiveapproach,evenwithratherlargedatasets. Other analytic strategies based on less approximate likelihood methods are also possible butwouldbeneflt fromfurtherstudy ofthe type summarized here.