Using the Effective Sample Size as the Stopping Criterion in Markov Chain Monte Carlo with the Bayes Module in Mplus

Steffen Zitzmann,Sebastian Weirich,Martin Hecht

doi:10.3390/psych3030025

Steffen Zitzmann, Sebastian Weirich + Show 1 more

Open Access

https://doi.org/10.3390/psych3030025

Copy DOI

Journal: Psych	Publication Date: Jul 30, 2021
Citations: 10	License type: CC BY 4.0

Affiliation: University of Tübingen, Humboldt-Universität zu Berlin

Abstract

Bayesian modeling using Markov chain Monte Carlo (MCMC) estimation requires researchers to decide not only whether estimation has converged but also whether the Bayesian estimates are well-approximated by summary statistics from the chain. On the contrary, software such as the Bayes module in Mplus, which helps researchers check whether convergence has been achieved by comparing the potential scale reduction (PSR) with a prespecified maximum PSR, the size of the MCMC error or, equivalently, the effective sample size (ESS), is not monitored. Zitzmann and Hecht (2019) proposed a method that can be used to check whether a minimum ESS has been reached in Mplus. In this article, we evaluated this method with a computer simulation. Specifically, we fit a multilevel structural equation model to a large number of simulated data sets and compared different prespecified minimum ESS values with the actual (empirical) ESS values. The empirical values were approximately equal to or larger than the prespecified minimum ones, thus indicating the validity of the method.

Highlights

Bayesian modeling has gained popularity in psychology [1] and related sciences [2]because Bayesian modeling can be beneficial in several respects; for example, it offers greater flexibility in modeling (e.g., [3]), fewer estimation problems (e.g., [4]), and more accurate estimates, (e.g., [5]), when models with latent variables are estimated.Most software uses Markov chain Monte Carlo (MCMC) methods, which generate samples from the distribution of the population parameters given the data
Zitzmann and Hecht [10] proposed a method that can be used to control the effective sample size (ESS) in Mplus, taking advantage of a correspondence between the ESS and the potential scale reduction (PSR), which exists when both coefficients are computed with the help of estimates from the analysis of variance (ANOVA) literature that are based on the mean squares
We begin with an inspection of the correspondence between the empirical and the prespecified minimum ESS, followed by an inspection of how the statistical properties vary depending on the prespecified minimum ESS

Summary

Introduction

Bayesian modeling has gained popularity in psychology [1] and related sciences [2]because Bayesian modeling can be beneficial in several respects; for example, it offers greater flexibility in modeling (e.g., [3]), fewer estimation problems (e.g., [4]), and more accurate estimates, (e.g., [5]), when models with latent variables are estimated.Most software uses Markov chain Monte Carlo (MCMC) methods, which generate samples from the distribution of the population parameters given the data (i.e., the posterior distribution). To ensure a high level of precision, the MCMC error needs to be small (e.g., [7]). Software such as the Bayes module in Mplus [8] routinely monitors convergence by checking whether a certain value for the potential scale reduction (PSR; [9]) has been reached. This value can be specified by the user, whereas a value for the amount of the MCMC error or, equivalently, for the effective sample size (ESS), cannot be specified directly. By “control”, we mean that users can specify a minimum ESS and a minimum level of precision as the stopping criterion before the estimation is carried out

Objectives

Methods

Results

Conclusion