An introduction to the Markov chain Monte Carlo method

Abstract

We present an intuitive, conceptual, and semi-rigorous introduction to the Markov Chain Monte Carlo method using a simple model of population dynamics and focusing on a few elementary distributions. We start from two states, then three states, and finally generalize to many states with both discrete and continuous distributions. Despite the mathematical simplicity, our examples include the essential concepts of the Markov Chain Monte Carlo method, including ergodicity, global balance and detailed balance, proposal or selection probability, acceptance probability, the underlying stochastic matrix, and error analysis. Our experience suggests that most senior undergraduate students in physics can follow these materials without much difficulty.