Markov chain Monte Carlo sampling using a reservoir method

Abstract

Markov chain Monte Carlo methods are widely used to draw a sample from a target distribution which is hard to characterize analytically, and reservoir sampling is developed to obtain a sample from a data stream sequentially in a single pass. A stochastic thinning algorithm using reservoir sampling is proposed, and it can be embedded in most Markov chain Monte Carlo methods to reduce the autocorrelation among the generated sample. The distribution of the sample generated by the proposed sampling algorithm converges in total variation to the target distribution in probability under mild conditions. A practical method is introduced to detect the convergence of the proposed sampling algorithm. Two simulation studies are conducted to compare the proposed sampling algorithm and the corresponding Markov chain Monte Carlo methods without thinning, and results show that the estimation bias of the proposed sampling algorithm is approximately the same as the corresponding Markov chain Monte Carlo method, but the proposed sampling algorithm has a smaller Monte Carlo variance. The proposed sampling algorithm saves computer memory in the sense that the storage of a small portion of the Markov chain is required in each iteration.