Balanced importance resampling for Markov chains

Abstract

In this paper, given a sequence of realizations of a discrete-time finite state Markov chain, we estimate both transition probabilities and invariant distributions using a bootstrap method. In its original form this technique requires independent and identically distributed samples. Therefore, since the sequence is “Markov”, it has to be adapted in order to fit into this framework. Block bootstrap is introduced to make this adaptation. In order to reduce computer time, balanced importance resampling is proposed, so that in the bootstrap procedure, resampling is not done uniformly , this distribution is modified in order to get variance reduction. Efficiency properties of this alternative distribution are shown, together with numerical data. A central limit theorem for multinomial random sums is also proved.