Importance resampling for markov chains

Abstract

Given a sequence of realizations of a discrete-time Markov chain, this paper aims at estimating both transition probabilities and invariant distribution. Bootstrap methods are proposed; this technique–in its original form–requires independent and identically distributed samples. Therefore, the sequence, being 'Markov', has to be adapted in order to fit into this framework. Two ways to achieve this (block bootstrap and nested bootstrap) are discussed. Importance resampling is proposed to reduce computer effort: do not resample (in the bootstrap procedure) uniformly from your sample, but modify this distribution in order to get variance reduction. Efficiency properties of this alternative distribution are shown, together with numerical evidence