Bootstrapping Markov chains: countable case

Abstract

Let X be an irreducible, aperiodic and positive recurrent Markov chain with a countably infinite state space S and transition probability matrix P. Let π be the stationary probability and Tk be the first hitting time of a state k. Given a realization {xj;0⩽j⩽n} of {Xj;0⩽j⩽n}, let P̂n be the maximum likelihood estimate of P. In this paper, the distribution of the naive bootstrap of the pivot √n(P̂n–P) is shown to appropriate that of the pivot as n↦∞. The approach used is via a double array of Markov chains for which a weak law and a central limit theorem are established. Next, in order to estimate analogous quantities for the stationary probability and hitting time distribution, two different bootstrap methods are discussed.