On the Markov renewal theorem

Abstract

Let ( S, £) be a measurable space with countably generated σ-field £ and ( M n , X n ) n ⩾0 a Markov chain with state space S × R and transition kernel P : S × ( L ⊗ B )→[0, 1]. Then ( M n , S n ) n⩾0 , where S n = X 0+⋯+ X n for n⩾0, is called the associated Markov random walk. Markov renewal theory deals with the asymptotic behavior of suitable functionals of ( M n , S n ) n⩾0 like the Markov renewal measure Σ n⩾0 P(( M n , S n )ϵ Ax ( t+ B)) as t→∞ where Aϵ L and B denotes a Borel subset of R . It is shown that the Markov renewal theorem as well as a related ergodic theorem for semi-Markov processes hold true if only Harris recurrence of ( M n ) n⩾0 is assumed. This was proved by purely analytical methods by Shurenkov [15] in the one-sided case where P ( x,S×[0,∞)) = 1 for all xϵ S. Our proof uses probabilistic arguments, notably the construction of regeneration epochs for ( M n ) n⩾0 such that ( M n , X n ) n⩾0 is at least nearly regenerative and an extension of Blackwell's renewal theorem to certain random walks with stationary, 1-dependent increments.