Mean Passage Times and Nearly Uncoupled Markov Chains

Abstract

Let $P ( 0 ) \in R^{n \times n} $ be a stochastic matrix representing transition probabilities in a Markov Chain. Also, for a matrix $A \in R^{n \times n} $ whose row-sums are zero, let $P( \varepsilon ) \equiv P ( 0 ) + \varepsilon A$ be stochastic and irreducible for all $0 < \varepsilon \leq \varepsilon _{\max } $, for some $\varepsilon_{\max } $. Finally, let $M( \varepsilon )$ be a matrix whose $( i, j )$ entry is the mean passage time from state i to state j when transitions are governed by $P( \varepsilon )$. When the Markov chain associated with $P( 0 )$ is decomposable into a number of independent chains plus a set of transient states, some of the entries of $M( \varepsilon )$ have singularities at zero. The orders of these poles define timescales associated with the process when $\varepsilon $ is small. An algorithm is developed for computing these orders. The only input required is the supports of $P( 0 )$ and A, making the problem a combinatorial one. Finally, it is shown how the orders of the poles of $M( \varepsilon )$ at zero play a role in developing series expansions for $\pi ( \varepsilon )$, the stationary distribution of $P( \varepsilon )$.