Multi-scale metastable dynamics and the asymptotic stationary distribution of perturbed Markov chains

Abstract

We assume that the transition matrix of a Markov chain depends on a parameter ε, and converges as ε→0. The chain is irreducible for ε>0 but may have several essential communicating classes when ε=0. This leads to metastable behavior, possibly on multiple time scales. For each of the relevant time scales, we derive two effective chains. The first one describes the (possibly irreversible) metastable dynamics, while the second one is reversible and describes metastable escape probabilities. Closed probabilistic expressions are given for the asymptotic transition probabilities of these chains. As a consequence, we obtain efficient algorithms for computing the committor function and the limiting stationary distribution.