Stationary Averaging for Multiscale Continuous Time Markov Chains Using Parallel Replica Dynamics

Abstract

We propose two algorithms for simulating continuous time Markov chains in the presence of metastability. We show that the algorithms correctly estimate, under the ergodicity assumption, stationary averages of the process. Both algorithms, based on the idea of the parallel replica method, use parallel computing in order to explore metastable sets more efficiently. The algorithms require no assumptions on the Markov chains beyond ergodicity and the presence of identifiable metastability. In particular, there is no assumption on reversibility. For simpler illustration of the algorithms, we assume that a synchronous architecture is used throughout the paper. We present error analyses, as well as numerical simulations on multiscale stochastic reaction network models in order to demonstrate consistency of the method and its efficiency.