Asymptotic expansions of backward equations for two-time-scale Markov chains in continuous time

Abstract

This work develops asymptotic expansions for solutions of systems of backward equations of time-inhomogeneous Markov chains in continuous time. Owing to the rapid progress in technology and the increasing complexity in modeling, the underlying Markov chains often have large state spaces, which make the computational tasks infeasible. To reduce the complexity, two-time-scale formulations are used. By introducing a small parameter ɛ > 0 and using suitable decomposition and aggregation procedures, it is formulated as a singular perturbation problem. Both Markov chains having recurrent states only and Markov chains including also transient states are treated. Under certain weak irreducibility and smoothness conditions of the generators, the desired asymptotic expansions are constructed. Then error bounds are obtained.