On the Numerical Analysis of Inhomogeneous Continuous-Time Markov Chains

Abstract

Inhomogeneous continuous-time Markov chains play an important role in different application areas. In contrast to homogeneous continuous-time Markov chains, where a large number of numerical analysis techniques are available and have been compared, few results about the performance of numerical techniques in the inhomogeneous case are known. This paper presents a new variant of the uniformization technique, the most efficient approach for homogeneous Markov chains. The new uniformization technique allows for the stable computation of strict bounds for the transient distribution of inhomogeneous continuous-time Markov chains, which is not possible with other numerical techniques that provide only an approximation of the distribution and asymptotic bounds. Furthermore, another variant of uniformization is presented that computes an approximation of the transient distribution and is shown to outperform standard differential equation solvers if transition rates change slowly.