A sequential direct hybrid algorithm to compute stationary distribution of continuous-time Markov chain

Abstract

In modeling a wide variety of problems of the real world using Markov processes, we usually get large sparse Markov chains. In the case of non-Markovian process, the behavior of a stochastic phenomenon or system can be approximated by continuous-time Markov chains (CTMC) using Phase-type distributions. However, this substantially increases the number of states, which requires a large amount of computation time and operational memory resources. Even the most recent computer hardware is not enough to provide a solution in almost real time, unless a specific algorithm is designed.In this paper, a new hybrid algorithm for the computation of a stationary distribution of a large ergodic homogeneous Markov chains with a finite state space and continuous time is suggested. This algorithm would solve the aforementioned challenges. Depending on model size and sparsity, it significantly reduces a computational time when compared with other recent methods, such as the SparseLU solver from Eigen library (version 3.9.9) and the MUMPS solver (MUltifrontal Massively Parallel sparse direct Solver, version 5.4.0). It is important to note that the algorithm works well independently on the structure of the generator matrix of a continuous-time Markov chain. For the study of the developed hybrid algorithm, the generator matrix is constructed randomly using a special algorithm, based on combining two methods: the slightly modified Grassmann, Taskar and Heyman (GTH) algorithm and the Gauss elimination method. The new idea is to remove the majority of states in a near-optimal order by the GTH method and complete computations for the remaining states by the Gauss method. The robust approach to identify a switching position from one algorithm to another one has been proposed and tested.