Multilevel Solutions for Structured Markov Chains

Abstract

In this paper, a new analysis approach for continuous time Markov chains (CTMCs) with a multidimensional structure is introduced. The presented solution technique employs the multidimensional structure to define aggregated CTMCs that can be analyzed more efficiently. Generator matrices of aggregated CTMCs are described in a compact form by exploiting the Kronecker structure of the generator matrix of a structured CTMC. The solution of aggregated systems is used to improve the solution of the original system. This idea of a multilevel solution is motivated by multigrid methods, which are efficient solvers for partial differential equations. The technique can be combined with different iterative solution techniques. It usually improves the convergence of these techniques significantly. Numerical results are given to illustrate that the new solution technique allows the fast and accurate analysis of very large CTMCs.