Computing approximate stationary distributions for discrete Markov processes with banded infinitesimal generators

Abstract

We develop an algorithm for computing approximations to the stationary distribution of a discrete birth-and-death process, provided that the infinitesimal generator is a banded matrix. We begin by computing stationary distributions for processes whose infinitesimal generators are Hessenberg. Our derivation in this special case is different from the classical case but it leads to the same result. We then show how to extend these ideas to processes where the infinitesimal generator is banded (or half-banded) and to quasi-birth–death processes. Finally, we give an example of the application of this method to a nearly completely decomposable Markov chain to demonstrate the general applicability of the technique.