Recursive Algorithm for the Fundamental/Group Inverse Matrix of a Markov Chain from an Explicit Formula

Abstract

We present a new accurate algorithm (REFUND) for computing the fundamental matrix (or closely related group inverse matrix) of a finite regular Markov chain. This algorithm is developed within the framework of the state reduction approach exemplified by the GTH (Grassmann, Taksar, Heyman)/S (Sheskin) algorithm for recursively finding invariant measure. The first (reduction) stage of the GTH/S algorithm is shared by REFUND, as well as by an earlier algorithm FUND developed for the fundamental matrix by Heyman in 1995, and by a modified version of Heyman and O'Leary in 1998. Unlike FUND, REFUND is recursive, being based on an explicit formula relating the group inverse matrix of an initial Markov chain and the group inverse matrix of a Markov chain with one state removed. Operation counts are approximately the same: $\Theta (\frac 73n^3)$ for REFUND versus $\Theta (\frac 83n^3)$ for FUND. Numerical tests indicate that REFUND is accurate. The structure of REFUND makes it easily combined with the other algorithms based on the state reduction approach. We also discuss the general properties of this approach, as well as connections to the optimal stopping problem and to tree decompositions of graphs related to Markov chains.