ON THE APPLICATION OF MATRIX SCALING AND POWERING METHODS OF SMALL STATE SPACES FOR SOLVING TRANSIENT DISTRIBUTION IN MARKOV CHAIN

Abstract

The iterative solution methods for transient distribution in Markov chain is the computation of state probability distributions at an arbitrary point of time, which in the case of a discrete-time Markov chain means, finding the distribution at some arbitrary time step denoted , a row vector whose component is the probability that the Markov chain is in state at time step . In this study, the solutions of transient distribution in Markov chain using matrix scaling and powering methods for small state spaces which produce a significantly more accurate response in less time for some types of situations and also, tries to get to the end result as quickly as possible has been investigated, in order to provide some insight into the solutions of transient distribution of Markov chain. Our goal is to compute solutions and algorithms for tiny state spaces utilizing matrix scaling and powering approaches, which begin with an initial estimate of the solution vector and then comes closer and closer to the true solution with each step or iteration. With the help of several existing Markov chain laws, theorems, and formulas, matrices operations such as multiplication with one or more vectors, Padé variant of the matrix-powering and scaling technique are used. While the algorithms are explained, the transient distribution vector’s Padé approximants , and a backward error analysis of the Padé approximation are obtained for certain illustrative examples..