On the Application of Successive Over-relaxation Algorithmic and Block Numerical Iterative Solutions for the Stationary Distribution in Markov Chain

Abstract

The evolution of a system is represented by transitions from one state to the next, and the system's physical or mathematical behavior can also be depicted by defining all of the numerous states it can be in and demonstrating how it moves between them. In this study, the iterative solution methods for the stationary distribution of Markov chains were investigated, which start with an initial estimate of the solution vector and then alter it in such a way that it gets closer and closer to the genuine solution with each step or iteration., and also involved matrices operations such as multiplication with one or more vectors, which leaves the transition matrices unchanged and saves time. Our goal is to use Successive Overrelaxation Algorithmic and Block Numerical Iterative Solution Methods to compute the solutions. With the help of some existing Markov chain laws, theorems, and formulas, the normalization principle and matric operations such as lower, upper, and diagonal matrices are used. The stationary distribution vector’s π^((k+1) )={■(π_1^((k+1) )&π_2^((k+1) )&■(π_3^((k+1) )&π_4^((k+1) )&π_5^((k+1) ) )),k=0,1,2,…,n} are obtained for the illustrative examples, taken the initial stationary solution to be π^((0) )=(■(0.2&0.2&■(0.2&0.2&0.2)))^T and it was observed that all subsequent iterations yield exactly the same result as π^((1) ), and this shows that, the block iterative method requires only a single iteration to obtain the solution to full machine precision.