The performance measure analysis on the states classification in Markov Chain

Abstract

The system being modelled is assumed to occupy one and only one state at any moment in time and its evolution is represented by transitions from state to state. Also, the physical or mathematical behaviour of this system may be represented by describing all the different states it may occupy and by indicating how it moves among these states. In this work, the concept of the classification of groups of states, between states that are recurrent, meaning that the Markov chain is guaranteed to return to these states infinitely often, and states that are transient, meaning that there is a nonzero probability that the Markov chain will never return to such a state are investigated, in order to provide some insight into the performance measure analysis such as the mean first passage time, 𝑅𝑖𝑗, the mean recurrence time of state 𝑅𝑗𝑗 as well as recurrence iterative matrix 𝑅(𝑘+1). Our quest is to demonstrate with illustrative examples on Markov chains with different classes of states, and the following results are obtained, the mean recurrence time of state 1 is infinite, as well as the mean first passage times from states 2 and 3 to state 1. The mean first passage time from state 2 to state 3 or vice versa is given as 1, while the mean recurrence time of both state 2 and state 3 is given as 2.