Discrete-time Finite Markov Chains Research Articles

Recursive filters for partially observable finite Markov chains

In this note, we consider discrete-time finite Markov chains and assume that they are only partly observed. We obtain finite-dimensional normalized filters for basic statistics associated with such processes. Recursive equations for these filters are derived by means of simple computations involving conditional expectations. An application to the estimation of parameters of the so-called discrete-time batch Markovian arrival process is outlined.

Recursive filters for partially observable finite Markov chains

In this note, we consider discrete-time finite Markov chains and assume that they are only partly observed. We obtain finite-dimensional normalized filters for basic statistics associated with such processes. Recursive equations for these filters are derived by means of simple computations involving conditional expectations. An application to the estimation of parameters of the so-called discrete-time batch Markovian arrival process is outlined.

Likelihood ratio test for testing order of continuous time finite markov chains

Anderson and Goodman ( 1957) have obtained the likelihood ratio tests and chi-square tests for testing the hypothesis about the order of discrete time finite Markov chains, On the similar lines we have obtained likeli¬hood ratio tests and chi-square tests (asymptotic) for testing hypotheses about the order of continuous time Markov chains (MC) with finite state space.

On Quasi-Stationary distributions in absorbing discrete-time finite Markov chains

The time to absorption from the set T of transient states of a Markov chain may be sufficiently long for the probability distribution over T to settle down in some sense to a “quasi-stationary” distribution. Various analogues of the stationary distribution of an irreducible chain are suggested and compared. The reverse process of an absorbing chain is found to be relevant.

On Quasi-Stationary distributions in absorbing discrete-time finite Markov chains

The time to absorption from the setTof transient states of a Markov chain may be sufficiently long for the probability distribution overTto settle down in some sense to a “quasi-stationary” distribution. Various analogues of the stationary distribution of an irreducible chain are suggested and compared. The reverse process of an absorbing chain is found to be relevant.