On Direct Convergence and Periodicity for Transition Probabilities of Markov Chains in Random Environments

Abstract

We study direct convergence of the products $P(\theta_0) \cdots P(\theta_n)$ of random stochastic matrices. These products can be interpreted as the transition probabilities of nonhomogeneous Markov chains selected at random by a stationary "environmental" sequence $\{\theta_n\}$, in other words, a Markov chain in a random environment. Rather than make assumptions analogous to irreducibility and aperiodicity for homogeneous Markov chains, we introduce equivalence relations that allow convergence results on the equivalence classes. The classical decomposition into a cycle of periodic sets is not possible in general, so the "periodicity" in the title is meant only to be suggestive. We also examine the frequency of times of positive probability of return to a state or set.