On the periodicity of non-homogeneous Markov chains and systems

Abstract

We firstly recall the concept of an NHMS and introduce concepts and known results necessary for the study of asymptotic periodicity of the vector of expectations, variances and covariances of the state sizes. This sequence of vectors is called the variability of population structures. We then proceed to prove some useful propositions and lemmas for the establishment of the basic results that follow. Note, that some of these lemmas have a power and usefulness of their own. Finally, we provide two theorems, where we study the asymptotic periodicity of the variability of population structures, when the inherent non-homogeneous Markov chain is uniquely determined by a sequence of arbitrary stochastic matrices all with the same incidence matrix. Following that an illustrative example from the area of manpower systems is provided. In this example the results in the basic theorems of the previous section are illustrated numerically. We conclude with an appendix, where we firstly introduce the necessary concepts and known results, needed in order to study the periodicity and asymptotic periodicity of an infinite product of finite arbitrary stochastic matrices. We then proceed to study the periodicity and asymptotic periodicity of non-homogeneous Markov chains in their full possible generality. It is very important to note that similar results hold also for an infinite product of non-negative matrices.