Finite Nonhomogeneous Markov Chains Research Articles

Markov approximation and the generalized entropy ergodic theorem for non-null stationary process

In an earlier work, we proved a generalized entropy ergodic theorem for finite nonhomogeneous Markov chains (NMC). In this paper, we establish a generalized strong law of large numbers for finite m-th order NMC. Then we deduce a generalized entropy ergodic theorem for finite m-th order NMC, under some assumptions on the continuity rate and of non-nullness. Explicit upper and lower bounds relating the generalized relative entropy density of the original finite non-null stationary sequence and its canonical m-order Markov approximation is obtained.

THE GENERALIZED ENTROPY ERGODIC THEOREM FOR NONHOMOGENEOUS MARKOV CHAINS INDEXED BY A HOMOGENEOUS TREE

In this paper, we extend the strong laws of large numbers and entropy ergodic theorem for partial sums for tree-indexed nonhomogeneous Markov chains fields to delayed versions of nonhomogeneous Markov chains fields indexed by a homogeneous tree. At first we study a generalized strong limit theorem for nonhomogeneous Markov chains indexed by a homogeneous tree. Then we prove the generalized strong laws of large numbers and the generalized asymptotic equipartition property for delayed sums of finite nonhomogeneous Markov chains indexed by a homogeneous tree. As corollaries, we can get the similar results of some current literatures. In this paper, the problem settings may not allow to use Doob's martingale convergence theorem, and we overcome this difficulty by using Borel–Cantelli Lemma so that our proof technique also has some new elements compared with the reference Yang and Ye (2007).

Limit theorems for asymptotic circular mth-order Markov chains indexed by an m-rooted homogeneous tree

In this paper, we give the definition of an asymptotic circularmth-order Markov chain indexed by an m rooted homogeneous tree. By applying the limit property for a sequence of multi-variables functions of a nonhomogeneous Markov chain indexed by such tree, we estabish the strong law of large numbers and the asymptotic equipartition property (AEP) for asymptotic circular mth-order finite Markov chains indexed by this homogeneous tree. As a corollary, we can obtain the strong law of large numbers and AEP about the mth-order finite nonhomogeneous Markov chain indexed by the m rooted homogeneous tree.

THE STRONG LIMIT THEOREM FOR RELATIVE ENTROPY DENSITY RATES BETWEEN TWO ASYMPTOTICALLY CIRCULAR MARKOV CHAINS

In this paper, we are going to study the strong limit theorem for the relative entropy density rates between two finite asymptotically circular Markov chains. Firstly, we prove some lammas on which the main result based. Then, we establish two strong limit theorem for non-homogeneous Markov chains. Finally, we obtain the main result of this paper. As corollaries, we get the strong limit theorem for the relative entropy density rates between two finite non-homogeneous Markov chains. We also prove that the relative entropy density rates between two finite non-homogeneous Markov chains are uniformly integrable under some conditions.

The Asymptotic Equipartition Property for Nonhomogeneous Markov Chains Indexed by a Homogeneous Tree

In this correspondence, we first establish a strong limit theorem for countable nonhomogeneous Markov chains indexed by a homogeneous tree. As corollaries, we obtain some strong limit theorems for frequencies of occurrence of states and ordered couple of states for these Markov chains. Finally, we prove the strong law of large numbers and the asymptotic equipartition property (AEP) for finite nonhomogeneous Markov chains indexed by a homogeneous tree.

Hitting time in a finite non-homogeneous Markov chain with applications

SUMMARY This paper deals with the computation of the hitting time for a non-homogeneous discrete time Markov chain (NHDTMC or NHMC). We first give the basic definitions of NHMC, then we analyse the hitting time and its survivor function. We also give the suƒcient conditions for the existence of the mean hitting time. Finally, a numerical example and an application of this development in reliability evaluation are given in order to illustrate our results. ( 1998 John Wiley & Sons, Ltd.

Some strong limit theorems relative to the geometric average of random transition probabilities of arbitrary finite nonhomogeneous Markov chains

Let { X n , n ⩾ 0} be nonhomogeneous Markov chain with state space S = {1,2, …, m} and transition probabilities p n ( i, j). In this paper some strong limit theorems relative to the geometric average of the random transition probabilities { p k ( X k−1 , X k ), 1⩽ k ⩽ n} are obtained.

On a paper by Doeblin on non-homogeneous Markov chains

In [5] Doeblin considered some classes of finite non-homogeneous Markov chains and gave without proofs several results concerning their asymptotic behaviour. In the present paper we first attempt to make Doeblin's results precise and try to reconstruct his arguments. Subsequently we investigate more general situations, where a state space decomposition is provided by the sets occurring in the representation of the atomic sets of the tail σ-field. We show that Doeblin's notion of an associated chain, as well as considerations regarding the tail σ-field structure of the chain, can be used to solve such cases.

On a paper by Doeblin on non-homogeneous Markov chains

In [5] Doeblin considered some classes of finite non-homogeneous Markov chains and gave without proofs several results concerning their asymptotic behaviour. In the present paper we first attempt to make Doeblin's results precise and try to reconstruct his arguments. Subsequently we investigate more general situations, where a state space decomposition is provided by the sets occurring in the representation of the atomic sets of the tail σ-field. We show that Doeblin's notion of an associated chain, as well as considerations regarding the tail σ-field structure of the chain, can be used to solve such cases.

A ratio limit theorem for the finite nonhomogeneous Markov chains

A new simple proof concerning the structure of the tail σ-field of a nonhomogeneous Markov chain as well as a result regarding the asymptotic behaviour of the ratio between the transition probabilities and the absolute probabilities of the chain are given. It is shown that the latter result can be used to characterize the atomic sets of the tail σ-field.

The ergodic properties of non-homogeneous finite Markov chains

ABSTRACTThe paper deals with the behaviour of finite non-homogeneous Markov chains having regular transition matrices. Two types of ergodic behaviour are distinguished, and sufficient conditions are given for each type. Chains with commutative transition matrices are specially investigated. By means of expansions in terms of the characteristic roots several types of behaviour of chains which are not ergodic are studied.