Markov Chains In Random Environments Research Articles

On the Ergodicity of Certain Markov Chains in Random Environments

We study the ergodic behaviour of a discrete-time process X which is a Markov chain in a stationary random environment. The laws of X_t are shown to converge to a limiting law in (weighted) total variation distance as trightarrow infty . Convergence speed is estimated, and an ergodic theorem is established for functionals of X. Our hypotheses on X combine the standard “drift” and “small set” conditions for geometrically ergodic Markov chains with conditions on the growth rate of a certain “maximal process” of the random environment. We are able to cover a wide range of models that have heretofore been intractable. In particular, our results are pertinent to difference equations modulated by a stationary (Gaussian) process. Such equations arise in applications such as discretized stochastic volatility models of mathematical finance.

Strong mixing properties of discrete-valued time series with exogenous covariates

We derive strong mixing conditions for many existing discrete-valued time series models that include exogenous covariates in the dynamic. Our main contribution is to study how a mixing condition on the covariate process transfers to a mixing condition for the response. Using a coupling method, we first derive mixing conditions for some Markov chains in random environments, which gives a first result for some autoregressive categorical processes with strictly exogenous regressors. Our result is then extended to some infinite memory categorical processes. In the second part of the paper, we study autoregressive models for which the covariates are sequentially exogenous. Using a general random mapping approach on finite sets, we get explicit mixing conditions that can be checked for many categorical time series found in the literature, including multinomial autoregressive processes, ordinal time series and dynamic multiple choice models. We also study some autoregressive count time series using a somewhat different contraction argument. Our contribution fills an important gap for such models, presented here under a more general form, since such a strong mixing condition is often assumed in some recent works but no general approach is available to check it.

Tree-Indexed Markov Chains in Random Environment and Some of Their Strong Limit Properties

Tree-Indexed Markov Chains in Random Environment and Some of Their Strong Limit Properties

Invariant measures for multidimensional fractional stochastic volatility models

We establish convergence to an invariant measure as time tends to infinity, for a large class of (possibly non-Markovian) stochastic volatility models. Our arguments are based on a novel coupling idea for Markov chains which also extends to Markov chains in random environments in an efficient way.

Markov chains in random environment with applications in queuing theory and machine learning

We prove the existence of limiting distributions for a large class of Markov chains on a general state space in a random environment. We assume suitable versions of the standard drift and minorization conditions. In particular, the system dynamics should be contractive on the average with respect to the Lyapunov function and large enough small sets should exist with large enough minorization constants. We also establish that a law of large numbers holds for bounded functionals of the process. Applications to queuing systems, to machine learning algorithms and to autoregressive processes are presented.

A class of small deviation theorems for functionals of random fields on double Cayley tree in random environment

ABSTRACT In this paper, we introduce the definition of tree-indexed Markov chains in random environment with countable state space, and then study the realization of Markov chain indexed by a tree in random environment. Meanwhile, we establish a class of small deviation theorems for functional of random fields on double Cayley tree in random environment. Finally, We prove the asymptotic equipartition property (AEP) for tree indexed Markov chains in Markovian environment with finite state space.

Cutoff at the “entropic time” for sparse Markov chains

We study convergence to equilibrium for a large class of Markov chains in random environment. The chains are sparse in the sense that in every row of the transition matrix P the mass is essentially concentrated on few entries. Moreover, the random environment is such that rows of P are independent and such that the entries are exchangeable within each row. This includes various models of random walks on sparse random directed graphs. The models are generally non reversible and the equilibrium distribution is itself unknown. In this general setting we establish the cutoff phenomenon for the total variation distance to equilibrium, with mixing time given by the logarithm of the number of states times the inverse of the average row entropy of P. As an application, we consider the case where the rows of P are i.i.d. random vectors in the domain of attraction of a Poisson-Dirichlet law with index α ∈ (0, 1). Our main results are based on a detailed analysis of the weight of the trajectory followed by the walker. This approach offers an interpretation of cutoff as an instance of the concentration of measure phenomenon.

THE SHANNON–MCMILLAN THEOREM FOR MARKOV CHAINS INDEXED BY A CAYLEY TREE IN RANDOM ENVIRONMENT

In this paper, we give the definition of tree-indexed Markov chains in random environment with countable state space, and then study the realization of Markov chain indexed by a tree in random environment. Finally, we prove the strong law of large numbers and Shannon–McMillan theorem for Markov chains indexed by a Cayley tree in a Markovian environment with countable state space.

The definition of tree-indexed Markov chains in random environment and their existence

ABSTRACTThis paper gives the definition of tree-indexed Markov chains in random environment with discrete state space, and then studies some equivalent theorems of tree-indexed Markov chains in random environment. Finally, we give the equivalence on tree-indexed Markov chains in Markov environment and double Markov chains indexed by a tree.

Stability and approximation of invariant measures of Markov chains in random environments

We consider finite-state Markov chains driven by stationary ergodic invertible processes representing random environments. Our main result is that the invariant measures of Markov chains in random environments (MCREs) are stable under a wide variety of perturbations. We prove stability in the sense of convergence in probability of the invariant measure of the perturbed MCRE to the original invariant measure. We also develop a new numerical scheme to construct rigorous approximations of the invariant measures, which converge in probability as the resolution of the scheme increases. This numerical approach is illustrated with an example of a random walk in a random environment.

The stability of conditional Markov processes and Markov chains in random environments

We consider a discrete time hidden Markov model where the signal is a stationary Markov chain. When conditioned on the observations, the signal is a Markov chain in a random environment under the conditional measure. It is shown that this conditional signal is weakly ergodic when the signal is ergodic and the observations are nondegenerate. This permits a delicate exchange of the intersection and supremum of σ-fields, which is key for the stability of the nonlinear filter and partially resolves a long-standing gap in the proof of a result of Kunita [J. Multivariate Anal. 1 (1971) 365–393]. A similar result is obtained also in the continuous time setting. The proofs are based on an ergodic theorem for Markov chains in random environments in a general state space.

Aspects of large random Markov kernels

We briefly review certain asymptotic properties of random Markov kernels on a finite state space. These models can be thought of as finite Markov chains in random environment. Here, the asymptotics are taken with respect to the cardinality of the state space. We study, for instance, the behaviour of the normalized invariant vector, the global behaviour of the spectrum and the extremal eigenvalues. The analysis of these models involves Random Matrix Theory, convex compact polytopes and finite graphs with random weights. We also give some open problems related to these models.

The classification of states for Markov chain in random environments and m-irreducible

First of all, we introduces the concept of m-irreducible of Markov chain in random environment. Then, under the condition of m-irreducible, the relationship of recurrent and positive recurrent between two states is studied. We also give several conditions that can imply a state is recurrent and positive recurrent. And then the period of a state is discussed and we obtained that under the condition of m-irreducible, for any two states in X, they have the same period.

The Invariance Principle for p- $$ \ifmmode\expandafter\vec\else\expandafter\vecabove\fi{\theta } $$ Chain

There are two parts in this paper. In the first part we construct the Markov chain in random environment (MCRE), the skew product Markov chain and p–\( \ifmmode\expandafter\vec\else\expandafter\vecabove\fi{\theta } \) chain from a random transition matrix and a two–dimensional probability distribution, and in the second part we prove that the invariance principle for p–\( \ifmmode\expandafter\vec\else\expandafter\vecabove\fi{\theta } \) chain, a more complex non–homogeneous Markov chain, is true under some reasonable conditions. This result is more powerful.

THE DECOMPOSITION OF STATE SPACE FOR MARKOV CHAIN IN RANDOM ENVIRONMENT

This paper is a continuation of [8] and [9]. The author obtains the decomposition of state space X of an Markov chain in random environment by making use of the results in [8] and [9], gives three examples, random walk in random environment, renewal process in random environment and queue process in random environment, and obtains the decompositions of the state spaces of these three special examples.

Generalized product-form stationary distributions for Markov chains in random environments with queueing applications

Consider a continuous-time Markov chain evolving in a random environment. We study certain forms of interaction between the process of interest and the environmental process, under which the stationary joint distribution is tractable. Moreover, we obtain necessary and sufficient conditions for a product-form stationary distribution. A number of examples that illustrate the applicability of our results in queueing and population growth models are also included.

Generalized product-form stationary distributions for Markov chains in random environments with queueing applications

Consider a continuous-time Markov chain evolving in a random environment. We study certain forms of interaction between the process of interest and the environmental process, under which the stationary joint distribution is tractable. Moreover, we obtain necessary and sufficient conditions for a product-form stationary distribution. A number of examples that illustrate the applicability of our results in queueing and population growth models are also included.

Stationary Distributions of Discrete-Time Markov Chains in Random Environment: Exact Computations and Bounds

A general model for the description of a discrete-time Markov chain in random environment is introduced. We study the stationary distribution of this model and its relationship with the Palm (embedded) distributions at certain environmental transition epochs. We also study the Events See Time Averages (ESTA) property for this model and we show that it is related to certain partial balance equations and a product-form stationary distribution. Next, we study certain forms of interaction between the environmental process and the process of interest that lead to tractable stationary distributions. More specifically, the stationary distributions of a large class of Markov chains in random environment can be computed efficiently using α-potentials of smaller Markov chains. This decomposability result is also the basis for developing approximations for the stationary distributions of intractable models that evolve in a slowly changing random environment. Moreover, explicit error bounds for these approximations are given. The model can be used for the performance evaluation of discrete time queueing systems in random environment.

The relations among various Markov chains

Some basic equations and the relations among various Markov chains are established. These works are the bases in the investigation of the theory of Markov chain in random environment.

Recurrence and invariant measure of Markov chains in double-infinite random environments

The concepts of π -irreduciblity, recurrence and transience are introduced into the research field of Markov chains in random environments. That a π -irreducible chain must be either recurrent or transient is proved, a criterion is shown for recurrent Markov chains in double-infinite random environments, the existence of invariant measure of π -irreducible chains in double-infinite environments is discussed, and then Orey’s open-questions are partially answered.