Monotone Markov Chains Research Articles

A Note on the Stability of Monotone Markov Chains

This note studies monotone Markov chains, a subclass of Markov chains with extensive applications in operations research and economics. While the properties that ensure the global stability of these chains are well studied, their establishment often relies on the fulfillment of a certain splitting condition. We address the challenges of verifying the splitting condition by introducing simple, applicable conditions that ensure global stability. The simplicity of these conditions is demonstrated through various examples including autoregressive processes, portfolio allocation problems and resource allocation dynamics.

Spectral gap and curvature of monotone Markov chains

Spectral gap and curvature of monotone Markov chains

On monotone Markov chains and properties of monotone matrix roots

Abstract Monotone matrices are stochastic matrices that satisfy the monotonicity conditions as introduced by Daley in 1968. Monotone Markov chains are useful in modeling phenomena in several areas. Most previous work examines the embedding problem for Markov chains within the entire set of stochastic transition matrices, and only a few studies focus on the embeddability within a specific subset of stochastic matrices. This article examines the embedding in a discrete-time monotone Markov chain, i.e., the existence of monotone matrix roots. Monotone matrix roots of ( 2 × 2 ) \left(2\times 2) monotone matrices are investigated in previous work. For ( 3 × 3 ) \left(3\times 3) monotone matrices, this article proves properties that are useful in studying the existence of monotone roots. Furthermore, we demonstrate that all ( 3 × 3 ) \left(3\times 3) monotone matrices with positive eigenvalues have an m m th root that satisfies the monotonicity conditions (for all values m ∈ N , m ≥ 2 m\in {\mathbb{N}},m\ge 2 ). For monotone matrices of order n > 3 n\gt 3 , diverse scenarios regarding the matrix roots are pointed out, and interesting properties are discussed for block diagonal and diagonalizable monotone matrices.

Swendsen‐Wang dynamics for general graphs in the tree uniqueness region

The Swendsen‐Wang (SW) dynamics is a popular Markov chain for sampling from the Gibbs distribution for the ferromagnetic Ising model on a graph G = (V,E). The dynamics is conjectured to converge to equilibrium in O(|V|1/4) steps at any (inverse) temperature β, yet there are few results providing o(|V|) upper bounds. We prove fast convergence of the SW dynamics on general graphs in the tree uniqueness region. In particular, when β < βc(d) where βc(d) denotes the uniqueness/nonuniqueness threshold on infinite d‐regular trees, we prove that the relaxation time (i.e., the inverse spectral gap) of the SW dynamics is Θ(1) on any graph of maximum degree d ≥ 3. Our proof utilizes a monotone version of the SW dynamics which only updates isolated vertices. We establish that this variant of the SW dynamics has mixing time and relaxation time Θ(1) on any graph of maximum degree d for all β < βc(d). Our proof technology can be applied to general monotone Markov chains, including for example the heat‐bath block dynamics, for which we obtain new tight mixing time bounds.

A unified stability theory for classical and monotone Markov chains

AbstractIn this paper we integrate two strands of the literature on stability of general state Markov chains: conventional, total-variation-based results and more recent order-theoretic results. First we introduce a complete metric over Borel probability measures based on ‘partial’ stochastic dominance. We then show that many conventional results framed in the setting of total variation distance have natural generalizations to the partially ordered setting when this metric is adopted.

Antiduality and Möbius monotonicity: generalized coupon collector problem

For a given absorbing Markov chain X* on a finite state space, a chain X is a sharp antidual of X* if the fastest strong stationary time (FSST) of X is equal, in distribution, to the absorption time of X*. In this paper, we show a systematic way of finding such an antidual based on some partial ordering of the state space. We use a theory of strong stationary duality developed recently for Möbius monotone Markov chains. We give several sharp antidual chains for Markov chain corresponding to a generalized coupon collector problem. As a consequence – utilizing known results on the limiting distribution of the absorption time – we indicate separation cutoffs (with their window sizes) in several chains. We also present a chain which (under some conditions) has a prescribed stationary distribution and its FSST is distributed as a prescribed mixture of sums of geometric random variables.

Rank-dependent Galton‒Watson processes and their pathwise duals

Abstract We introduce a modified Galton‒Watson process using the framework of an infinite system of particles labelled by (x,t), where x is the rank of the particle born at time t. The key assumption concerning the offspring numbers of different particles is that they are independent, but their distributions may depend on the particle label (x,t). For the associated system of coupled monotone Markov chains, we address the issue of pathwise duality elucidated by a remarkable graphical representation in which the trajectories of the primary Markov chains and their duals coalesce to form forest graphs on a two-dimensional grid.

Necessary embedding conditions for state-wise monotone Markov chains

In previous work, the embedding problem is examined within the entire set of discrete-time Markov chains. However, for several phenomena, the states of a Markov model are ordered categories and the transition matrix is state-wise monotone. The present paper investigates the embedding problem for the specific subset of state-wise monotone Markov chains. We prove necessary conditions on the transition matrix of a discrete-time Markov chain with ordered states to be embeddable in a state-wise monotone Markov chain regarding time-intervals with length 0.5: A transition matrix with a square root within the set of state-wise monotone matrices has a trace at least equal to 1.

Some Results on the Embeddable Problem for Discrete-Time Markov Models in Manpower Planning

For discrete-time 2-stated Markov chains the embeddable problem as well as the inverse problem are discussed in detail based on the concept of m-th root probability matrix: necessary and sufficient conditions are formulated under which the considered Markov chain is compatible with a transition matrix regarding time unity . In case of an embeddable Markov chain, the m-th root probability matrices are expressed in analytic form. A Markov chain with states that are ordered, is expected to be embedded in a discrete-time state-wise monotone Markov chain. For this type of Markov chains with the number of states equal to 2, 3, or 4, the trace of the transition matrix at least equal to 1 is proved to be a necessary condition for the embeddability in a Markov chain regarding time unity , for m an even number.

Strong stationary duality for Möbius monotone Markov chains

For Markov chains with a finite, partially ordered state space, we show strong stationary duality under the condition of Möbius monotonicity of the chain. We give examples of dual chains in this context which have no downwards transitions. We illustrate general theory by an analysis of nonsymmetric random walks on the cube with an interpretation for unreliable networks of queues.

An order-theoretic mixing condition for monotone Markov chains

We discuss the stability of discrete-time Markov chains satisfying monotonicity and an order-theoretic mixing condition that can be seen as an alternative to irreducibility. A chain satisfying these conditions has at most one stationary distribution. Moreover, if there is a stationary distribution, then the chain is stable in an order-theoretic sense.

Functional Convergence to Stable Lévy Motions for Iterated Random Lipschitz Mappings

It is known that, in the dependent case, partial sums processes which are elements of $D([0,1])$ (the space of right-continuous functions on $[0,1]$ with left limits) do not always converge weakly in the $J_1$-topology sense. The purpose of our paper is to study this convergence in $D([0,1])$ equipped with the $M_1$-topology, which is weaker than the $J_1$ one. We prove that if the jumps of the partial sum process are associated then a functional limit theorem holds in $D([0,1])$ equipped with the $M_1$-topology, as soon as the convergence of the finite-dimensional distributions holds. We apply our result to some stochastically monotone Markov chains arising from the family of iterated Lipschitz models.

Subgeometric ergodicity for continuous-time Markov chains

In this paper, subgeometric ergodicity is investigated for continuous-time Markov chains. Several equivalent conditions, based on the first hitting time or the drift function, are derived as the main theorem. In its corollaries, practical drift criteria are given for ℓ-ergodicity and computable bounds on subgeometric convergence rates are obtained for stochastically monotone Markov chains. These results are illustrated by examples.

MONOTONICITY IN LAG FOR NONMONOTONE MARKOV CHAINS

It is well known that for a stochastically monotone Markov chain {Jn}n≥1 a function γ(n) = Cov[f (J1),g(Jn)] is decreasing if f and g are increasing. We prove this property for a special subclass of nonmonotone double stochastic Markov chains.

Bounds for the Reliability of Multistate Systems with Partially Ordered State Spaces and Stochastically Monotone Markov Transitions

Consider a multistate system with partially ordered state space E, which is divided into a set C of working states and a set D of failure states. Let X(t) be the state of the system at time t and suppose {X(t)} is a stochastically monotone Markov chain on E. Let T be the failure time, i.e., the hitting time of the set D. We derive upper and lower bounds for the reliability of the system, defined as Pm(T > t) where m is the state of perfect system performance.

On Strassen's Theorem on Stochastic Domination

The purpose of this note is to make available a reasonably complete and straightforward proof of Strassen's theorem on stochastic domination, and to draw attention to the original paper. We also point out that the maximal possible value of $P(Z = Z')$ is actually not reduced by the requirement $Z \leq Z'$. Here, $Z,Z'$ are stochastic elements that Strassen's theorem states exist under a stochastic domination condition. The consequence of that observation to stochastically monotone Markov chains is pointed out. Usually the theorem is formulated with the assumption that $\leq$ is a partial ordering; the proof reveals that a pre-ordering suffices.

Income distribution dynamics: monotone Markov chains make light work

This paper considers some aspects of the dynamics of income distributions by employing a simple Markov chain model of income mobility. The main motivation of the paper is to introduce the techniques of “monotone” Markov chains to this field. The transition matrix of a discrete Markov chain is called monotone if each row stochastically dominates the row above it. It will be shown that by embedding the dynamics of the income distribution in a monotone Markov chain, a number of interesting results may be obtained in a straightforward and intuitive fashion.

Coefficients of ergodicity for stochastically monotone Markov chains

In this paper we show that to each distance d defined on the finite state space S of a strongly ergodic Markov chain there corresponds a coefficient ρd of ergodicity based on the Wasserstein metric. For a class of stochastically monotone transition matrices P, the infimum over all such coefficients is given by the spectral radius of P – R, where R = lim k Pk and is attained. This result has a probabilistic interpretation of a control of the speed of convergence of by the metric d and is linked to the second eigenvalue of P.

Coefficients of ergodicity for stochastically monotone Markov chains

In this paper we show that to each distance d defined on the finite state space S of a strongly ergodic Markov chain there corresponds a coefficient ρd of ergodicity based on the Wasserstein metric. For a class of stochastically monotone transition matrices P, the infimum over all such coefficients is given by the spectral radius of P – R, where R = limkPk and is attained. This result has a probabilistic interpretation of a control of the speed of convergence of by the metric d and is linked to the second eigenvalue of P.

Relative stability and weak convergence in non‐decreasingstochastically monotone Markov chains

Let {ξn} be a non‐decreasing stochastically monotone Markov chain whose transition probability Q(..) has Q(x, {x}) = β(x) > 0 for some function β(.) that is non‐decreasing with β(x)↑1 as x → +∞, and each Q(x.) is non‐atomic otherwise. A typical realization of {ξn} is a Markov renewal process {(Xn, Tn)}, where ξj = Xn, for Tn consecutive values of j, Tn geometric on {1, 2, …} with parameter β(Xn). Conditions are given for Xn, to be relatively stable and for Tn to be weakly convergent.