Positive Recurrent Research Articles

Sufficient conditions for regularity, positive recurrence, and absorption in level‐dependent QBD processes and related block‐structured Markov chains

This paper is concerned with level‐dependent quasi‐birth‐death (LD‐QBD) processes, i.e., multi‐variate Markov chains with a block‐tridiagonal ‐matrix, and a more general class of block‐structured Markov chains, which can be seen as LD‐QBD processes with total catastrophes. Arguments from univariate birth‐death processes are combined with existing matrix‐analytic formulations to obtain sufficient conditions for these block‐structured processes to be regular, positive recurrent, and absorbed with certainty in a finite mean time. Specifically, it is our purpose to show that, as is the case for competition processes, these sufficient conditions are inherently linked to a suitably defined birth‐death process. Our results are exemplified with two Markov chain models: a study of target cells and viral dynamics and one of kinetic proof‐reading in T cell receptor signal transduction.

Some Network Conditions for Positive Recurrence of Stochastically Modeled Reaction Networks

We consider discrete-space continuous-time Markov models of reaction networks and provide sufficient conditions for the following stability condition to hold: each state in a closed, irreducible component of the state space is positive recurrent; moreover the time required for a trajectory to enter such a component has finite expectation. The provided analytical results depend solely on the underlying structure of the reaction network and not on the specific choice of model parameters. Our main results apply to binary systems, and our main analytical tool is the “tier structure” previously utilized successfully in the study of deterministic models of reaction networks.

Dependent double branching annihilating random walk

Double (or parity conserving) branching annihilating random walk, introduced by Sudbury in '90, is a one-dimensional non-attractive particle system in which positive and negative particles perform nearest neighbor hopping, produce two offsprings to neighboring lattice points and annihilate when they meet. Given an odd number of initial particles, positive recurrence as seen from the leftmost particle position was first proved by Belitsky, Ferrari, Menshikov and Popov in '01 and, subsequently in a much more general setup, in the article by Sturm and Swart (Tightness of voter model interfaces) in '08. These results assume that jump rates of the various moves do not depend on the configuration of the particles not involved in these moves. The present article deals with the case when the jump rates are affected by the locations of several particles in the system. Motivation for such models comes from non-attractive interacting particle systems with particle conservation. Under suitable assumptions we establish the existence of the process, and prove that the one-particle state is positive recurrent. We achieve this by arguments similar to those appeared in the previous article by Sturm and Swart. We also extend our results to some cases of long range jumps, when branching can also occur to non-neighboring sites. We outline and discuss several particular examples of models where our results apply.

Stability of a Markov-modulated Markov Chain, with Application to a Wireless Network Governed by two Protocols

We consider a discrete-time Markov chain (Xt,Yt), t = 0, 1, 2,…, where the X-component forms a Markov chain itself. Assume that (Xt) is Harris-ergodic and consider an auxiliary Markov chain [Formula: see text] whose transition probabilities are the averages of transition probabilities of the Y-component of the (X, Y)-chain, where the averaging is weighted by the stationary distribution of the X-component. We first provide natural conditions in terms of test functions ensuring that the [Formula: see text]-chain is positive recurrent and then prove that these conditions are also sufficient for positive recurrence of the original chain (Xt, Yt). The we prove a “multi-dimensional” extension of the result obtained. In the second part of the paper, we apply our results to two versions of a multi-access wireless model governed by two randomised protocols.

Stability of a Markov-modulated Markov chain, with application to a wireless network governed by two protocols

We consider a discrete-time Markov chain (Xt,Yt), t = 0, 1, 2,…, where the X-component forms a Markov chain itself. Assume that (Xt) is Harris-ergodic and consider an auxiliary Markov chain {Yt} whose transition probabilities are the averages of transition probabilities of the Y-component of the (X, Y)-chain, where the averaging is weighted by the stationary distribution of the X-component. We first provide natural conditions in terms of test functions ensuring that the Y-chain is positive recurrent and then prove that these conditions are also sufficient for positive recurrence of the original chain (Xt, Yt). The we prove a “multi-dimensional” extension of the result obtained. In the second part of the paper, we apply our results to two versions of a multi-access wireless model governed by two randomised protocols.

A positive recurrent reflecting Brownian motion with divergent fluid path

Semimartingale reflecting Brownian motions (SRBMs) are diffusion processes with state space the d-dimensional nonnegative orthant, in the interior of which the processes evolve according to a Brownian motion, and that reflect against the boundary in a specified manner. The data for such a process are a drift vector θ, a nonsingular d×d covariance matrix Σ, and a d×d reflection matrix R. A standard problem is to determine under what conditions the process is positive recurrent. Necessary and sufficient conditions for positive recurrence are easy to formulate for d=2, but not for d>2. Associated with the pair (θ, R) are fluid paths, which are solutions of deterministic equations corresponding to the random equations of the SRBM. A standard result of Dupuis and Williams [Ann. Probab. 22 (1994) 680–702] states that when every fluid path associated with the SRBM is attracted to the origin, the SRBM is positive recurrent. Employing this result, El Kharroubi, Ben Tahar and Yaacoubi [Stochastics Stochastics Rep. 68 (2000) 229–253, Math. Methods Oper. Res. 56 (2002) 243–258] gave sufficient conditions on (θ, Σ, R) for positive recurrence for d=3; Bramson, Dai and Harrison [Ann. Appl. Probab. 20 (2009) 753–783] showed that these conditions are, in fact, necessary. Relatively little is known about the recurrence behavior of SRBMs for d>3. This pertains, in particular, to necessary conditions for positive recurrence. Here, we provide a family of examples, in d=6, with θ=(−1, −1, …, −1)T, Σ=I and appropriate R, that are positive recurrent, but for which a linear fluid path diverges to infinity. These examples show in particular that, for d≥6, the converse of the Dupuis–Williams result does not hold.

Positive recurrence for reflecting Brownian motion in higher dimensions

Semimartingale reflecting Brownian motions (SRBMs) are diffusion processes with state space the d-dimensional nonnegative orthant, in the interior of which the processes evolve according to a Brownian motion, and that reflect against the boundary in a specified manner. A standard problem is to determine under what conditions the process is positive recurrent. Necessary and sufficient conditions for positive recurrence are easy to formulate in d=2, but not in d?3. Fluid paths are solutions of deterministic equations that correspond to the random equations of the SRBM. A standard result of Dupuis and Williams (in Ann. Probab. 22:680---702, 1994) states that when every fluid path associated with the SRBM is attracted to the origin, the SRBM is positive recurrent. Employing this result, El Kharroubi et al. (in Stoch. Stoch. Rep. 68:229---253, 2000; Math. Methods Oper. Res. 56:243---258, 2002) gave sufficient conditions involving fluid paths for positive recurrence of SRBM in d=3. Here, we discuss two recent results regarding necessary conditions for positive recurrence of SRBM in d?3. Bramson et al. (in Ann. Appl. Probab. 20:753---783, 2010) showed that the conditions in El Kharroubi et al. (Math. Methods Oper. Res. 56:243---258, 2002) are, in fact, necessary in d=3. On the other hand, Bramson (in Ann. Appl. Probab., to appear, 2011) provided a family of positive recurrent SRBMs, in d?6, with linear fluid paths that diverge to infinity. The latter result shows in particular that the converse of the Dupuis---Williams result does not hold.

Positive recurrence of reflecting Brownian motion in three dimensions

Consider a semimartingale reflecting Brownian motion (SRBM) Z whose state space is the d-dimensional nonnegative orthant. The data for such a process are a drift vector θ, a nonsingular d×d covariance matrix Σ, and a d×d reflection matrix R that specifies the boundary behavior of Z. We say that Z is positive recurrent, or stable, if the expected time to hit an arbitrary open neighborhood of the origin is finite for every starting state. In dimension d=2, necessary and sufficient conditions for stability are known, but fundamentally new phenomena arise in higher dimensions. Building on prior work by El Kharroubi, Ben Tahar and Yaacoubi [Stochastics Stochastics Rep. 68 (2000) 229–253, Math. Methods Oper. Res. 56 (2002) 243–258], we provide necessary and sufficient conditions for stability of SRBMs in three dimensions; to verify or refute these conditions is a simple computational task. As a byproduct, we find that the fluid-based criterion of Dupuis and Williams [Ann. Probab. 22 (1994) 680–702] is not only sufficient but also necessary for stability of SRBMs in three dimensions. That is, an SRBM in three dimensions is positive recurrent if and only if every path of the associated fluid model is attracted to the origin. The problem of recurrence classification for SRBMs in four and higher dimensions remains open.

Recurrence for branching Markov chains

The question of recurrence and transience of branching Markov chains is more subtle than for ordinary Markov chains; they can be classified in transience, weak recurrence, and strong recurrence. We review criteria for transience and weak recurrence and give several new conditions for weak recurrence and strong recurrence. These conditions make a unified treatment of known and new examples possible and provide enough information to distinguish between weak and strong recurrence. This represents a step towards a general classification of branching Markov chains. In particular, we show that in homogeneous cases weak recurrence and strong recurrence coincide. Furthermore, we discuss the generalization of positive and null recurrence to branching Markov chains and show that branching random walks on $Z$ are either transient or positive recurrent.

On the notion of weak stability and related issues of hybrid diffusion systems

This work is concerned with the weak stability of hybrid diffusion systems, which consist of a number of diffusions modulated by a jump process. First, we show that when the jump component is nearly completely decomposable into a number of ergodic jump processes, the overall system is still weakly stable (or positive recurrent). Next we show that even if the process contains transient states, the positive recurrence can still be preserved. Then, we examine the asymptotic distribution when only one ergodic group of states is involved in the jump process and the state space of the continuous state belongs to a compact set. Our attention is devoted to asymptotic distribution in this case. The distribution is obtained by utilizing a spectrum gap property of the underlying process.

Asymptotic Properties of Hybrid Diffusion Systems

In response to the increasing needs for control and optimization of hybrid systems, this work is concerned with such asymptotic properties as recurrence (also known as weak stochastic stability in the literature) and ergodicity of regime-switching diffusions. Using Liapunov functions, necessary and sufficient conditions for positive recurrence are developed. Then, ergodicity of positive recurrent regime-switching diffusions is obtained by constructing cycles using the associated discrete-time Markov chains.

RANDOM DYNAMICAL SYSTEMS ON ORDERED TOPOLOGICAL SPACES

Let (Xn, n ≥ 0) be a random dynamical system and its state space be endowed with a reasonable topology. Instead of completing the structure as common by some linearity, this study stresses — motivated in particular by economic applications — order aspects. If the underlying random transformations are supposed to be order-preserving, this results in a fairly complete theory. First of all, the classical notions of and familiar criteria for recurrence and transience can be extended from discrete Markov chain theory. The most important fact is provided by the existence and uniqueness of a locally finite-invariant measure for recurrent systems. It allows to derive ergodic theorems as well as to introduce an attract or in a natural way. The classification is completed by distinguishing positive and null recurrence corresponding, respectively, to the case of a finite or infinite invariant measure; equivalently, this amounts to finite or infinite mean passage times. For positive recurrent systems, moreover, strengthened versions of weak convergence as well as generalized laws of large numbers are available.

Positive recurrence of processes associated to crystal growth models

We show that certain Markov jump processes associated to crystal growth models are positive recurrent when the parameters satisfy a rather natural condition.

On Positive Recurrence of Constrained Diffusion Processes

Let $G \subset \mathbb{R}^k$ be a convex polyhedral cone with vertex at the origin given as the intersection of half spaces $\{G_i, i =1,\ldots, N\}$, where $n_i$ and $d_i$ denote the inward normal and direction of constraint associated with $G_i$, respectively. Stability properties of a class of diffusion processes, constrained to take values in $G$, are studied under the assumption that the Skorokhod problem defined by the data $\{(n_i,d_i),i = 1,\ldots,N\}$ is well posed and the Skorokhod map is Lipschitz continuous. Explicit conditions on the drift coefficient, $b(\cdot)$, of the diffusion process are given under which the constrained process is positive recurrent and has a unique invariant measure. Define $$\mathscr{C}\doteq \left\{ - \sum^{N}_{i=1} a_i d_i;a_i \ge 0, i \in \{1,\ldots,N\} \right\}.$$ Then the key condition for stability is that there exists $\delta \in (0,\infty)$ and a bounded subset $A$ of $G$ such that for all $x \in G\setminus A, b(x) \in \mathscr{C}$ and $\mathrm{dist} (b(x),\partial\mathscr{C}) \ge \delta$, where $\partial\mathscr{C}$denotes the boundary of $\mathscr{C}$.

Recurrence-transience criteria for storage processes

It is shown that general storage processes are classified into three types: positive recurrent, null recurrent, and transient processes. Sufficient conditions are given for transience and recurrence (null or positive recurrence), respectively. The sufficient condition for transience is shown to be necessary for transience in a special case (linear release rate case). This is a slight simplification of a known necessary and sufficient condition.

Classification of Markov Processes of Matrix M/G/l type with a Tree Structure and its Applications to the MMAP[K]/G[K]/1 Queues

This paper studies the classification problem of discrete time and continuous time Markov processes of matrix M/G/l type with a tree structure. It is shown that the Perron-Frobenius eigenvalue of a nonnegative matrix provides information for a complete classification of the Markov process of interest. A computational method is developed to find whether a Markov process of matrix M/G/l type with a tree structure is positive recurrent, null recurrent, or transient. The method is then used to study the impact of the last-come-first-served general preemptive resume (LCFS-GPR) service discipline on the stability of a MAP/PHA queue. Two sufficient conditions are identified for the positive recurrence and transience of the Markov processes of interest, respectively. As an example, the results are used to show that the discrete time (and continuous time) MMAP[K]/G[K]/1queue with a work conserving service discipline is stable if and only if its traffic intensity is less than one, unstable if its traffic intensity is larger than one

Thermodynamic formalism for countable Markov shifts

We establish a generalized thermodynamic formalism for topological Markov shifts with a countable number of states. We offer a definition of topological pressure and show that it satisfies a variational principle for the metric entropies. The pressure of $\phi =0$ is the Gurevic entropy. This pressure may be finite even if the topological entropy is infinite. Let $L_\phi$ denote the Ruelle operator for $\phi$. We offer a definition of positive recurrence for $\phi$ and show that it is a necessary and sufficient condition for a Ruelle–Perron–Frobenius theorem to hold: there exist a $\sigma$-finite measure $\nu $, a continuous function $h>0$ and $\lambda >0$ such that $L_\phi ^{*}\nu =\lambda \nu$, $L_\phi h=\lambda h $ and $\lambda ^{-n}L_\phi ^nf\rightarrow h\int f\,d\nu$ for suitable functions $f$. We show that under certain conditions this convergence is uniform and exponential. We prove a decomposition theorem for positive recurrent functions and construct conformal measures and equilibrium measures. We give complete characterization of the situation when the equilibrium measure is a Gibbs measure. We end by giving examples where positive recurrence can be verified. These include functions of the form $$ \phi =\log f\left( \cfrac{1}{x_0+ \cfrac{1}{x_1+\dotsb }}\right), $$ where $f$ is a suitable function on a suitable shift $X$.

Recurrence and Transience of Diffusions in a Half-Space

For non-degenerate diffusions in the half-space with oblique reflection, a dichotomy between recurrence and transience is established; convenient characterizations of recurrence and transience are given. Verifiable criteria for recurrence/transience are derived in terms of the generator and the boundary operator. Using these criteria, `real variables proofs' of some results due to Rogers, concerning reflecting Brownian motion in a half-plane, are obtained. The problem of transience down a side in the case of diffusions in the half-plane is dealt with. Positive recurrence of diffusions in half-space is also considered; it is shown that the hitting time of any open set has finite expectation if there is just one positive recurrent point.

Recurrence of Ancestral Lines and Offspring Trees in Time Stationary Branching Populations

AbstractFor time stationary Galton‐ Watson‐branching populations on a general type space, the structure of the “individually positive recurrent part” of the system is described: its building blocks consist of finitely many “clans” with positive recurrent trunks. Conditions are given when this nubsystem is void, and when it equals the full system. In addition, positive recurrence on the clan level is characterized. Whereas individual positive recurrence turns out to be a symmetric concept with respect to forward and backward time direction (i. e., with respect to anceatral lines and offspring trees), with individual null recurrence this symmetry can fail even in the absence of branching, i.e., for independently migrating particle systems (Example 13.1). For discrete type spaces a classification of types as to the various individual recurrence concepts (positive, null, forward and backward in time) is proposed and illustrated by a couple of results and examples. For finite type spaces conditiom on the branching dynamics and its mean matrix for the existence of nontrivial equilibria are given.

Recurrence and ergodicity of diffusions

This article attempts to lay a proper foundation for studying asymptotic properties of nonhomogeneous diffusions, extends earlier criteria for transience, recurrence, and positive recurrence, and provides sufficient conditions for the weak convergence of a shifted nonhomogeneous diffusion to a limiting stationary homogenous diffusion. A functional central limit theorem is proved for the class of positive recurrent homogeneous diffusions. Upper and lower functions for positive recurrent nonhomogeneous diffusions are also studied.