Positive Recurrent Markov Chain Research Articles

On the Power of Randomization for Scheduling Real-Time Traffic in Wireless Networks

In this paper, we consider the problem of scheduling real-time traffic in wireless networks under a conflict-graph interference model and single-hop traffic. The objective is to guarantee that at least a certain fraction of packets of each link are delivered within their deadlines, which is referred to as delivery ratio. This problem has been studied before under restrictive frame-based traffic models, or greedy maximal scheduling schemes like LDF (Largest-Deficit First) that can lead to poor delivery ratio for general traffic patterns. In this paper, we pursue a different approach through randomization over the choice of maximal links that can transmit at each time. We design randomized policies in collocated networks, multi-partite networks, and general networks, that can achieve delivery ratios much higher than what is achievable by LDF. Further, our results apply to any traffic (arrival and deadline) process that evolves as an unknown positive recurrent Markov chain. Hence, this work is an improvement with respect to both efficiency and traffic assumptions compared to the past work. We further present extensive simulation results over various traffic patterns and interference graphs to illustrate the gains of our randomized policies over LDF variants.

Kemeny's constant for countable Markov chains

In this paper, we aim to investigate Kemeny's constant for discrete-time or continuous-time, irreducible and positive recurrent Markov chains on a countable state space. We show that Kemeny's constant is infinite for a discrete-time Markov chain on an infinitely countable state space, which confirms a conjecture in page 1031 of [3]. Moreover, it is shown that the study on Kemeny's constant for a continuous-time Markov chain with a bounded generator can be converted to that for a discrete-time Markov chain through the uniformization technique. It is different and challenging to analyze Kemeny's constant for a chain with unbounded generator. Finally, a upward skip-free process is considered and some discussions for Kemeny's constant are given.

Error bounds for augmented truncation approximations of Markov chains via the perturbation method

AbstractLetPbe the transition matrix of a positive recurrent Markov chain on the integers with invariant probability vectorπT, and let(n)P̃ be a stochastic matrix, formed by augmenting the entries of the (n+ 1) x (n+ 1) northwest corner truncation ofParbitrarily, with invariant probability vector(n)πT. We derive computableV-norm bounds on the error betweenπTand(n)πTin terms of the perturbation method from three different aspects: the Poisson equation, the residual matrix, and the norm ergodicity coefficient, which we prove to be effective by showing that they converge to 0 asntends to ∞ under suitable conditions. We illustrate our results through several examples. Comparing our error bounds with the ones of Tweedie (1998), we see that our bounds are more applicable and accurate. Moreover, we also consider possible extensions of our results to continuous-time Markov chains.

Berry–Esseen estimates for regenerative processes under weak moment assumptions

We prove Berry–Esseen type rates of convergence for central limit theorems (CLTs) of regenerative processes which generalize previous results of Bolthausen under weaker moment assumptions. We then show how this general result can be applied to obtain rates of convergence for (1) CLTs for additive functionals of positive recurrent Markov chains under certain conditions on the strong mixing coefficients, and (2) annealed CLTs for certain ballistic random walks in random environments.

An arcsine law for Markov random walks

The classic arcsine law for the number Nn>≔n−1∑k=1n1{Sk>0} of positive terms, as n→∞, in an ordinary random walk (Sn)n≥0 is extended to the case when this random walk is governed by a positive recurrent Markov chain (Mn)n≥0 on a countable state space S, that is, for a Markov random walk (Mn,Sn)n≥0 with positive recurrent discrete driving chain. More precisely, it is shown that n−1Nn> converges in distribution to a generalized arcsine law with parameter ρ∈[0,1] (the classic arcsine law if ρ=1∕2) iff the Spitzer condition limn→∞1n∑k=1nPi(Sn>0)=ρholds true for some and then all i∈S, where Pi≔P(⋅|M0=i) for i∈S. It is also proved, under an extra assumption on the driving chain if 0<ρ<1, that this condition is equivalent to the stronger variant limn→∞Pi(Sn>0)=ρ.For an ordinary random walk, this was shown by Doney (1995) for 0<ρ<1 and by Bertoin and Doney (1997) for ρ∈{0,1}.

Limiting spectral distribution of Gram matrices associated with functionals of β-mixing processes

We give asymptotic spectral results for Gram matrices of the form n−1XnXnT where the entries of Xn are dependent across both rows and columns. More precisely, they consist of short or long range dependent random variables having moments of second order and that are functionals of an absolutely regular sequence. We also give a concentration inequality of the Stieltjes transform and we prove that, under an arithmetical decay condition on the β-mixing coefficients, it is almost surely concentrated around its expectation. Applications to examples of positive recurrent Markov chains and dynamical systems are also given.

MaxWeight Scheduling

The model is a "generalized switch", serving multiple traffic flows in discrete time. The switch uses MaxWeight algorithm to make a service decision (scheduling choice) at each time step, which determines the probability distribution of the amount of service that will be provided. We are primarily motivated by the following question: in the heavy traffic regime, when the switch load approaches critical level, will the service processes provided to each flow remain "smooth" (i.e., without large gaps in service)? Addressing this question reduces to the analysis of the asymptotic behavior of the unscaled queue-differential process in heavy traffic. We prove that the stationary regime of this process converges to that of a positive recurrent Markov chain, whose structure we explicitly describe. This in turn implies asymptotic "smoothness" of the service processes.

Almost sure central limit theory for positive recurrent Markov chain

Consider an irreducible and positive recurrent Markov chain. A universal result in almost sure limit central theorem for the partial sums of its function is established. Our results realize the extension of almost sure central limit theorem for a sequence of independent and identically distributed (i.i.d.) random variables to Markov chain.

Clustering of Markov chain exceedances

The tail chain of a Markov chain can be used to model the dependence between extreme observations. For a positive recurrent Markov chain, the tail chain aids in describing the limit of a sequence of point processes $\{N_{n},n\geq1\}$, consisting of normalized observations plotted against scaled time points. Under fairly general conditions on extremal behaviour, $\{N_{n}\}$ converges to a cluster Poisson process. Our technique decomposes the sample path of the chain into i.i.d. regenerative cycles rather than using blocking argument typically employed in the context of stationarity with mixing.

Potential analysis for positive recurrent Markov chains with asymptotically zero drift: Power-type asymptotics

We consider a positive recurrent Markov chain on R+ with asymptotically zero drift which behaves like −c1/x at infinity; this model was first considered by Lamperti. We are interested in tail asymptotics for the stationary measure. Our analysis is based on construction of a harmonic function which turns out to be regularly varying at infinity. This harmonic function allows us to perform non-exponential change of measure. Under this new measure Markov chain is transient with drift like c2/x at infinity and we compute the asymptotics for its Green function. Applying further the inverse transform of measure we deduce a power-like asymptotic behaviour of the stationary tail distribution. Such a heavy-tailed stationary measure happens even if the jumps of the chain are bounded. This model provides an example where possibly bounded input distributions produce non-exponential output.

Asymptotic properties of covariate-adaptive randomization

Balancing treatment allocation for influential covariates is critical in clinical trials. This has become increasingly important as more and more biomarkers are found to be associated with different diseases in translational research (genomics, proteomics and metabolomics). Stratified permuted block randomization and minimization methods [Pocock and Simon Biometrics 31 (1975) 103-115, etc.] are the two most popular approaches in practice. However, stratified permuted block randomization fails to achieve good overall balance when the number of strata is large, whereas traditional minimization methods also suffer from the potential drawback of large within-stratum imbalances. Moreover, the theoretical bases of minimization methods remain largely elusive. In this paper, we propose a new covariate-adaptive design that is able to control various types of imbalances. We show that the joint process of within-stratum imbalances is a positive recurrent Markov chain under certain conditions. Therefore, this new procedure yields more balanced allocation. The advantages of the proposed procedure are also demonstrated by extensive simulation studies. Our work provides a theoretical tool for future research in this area.

Asymptotic Income Distribution in the German Socio-Economic Panel (SOEP): Income Inequality and Darwinian Fitness

In this paper, the relation between income inequality and population growth is analyzed from a Darwinian perspective. A Markov chain population growth model is presented and estimated using data from the German Socio-Economic Panel (SOEP). We estimate both population growth rates and steady-state income distribution for males and females. The results are compatible with the traditional age-based population growth models of demography, in so far as these are actually irreducible, positive-recurrent Markov chains. It is concluded that, from the perspective of individuals and family lines, income inequality may improve reproductive fitness of high-income individuals, and foster adaptive reproductive strategies for a given income level.

Almost sure central limit theorem for partial sums of Markov chain

The authors prove an almost sure central limit theorem for partial sums based on an irreducible and positive recurrent Markov chain using logarithmic means, which realizes the extension of the almost sure central limit theorem for partial sums from an i.i.d. sequence of random variables to a Markov chain.

Regenerative block empirical likelihood for Markov chains

Empirical likelihood (EL) is a powerful semi-parametric method increasingly investigated in the literature. However, most authors essentially focus on an i.i.d. setting. In the case of dependent data, the classical EL method cannot be directly applied on the data but rather on blocks of consecutive data catching the dependence structure. Generalisation of EL based on the construction of blocks of increasing random length have been proposed for time series satisfying mixing conditions. Following some recent developments in the bootstrap literature, we propose a generalisation for a large class of Markov chains, based on small blocks of various lengths. Our approach makes use of the regenerative structure of Markov chains, which allows us to construct blocks which are almost independent (independent in the atomic case). We obtain the asymptotic validity of the method for positive recurrent Markov chains and present some simulation results.

Asymptotic Optimality of Isoperimetric Constants

In this paper, we investigate the existence of L 2(π)-spectral gaps for π-irreducible, positive recurrent Markov chains with a general state space Ω. We obtain necessary and sufficient conditions for the existence of L 2(π)-spectral gaps in terms of a sequence of isoperimetric constants. For reversible Markov chains, it turns out that the spectral gap can be understood in terms of convergence of an induced probability flow to the uniform flow. These results are used to recover classical results concerning uniform ergodicity and the spectral gap property as well as other new results. As an application of our result, we present a rather short proof for the fact that geometric ergodicity implies the spectral gap property. Moreover, the main result of this paper suggests that sharp upper bounds for the spectral gap should be expected when evaluating the isoperimetric flow for certain sets. We provide several examples where the obtained upper bounds are exact.

Sharp Bounds for the Tails of Functionals of Markov Chains

This paper is devoted to establishing sharp bounds for deviation probabilities of partial sums $\sum_{i=1}^{n}f(X_{i})$, where $X=(X_{n})_{n\in{\bf N}}$ is a positive recurrent Markov chain and f is a real valued function defined on its state space. Combining the regenerative method to the Esscher transformation, these estimates are shown in particular to generalize probability inequalities proved in the independent i.i.d. case to the Markovian setting for (not necessarily uniformly) geometrically ergodic chains.

Perfect simulation for a class of positive recurrent Markov chains

In [1] we introduced a class of positive recurrent Markov chains, named tame chains. A perfect simulation algorithm, based on the method of dominated CFTP, was then shown to exist in principle for such chains. The construction of a suitable dominating process was flawed, in that it relied on an incorrectly stated lemma ([1], Lemma 6). This claimed that a geometrically ergodic chain, subsampled at a stopping time σ , satisfies a geometric Foster–Lyapunov drift condition with coefficients not depending on σ. This is true if σ is a stopping time independent of the chain, but not if this independence does not hold. Reference [1], Lemma 6 is therefore false as stated.

Perfect simulation for a class of positive recurrent Markov chains

This paper generalizes the work of Kendall [Electron. Comm. Probab. 9 (2004) 140--151], which showed that perfect simulation, in the form of dominated coupling from the past, is always possible (although not necessarily practical) for geometrically ergodic Markov chains. Here, we consider the more general situation of positive recurrent chains and explore when it is possible to produce such a simulation algorithm for these chains. We introduce a class of chains which we name tame, for which we show that perfect simulation is possible.

Regenerative block bootstrap for Markov chains

A specific bootstrap method is introduced for positive recurrent Markov chains, based on the regenerative method and the Nummelin splitting technique. This construction involves generating a sequence of approximate pseudo-renewal times for a Harris chain X from data X1,..., Xn and the parameters of a minorization condition satisfied by its transition probability kernel and then applying a variant of the methodology proposed by Datta and McCormick for bootstrapping additive functionals of type n-1∑i=1nf(Xi) when the chain possesses an atom. This novel methodology mainly consists in dividing the sample path of the chain into data blocks corresponding to the successive visits to the atom and resampling the blocks until the (random) length of the reconstructed trajectory is at least n, so as to mimic the renewal structure of the chain. In the atomic case we prove that our method inherits the accuracy of the bootstrap in the independent and identically distributed case up to OP(n-1) under weak conditions. In the general (not necessarily stationary) case asymptotic validity for this resampling procedure is established, provided that a consistent estimator of the transition kernel may be computed. The second-order validity is obtained in the stationary case (up to a rate close to OP(n-1) for regular stationary chains). A data-driven method for choosing the parameters of the minorization condition is proposed and applications to specific Markovian models are discussed.

Strong Disorder for a Certain Class of Directed Polymers in a Random Environment

We study a model of directed polymers in a random environment with a positive recurrent Markov chain, taking values in a countable space Σ. The random environment is a family ( $$g(i,x), i \geq 1,x \in \Sigma$$ ) of independent and identically distributed real-valued variables. The asymptotic behaviour of the normalized partition function is characterized: when the common law of the g(·,·) is infinitely divisible and the Markov chain is exponentially recurrent we prove that the normalized partition function converges exponentially fast towards zero at all temperatures.