Uniform Ergodicity Research Articles

Uniform ergodicity of continuous-time controlled Markov chains: A survey and new results

We make a review of several variants of ergodicity for continuous-time Markov chains on a countable state space. These include strong ergodicity, ergodicity in weighted-norm spaces, exponential and subexponential ergodicity. We also study uniform exponential ergodicity for continuous-time controlled Markov chains, as a tool to deal with average reward and related optimality criteria. A discussion on the corresponding ergodicity properties is made, and an application to a controlled population system is shown.

Explicit error bounds for Markov chain Monte Carlo

We prove explicit, i.e. non-asymptotic, error bounds for Markov chain Monte Carlo methods. The problem is to compute the expectation of a function f with respect to a measure {\pi}. Different convergence properties of Markov chains imply different error bounds. For uniformly ergodic and reversible Markov chains we prove a lower and an upper error bound with respect to the L2 -norm of f . If there exists an L2 -spectral gap, which is a weaker convergence property than uniform ergodicity, then we show an upper error bound with respect to the Lp -norm of f for p > 2. Usually a burn-in period is an efficient way to tune the algorithm. We provide and justify a recipe how to choose the burn-in period. The error bounds are applied to the problem of the integration with respect to a possibly unnormalized density. More precise, we consider the integration with respect to log-concave densities and the integration over convex bodies. By the use of the Metropolis algorithm based on a ball walk and the hit-and-run algorithm it is shown that both problems are polynomial tractable.

Perturbation Bounds for the Stationary Distributions of Markov Chains

In this paper, we are interested in investigating the perturbation bounds for the stationary distributions for discrete-time or continuous-time Markov chains on a countable state space. For discrete-time Markov chains, two new normwise bounds are obtained. The first bound is rather easy to obtain since the needed condition, equivalent to uniform ergodicity, is imposed on the transition matrix directly. The second bound, which holds for a general (possibly periodic) Markov chain, involves finding a drift function. This drift function is closely related to the mean first hitting times. Some $V$-normwise bounds are also derived based on the results in [N. V. Kartashov, J. Soviet Math., 34 (1986), pp. 1493--1498]. Moreover, we show how the bounds developed in this paper and one bound given in [E. Seneta, Adv. Appl. Probab., 20 (1988), pp. 228--230] can be extended to continuous-time Markov chains. Several examples are shown to illustrate our results or to compare our bounds with the known ones in the literature.

Jump-diffusions with state-dependent switching: existence and uniqueness, Feller property, linearization, and uniform ergodicity

This work focuses on a class of jump-diffusions with state-dependent switching. First, compared with the existing results in the literature, in our model, the characteristic measure is allowed to be σ-finite. The existence and uniqueness of the underlying process are obtained by representing the switching component as a stochastic integral with respect to a Poisson random measure and by using a successive approximation method. Then, the Feller property is proved by means of introducing auxiliary processes and by making use of Radon-Nikodym derivatives. Furthermore, the irreducibility and all compact sets being petite are demonstrated. Based on these results, the uniform ergodicity is established under a general Lyapunov condition. Finally, easily verifiable conditions for uniform ergodicity are established when the jump-diffusions are linearizable with respect to the variable x (the state variable corresponding to the jump-diffusion component) in a neighborhood of the infinity, and some examples are presented to illustrate the results.

Asymptotic Optimality of Isoperimetric Constants

In this paper, we investigate the existence of L 2(π)-spectral gaps for π-irreducible, positive recurrent Markov chains with a general state space Ω. We obtain necessary and sufficient conditions for the existence of L 2(π)-spectral gaps in terms of a sequence of isoperimetric constants. For reversible Markov chains, it turns out that the spectral gap can be understood in terms of convergence of an induced probability flow to the uniform flow. These results are used to recover classical results concerning uniform ergodicity and the spectral gap property as well as other new results. As an application of our result, we present a rather short proof for the fact that geometric ergodicity implies the spectral gap property. Moreover, the main result of this paper suggests that sharp upper bounds for the spectral gap should be expected when evaluating the isoperimetric flow for certain sets. We provide several examples where the obtained upper bounds are exact.

Asymptotic properties of nonlinear autoregressive Markov processes with state-dependent switching

In this paper, we consider a class of nonlinear autoregressive (AR) processes with state-dependent switching, which are two-component Markov processes. The state-dependent switching model is a nontrivial generalization of Markovian switching formulation and it includes the Markovian switching as a special case. We prove the Feller and strong Feller continuity by means of introducing auxiliary processes and making use of the Radon–Nikodym derivatives. Then, we investigate the geometric ergodicity by the Foster–Lyapunov inequality. Moreover, we establish the V -uniform ergodicity by means of introducing additional auxiliary processes and by virtue of constructing certain order-preserving couplings of the original as well as the auxiliary processes. In addition, illustrative examples are provided for demonstration.

Uniform Conditional Ergodicity and Intrinsic Ultracontractivity

We study two properties of semigroups of sub-Markov kernels, namely uniform conditional ergodicity and intrinsic ultracontractivity. In this paper we investigate the relationship between these two properties and we provide sufficient criteria as well as characterisations of them. In particular, our considerations show that, under suitable assumptions, the second property implies the first one. We also introduce a property called compact domination and show how this property and the parabolic boundary Harnack principle are related to the aforementioned properties. Furthermore, we apply these results in some special cases.

An Adaptive Algorithm for Finding the Optimal Base-Stock Policy in Lost Sales Inventory Systems with Censored Demand

We consider a periodic-review, single-location, single-product inventory system with lost sales and positive replenishment lead times. It is well known that the optimal policy does not possess a simple structure. Motivated by recent results showing that base-stock policies perform well in these systems, we study the problem of finding the best base-stock policy in such a system. In contrast to the classical inventory literature, we assume that the manager does not know the demand distribution a priori but must make the replenishment decision in each period based only on the past sales (censored demand) data. We develop a nonparametric adaptive algorithm that generates a sequence of order-up-to levels whose running average of the inventory holding and lost sales penalty cost converges to the cost of the optimal base-stock policy, and we establish the cubic-root convergence rate of the algorithm. Our analysis is based on recent advances in stochastic online convex optimization and on the uniform ergodicity of Markov chains associated with bases-stock policies.

Mean ergodicity of multiplication operators in weighted spaces of holomorphic functions

Multiplication operators in weighted Banach (and locally convex) spaces of functions holomorphic in the unit disc are well known. In this note we investigate the connection between power boundedness, mean ergodicity and uniform mean ergodicity of such operators.

Asymptotic Properties of a Mean-Field Model with a Continuous-State-Dependent Switching Process

This work is concerned with a class of mean-field models given by a switching diffusion with a continuous-state-dependent switching process. Focusing on asymptotic properties, the regularity or nonexplosiveness, Feller continuity, and strong Feller continuity are established by means of introducing certain auxiliary processes and by making use of the truncations. Based on these results, exponential ergodicity is obtained under the Foster–Lyapunov drift conditions. By virtue of the coupling methods, the strong ergodicity or uniform ergodicity in the sense of convergence in the variation norm is established for the mean-field model with a Markovian switching process. Besides this, several examples are presented for demonstration and illustration.

Asymptotic Properties of a Mean-Field Model with a Continuous-State-Dependent Switching Process

This work is concerned with a class of mean-field models given by a switching diffusion with a continuous-state-dependent switching process. Focusing on asymptotic properties, the regularity or nonexplosiveness, Feller continuity, and strong Feller continuity are established by means of introducing certain auxiliary processes and by making use of the truncations. Based on these results, exponential ergodicity is obtained under the Foster–Lyapunov drift conditions. By virtue of the coupling methods, the strong ergodicity or uniform ergodicity in the sense of convergence in the variation norm is established for the mean-field model with a Markovian switching process. Besides this, several examples are presented for demonstration and illustration.

Quasi-compactness and mean ergodicity for Markov kernels acting on weighted supremum normed spaces

Soit P un noyau markovien sur un espace mesurable E muni d’une tribu a base denombrable, soit w:E→[1, +∞[ tel que Pw≤Cw, avec C≥0, et soit $\mathcal {B}_{w}$ l’espace des fonctions f mesurables de E dans ℂ telles que ‖f‖w=sup{w(x)−1|f(x)|, x∈E}<+∞. Nous demontrons que P est quasi-compact sur $(\mathcal {B}_{w},\|\cdot\|_{w})$ si et seulement si, pour tout $f\in \mathcal {B}_{w}$, $(\frac{1}{n}\sum_{k=1}^{n}P^{k}f)_{n}$ contient une sous-suite convergeant dans $\mathcal {B}_{w}$ vers Πf=∑di=1μi(f)vi, ou vi est une fonction mesurable positive bornee sur E et μi une probabilite sur E. En particulier, quand le sous-espace de $\mathcal {B}_{w}$ constitue des fonctions P-invariantes est de dimension finie, la convergence uniforme des moyennes est equivalente a la convergence ponctuelle.

Population-Based Reversible Jump Markov Chain Monte Carlo

We present an extension of population-based Markov chain Monte Carlo to the transdimensional case. A major challenge is that of simulating from high- and transdimensional target measures. In such cases, Markov chain Monte Carlo methods may not adequately traverse the support of the target; the simulation results will be unreliable. We develop population methods to deal with such problems, and give a result proving the uniform ergodicity of these population algorithms, under mild assumptions. This result is used to demonstrate the superiority, in terms of convergence rate, of a population transition kernel over a reversible jump sampler for a Bayesian variable selection problem. We also give an example of a population algorithm for a Bayesian multivariate mixture model with an unknown number of components. This is applied to gene expression data of 1000 data points in six dimensions and it is demonstrated that our algorithm outperforms some competing Markov chain samplers. In this example, we show how to combine the methods of parallel chains (Geyer, 1991), tempering (Geyer & Thompson, 1995), snooker algorithms (Gilks et al., 1994), constrained sampling and delayed rejection (Green & Mira, 2001).

Lower estimates of transition densities and bounds on exponential ergodicity for stochastic PDEs

A formula for the transition density of a Markov process defined by an infinite-dimensional stochastic equation is given in terms of the Ornstein–Uhlenbeck bridge and a useful lower estimate on the density is provided. As a consequence, uniform exponential ergodicity and V ergodicity are proved for a large class of equations. We also provide computable bounds on the convergence rates and the spectral gap for the Markov semigroups defined by the equations. The bounds turn out to be uniform with respect to a large family of nonlinear drift coefficients. Examples of finite-dimensional stochastic equations and semilinear parabolic equations are given.

A note on a variation of Doeblin's condition for uniform ergodicity of Markov chains

We introduce a simple variation of Doeblin's condition, Condition D*, that assures the uniform ergodicity of a Markov chain. It is also shown that for non-homogeneous chains our conditions are equivalent to Dobrushin's weak ergodic coefficient.

Bias Optimality for Continuous-Time Controlled Markov Chains

In this paper we give conditions for the existence of bias optimal policies in a class of continuous-time controlled Markov chains with unbounded reward and transition rates. Several characterizations of bias optimality are proposed. We also introduce new sets of conditions ensuring uniform exponential ergodicity of continuous-time controlled Markov chains.

Stability of asymptotic properties of Hille-yosida operators under perturbations and retarded differential equations

We give conditions on the strongly continuous semigroup (T 0(t)) t≥0, generated by the part of a Hille-Yosida operator A on X 0 := D(A), and a non-autonomous family of operators (B(t)) t≥0 such that the evolution family (U(t,s)) t≥s≥0, generated by the part of (A + B(t)) t≥0 in X 0, inherits some asymptotic properties of (T 0(t)) t≥0 as boundedness, asymptotic almost periodicity, uniform ergodicity and total uniform ergodicity. Our main result is obtained via an extended result of Batty-Chill [7], which we prove here. We illustrate our abstract hypotheses and results by an example from population dynamics. Also, an application to non-autonomous retarded differential equations is given.

A condition equivalent to uniform ergodicity

A condition equivalent to uniform ergodicity

Numerical integration using V-uniformly ergodic Markov chains

We study numerical integration based on Markov chains. Our focus is on establishing error bounds uniformly on classes of integrands. Since in general state space the concept of uniform ergodicity is too restrictive to cover important cases, we analyze the error of V-uniformly ergodic Markov chains. We place emphasis on the interplay between ergodicity properties of the transition kernel, the initial distributions and the classes of integrands. Our analysis is based on arguments from interpolation theory.

Numerical integration using V-uniformly ergodic Markov chains

We study numerical integration based on Markov chains. Our focus is on establishing error bounds uniformly on classes of integrands. Since in general state space the concept of uniform ergodicity is too restrictive to cover important cases, we analyze the error of V-uniformly ergodic Markov chains. We place emphasis on the interplay between ergodicity properties of the transition kernel, the initial distributions and the classes of integrands. Our analysis is based on arguments from interpolation theory.