Convergence Rates Of Markov Chains Research Articles

Dimension-Free Mixing for High-Dimensional Bayesian Variable Selection

AbstractYang et al. proved that the symmetric random walk Metropolis–Hastings algorithm for Bayesian variable selection is rapidly mixing under mild high-dimensional assumptions. We propose a novel Markov chain Monte Carlo (MCMC) sampler using an informed proposal scheme, which we prove achieves a much faster mixing time that is independent of the number of covariates, under the assumptions of Yang et al. To the best of our knowledge, this is the first high-dimensional result which rigorously shows that the mixing rate of informed MCMC methods can be fast enough to offset the computational cost of local posterior evaluation. Motivated by the theoretical analysis of our sampler, we further propose a new approach called ‘two-stage drift condition’ to studying convergence rates of Markov chains on general state spaces, which can be useful for obtaining tight complexity bounds in high-dimensional settings. The practical advantages of our algorithm are illustrated by both simulation studies and real data analysis.

Convergence Rate of Markov Chains and Hybrid Numerical Schemes to Jump-Diffusion with Application to the Bates Model

Convergence Rate of Markov Chains and Hybrid Numerical Schemes to Jump-Diffusion with Application to the Bates Model

Exponential Convergence Rates of Markov Chains under a Weaken Minorization Condition

In this paper, we obtain the quantitative bound of the exponential convergence rates of Markov chains under a weaken minorization condition, using the coupling method and the analytic approach. And also, we obtain the convergence rates for continuous time Markov processes.

Quantitative bounds of convergence for geometrically ergodic Markov chain in the Wasserstein distance with application to the Metropolis Adjusted Langevin Algorithm

In this paper, we establish explicit convergence rates for Markov chains in Wasserstein distance. Compared to the more classical total variation bounds, the proposed rate of convergence leads to useful insights for the analysis of MCMC algorithms, and suggests ways to construct sampler with good mixing rate even if the dimension of the underlying sampling space is large. We illustrate these results by analyzing the Exponential Integrator version of the Metropolis Adjusted Langevin Algorithm. We illustrate our findings using a Bayesian linear inverse problem.

Convergence rates of Markov chains on spaces of partitions

We study the convergence rate to stationarity for a class of exchangeable partition-valued Markov chains called cut-and-paste chains. The law governing the transitions of a cut-and-paste chain are determined by products of i.i.d. stochastic matrices, which describe the chain induced on the simplex by taking asymptotic frequencies. Using this representation, we establish upper bounds for the mixing times of ergodic cut-and-paste chains, and under certain conditions on the distribution of the governing random matrices we show that the cutoff phenomenon holds.

Convergence rates of Markov chains for some self-assembly and non-saturated Ising models

Algorithms based on Markov chains are ubiquitous across scientific disciplines as they provide a method for extracting statistical information about large, complicated systems. For some self-assembly models, Markov chains can be used to predict both equilibrium and non-equilibrium dynamics. In fact, the efficiency of these self-assembly algorithms can be related to the rate at which simple chains converge to their stationary distribution. We give an overview of the theory of Markov chains and show how many natural chains, including some relevant in the context of self-assembly, undergo a phase transition as a parameter representing temperature is varied in the model. We illustrate this behavior for the non-saturated Ising model in which there are two types of tiles that prefer to be next to other tiles of the same type. Unlike the standard Ising model, we also allow empty spaces that are not occupied by either type of tile. We prove that for a local Markov chain that allows tiles to attach and detach from the lattice, the rate of convergence is fast at high temperature and slow at low temperature.

Convergence rates in the local renewal theorem

We improve the results from [Lindvall, T., 1979. On coupling of discrete renewal processes. Z. Wahrscheinlichkeitsth. 48, 57–70] on subgeometric convergence rates in the local renewal theorem. The results are used in [Sapozhnikov, A., 2006. Subgeometric rates of convergence of f -ergodic Markov chains (submitted for publication)] to generalize the previous results on convergence rates for Markov chains [Tuominen, P., Tweedie, R.L., 1994. Subgeometric rates of convergence of f -ergodic Markov chains. Adv. Appl. Probab. 26, 775–798].

Expander graphs and their applications

A major consideration we had in writing this survey was to make it accessible to mathematicians as well as to computer scientists, since expander graphs, the protagonists of our story, come up in numerous and often surprising contexts in both fields. But, perhaps, we should start with a few words about graphs in general. They are, of course, one of the prime objects of study in Discrete Mathematics. However, graphs are among the most ubiquitous models of both natural and human-made structures. In the natural and social sciences they model relations among species, societies, companies, etc. In computer science, they represent networks of communication, data organization, computational devices as well as the flow of computation, and more. In mathematics, Cayley graphs are useful in Group Theory. Graphs carry a natural metric and are therefore useful in Geometry, and though they are “just” one-dimensional complexes, they are useful in certain parts of Topology, e.g. Knot Theory. In statistical physics, graphs can represent local connections between interacting parts of a system, as well as the dynamics of a physical process on such systems. The study of these models calls, then, for the comprehension of the significant structural properties of the relevant graphs. But are there nontrivial structural properties which are universally important? Expansion of a graph requires that it is simultaneously sparse and highly connected. Expander graphs were first defined by Bassalygo and Pinsker, and their existence first proved by Pinsker in the early ’70s. The property of being an expander seems significant in many of these mathematical, computational and physical contexts. It is not surprising that expanders are useful in the design and analysis of communication networks. What is less obvious is that expanders have surprising utility in other computational settings such as in the theory of error correcting codes and the theory of pseudorandomness. In mathematics, we will encounter e.g. their role in the study of metric embeddings, and in particular in work around the Baum-Connes Conjecture. Expansion is closely related to the convergence rates of Markov Chains, and so they play a key role in the study of Monte-Carlo algorithms in statistical mechanics and in a host of practical computational applications. The list of such interesting and fruitful connections goes on and on with so many applications we will not even

Quantitative bounds on convergence of time-inhomogeneous Markov chains

Convergence rates of Markov chains have been widely studied in recent years. In particular, quantitative bounds on convergence rates have been studied in various forms by Meyn and Tweedie [Ann. Appl. Probab. 4 (1994) 981-1101], Rosenthal [J. Amer. Statist. Assoc. 90 (1995) 558-566], Roberts and Tweedie [Stochastic Process. Appl. 80 (1999) 211-229], Jones and Hobert [Statist. Sci. 16 (2001) 312-334] and Fort [Ph.D. thesis (2001) Univ. Paris VI]. In this paper, we extend a result of Rosenthal [J. Amer. Statist. Assoc. 90 (1995) 558-566] that concerns quantitative convergence rates for time-homogeneous Markov chains. Our extension allows us to consider f-total variation distance (instead of total variation) and time-inhomogeneous Markov chains. We apply our results to simulated annealing.

Bounding spectral gaps of Markov chains: a novel exact multi-decomposition technique

We propose an exact technique to calculate lower bounds of spectral gaps of discrete time reversible Markov chains on finite state sets. Spectral gaps are a common tool for evaluating convergence rates of Markov chains. As an illustration, we successfully use this technique to evaluate the ``absorption time'' of the ``Backgammon model'', a paradigmatic model for glassy dynamics. We also discuss the application of this technique to the ``Contingency table problem'', a notoriously difficult problem from probability theory. The interest of this technique is that it connects spectral gaps, which are quantities related to dynamics, with static quantities, calculated at equilibrium.

Polynomial ergodicity of Markov transition kernels

This paper discusses quantitative bounds on the convergence rates of Markov chains, under conditions implying polynomial convergence rates. This paper extends an earlier work by Roberts and Tweedie (Stochastic Process. Appl. 80(2) (1999) 211), which provides quantitative bounds for the total variation norm under conditions implying geometric ergodicity. Explicit bounds for the total variation norm are obtained by evaluating the moments of an appropriately defined coupling time, using a set of drift conditions, adapted from an earlier work by Tuominen and Tweedie (Adv. Appl. Probab. 26(3) (1994) 775). Applications of this result are then presented to study the convergence of random walk Hastings Metropolis algorithm for super-exponential target functions and of general state-space models. Explicit bounds for f-ergodicity are also given, for an appropriately defined control function f.

Polynomial Convergence Rates of Markov Chains

In this paper we consider Foster–Liapounov-type drift conditions for Markov chains which imply polynomial rate convergence to stationarity in appropriate V-norms. We also show how these results can be used to prove central limit theorems for functions of the Markov chain. We consider two examples concerning random walks on the half line and the independence sampler.

$\varepsilon$-Equilibria for Stochastic Games with Uncountable State Space and Unbounded Costs

We study a class of noncooperative stochastic games with unbounded cost functions and an uncountable state space. It is assumed that the transition law is absolutely continuous with respect to some probability measure on the state space. Undiscounted stochastic games with expected average costs are considered first. It is shown under a uniform geometric ergodicity assumption that there exists a stationary $\varepsilon$-equilibrium for each $\varepsilon > 0 $. The proof is based on recent results on uniform bounds for convergence rates of Markov chains [S. P. Meyn and R. L. Tweedie, Ann. Appl. Probab., 4 (1994), pp. 981--1011] and on an approximation method similar to that used in [A. S. Nowak, J. Optim. Theory Appl., 45 (1985), pp. 591--602], where an $\varepsilon$-equilibrium in stationary policies was shown to exist for the bounded discounted costs. The stochastic game is approximated by one with a countable state space for which a stationary Nash equilibrium exists (see [E. Altman, A. Hordijk, and F. M. Spieksma, Math. Oper. Res., 22 (1997), pp. 588--618]); this equilibrium determines an $\epsilon$-equilibrium for the original game. Finally, new results for the existence of stationary $\varepsilon$-equilibrium for discounted stochastic games are given.

Geometric renewal convergence rates from hazard rates

This paper studies the geometric convergence rate of a discrete renewal sequence to its limit. A general convergence rate is first derived from the hazard rates of the renewal lifetimes. This result is used to extract a good convergence rate when the lifetimes are ordered in the sense of new better than used or increasing hazard rate. A bound for the best possible geometric convergence rate is derived for lifetimes having a finite support. Examples demonstrating the utility and sharpness of the results are presented. Several of the examples study convergence rates for Markov chains.

Geometric renewal convergence rates from hazard rates

This paper studies the geometric convergence rate of a discrete renewal sequence to its limit. A general convergence rate is first derived from the hazard rates of the renewal lifetimes. This result is used to extract a good convergence rate when the lifetimes are ordered in the sense of new better than used or increasing hazard rate. A bound for the best possible geometric convergence rate is derived for lifetimes having a finite support. Examples demonstrating the utility and sharpness of the results are presented. Several of the examples study convergence rates for Markov chains.

Bounding convergence rates for Markov chains: An example of the use of computer algebra

Kolassa and Tanner (J. Am. Stat. Assoc. (1994) 89, 697–702) present the Gibbs-Skovgaard algorithm for approximate conditional inference. Kolassa (Ann Statist. (1999), 27, 129–142) gives conditions under which their Markov chain is known to converge. This paper calculates explicity bounds on convergence rates in terms calculable directly from chain transition operators. These results are useful in cases like those considered by Kolassa (1999).

Explicit bounds for geometric convergence of Markov chains

This paper presents bounds on convergence rates of Markov chains in terms of quantities calculable directly from chain transition operators. Bounds are constructed by creating a probability distribution that minorizes the transition kernel over some region, and by examining bounds on an expectation conditional on lying within and without this region. These are shown to be sharper in most cases than previous similar results. These bounds are applied to a Markov chain useful in frequentist conditional inference in canonical generalized linear models.

Explicit bounds for geometric convergence of Markov chains

This paper presents bounds on convergence rates of Markov chains in terms of quantities calculable directly from chain transition operators. Bounds are constructed by creating a probability distribution that minorizes the transition kernel over some region, and by examining bounds on an expectation conditional on lying within and without this region. These are shown to be sharper in most cases than previous similar results. These bounds are applied to a Markov chain useful in frequentist conditional inference in canonical generalized linear models.

Applications of geometric bounds to the convergence rate of Markov chains on [formula omitted

Quantitative geometric rates of convergence for reversible Markov chains are closely related to the spectral gap of the corresponding operator, which is hard to calculate for general state spaces. This article describes a geometric argumen t to give different types of bounds for spectral gaps of Markov chains on bounded subsets of R n and to compare the rates of convergence of different Markov chains.

Sensitive equilibria for ergodic stochastic games with countable state spaces

We consider stochastic games with countable state spaces and unbounded immediate payoff functions. Our assumptions on the transition structure of the game are based on a recent work by Meyn and Tweedie [19] on computable bounds for geometric convergence rates of Markov chains. The main results in this paper concern the existence of sensitive optimal strategies in some classes of zero-sum stochastic games. By sensitive optimality we mean overtaking or 1-optimality. We also provide a new Nash equilibrium theorem for a class of ergodic nonzero-sum stochastic games with denumerable state spaces.