On the Convergence of Reversible Markov Chains

Abstract

A simple upper bound for the second-largest eigenvalue of a finite reversible time-homogeneous Markov chain is presented as a function of the transition probabilities, the equilibrium distribution, and the underlying structure of the chain. This work is extended to a relation between the second-largest eigenvalues of any two reversible Markov chains with the same underlying structure. Furthermore, a lower bound for the smallest eigenvalue of a reversible chain is also presented, thereby providing a bound on the spectral gap of such chains. These eigenvalue bounds are fairly easy to compute for a variety of reversible chains by using known results on eigenvalues of certain matrices associated with graphs or random walks on graphs. The results on the spectral gap lead to a bound on the time constant of a reversible Markov chain converging to its equilibrium distribution. As an application, the temperature asymptotics of simulated annealing, which is a probabilistic algorithm widely used for solving combinat...