Reversible Markov Chain Research Articles

Multivariate moment least-squares variance estimators for reversible Markov chains

Markov chain Monte Carlo (MCMC) is a commonly used method for approximating expectations with respect to probability distributions. Uncertainty assessment for MCMC estimators is essential in practical applications. Moreover, for multivariate functions of a Markov chain, it is important to estimate not only the auto-correlation for each component but also to estimate cross-correlations, in order to better assess sample quality, improve estimates of effective sample size, and use more effective stopping rules. Berg and Song (2023) introduced the moment least squares (momentLS) estimator, a shape-constrained estimator for the autocovariance sequence from a reversible Markov chain, for univariate functions of the Markov chain. Based on this sequence estimator, they proposed an estimator of the asymptotic variance of the sample mean from MCMC samples. In this study, we propose novel autocovariance sequence and asymptotic variance estimators for Markov chain functions with multiple components, based on the univariate momentLS estimators from Berg and Song (2023). We demonstrate strong consistency of the proposed auto(cross)-covariance sequence and asymptotic variance matrix estimators. We conduct empirical comparisons of our method with other state-of-the-art approaches on simulated and real-data examples, using popular samplers including the random-walk Metropolis sampler and the No-U-Turn sampler from STAN. Supplemental materials for this article are available online.

A necessary and sufficient condition for double coset lumping of Markov chains on groups with an application to the random to top shuffle

Let Q Q be a probability measure on a finite group G G , and let H H be a subgroup of G G . We show that a necessary and sufficient condition for the random walk driven by Q Q on G G to induce a Markov chain on the double coset space H ∖ G / H H\backslash G/H is that Q ( g H ) Q(gH) is constant as g g ranges over any double coset of H H in G G . We obtain this result as a corollary of a more general theorem on the double cosets H ∖ G / K H \backslash G / K for K K an arbitrary subgroup of G G . As an application we study a variation on the r r -top to random shuffle which we show induces an irreducible, recurrent, reversible and ergodic Markov chain on the double cosets of S y m r × S y m n − r \mathrm {Sym}_r \times \mathrm {Sym}_{n-r} in S y m n \mathrm {Sym}_n . The transition matrix of the induced walk has remarkable spectral properties: we find its invariant distribution and its eigenvalues and hence determine its rate of convergence.

Identifying the generator matrix of a stationary Markov chain using partially observable data.

Given that most states in real-world systems are inaccessible, it is critical to study the inverse problem of an irreversibly stationary Markov chain regarding how a generator matrix can be identified using minimal observations. The hitting-time distribution of an irreversibly stationary Markov chain is first generalized from a reversible case. The hitting-time distribution is then decoded via the taboo rate, and the results show remarkably that under mild conditions, the generator matrix of a reversible Markov chain or a specific case of irreversibly stationary ones can be identified by utilizing observations from all leaves and two adjacent states in each cycle. Several algorithms are proposed for calculating the generator matrix accurately, and numerical examples are presented to confirm their validity and efficiency. An application to neurophysiology is provided to demonstrate the applicability of such statistics to real-world data. This means that partially observable data can be used to identify the generator matrix of a stationary Markov chain.

Spectral clustering of Markov chain transition matrices with complex eigenvalues

The Robust Perron Cluster Analysis (PCCA+) has become a popular spectral clustering algorithm for coarse-graining transition matrices of nearly decomposable Markov chains with transition states. Originally developed for reversible Markov chains, the algorithm only worked for transition matrices with real eigenvalues. In this paper, we therefore extend the theoretical framework of PCCA+ to Markov chains with a complex eigen-decomposition. We show that by replacing a complex conjugate pair of eigenvectors by their real and imaginary components, a real representation of the same subspace is obtained, which is suitable for the cluster analysis. We show that our approach leads to the same results as the generalized PCCA+ (GPCCA), which replaces the complex eigen-decomposition by a conceptually more difficult real Schur decomposition. We apply the method on non-reversible Markov chains, including circular chains, and demonstrate its efficiency compared to GPCCA. The experiments are performed in the Matlab programming language and codes are provided.

Mixing reversible Markov chains in the max-ℓ2-distance

Mixing reversible Markov chains in the max-ℓ2-distance

Efficient shape-constrained inference for the autocovariance sequence from a reversible Markov chain

Efficient shape-constrained inference for the autocovariance sequence from a reversible Markov chain

Mixing times and hitting times for general Markov processes

The hitting and mixing times are two fundamental quantities associated with Markov chains. In Peres and Sousi [PS15] and Oliveira [Oli12], the authors show that the mixing times and “worst-case” hitting times of reversible Markov chains on finite state spaces are equal up to some universal multiplicative constant. We use tools from nonstandard analysis to extend this result to reversible Markov chains on general state spaces that satisfy the strong Feller property. Finally, we show that this asymptotic equivalence can be used to find bounds on the mixing times of a large class of Markov chains used in MCMC, such as typical Gibbs samplers and Metropolis–Hastings chains, even though they usually do not satisfy the strong Feller property.

Improvement of quantum walks search algorithm in single-marked vertex graph

Quantum walks are powerful tools for building quantum search algorithms. However, these success probabilities are far below 1. Amplitude amplification is usually used to amplify success probability, but the soufflé problem follows. Only stop at the right step can we achieve a maximum success probability. Otherwise, as the number of steps increases, the success probability may decrease, which will cause troubles in the practical application of the algorithm when the optimal number of steps is unknown. In this work, we define generalized interpolated quantum walks instead of amplitude amplification, which can not only improve the success probability but also avoid the soufflé problem. We choose a special case of generalized interpolated quantum walks and construct a series of new search algorithms based on phase estimation and quantum fast-forwarding, respectively. Especially, by combining our interpolated quantum walks with quantum fast-forwarding, we both reduce the time complexity of the search algorithm from to and reduce the number of ancilla qubits required from to , where ɛ denotes the precision and denotes the classical hitting time. In addition, we show that our generalized interpolated quantum walks can improve the construction of quantum stationary states corresponding to reversible Markov chains. Finally, we give an application to construct a slowly evolving Markov chain sequence by applying generalized interpolated quantum walks, which is the necessary premise in adiabatic stationary state preparation.

Upgrading MLSI to LSI for reversible Markov chains

For reversible Markov chains on finite state spaces, we show that the modified log-Sobolev inequality (MLSI) can be upgraded to a log-Sobolev inequality (LSI) at the surprisingly low cost of degrading the associated constant by log⁡(1/p), where p is the minimum non-zero transition probability. We illustrate this by providing the first log-Sobolev estimate for Zero-Range processes on arbitrary graphs. As another application, we determine the modified log-Sobolev constant of the Lamplighter chain on all bounded-degree graphs, and use it to provide negative answers to two open questions by Montenegro and Tetali (2006) [27] and Hermon and Peres (2018) [17]. Our proof builds upon the ‘regularization trick’ recently introduced by the last two authors.

Convergence of series of conditional expectations

This paper deals with almost sure convergence for partial sums of Banach space valued random variables. The results are then applied to solve similar problems for weighted partial sums of conditional expectations. They are further used to treat partial sums of powers of a reversible Markov chain operator. The method of proof is based on martingale approximation. The conditions are expressed in terms of moments of individual summands.

Convergence of spherical averages for actions of Fuchsian groups

We prove pointwise convergence of spherical averages for a measure-preserving action of a Fuchsian group. The proof is based on a new variant of the Bowen–Series symbolic coding for Fuchsian groups that, developing a method introduced by Wroten, simultaneously encodes all possible shortest paths representing a given group element. The resulting coding is self-inverse, giving a reversible Markov chain to which methods previously introduced by the first author for the case of free groups may be applied.

Modified log-Sobolev inequalities for strong-Rayleigh measures

We establish universal modified log-Sobolev inequalities for reversible Markov chains on the boolean lattice $\{0,1\}^n$, under the only assumption that the invariant law $\pi$ satisfies a form of negative dependence known as the stochastic covering property. This condition is strictly weaker than the strong Rayleigh property, and is satisfied in particular by all determinantal measures, as well as any product measure over the set of bases of a balanced matroid. In the special case where $\pi$ is $k-$homogeneous, our results imply the celebrated concentration inequality for Lipschitz functions due to Pemantle & Peres (2014). As another application, we deduce that the natural Monte-Carlo Markov Chain used to sample from $\pi$ has mixing time at most $kn\log\log\frac{1}{\pi(x)}$ when initialized in state $x$. To the best of our knowledge, this is the first work relating negative dependence and modified log-Sobolev inequalities.

Faster quantum sampling of Markov chains in nonregular graphs with fewer qubits

Sampling from the stationary distribution is one of the fundamental tasks of Markov chain-based algorithms and has important applications in machine learning, combinatorial optimization and network science. For the quantum case, qsampling from Markov chains can be constructed as preparing quantum states with amplitudes arbitrarily close to the square root of a stationary distribution instead of classical sampling from a stationary distribution. In this paper, a new qsampling algorithm for all reversible Markov chains is constructed by discrete-time quantum walks and works without any limit compared with existing results. In detail, we build a qsampling algorithm that not only accelerates non-regular graphs but also keeps the speed-up of existing quantum algorithms for regular graphs. In non-regular graphs, the invocation of the quantum fast-forward algorithm accelerates existing state-of-the-art qsampling algorithms for both discrete-time and continuous-time cases, especially on sparse graphs. Compared to existing algorithms we reduce log n, where n is the number of graph vertices. In regular graphs, our result matches other quantum algorithms, and our reliance on the gap of Markov chains achieves quadratic speedup compared with classical cases. For both cases, we reduce the number of ancilla qubits required compared to the existing results. In some widely used graphs and a series of sparse graphs where stationary distributions are difficult to reach quickly, our algorithm is the first algorithm to achieve complete quadratic acceleration (without log factor) over the classical case without any limit. To enlarge success probability amplitude amplification is introduced. We construct a new reflection on stationary state with fewer ancilla qubits and think it may have independent application.

Some inequalities for reversible Markov chains and branching random walks via spectral optimization

Nous présentons des résultats qui relient les temps de mélange au temps d’intersection d’une marche aléatoire branchante (BRW) dans laquelle le logarithme du nombre moyen de particules croît au taux du trou spectral gap. Il s’agit d’un analogue en espace d’état fini d’un processus de branchement critique. Plus précisément, nous montrons que le maximum de l’espérance du temps d’atteinte d’un état par une telle BRW est à une constante universelle près, plus grand que le temps de mélange L∞, alors que sous transitivité le même résultat est vrai pour le temps d’intersection de deux telles BRW indépendantes. En utilisant la même méthodologie, nous montrons que pour une suite de chaînes de Markov réversibles, les temps de mélange L∞ tmix(∞) sont d’ordre inférieur aux temps d’atteinte maximaux thit si et seulement si le produit du trou spectral et de thit diverge, en établissant l’inégalité tmix(∞)≤1gaplog(ethit·gap). Ceci résout une conjecture d’Aldous et Fill “Reversible Markov chains and random walks on graphs” Open Problem 14.12 affirmant que sous transitivité la condition thit≫1gap implique un comportement de champ moyen pour le temps de coalescence des marches aléatoires coalescentes.

A combinatorial proof of Aldous–Broder theorem for general Markov chains

AbstractAldous–Broder algorithm is a famous algorithm used to sample a uniform spanning tree of any finite connected graph , but it is more general: given an irreducible and reversible Markov chain on started at , the tree rooted at formed by the first entrance steps in each node (different from the root) has a probability proportional to , where the edges are directed toward . In this article we give proofs of Aldous–Broder theorem in the general case, where the kernel is irreducible but not assumed to be reversible (this generalized version appeared recently in Hu, Lyons, and Tang [5]). We provide two new proofs: an adaptation of the classical argument, which is purely probabilistic, and a new proof based on combinatorial arguments. On the way we introduce a new combinatorial object that we call the golf sequences.

On multiple acceleration of reversible Markov chain

Reversible chains such as Gibbs sampler and Metropolis Hasting are popular in Markov chain Monte Carlo algorithms. However, it has been shown that they can be easily improved by adding an antisymmetric perturbation. Since the perturbed Markov chain is no longer reversible, adding another antisymmetric perturbation is not guaranteed to be better. Chen and Hwang (2013) proposed a way for multiple acceleration. However, there is a mistake in their proof, and the statement does not always hold. In this paper, we will first point out the mistake and show a counterexample. Then we will give a sufficient condition such that multiple acceleration is guaranteed.

On the Lyapunov Foster Criterion and Poincaré Inequality for Reversible Markov Chains

This paper presents an elementary proof of stochastic stability of a discrete-time reversible Markov chain starting from a Foster-Lyapunov drift condition. Besides its relative simplicity, there are two salient features of the proof: (i) it relies entirely on functional-analytic non-probabilistic arguments; and (ii) it makes explicit the connection between a Foster-Lyapunov function and Poincare inequality. The proof is used to derive an explicit bound for the spectral gap. An extension to the non-reversible case is also presented.

Networks with Complex Weights: Green Function and Power Series

We introduce a Green function and analogues of other related kernels for finite and infinite networks whose edge weights are complex-valued admittances with positive real part. We provide comparison results with the same kernels associated with corresponding reversible Markov chains, i.e., where the edge weights are positive. Under suitable conditions, these lead to comparison of series of matrix powers which express those kernels. We show that the notions of transience and recurrence extend by analytic continuation to the complex-weighted case even when the network is infinite. Thus, a variety of methods known for Markov chains extend to that setting.

Direction-sweep Markov chains

We discuss a non-reversible, lifted Markov-chain Monte Carlo (MCMC) algorithm for particle systems in which the direction of proposed displacements is changed deterministically. This algorithm sweeps through directions analogously to the popular MCMC sweep methods for particle or spin indices. Direction-sweep MCMC can be applied to a wide range of reversible or non-reversible Markov chains, such as the Metropolis algorithm or the event-chain Monte Carlo algorithm. For a single two-dimensional tethered hard-disk dipole, we consider direction-sweep MCMC in the limit where restricted equilibrium is reached among the accessible configurations for a fixed direction before incrementing it. We show rigorously that direction-sweep MCMC leaves the stationary probability distribution unchanged and that it profoundly modifies the Markov-chain trajectory. Long excursions, with persistent rotation in one direction, alternate with long sequences of rapid zigzags resulting in persistent rotation in the opposite direction in the limit of small direction increments. The mapping to a Langevin equation then yields the exact scaling of excursions while the zigzags are described through a non-linear differential equation that is solved exactly. We show that the direction-sweep algorithm can have shorter mixing times than the algorithms with random updates of directions. We point out possible applications of direction-sweep MCMC in polymer physics and in molecular simulation.

Search Graph Magnification in Rapid Mixing of Markov Chains Associated with the Local Search-Based Metaheuristics

The structural property of the search graph plays an important role in the success of local search-based metaheuristic algorithms. Magnification is one of the structural properties of the search graph. This study builds the relationship between the magnification of a search graph and the mixing time of Markov Chain (MC) induced by the local search-based metaheuristics on that search space. The result shows that the ergodic reversible Markov chain induced by the local search-based metaheuristics is inversely proportional to magnification. This result indicates that it is desirable to use a search space with large magnification for the optimization problem in hand rather than using any search spaces. The performance of local search-based metaheuristics may be good on such search spaces since the mixing time of the underlying Markov chain is inversely proportional to the magnification of search space. Using these relations, this work shows that MC induced by the Metropolis Algorithm (MA) mixes rapidly if the search graph has a large magnification. This indicates that for any combinatorial optimization problem, the Markov chains associated with the MA mix rapidly i.e., in polynomial time if the underlying search graph has large magnification. The usefulness of the obtained results is illustrated using the 0/1-Knapsack Problem, which is a well-studied combinatorial optimization problem in the literature and is NP-Complete. Using the theoretical results obtained, this work shows that Markov Chains (MCs) associated with the local search-based metaheuristics like random walk and MA for 0/1-Knapsack Problem mixes rapidly.