Reversible Markov Processes Research Articles

Controlling Uncertainty of Empirical First-Passage Times in the Small-Sample Regime.

We derive general bounds on the probability that the empirical first-passage time τ[over ¯]_{n}≡∑_{i=1}^{n}τ_{i}/n of a reversible ergodic Markov process inferred from a sample of n independent realizations deviates from the true mean first-passage time by more than any given amount in either direction. We construct nonasymptotic confidence intervals that hold in the elusive small-sample regime and thus fill the gap between asymptotic methods and the Bayesian approach that is known to be sensitive to prior belief and tends to underestimate uncertainty in the small-sample setting. We prove sharp bounds on extreme first-passage times that control uncertainty even in cases where the mean alone does not sufficiently characterize the statistics. Our concentration-of-measure-based results allow for model-free error control and reliable error estimation in kinetic inference, and are thus important for the analysis of experimental and simulation data in the presence of limited sampling.

On the Kemeny time for continuous-time reversible and irreversible Markov processes with applications to stochastic resetting and to conditioning towards forever-survival

For continuous-time ergodic Markov processes, the Kemeny time is the characteristic time needed to converge towards the steady state : in real-space, the Kemeny time corresponds to the average of the mean-first-passage-time over the final configuration x drawn with the steady state , which turns out to be independent of the initial configuration x 0; in the spectral domain, the Kemeny time corresponds to the sum of the inverses of all the non-vanishing eigenvalues of the opposite generator. We describe many illustrative examples involving jumps and/or diffusion in one dimension, where the Kemeny time can be explicitly computed as a function of the system-size, via its real-space definition and/or via its spectral definition: we consider both reversible processes satisfying detailed-balance where the eigenvalues are real, and irreversible processes characterized by non-vanishing steady currents where the eigenvalues can be complex. In particular, we study the specific properties of the Kemeny times for Markov processes with stochastic resetting, and for absorbing Markov processes conditioned to survive forever.

Variational Principles for Asymptotic Variance of General Markov Processes

A variational formula for the asymptotic variance of general Markov processes is obtained. As application, we get an upper bound of the mean exit time of reversible Markov processes, and some comparison theorems between the reversible and non-reversible diffusion processes.

Collisions of random walks in dynamic random environments

We study dynamic random conductance models on Z2 in which the environment evolves as a reversible Markov process that is stationary under space-time shifts. We prove under a second moment assumption that two conditionally independent random walks in the same environment collide infinitely often almost surely. These results apply in particular to random walks on dynamical percolation.

Duality for a class of continuous-time reversible Markov models

ABSTRACT Using a conditional probability structure we build transition probabilities that drive appealing classes of reversible Markov processes. The mechanism used in such a construction allows to find a dual Markov process. This kind of duality is then used to compute the predictor operator of one process via its dual. In particular, we identify the dual of some non-conjugate models, namely the queue model and a simple birth, death and immigration process. Such duals ensure that the computation of the predictor operators can be done via finite sums.

A Multi-Service Model of Resources With the Neighboring Choice of Allocation Units

This article discusses a discretized model of resources to which a mixture of multi-service traffic streams is offered. In the model the connections of individual traffic classes that require a large number of allocation units are always executed using adjacent allocation units. The necessity of occupying the adjacent allocation units is typical, for example, for elastic optical networks, in which these units are called frequency slots. The article proposes a new occupancy distribution specifically designed for these systems. The new occupancy distribution is based on an approximation of the service process by a reversible Markov process. Particular attention is given to the method for calculating the so-called conditional transition probability of the transitions between neighboring states in the Markov process. The analytical results of the modeling are compared with the results of simulations of corresponding network systems.

Spectral gaps for reversible Markov processes with chaotic invariant measures: The Kac process with hard sphere collisions in three dimensions

We develop a method for producing estimates on the spectral gaps of reversible Markov jump processes with chaotic invariant measures, that is effective in the case of degenerate jump rates, and we apply it to prove the Kac conjecture for hard sphere collision in three dimensions.

Continuous-time Markov processes, orthogonal polynomials and Lancaster probabilities

This work links the conditional probability structure of Lancaster probabilities to a construction of reversible continuous-time Markov processes. Such a task is achieved by using the spectral expansion of the corresponding transition probabilities in order to introduce a continuous time dependence in the orthogonal representation inherent to Lancaster probabilities. This relationship provides a novel methodology to build continuous-time Markov processesviaLancaster probabilities. Particular cases of well-known models are seen to fall within this approach. As a byproduct, it also unveils new identities associated to well known orthogonal polynomials.

Convergence Time to Equilibrium of the Metropolis Dynamics for the GREM

We study the convergence time to equilibrium of the Metropolis dynamics for the Generalized Random Energy Model with an arbitrary number of hierarchical levels, a finite and reversible continuous-time Markov process, in terms of the spectral gap of its transition probability matrix. This is done by deducing bounds to the inverse of the gap using a Poincar\'e inequality and a path technique. We also apply convex analysis tools to give the bounds in the most general case of the model.

The convergence rate of the Gibbs sampler for generalized 1-D Ising model

The rate of convergence of the Gibbs sampler for the generalized one-dimensional Ising model is determined by the second largest eigenvalue of its transition matrix in absolute value denoted by β∗. In this paper we generalize a bound for β∗ from Shiu and Chen (2015) for the one-dimensional Ising model with two states to a multiple state situation. The method is based on Diaconis and Stroock bound for reversible Markov processes. The new bound presented in this paper improves Ingrassia’s (1994) result.

Extreme value statistics of ergodic Markov processes from first passage times in the large deviation limit

Extreme value functionals of stochastic processes are inverse functionals of the first passage time—a connection that renders their probability distribution functions equivalent. Here, we deepen this link and establish a framework for analyzing extreme value statistics of ergodic reversible Markov processes in confining potentials on the hand of the underlying relaxation eigenspectra. We derive a chain of inequalities, which bounds the long-time asymptotics of first passage densities, and thereby extrema, from above and from below. The bounds involve a time integral of the transition probability density describing the relaxation towards equilibrium. We apply our general results to the analysis of extreme value statistics at long times in the case of Ornstein–Uhlenbeck process and a 3D Brownian motion confined to a sphere, also known as Bessel process. We find that even on time-scales that are shorter than the equilibration time, the large deviation limit characterizing long-time asymptotics can approximate the statistics of extreme values remarkably well. Our findings provide a novel perspective on the study of extrema beyond the established limit theorems for sequences of independent random variables and for asymmetric diffusion processes beyond a constant drift.

Realizations and Factorizations of Positive Definite Kernels

Given a fixed sigma-finite measure space $$\left( X,\mathscr {B},\nu \right) $$ , we shall study an associated family of positive definite kernels K. Their factorizations will be studied with view to their role as covariance kernels of a variety of stochastic processes. In the interesting cases, the given measure $$\nu $$ is infinite, but sigma-finite. We introduce such positive definite kernels $$K\left( \cdot ,\cdot \right) $$ with the two variables from the subclass of the sigma-algebra $$\mathscr {B}$$ whose elements are sets with finite $$\nu $$ measure. Our setting and results are motivated by applications. The latter are covered in the second half of the paper. We first make precise the notions of realizations and factorizations for K, and we give necessary and sufficient conditions for K to have realizations and factorizations in $$L^{2}\left( \nu \right) $$ . Tools in the proofs rely on probability theory and on spectral theory for unbounded operators in Hilbert space. Applications discussed here include the study of reversible Markov processes, and realizations of Gaussian fields, and their Ito-integrals.

Duality between relaxation and first passage in reversible Markov dynamics: rugged energy landscapes disentangled

Relaxation and first passage processes are the pillars of kinetics in condensed matter, polymeric and single-molecule systems. Yet, an explicit connection between relaxation and first passage time-scales so far remained elusive. Here we prove a duality between them in the form of an interlacing of spectra. In the basic form the duality holds for reversible Markov processes to effectively one-dimensional targets. The exploration of a triple-well potential is analyzed to demonstrate how the duality allows for an intuitive understanding of first passage trajectories in terms of relaxational eigenmodes. More generally, we provide a comprehensive explanation of the full statistics of reactive trajectories in rugged potentials, incl. the so-called ‘few-encounter limit’. Our results are required for explaining quantitatively the occurrence of diseases triggered by protein misfolding.

About the entropic structure of detailed balanced multi-species cross-diffusion equations

This paper links at the formal level the entropy structure of a multi-species cross-diffusion system of Shigesada–Kawasaki–Teramoto (SKT) type (cf. [1]) satisfying the detailed balance condition with the entropy structure of a reversible microscopic many-particle Markov process on a discretised space. The link is established by first performing a mean-field limit to a master equation over discretised space. Then the spatial discretisation limit is performed in a completely rigorous way. This by itself provides a novel strategy for proving global existence of weak solutions to a class of cross-diffusion systems.

Infinite weighted graphs with bounded resistance metric

We consider infinite weighted graphs $G$, i.e., sets of vertices $V$, and edges $E$ assumed countably infinite. An assignment of weights is a positive symmetric function $c$ on $E$ (the edge-set), conductance. From this, one naturally defines a reversible Markov process, and a corresponding Laplace operator acting on functions on $V$, voltage distributions. The harmonic functions are of special importance. We establish explicit boundary representations for the harmonic functions on $G$ of finite energy.We compute a resistance metric $d$ from a given conductance function. (The resistance distance $d(x,y)$ between two vertices $x$ and $y$ is the voltage drop from $x$ to $y$, which is induced by the given assignment of resistors when $1$ amp is inserted at the vertex $x$, and then extracted again at $y$.)We study the class of models where this resistance metric is bounded. We show that then the finite-energy functions form an algebra of ${1}/{2}$-Lipschitz-continuous and bounded functions on $V$, relative to the metric $d$. We further show that, in this case, the metric completion $M$ of $(V,d)$ is automatically compact, and that the vertex-set $V$ is open in $M$. We obtain a Poisson boundary-representation for the harmonic functions of finite energy, and an interpolation formula for every function on $V$ of finite energy. We further compare $M$ to other compactifications; e.g., to certain path-space models.

The Bouncy Particle Sampler: A Nonreversible Rejection-Free Markov Chain Monte Carlo Method

ABSTRACTMany Markov chain Monte Carlo techniques currently available rely on discrete-time reversible Markov processes whose transition kernels are variations of the Metropolis–Hastings algorithm. We explore and generalize an alternative scheme recently introduced in the physics literature (Peters and de With 2012) where the target distribution is explored using a continuous-time nonreversible piecewise-deterministic Markov process. In the Metropolis–Hastings algorithm, a trial move to a region of lower target density, equivalently of higher “energy,” than the current state can be rejected with positive probability. In this alternative approach, a particle moves along straight lines around the space and, when facing a high energy barrier, it is not rejected but its path is modified by bouncing against this barrier. By reformulating this algorithm using inhomogeneous Poisson processes, we exploit standard sampling techniques to simulate exactly this Markov process in a wide range of scenarios of interest. Additionally, when the target distribution is given by a product of factors dependent only on subsets of the state variables, such as the posterior distribution associated with a probabilistic graphical model, this method can be modified to take advantage of this structure by allowing computationally cheaper “local” bounces, which only involve the state variables associated with a factor, while the other state variables keep on evolving. In this context, by leveraging techniques from chemical kinetics, we propose several computationally efficient implementations. Experimentally, this new class of Markov chain Monte Carlo schemes compares favorably to state-of-the-art methods on various Bayesian inference tasks, including for high-dimensional models and large datasets. Supplementary materials for this article are available online.

Selection originating from protein stability/foldability: Relationships between protein folding free energy, sequence ensemble, and fitness

Assuming that mutation and fixation processes are reversible Markov processes, we prove that the equilibrium ensemble of sequences obeys a Boltzmann distribution with exp(4Nem(1−1/(2N))), where m is Malthusian fitness and Ne and N are effective and actual population sizes. On the other hand, the probability distribution of sequences with maximum entropy that satisfies a given amino acid composition at each site and a given pairwise amino acid frequency at each site pair is a Boltzmann distribution with exp(−ψN), where the evolutionary statistical energy ψN is represented as the sum of one body (h) (compositional) and pairwise (J) (covariational) interactions over all sites and site pairs. A protein folding theory based on the random energy model (REM) indicates that the equilibrium ensemble of natural protein sequences is well represented by a canonical ensemble characterized by exp(−ΔGND/kBTs) or by exp(−GN/kBTs) if an amino acid composition is kept constant, where ΔGND≡GN−GD,GN and GD are the native and denatured free energies, and Ts is the effective temperature representing the strength of selection pressure. Thus, 4Nem(1−1/(2N)),−ΔψND(≡−ψN+ψD), and −ΔGND/kBTs must be equivalent to each other. With h and J estimated by the DCA program, the changes (ΔψN) of ψN due to single nucleotide nonsynonymous substitutions are analyzed. The results indicate that the standard deviation of ΔGN(=kBTsΔψN) is approximately constant irrespective of protein families, and therefore can be used to estimate the relative value of Ts. Glass transition temperature Tg and ΔGND are estimated from estimated Ts and experimental melting temperature (Tm) for 14 protein domains. The estimates of ΔGND agree with their experimental values for 5 proteins, and those of Ts and Tg are all within a reasonable range. In addition, approximating the probability density function (PDF) of ΔψN by a log-normal distribution, PDFs of ΔψN and Ka/Ks, which is the ratio of nonsynonymous to synonymous substitution rate per site, in all and in fixed mutants are estimated. The equilibrium values of ψN, at which the average of Δψ in fixed mutants is equal to zero, well match ψN averaged over homologous sequences, confirming that the present methods for a fixation process of mutations and for the equilibrium ensemble of ψN give a consistent result with each other. The PDFs of Ka/Ks at equilibrium confirm that Ts negatively correlates with the amino acid substitution rate (the mean of Ka/Ks) of protein. Interestingly, stabilizing mutations are significantly fixed by positive selection, and balance with destabilizing mutations fixed by random drift, although most of them are removed from population. Supporting the nearly neutral theory, neutral selection is not significant even in fixed mutants.

Remarks on limit theorems for reversible Markov processes and their applications

We propose some backward–forward martingale decompositions for functions of reversible Markov chains. These decompositions are used to prove the functional Central limit theorem for reversible Markov chains with asymptotically linear variance of partial sums. We also provide a proof of the equivalence between asymptotic linearity of the variance and convergence of the integral of 1/(1−t) with respect to the associated spectral measure ρ. We show a result on uniform integrability of the supremum of the average sum of squares of martingale differences. We also study the asymptotic behavior of linear processes having as innovations mean zero square integrable functions of stationary reversible Markov chains. We include in our study the long-range dependence case. We apply this study to several cases of reversible stationary Markov chains that arise in regression estimation.

Logarithmic Sobolev inequalities in non-commutative algebras

We study the relations between (tight) logarithmic Sobolev inequalities, entropy decay and spectral gap inequalities for Markov evolutions on von Neumann algebras. We prove that log-Sobolev inequalities (in the non-commutative form defined by Olkiewicz and Zegarlinski in Ref. 25) imply spectral gap inequalities, with optimal relation between the constants. Furthermore, we show that a uniform exponential decay of a proper relative entropy is equivalent to a modified version of log-Sobolev inequalities. The relations among the mentioned inequalities are investigated and often depend on some regularity conditions, which are also discussed. With regard to this aspect, we provide an example of a positive identity-preserving semigroup not verifying the usually requested regularity conditions (which are always fulfilled for reversible classical Markov processes).

Asymptotic variance of stationary reversible and normal Markov processes

We obtain necessary and sufficient conditions for the regularvariation of the variance of partial sums of functionals of discrete and continuous-time stationary Markov processes with normal transition operators. We also construct a class of Metropolis-Hastings algorithms which satisfy a central limit theorem and invariance principle when the variance is not linear in n.