Markov Generator Research Articles

Markov generators as non-Hermitian supersymmetric quantum Hamiltonians: spectral properties via bi-orthogonal basis and singular value decompositions

Continuity equations associated with continuous-time Markov processes can be considered as Euclidean Schrödinger equations, where the non-Hermitian quantum Hamiltonian H=divJ is naturally factorized into the product of the divergence operator div and the current operator J . For non-equilibrium Markov jump processes in a space of N configurations with M links and C=M−(N−1)⩾1 independent cycles, this factorization of the N × N Hamiltonian H=I†J involves the incidence matrix I and the current matrix J of size M × N, so that the supersymmetric partner H^=JI† governing the dynamics of the currents existing on the M links is of size M × M. To better understand the relations between the spectral decompositions of these two Hamiltonians H=I†J and H^=JI† with respect to their bi-orthogonal basis of right and left eigenvectors that characterize the relaxation dynamics towards the steady state and the steady currents, it is useful to analyze the properties of the singular value decomposition of the two rectangular matrices I and J of size M × N and the interpretations in terms of discrete Helmholtz decompositions. This general framework concerning Markov jump processes can be adapted to non-equilibrium diffusion processes governed by Fokker–Planck equations in dimension d, where the number N of configurations, the number M of links and the number C=M−(N−1) of independent cycles become infinite, while the two matrices I and J become first-order differential operators acting on scalar functions to produce vector fields.

A supersymmetric quantum perspective on the explicit large deviations for reversible Markov jump processes, with applications to pure and random spin chains

The large deviations at various levels that are explicit for Markov jump processes satisfying detailed balance are revisited in terms of the supersymmetric quantum Hamiltonian H that can be obtained from the Markov generator via a similarity transformation. We first focus on the large deviations at level 2 for the empirical density p^(C) of the configurations C seen during a trajectory over the large time window [0,T] , and rewrite the explicit Donsker–Varadhan rate function as the matrix element I[2][p^(.)]=⟨p^|H|p^⟩ involving the square-root ket |p^⟩ . (The analog formula is also discussed for reversible diffusion processes as a comparison.) We then consider the explicit rate functions at higher levels, in particular for the joint probability of the empirical density p^(C) and the empirical local activities a^(C,C′) characterizing the density of jumps between two configurations (C,C′) . Finally, the explicit rate function for the joint probability of the empirical density p^(C) and of the empirical total activity A^ that represents the total density of jumps of a long trajectory is written in terms of the two matrix elements ⟨p^|H|p^⟩ and ⟨p^|Hoff|p^⟩ , where Hoff represents the off-diagonal part of the supersymmetric Hamiltonian H. This general framework is then applied to pure or random spin chains with single-spin-flip or two-spin-flip transition rates, where the supersymmetric Hamiltonian H corresponds to quantum spin chains with local interactions involving Pauli matrices of two or three neighboring sites. It is then useful to introduce the quantum density matrix ρ^=|p^⟩⟨p^| associated with the empirical density p^(.) in order to rewrite the various rate functions in terms of reduced density matrices involving only two or three neighboring sites.

Coercive inequalities on Carnot groups: taming singularities

In the setting of Carnot groups, we propose an approach of taming singularities to get coercive inequalities. To this end, we develop a technique to introduce natural singularities in the energy function U in order to force one of the coercivity conditions. In particular, we explore explicit constructions of probability measures on Carnot groups which secure Poincaré and even Logarithmic Sobolev inequalities. As applications, we get analogues of the Dyson–Ornstein–Uhlenbeck model on the Heisenberg group and obtain results on the discreteness of the spectrum of related Markov generators.

Local Detailed Balance Invariant States of Markov Generators of Low Density Limit

We introduce the notion of local detailed balance for low density limit generators of quantum Markov semigroups and explore the consequences of this condition on the structure of the corresponding set of invariant states. The local detailed balance condition allows us to include the case of degenerate reference Hamiltonian and due to this degeneracy, we can compute nonequilibrium invariant states by combining the local detailed balance with generic, quasi-generic and [Formula: see text]-generic conditions, obtaining LDL Markov generators with pure invariant states.

Inverse problem in the conditioning of Markov processes on trajectory observables: what canonical conditionings can connect two given Markov generators?

In the field of large deviations for stochastic dynamics, the canonical conditioning of a given Markov process with respect to a given time-local trajectory observable over a large time-window has attracted a lot of interest recently. In the present paper, we analyze the following inverse problem: when two Markov generators are given, is it possible to connect them via some canonical conditioning and to construct the corresponding time-local trajectory observable? We focus on continuous-time Markov processes and obtain the following necessary and sufficient conditions: (i) for continuous-time Markov jump processes, the two generators should involve the same possible elementary jumps in configuration space, i.e. only the values of the corresponding rates can differ; (ii) for diffusion processes, the two Fokker–Planck generators should involve the same diffusion coefficients, i.e. only the two forces can differ. In both settings, we then construct explicitly the various time-local trajectory observables that can be used to connect the two given generators via canonical conditioning. This general framework is illustrated with various applications involving a single particle or many-body spin models. In particular, we describe several examples to show how non-equilibrium Markov processes with non-vanishing steady currents can be interpreted as the canonical conditionings of detailed-balance processes with respect to explicit time-local trajectory observables.

Improved Bound of Four Moment Theorem and Its Application to Orthogonal Polynomials Associated with Laws

In the case where the square of an eigenfunction F with respect to an eigenvalue of Markov generator L can be expressed as a sum of eigenfunctions, we find the largest number excluding zero among the eigenvalues in the terms of the sum. Using this number, we obtain an improved bound of the fourth moment theorem for Markov diffusion generators. To see how this number depends on an improved bound, we give some examples of eigenfunctions of the diffusion generators L such as Ornstein–Uhlenbeck, Jacobi, and Romanovski–Routh.

Revisiting boundary-driven non-equilibrium Markov dynamics in arbitrary potentials via supersymmetric quantum mechanics and via explicit large deviations at various levels

For boundary-driven non-equilibrium Markov models of non-interacting particles in one dimension, either in continuous space with the Fokker–Planck dynamics involving an arbitrary force F(x) and an arbitrary diffusion coefficient D(x), or in discrete space with the Markov jump dynamics involving arbitrary nearest-neighbor transition rates , the Markov generator can be transformed via an appropriate similarity transformation into a quantum supersymmetric Hamiltonian with many remarkable properties. We first describe how the mapping from the boundary-driven non-equilibrium dynamics towards some dual equilibrium dynamics (see Tailleur et al 2008 J. Phys. A: Math. Theor. 41 505001) can be reinterpreted via the two corresponding quantum Hamiltonians that are supersymmetric partners of each other, with the same energy spectra. We describe the consequences for the spectral decomposition of the boundary-driven dynamics, and we give explicit expressions for the Kemeny times needed to converge towards the non-equilibrium steady states. We then focus on the large deviations at various levels for empirical time-averaged observables over a large time-window T. We start with the always explicit Level 2.5 concerning the joint distribution of the empirical density and of the empirical flows before considering the contractions towards lower levels. In particular, the rate function for the empirical current alone can be explicitly computed via the contraction from the Level 2.5 using the properties of the associated quantum supersymmetric Hamiltonians.

The embedding problem for Markov matrices

Characterizing whether a Markov process of discrete random variables has an homogeneous continuous-time realization is a hard problem. In practice, this problem reduces to deciding when a given Markov matrix can be written as the exponential of some rate matrix (a Markov generator). This is an old question known in the literature as the embedding problem (Elfving37), which has been only solved for matrices of size $2\times 2$ or $3\times 3$. In this paper, we address this problem and related questions and obtain results in two different lines. First, for matrices of any size, we give a bound on the number of Markov generators in terms of the spectrum of the Markov matrix. Based on this, we establish a criterion for deciding whether a generic Markov matrix (different eigenvalues) is embeddable and propose an algorithm that lists all its Markov generators. Then, motivated and inspired by recent results on substitution models of DNA, we focus in the $4\times 4$ case and completely solve the embedding problem for any Markov matrix. The solution in this case is more concise as the embeddability is given in terms of a single condition.

Feynman-Kac formula under a finite entropy condition

Motivated by entropic optimal transport, we investigate an extended notion of solution to the parabolic equation $( \partial_t + b\cdot \nabla + \Delta _{ a}/2 +V)g =0$ with a final boundary condition. It is well-known that the viscosity solution $g$ of this PDE is represented by the Feynman-Kac formula when the drift $b$, the diffusion matrix $a$ and the scalar potential $V$ are regular enough and not growing too fast. In this article, $b$ and $V$ are not assumed to be regular and their growth is controlled by a finite entropy condition, allowing for instance $V$ to belong to some Kato class. We show that the Feynman-Kac formula represents a solution, in an extended sense, to the parabolic equation. This notion of solution is trajectorial and expressed with the semimartingale extension of the Markov generator $ b\cdot \nabla + \Delta _{ a}/2.$ Our probabilistic approach relies on stochastic derivatives, semimartingales, Girsanov's theorem and the Hamilton-Jacobi-Bellman equation satisfied by $\log g$.

Markov trajectories: Microcanonical Ensembles based on empirical observables as compared to Canonical Ensembles based on Markov generators

The Ensemble of trajectories $x(0 \leq t \leq T)$ produced by the Markov generator $M$ can be considered as 'Canonical' for the following reasons : (C1) the probability of the trajectory $x(0 \leq t \leq T)$ can be rewritten as the exponential of a linear combination of its relevant empirical time-averaged observables $E_n$, where the coefficients involving the Markov generator are their fixed conjugate parameters; (C2) the large deviations properties of these empirical observables $E_n$ for large $T$ are governed by the explicit rate function $I^{[2.5]}_M (E_.) $ at Level 2.5, while in the thermodynamic limit $T=+\infty$, they concentrate on their typical values $E_n^{typ[M]}$ determined by the Markov generator $M$. This concentration property in the thermodynamic limit $T=+\infty$ suggests to introduce the notion of the 'Microcanonical Ensemble' at Level 2.5 for stochastic trajectories $x(0 \leq t \leq T)$, where all the relevant empirical variables $E_n$ are fixed to some values $E^*_n$ and cannot fluctuate anymore for finite $T$. The goal of the present paper is to discuss its main properties : (MC1) when the long trajectory $x(0 \leq t \leq T) $ belongs the Microcanonical Ensemble with the fixed empirical observables $E_n^*$, the statistics of its subtrajectory $x(0 \leq t \leq \tau) $ for $1 \ll \tau \ll T $ is governed by the Canonical Ensemble associated to the Markov generator $M^*$ that would make the empirical observables $E_n^*$ typical ; (MC2) in the Microcanonical Ensemble, the central role is played by the number $\Omega^{[2.5]}_T(E^*_.) $ of stochastic trajectories of duration $T$ with the given empirical observables $E^*_n$, and by the corresponding explicit Boltzmann entropy $S^{[2.5]}( E^*_. ) = [\ln \Omega^{[2.5]}_T(E^*_.)]/T $. This general framework is applied to continuous-time Markov Jump processes and to discrete-time Markov chains with illustrative examples.

Finite Time Large Deviations via Matrix Product States.

Recent work has shown the effectiveness of tensor network methods for computing large deviation functions in constrained stochastic models in the infinite time limit. Here we show that these methods can also be used to study the statistics of dynamical observables at arbitrary finite time. This is a harder problem because, in contrast to the infinite time case, where only the extremal eigenstate of a tilted Markov generator is relevant, for finite time the whole spectrum plays a role. We show that finite time dynamical partition sums can be computed efficiently and accurately in one dimension using matrix product states and describe how to use such results to generate rare event trajectories on demand. We apply our methods to the Fredrickson-Andersen and East kinetically constrained models and to the symmetric simple exclusion process, unveiling dynamical phase diagrams in terms of counting field and trajectory time. We also discuss extensions of this method to higher dimensions.

Optimal sampling of dynamical large deviations via matrix product states.

The large deviation statistics of dynamical observables is encoded in the spectral properties of deformed Markov generators. Recent works have shown that tensor network methods are well suited to compute accurately the relevant leading eigenvalues and eigenvectors. However, the efficient generation of the corresponding rare trajectories is a harder task. Here, we show how to exploit the matrix product state approximation of the dominant eigenvector to implement an efficient sampling scheme which closely resembles the optimal (so-called "Doob") dynamics that realizes the rare events. We demonstrate our approach on three well-studied lattice models, the Fredrickson-Andersen and East kinetically constrained models, and the symmetric simple exclusion process. We discuss how to generalize our approach to higher dimensions.

An open set of 4×4 embeddable matrices whose principal logarithm is not a Markov generator

ABSTRACT A Markov matrix is embeddable if it can represent a homogeneous continuous-time Markov process. It is well known that if a Markov matrix has real and pairwise-different eigenvalues, then the embeddability can be determined by checking whether its principal logarithm is a rate matrix or not. The same holds for Markov matrices that are close enough to the identity matrix. In this paper we exhibit open sets of Markov matrices that are embeddable and whose principal logarithm is not a rate matrix, thus proving that the principal logarithm test above does not suffice generically.

Breaking of the Similarity Principle in Markov Generators of Low Density Limit Type and the Role of Degeneracies in the Landscape of Invariant States

The similarity principle is an extension of the principle of thermal relaxation that naturally arises in the stochastic limit of quantum theory. We construct examples of Low Density Limit (LDL) generators, associated to an environment state in equilibrium at inverse temperature β, which admit non-(β, HS)-equilibrium states. We prove that in some cases, the attraction domain of the (β, HS)-equilibrium state is empty. This means that the similarity principle, in its original thermodynamical formulation, can be broken in the LDL limit. This result is obtained as a consequence of a more general phenomenon: the role of degeneracies in the spectrum of the Liouvillian of the system Hamiltonian associated to the generator. We start from the definition of LDL type generators given in [5] and we introduce a finer classification of these generators based on the above degeneracies. The simplest subclass, called 2-generic, is a nontrivial extension of the generators associated to the so-called Λ and V configurations, widely used in quantum optics and involving 2 levels of the system Hamiltonian. Since each 2-generic block involves 3 or 4 levels of the system Hamiltonian we expect that they can reveal some interesting new physical phenomenon, as it happened in the 2-level case. In the last section, we restrict our attention to a 3-level system with a Hamiltonian that is associated to a class of 2-generic LDL generators. Finally, we prove that, for some LDL generators in this class the statement formulated at the beginning holds true.

Matrix Poincaré inequalities and concentration

We show that any probability measure satisfying a Matrix Poincaré inequality with respect to some reversible Markov generator satisfies an exponential matrix concentration inequality depending on the associated matrix carré du champ operator. This extends to the matrix setting a classical phenomenon in the scalar case. Moreover, the proof gives rise to new matrix trace inequalities which could be of independent interest. We then apply this general fact by establishing matrix Poincaré inequalities to derive matrix concentration inequalities for Gaussian measures, product measures and for Strong Rayleigh measures. The latter represents the first instance of matrix concentration for general matrix functions of negatively dependent random variables.

Estimation of state-dependent jump activity and drift for Markovian semimartingales

The jump behavior of an infinitely active Itô semimartingale can be conveniently characterized by a jump activity index of Blumenthal–Getoor type, typically assumed to be constant in time. We study Markovian semimartingales with a non-constant, state-dependent jump activity index and a non-vanishing continuous diffusion component. A nonparametric estimator for the functional jump activity index is proposed and shown to be asymptotically normal under combined high-frequency and long-time-span asymptotics. Furthermore, we propose a nonparametric drift estimator which is robust to symmetric jumps of infinite variance and infinite variation, and which attains the same asymptotic variance as for a continuous diffusion process. Simulations demonstrate the finite sample behavior of our proposed estimators. The mathematical results are based on a novel uniform bound on the Markov generator of the jump diffusion.

Eigenstates of triangularisable open XXX spin chains and closed-form solutions for the steady state of the open SSEP

In this article we study the relation between the eigenstates of open rational spin Heisenberg chains with different boundary conditions. The focus lies on the relation between the spin chain with diagonal boundary conditions and the spin chain with triangular boundary conditions as well as the class of spin chains that can be brought to such form by certain similarity transformations in the physical space. The boundary driven symmetric simple exclusion process (open SSEP) belongs to the latter. We derive a transformation that maps the eigenvectors of the diagonal spin chain to the eigenvectors of the triangular chain. This transformation yields an essential simplification for determining the states beyond half-filling. It allows to first determine the eigenstates of the diagonal chain through the Bethe ansatz on the fully excited reference state and subsequently map them to the triangular chain for which only the vacuum serves as a reference state. In particular the transformed reference state, i.e. the fully excited eigenstate of the triangular chain, is presented at any length of the chain. It can be mapped to the steady state of the open SSEP. This results in a concise closed-form expression for the probabilities of particle distributions and correlation functions in the steady state. Further, the complete set of eigenstates of the Markov generator is expressed in terms of the eigenstates of the diagonal open chain.

Stationary states of weak coupling limit-type Markov generators and quantum transport models

We prove that every stationary state in the annihilator of all Kraus operators of a weak coupling limit-type Markov generator consists of two pieces, one of them supported on the interaction-free subspace and the second one on its orthogonal complement. In particular, we apply the previous result to describe in detail the structure of a slightly modified quantum transport model due to Arefeva et al. (modified AKV’s model) studied first in [J. C. García et al., Entangled and dark stationary states of excitation energy transport models in many-particles systems and photosynthesis, Infin. Dimens. Anal. Quantum Probab. Relat. Top. 21(3) (2018), Article ID: 1850018, p. 21, doi:10.1142/S0219025718500182], in terms of generalized annihilation and creation operators.

Quantum Markov Semigroups of Low Density Limit: the Generic Case

We investigate the structure of quantum Markov generators that describe the reduced dynamics of a test particle interacting with a dilute Bose gas in the low density limit. These generators, called low density limit type generators (LDL), differ from the weak coupling limit type generators (WCL) studied in [4, 5] because of the presence of the T-operator that describes the change in momentum of the gas particle due to collisions with the test particle. We propose a general definition of Markov generator of stochastic limit type that includes (almost) all generators arising in the stochastic limit approach and we prove that the associated semigroups as well as their invariant states (or weights) have a very special structure which extends the explicit representation for the generic quantum Markov semigroups of weak coupling limit type, due to Accardi, Hachicha and Ouerdiane [7]. We also investigate the structure of invariant states and give conditions on the T - operator for the existence of a unique invariant state and of equilibrium states.

Maximum principles for nonlocal parabolic Waldenfels operators

As a class of Lévy type Markov generators, nonlocal Waldenfels operators appear naturally in the context of investigating stochastic dynamics under Lévy fluctuations and constructing Markov processes with boundary conditions (in particular the construction with jumps). This work is devoted to prove the weak and strong maximum principles for ‘parabolic’ equations with nonlocal Waldenfels operators. Applications in stochastic differential equations with [Formula: see text]-stable Lévy processes are presented to illustrate the maximum principles.