Jump Markov Processes Research Articles

Model-free inference of memory in conformational dynamics of a multi-domain protein

Abstract Single-molecule experiments provide insight into the motion (conformational dynamics) of individual protein molecules. Usually, a well-defined but coarse-grained intramolecular coordinate is measured and subsequently analysed with the help of hidden Markov models to deduce the kinetics of protein conformational changes. Such approaches rely on the assumption that the microscopic dynamics of the protein evolve according to a Markov-jump process on some network. However, the manifestation and extent of memory in the dynamics of the observable strongly depends on the chosen underlying Markov model, which is generally not known and therefore can lead to misinterpretations. Here, we combine extensive single-molecule plasmon ruler experiments on the heat shock protein Hsp90, computer simulations, and theory to infer and quantify memory in a model-free fashion. Our analysis is based on the bare definition of non-Markovian behaviour and does not require any underlying model. In the case of Hsp90 probed by a plasmon ruler, the Markov assumption is found to be clearly and conclusively violated on timescales up to roughly 50 s, which corresponds roughly to ∼50% of the inferred correlation time of the signal. The extent of memory is striking and reaches biologically relevant timescales. This implies that memory effects penetrate even the slowest observed motions. We provide clear and reproducible guidelines on how to test for the presence and duration of memory in experimental single-molecule data.

Modeling Length Changes in De Novo Open Reading Frames during Neutral Evolution.

For protein coding genes to emerge de novo from a non-genic DNA, the DNA sequence must gain an open reading frame (ORF) and the ability to be transcribed. The newborn de novo gene can further evolve to accumulate changes in its sequence. Consequently, it can also elongate or shrink with time. Existing literature shows that older de novo genes have longer ORF, but it is not clear if they elongated with time or remained of the same length since their inception. To address this question we developed a mathematical model of ORF elongation as a Markov-jump process, and show that ORFs tend to keep their length in short evolutionary timescales. We also show that if change occurs it is likely to be a truncation. Our genomics and transcriptomics data analyses of seven Drosophila melanogaster populations are also in agreement with the model's prediction. We conclude that selection could facilitate ORF length extension that may explain why longer ORFs were observed in old de novo genes in studies analysing longer evolutionary time scales. Alternatively, shorter ORFs may be purged because they may be less likely to yield functional proteins.

Strongly Convergent Homogeneous Approximations to Inhomogeneous Markov Jump Processes and Applications

The study of time-inhomogeneous Markov jump processes is a traditional topic within probability theory that has recently attracted substantial attention in various applications. However, their flexibility also incurs a substantial mathematical burden which is usually circumvented by using well-known generic distributional approximations or simulations. This article provides a novel approximation method that tailors the dynamics of a time-homogeneous Markov jump process to meet those of its time-inhomogeneous counterpart on an increasingly fine Poisson grid. Strong convergence of the processes in terms of the Skorokhod J1 metric is established, and convergence rates are provided. Under traditional regularity assumptions, distributional convergence is established for unconditional proxies, to the same limit. Special attention is devoted to the case where the target process has one absorbing state and the remaining ones transient, for which the absorption times also converge. Some applications are outlined, such as univariate hazard-rate density estimation, ruin probabilities, and multivariate phase-type density evaluation. Funding: M. Bladt and O. Peralta would like to acknowledge financial support from the Swiss National Science Foundation Project 200021_191984. O. Peralta acknowledges financial support from NSF Award #1653354 and AXA Research Fund Award on “Mitigating risk in the wake of the pandemic”.

On Forward and Backward Kolmogorov Equations for Pure Jump Markov Processes and Their Generalizations

On Forward and Backward Kolmogorov Equations for Pure Jump Markov Processes and Their Generalizations

Nonequilibrium Response for Markov Jump Processes: Exact Results and Tight Bounds.

Generalizing response theory of open systems far from equilibrium is a central quest of nonequilibrium statistical physics. Using stochastic thermodynamics, we develop an algebraic method to study the static response of nonequilibrium steady state to arbitrary perturbations. This allows us to derive explicit expressions for the response of edge currents as well as traffic to perturbations in kinetic barriers and driving forces. We also show that these responses satisfy very simple bounds. For the response to energy perturbations, we straightforwardly recover results obtained using nontrivial graph-theoretical methods.

SIR model with social gatherings

AbstractWe introduce an extension to Kermack and McKendrick’s classic susceptible–infected–recovered (SIR) model in epidemiology, whose underlying mechanism of infection consists of individuals attending randomly generated social gatherings. This gives rise to a system of ordinary differential equations (ODEs) where the force of the infection term depends non-linearly on the proportion of infected individuals. Some specific instances yield models already studied in the literature, to which the present work provides a probabilistic foundation. The basic reproduction number is seen to depend quadratically on the average size of the gatherings, which may be helpful in understanding how restrictions on social gatherings affect the spread of the disease. We rigorously justify our model by showing that the system of ODEs is the mean-field limit of the jump Markov process corresponding to the evolution of the disease in a finite population.

Explicit expressions for stationary states of the Lindblad equation for a finite state space

The Lindblad equation describes the time evolution of a density matrix of a quantum mechanical system. Stationary solutions are obtained by time-averaging the solution, which will in general depend on the initial state. We provide an analytical expression for the steady states of the Lindblad equation using the quantum jump unraveling, a version of an ergodic theorem, and the stationary probabilities of the corresponding discrete-time Markov chains. Our result is valid when the number of states appearing the in quantum trajectory is finite. The classical case of a Markov jump-process is recovered as a special case, and differences between the two are discussed.

Large deviations for Markov jump processes in periodic and locally periodic environments

The paper deals with a family of jump Markov process defined in a medium with a periodic or locally periodic microstructure. We assume that the generator of the process is a zero order convolution type operator with rapidly oscillating locally periodic coefficient and, under natural ellipticity and localization conditions, show that the family satisfies the large deviation principle in the path space equipped with Skorokhod topology. The corresponding rate function is defined in terms of a family of auxiliary periodic spectral problems. It is shown that the corresponding Lagrangian is a convex function of velocity that has a superlinear growth at infinity. However, neither the Lagrangian nor the corresponding Hamiltonian need not be strictly convex, we only claim their strict convexity in some neighbourhood of infinity. It then depends on the profile of the generator kernel whether the Lagrangian is strictly convex everywhere or not.

Reflected Backward Stochastic Differential Equations Associated to Jump Markov Processes and Application to Partial Differential Equations

Reflected Backward Stochastic Differential Equations Associated to Jump Markov Processes and Application to Partial Differential Equations

A new sorting algorithm-based merging weighted fraction Monte Carlo method for solving the population balance equation for particle coagulation dynamics

PurposeThe purpose of this study is to present a newly proposed and developed sorting algorithm-based merging weighted fraction Monte Carlo (SAMWFMC) method for solving the population balance equation for the weighted fraction coagulation process in aerosol dynamics with high computational accuracy and efficiency.Design/methodology/approachIn the new SAMWFMC method, the jump Markov process is constructed as the weighted fraction Monte Carlo (WFMC) method (Jiang and Chan, 2021) with a fraction function. Both adjustable and constant fraction functions are used to validate the computational accuracy and efficiency. A new merging scheme is also proposed to ensure a constant-number and constant-volume scheme.FindingsThe new SAMWFMC method is fully validated by comparing with existing analytical solutions for six benchmark test cases. The numerical results obtained from the SAMWFMC method with both adjustable and constant fraction functions show excellent agreement with the analytical solutions and low stochastic errors. Compared with the WFMC method (Jiang and Chan, 2021), the SAMWFMC method can significantly reduce the stochastic error in the total particle number concentration without increasing the stochastic errors in high-order moments of the particle size distribution at only slightly higher computational cost.Originality/valueThe WFMC method (Jiang and Chan, 2021) has a stringent restriction on the fraction functions, making few fraction functions applicable to the WFMC method except for several specifically selected adjustable fraction functions, while the stochastic error in the total particle number concentration is considerably large. The newly developed SAMWFMC method shows significant improvement and advantage in dealing with weighted fraction coagulation process in aerosol dynamics and provides an excellent potential to deal with various fraction functions with higher computational accuracy and efficiency.

Heat kernel estimates for symmetric jump processes with anisotropic jumping kernels

We show two-sided bounds of heat kernel for symmetric pure jump Markov process in R d \mathbb {R}^d with jumping kernel J ( x , y ) J(x,y) that is comparable to \frac {\bbone _{\mathcal {V}}(x-y)}{|x-y|^{d+\alpha }}, where V \mathcal {V} is a union of symmetric cones and 0 > α > 2 0>\alpha >2 .

Exact Bayesian Inference for Discretely Observed Markov Jump Processes Using Finite Rate Matrices

ABSTRACT We present new methodologies for Bayesian inference on the rate parameters of a discretely observed continuous-time Markov jump process with a countably infinite statespace. The usual method of choice for inference, particle Markov chain Monte Carlo (particle MCMC), struggles when the observation noise is small. We consider the most challenging regime of exact observations and provide two new methodologies for inference in this case: the minimal extended statespace algorithm (MESA) and the nearly minimal extended statespace algorithm (nMESA). By extending the Markov chain Monte Carlo statespace, both MESA and nMESA use the exponentiation of finite rate matrices to perform exact Bayesian inference on the Markov jump process even though its statespace is countably infinite. Numerical experiments show improvements over particle MCMC of between a factor of three and several orders of magnitude. Supplementary materials for this article are available online.

Markov trajectories: Microcanonical Ensembles based on empirical observables as compared to Canonical Ensembles based on Markov generators

The Ensemble of trajectories $x(0 \leq t \leq T)$ produced by the Markov generator $M$ can be considered as 'Canonical' for the following reasons : (C1) the probability of the trajectory $x(0 \leq t \leq T)$ can be rewritten as the exponential of a linear combination of its relevant empirical time-averaged observables $E_n$, where the coefficients involving the Markov generator are their fixed conjugate parameters; (C2) the large deviations properties of these empirical observables $E_n$ for large $T$ are governed by the explicit rate function $I^{[2.5]}_M (E_.) $ at Level 2.5, while in the thermodynamic limit $T=+\infty$, they concentrate on their typical values $E_n^{typ[M]}$ determined by the Markov generator $M$. This concentration property in the thermodynamic limit $T=+\infty$ suggests to introduce the notion of the 'Microcanonical Ensemble' at Level 2.5 for stochastic trajectories $x(0 \leq t \leq T)$, where all the relevant empirical variables $E_n$ are fixed to some values $E^*_n$ and cannot fluctuate anymore for finite $T$. The goal of the present paper is to discuss its main properties : (MC1) when the long trajectory $x(0 \leq t \leq T) $ belongs the Microcanonical Ensemble with the fixed empirical observables $E_n^*$, the statistics of its subtrajectory $x(0 \leq t \leq \tau) $ for $1 \ll \tau \ll T $ is governed by the Canonical Ensemble associated to the Markov generator $M^*$ that would make the empirical observables $E_n^*$ typical ; (MC2) in the Microcanonical Ensemble, the central role is played by the number $\Omega^{[2.5]}_T(E^*_.) $ of stochastic trajectories of duration $T$ with the given empirical observables $E^*_n$, and by the corresponding explicit Boltzmann entropy $S^{[2.5]}( E^*_. ) = [\ln \Omega^{[2.5]}_T(E^*_.)]/T $. This general framework is applied to continuous-time Markov Jump processes and to discrete-time Markov chains with illustrative examples.

Large deviations for Markov jump processes with uniformly diminishing rates

We prove a large-deviation principle (ldp) for the sample paths of jump Markov processes in the small noise limit when, possibly, all the jump rates vanish uniformly, but slowly enough, in a region of the state space. We further discuss the optimality of our assumptions on the decay of the jump rates. As a direct application of this work we relax the assumptions needed for the application of ldps to, e.g., Chemical Reaction Network dynamics, where vanishing reaction rates arise naturally particularly the context of mass action kinetics.

Backward Stochastic Differential Equations Driven by a Jump Markov Process with Continuous and Non-Necessary Continuous Generators

We deal with backward stochastic differential equations driven by a pure jump Markov process and an independent Brownian motion (BSDEJs for short). We start by proving the existence and uniqueness of the solutions for this type of equation and present a comparison of the solutions in the case of Lipschitz conditions in the generator. With these tools in hand, we study the existence of a (minimal) solution for BSDE where the coefficient is continuous and satisfies the linear growth condition. An existence result for BSDE with a left-continuous, increasing and bounded generator is also discussed. Finally, the general result is applied to solve one kind of quadratic BSDEJ.

Locally Lipschitz BSDE with jumps and related Kolmogorov equation

We study a backward SDE driven by a jump Markov process (BSDEJ for short) whose generator may be locally Lipschitz or of logarithmic growth in [Formula: see text]-variables. The existence, uniqueness and stability theorems to such BSDEJs are established. We essentially approximate the initial problem by constructing a suitable sequence of BSDEJs with globally Lipschitz generators for which the existence and uniqueness of solutions hold. By passing to the limits, we show the existence and uniqueness of solutions to the original problems. We apply our main results to prove the existence of a unique solution to the Kolmogorov equation of the Markov process.

Averaging of semigroups associated to diffusion processes on a simplex

We study the averaging of a diffusion process living in a simplex K of Rn, n≥1. We assume that its infinitesimal generator can be decomposed as a sum of two generators corresponding to two distinct timescales and that the one corresponding to the fastest timescale is pure noise with a diffusion coefficient vanishing exactly on the vertices of K. We show that this diffusion process averages to a pure jump Markov process living on the vertices of K for the Meyer–Zheng topology. The role of the geometric assumptions done on K is also discussed.

Risk-sensitive discounted cost criterion for continuous-time Markov decision processes on a general state space

In this paper, we consider risk-sensitive discounted control problem for continuous-time jump Markov processes taking values in general state space. The transition rates of underlying continuous-time jump Markov processes and the cost rates are allowed to be unbounded. Under certain Lyapunov condition, we establish the existence and uniqueness of the solution to the Hamilton–Jacobi–Bellman equation. Also, we prove the existence of optimal risk-sensitive control in the class of Markov control and completely characterized the optimal control.

Exact targeting of gibbs distributions using velocity-jump processes

This work introduces and studies a new family of velocity jump Markov processes directly amenable to exact simulation with the following two properties: (i) trajectories converge in law, when a time-step parameter vanishes, towards a given Langevin or Hamiltonian dynamics; (ii) the stationary distribution of the process is always exactly given by the product of a Gaussian (for velocities) by any target log-density. The simulation itself, in addition to the computability of the gradient of the log-density, depends on the knowledge of appropriate explicit upper bounds on lower order derivatives of this log-density. The process does not exhibit any velocity reflections (maximum size of jumps can be controlled) and is suitable for the ’factorization method’. We provide rigorous mathematical proofs of the convergence towards Hamiltonian/Langevin dynamics when the time step vanishes, and of the exponentially fast convergence towards the target distribution when a suitable noise on velocities is present. Numerical implementation is detailed and illustrated.

Kolmogorov's Equations for Jump Markov Processes and Their Applications to Control Problems

Kolmogorov's Equations for Jump Markov Processes and Their Applications to Control Problems