Piecewise Deterministic Markov Processes Research Articles

Global dynamical behavior of a multigroup SVIR epidemic model with Markovian switching

In this paper, we are concerned with the global dynamical behavior of a multigroup SVIR epidemic model, which is formulated as a piecewise-deterministic Markov process. We first obtain sufficient criteria for extinction of the diseases. Then we establish sufficient criteria for persistence in the mean of the diseases. Moreover, in the case of persistence, we find a domain which is positive recurrence for the solution of the stochastic system by constructing an appropriate Lyapunov function with regime switching.

Stationary distributions of persistent ecological systems.

We analyze ecological systems that are influenced by random environmental fluctuations. We first provide general conditions which ensure that the species coexist and the system converges to a unique invariant probability measure (stationary distribution). Since it is usually impossible to characterize this invariant probability measure analytically, we develop a powerful method for numerically approximating invariant probability measures. This allows us to shed light upon how the various parameters of the ecosystem impact the stationary distribution. We analyze different types of environmental fluctuations. At first we study ecosystems modeled by stochastic differential equations. In the second setting we look at piecewise deterministic Markov processes. These are processes where one follows a system of differential equations for a random time, after which the environmental state changes, and one follows a different set of differential equations-this procedure then gets repeated indefinitely. Finally, we look at stochastic differential equations with switching, which take into account both the white noise fluctuations and the random environmental switches. As applications of our theoretical and numerical analysis, we look at competitive Lotka-Volterra, Beddington-DeAngelis predator-prey, and rock-paper-scissors dynamics. We highlight new biological insights by analyzing the stationary distributions of the ecosystems and by seeing how various types of environmental fluctuations influence the long term fate of populations.

Push-forward method for piecewise deterministic biochemical simulations

A biochemical network can be simulated by a set of ordinary differential equations (ODE) under well-stirred reactor conditions, for large numbers of molecules, and frequent reactions. This is no longer a robust representation when some molecular species are in small numbers and reactions changing them are infrequent. In this case, discrete stochastic events trigger changes of the smooth deterministic dynamics of the biochemical network. Piecewise-deterministic Markov processes (PDMP) are well adapted for describing such situations. Although PDMP models are now well established in biology, these models remain computationally challenging. Previously we have introduced the push-forward method to compute how the probability measure is spread by the deterministic ODE flow of PDMPs, through the use of analytic expressions of the corresponding semigroup. In this paper we provide a more general simulation algorithm that works also for non-integrable systems. The method can be used for biochemical simulations with applications in fundamental biology, biotechnology and biocomputing. This work is an extended version of the work presented at the conference CMSB2019.

Gradient-Based Markov Chain Monte Carlo for Bayesian Inference With Non-differentiable Priors

The use of nondifferentiable priors in Bayesian statistics has become increasingly popular, in particular in Bayesian imaging analysis. Current state-of-the-art methods are approximate in the sense that they replace the posterior with a smooth approximation via Moreau-Yosida envelopes, and apply gradient-based discretized diffusions to sample from the resulting distribution. We characterize the error of the Moreau-Yosida approximation and propose a novel implementation using underdamped Langevin dynamics. In misson-critical cases, however, replacing the posterior with an approximation may not be a viable option. Instead, we show that piecewise-deterministic Markov processes (PDMP) can be used for exact posterior inference from distributions satisfying almost everywhere differentiability. Furthermore, in contrast with diffusion-based methods, the suggested PDMP-based samplers place no assumptions on the prior shape, nor require access to a computationally cheap proximal operator, and consequently have a much broader scope of application. Through detailed numerical examples, including a nondifferentiable circular distribution and a nonconvex genomics model, we elucidate the relative strengths of these sampling methods on problems of moderate to high dimensions, underlining the benefits of PDMP-based methods when accurate sampling is decisive. Supplementary materials for this article are available online.

Extreme first passage times of piecewise deterministic Markov processes

The time it takes the fastest searcher out of N ≫ 1 searchers to find a target determines the timescale of many physical, chemical, and biological processes. This time is called an extreme first passage time (FPT) and is typically much faster than the FPT of a single searcher. Extreme FPTs of diffusion have been studied for decades, but little is known for other types of stochastic processes. In this paper, we study the distribution of extreme FPTs of piecewise deterministic Markov processes (PDMPs). PDMPs are a broad class of stochastic processes that evolve deterministically between random events. Using classical extreme value theory, we prove general theorems which yield the distribution and moments of extreme FPTs in the limit of many searchers based on the short time distribution of the FPT of a single searcher. We then apply these theorems to some canonical PDMPs, including run and tumble searchers in one, two, and three space dimensions. We discuss our results in the context of some biological systems and show how our approach accounts for an unphysical property of diffusion which can be problematic for extreme statistics.

Asymptotics and renewal approximation in the online selection of increasing subsequence

We revisit the problem of maximising the expected length of increasing subsequence that can be selected from a marked Poisson process by an online strategy. Resorting to a natural size variable, we represent the problem in terms of a controlled piecewise deterministic Markov process with decreasing paths. We apply a comparison method to the optimality equation to obtain fairly complete asymptotic expansions for the moments of the maximal length, and, with the aid of a renewal approximation, give a novel proof to the central limit theorem for the length of selected subsequence under either the optimal strategy or a strategy sufficiently close to optimality.

Stochastic Gene Expression Revisited.

We investigate the model of gene expression in the form of Iterated Function System (IFS), where the probability of choice of any iterated map depends on the state of the phase space. Random jump times of the process mark activation periods of the gene when pre-mRNA molecules are produced before mRNA and protein processing phases occur. The main idea is inspired by the continuous-time piecewise deterministic Markov process describing stochastic gene expression. We show that for our system there exists a unique invariant limit measure. We provide full probabilistic description of the process with a comparison of our results to those obtained for the model with continuous time.

A piecewise deterministic Monte Carlo method for diffusion bridges

We introduce the use of the Zig-Zag sampler to the problem of sampling conditional diffusion processes (diffusion bridges). The Zig-Zag sampler is a rejection-free sampling scheme based on a non-reversible continuous piecewise deterministic Markov process. Similar to the Lévy–Ciesielski construction of a Brownian motion, we expand the diffusion path in a truncated Faber–Schauder basis. The coefficients within the basis are sampled using a Zig-Zag sampler. A key innovation is the use of the fully local algorithm for the Zig-Zag sampler that allows to exploit the sparsity structure implied by the dependency graph of the coefficients and by the subsampling technique to reduce the complexity of the algorithm. We illustrate the performance of the proposed methods in a number of examples.

Dynamic Reliability Assessment of Multi-cracked Structure under Fatigue Loading via Multi-State Physics Model

The structure strength abruptly decreases due to the propagation of multiple cracks. The crack interaction renders the multiple crack reliability problem being a challenging one. Most of the existing methods cannot fit the whole multiple crack growth process and quantify the crack linkup effects on reliability estimation. To address this problem, we extend the multi-state physics modelling and computation framework to evaluate the dynamic reliability of multi-cracked structure via implementing piecewise deterministic Markov processes (PDMP). The proposed multi-state physics model (MSPM) incorporates the crack propagation state transitions and the Forman model that describes the crack growth rate within the states. For the former, the probability of the linkup of adjacent cracks is calculated to identify the multiple crack systems and define the discrete states of each system. For the latter, the Forman model is employed to calculate the crack growth rate within the states. Therefore, each multiple crack system in the structure can be quantified by PDMP and a Monte Carlo algorithm is presented to evaluate the corresponding dynamic reliability values. Ultimately, the dynamic reliability formulation of the multi-cracked structure is developed. The accuracy of the proposed approach is verified by using the sample data from the literature and simulation example.

Coherent spin states and stochastic hybrid path integrals

Stochastic hybrid systems involve a coupling between a discrete Markov chain and a continuous stochastic process. If the latter evolves deterministically between jumps in the discrete state, then the system reduces to a piecewise deterministic Markov process. Well known examples include stochastic gene expression, voltage fluctuations in neurons, and motor-driven intracellular transport. In this paper we use coherent spin states to construct a new path integral representation of the probability density functional for stochastic hybrid systems, which holds outside the weak noise regime. We use the path integral to derive a system of Langevin equations in the semi-classical limit, which extends previous diffusion approximations based on a quasi-steady-state reduction. We then show how in the weak noise limit the path integral is equivalent to an alternative representation that was previously derived using Doi–Peliti operators. The action functional of the latter is related to a large deviation principle for stochastic hybrid systems.

Piecewise deterministic Markov process for condition-based maintenance models — Application to critical infrastructures with discrete-state deterioration

In recent decades, the technology and techniques for condition monitoring have experienced a rapid development. However, there is still a need for reducing unnecessary inspections and/or preventive maintenance actions and their associated cost, through optimal design of condition-based maintenance (CBM) strategies. Accordingly, mathematical modelling and optimization of CBM has become of interest for industry managers and researchers. This work explores on the application of a piecewise deterministic Markov process (PDMP) to encompass different modelling assumptions as non-negligible maintenance delays and inspection-based condition monitoring. These assumptions are relevant for many critical infrastructures in civil engineering or in oil & gas industry whose deterioration states are classified at a very high level of abstraction among a finite and small set of possible states. A formalism to model this type of problems is proposed in which the deterministic motion of the PDMP is reduced to a trivial differential equation to track the time elapsed between events. A numerical scheme for quantification, as an approximation of the Chapman–Kolmogorov equation, is presented. Later, an illustration case dealing with CBM of road bridges by the NPRA (Norwegian Public Roads Administration) is presented, guiding through the modelling and quantification approach.

Large-scale inference of correlation among mixed-type biological traits with phylogenetic multivariate probit models

Inferring concerted changes among biological traits along an evolutionary history remains an important yet challenging problem. Besides adjusting for spurious correlation induced from the shared history, the task also requires sufficient flexibility and computational efficiency to incorporate multiple continuous and discrete traits as data size increases. To accomplish this, we jointly model mixed-type traits by assuming latent parameters for binary outcome dimensions at the tips of an unknown tree informed by molecular sequences. This gives rise to a phylogenetic multivariate probit model. With large sample sizes, posterior computation under this model is problematic, as it requires repeated sampling from a high-dimensional truncated normal distribution. Current best practices employ multiple-try rejection sampling that suffers from slow-mixing and a computational cost that scales quadratically in sample size. We develop a new inference approach that exploits: (1) the bouncy particle sampler (BPS) based on piecewise deterministic Markov processes to simultaneously sample all truncated normal dimensions, and (2) novel dynamic programming that reduces the cost of likelihood and gradient evaluations for BPS to linear in sample size. In an application with 535 HIV viruses and 24 traits that necessitates sampling from a 12,840-dimensional truncated normal, our method makes it possible to estimate the across-trait correlation and detect factors that affect the pathogen’s capacity to cause disease. This inference framework is also applicable to a broader class of covariance structures beyond comparative biology.

Fluid Approximation–based Analysis for Mode-switching Population Dynamics

Fluid approximation results provide powerful methods for scalable analysis of models of population dynamics with large numbers of discrete states and have seen wide-ranging applications in modelling biological and computer-based systems and model checking. However, the applicability of these methods relies on assumptions that are not easily met in a number of modelling scenarios. This article focuses on one particular class of scenarios in which rapid information propagation in the system is considered. In particular, we study the case where changes in population dynamics are induced by information about the environment being communicated between components of the population via broadcast communication. We see how existing hybrid fluid limit results, resulting in piecewise deterministic Markov processes, can be adapted to such models. Finally, we propose heuristic constructions for extracting the mean behaviour from the resulting approximations without the need to simulate individual trajectories.

Integro-differential optimality equations for the risk-sensitive control of piecewise deterministic Markov processes

In this paper we study the minimization problem of the infinite-horizon expected exponential utility total cost for continuous-time piecewise deterministic Markov processes with the control acting continuously on the jump intensity $$\lambda $$ and on the transition measure Q of the process. The action space is supposed to depend on the state variable and the state space is considered to have a frontier such that the process jumps whenever it touches this boundary. We characterize the optimal value function as the minimal solution of an integro-differential optimality equation satisfying some boundary conditions, as well as the existence of a deterministic stationary optimal policy. These results are obtained by using the so-called policy iteration algorithm, under some continuity and compactness assumptions on the parameters of the problem, as well as some non-explosive conditions for the process.

Cores for piecewise-deterministic Markov processes used in Markov chain Monte Carlo

We show fundamental properties of the Markov semigroup of recently proposed MCMC algorithms based on Piecewise-deterministic Markov processes (PDMPs) such as the Bouncy Particle Sampler, the Zig-Zag process or the Randomized Hamiltonian Monte Carlo method. Under assumptions typically satisfied in MCMC settings, we prove that PDMPs are Feller and that their generator admits the space of infinitely differentiable functions with compact support as a core. As we illustrate via martingale problems and a simplified proof of the invariance of target distributions, these results provide a fundamental tool for the rigorous analysis of these algorithms and corresponding stochastic processes.

Dynamics of a vector-host model under switching environments

<p style='text-indent:20px;'>In this paper, the stochastic vector-host model has been proposed and analysed using nice properties of piecewise deterministic Markov processes (PDMPs). A threshold for the stochastic model is derived whose sign determines whether the disease will eventually disappear or persist. We show mathematically the existence of scenarios where switching plays a significant role in surprisingly reversing the long-term properties of deterministic systems.</p>

A multiscale stochastic criminal behavior model under a hybrid scheme

<p style='text-indent:20px;'>Crime in urban environment is a major social problem nowadays. As such, many efforts have been made to develop mathematical models for this type of crime. The pioneering work [M. B. Short, M. R. D'Orsogna, V. B. Pasour, G. E. Tita, P. J. Brantingham, A. L. Bertozzi and L. B. Chayes, Math. Models Methods Appl. Sci., 18, (2008), pp. 1249-1267] establishes an agent-based human-environment interaction model of criminal behavior for residential burglary, where aggregate pattern formation of "hotspots" is quantitatively studied for the first time. Potential offenders are assumed to interact with environment according to well-known criminology and sociology notions. However long-term simulations for the coupled dynamics are computationally costly due to all components evolving on slow time scales. In this paper, we introduce a new-generation criminal behavior model with separated spatio-temporal scales for the agent actions and the environment parameter reactions. The computational cost is reduced significantly, while the essential stochastic features of the pioneering model are preserved. Moreover, the separation of scales brings the model into the theoretical framework of piecewise deterministic Markov processes (PDMP). A martingale approach is applicable which will be useful to analyze both stochastic and statistical features of the model in subsequent studies.</p>

Subgeometric hypocoercivity for piecewise-deterministic Markov process Monte Carlo methods

We extend the hypocoercivity framework for piecewise-deterministic Markov process (PDMP) Monte Carlo established in [Andrieu et. al. (2018)] to heavy-tailed target distributions, which exhibit subgeometric rates of convergence to equilibrium. We make use of weak Poincar\'e inequalities, as developed in the work of [Grothaus and Wang (2019)], the ideas of which we adapt to the PDMPs of interest. On the way we report largely potential-independent approaches to bounding explicitly solutions of the Poisson equation of the Langevin diffusion and its first and second derivatives, required here to control various terms arising in the application of the hypocoercivity result.

On path-independent Girsanov transform

This article provides a unified and more direct approach to path-independent Girsanov transformation for (possibly time-inhomogeneous) Markov processes and their corresponding parabolic partial differential equations. The approach is applicable to a large class of Markov processes, including diffusion processes determined by stochastic differential equations, and Markov processes with discontinuous sample paths. In particular, we apply it to reflected diffusions, diffusions with jumps and a class of piecewise deterministic Markov processes that are Markov. The approach of this paper is also applicable to a class of time-inhomogeneous Markov processes that have jumps only at integer times, where the path-independent Girsanov transform have to be given by discontinuous but piecewise differentiable functions. In the last section of this paper, we illustrate this by a simplified stochastic model from insurance mathematics.

Low-Lying Eigenvalues and Convergence to the Equilibrium of Some Piecewise Deterministic Markov Processes Generators in the Small Temperature Regime

In this work we study the number of small eigenvalues and the convergence to the equilibrium of the Bouncy Particle Sampler process and the Zig-Zag process generators in the small temperature regime. Such processes, which fall in the class of Piecewise Deterministic Markov Processes, are non diffusive and non reversible.