Piecewise Deterministic Markov Processes Research Articles

In silico modelling of CD8 T cell immune response links genetic regulation to population dynamics

The CD8 T cell immune response operates at multiple temporal and spatial scales, including all the early complex biochemical and biomechanical processes, up to long term cell population behavior.In order to model this response, we devised a multiscale agent-based approach using Simuscale software. Within each agent (cell) of our model, we introduced a gene regulatory network (GRN) based upon a piecewise deterministic Markov process formalism. Cell fate – differentiation, proliferation, death – was coupled to the state of the GRN through rule-based mechanisms. Cells interact in a 3D computational domain and signal to each other via cell–cell contacts, influencing the GRN behavior.Results show the ability of the model to correctly capture both population behavior and molecular time-dependent evolution. We examined the impact of several parameters on molecular and population dynamics, and demonstrated the add-on value of using a multiscale approach by showing the influence of molecular parameters, particularly protein degradation rates, on the outcome of the response, such as effector and memory cell counts.

Zigzag path connects two Monte Carlo samplers: Hamiltonian counterpart to a piecewise deterministic Markov process

Zigzag and other piecewise deterministic Markov process samplers have attracted significant interest for their non-reversibility and other appealing properties for Bayesian posterior computation. Hamiltonian Monte Carlo is another state-of-the-art sampler, exploiting fictitious momentum to guide Markov chains through complex target distributions. We establish an important connection between the zigzag sampler and a variant of Hamiltonian Monte Carlo based on Laplace-distributed momentum. The position and velocity component of the corresponding Hamiltonian dynamics travels along a zigzag path paralleling the Markovian zigzag process; however, the dynamics is non-Markovian in this position-velocity space as the momentum component encodes non-immediate pasts. This information is partially lost during a momentum refreshment step, in which we preserve its direction but re-sample magnitude. In the limit of increasingly frequent momentum refreshments, we prove that Hamiltonian zigzag converges strongly to its Markovian counterpart. This theoretical insight suggests that, when retaining full momentum information, Hamiltonian zigzag can better explore target distributions with highly correlated parameters by suppressing the diffusive behavior of Markovian zigzag. We corroborate this intuition by comparing performance of the two zigzag cousins on high-dimensional truncated multivariate Gaussians, including a 11,235-dimensional target arising from a Bayesian phylogenetic multivariate probit modeling of hiv virus data.

Multilevel Particle Filters for a Class of Partially Observed Piecewise Deterministic Markov Processes

Multilevel Particle Filters for a Class of Partially Observed Piecewise Deterministic Markov Processes

On the Existence and Uniqueness of Stationary Distributions for Some Piecewise Deterministic Markov Processes with State-Dependent Jump Intensity

In this paper, we consider a subclass of piecewise deterministic Markov processes with a Polish state space that involve a deterministic motion punctuated by random jumps, occurring in a Poisson-like fashion with some state-dependent rate, between which the trajectory is driven by one of the given semiflows. We prove that there is a one-to-one correspondence between stationary distributions of such processes and those of the Markov chains given by their post-jump locations. Using this result, we further establish a criterion guaranteeing the existence and uniqueness of the stationary distribution in a particular case, where the post-jump locations result from the action of a random iterated function system with an arbitrary set of transformations.

“PEOPLE” MEET “MARKOVIANS” — INDIVIDUAL-BASED MODELING WITH HYBRID STOCHASTIC SYSTEMS

Individual-based models (IBMs) enable modelers to avoid far-reaching abstractions and strong simplifications by allowing for a state-based representation of individuals. The fact that IBMs are not represented using a standardized mathematical framework such as differential equations makes it harder to reproduce IBMs and introduces difficulties in the analysis of IBMs. We propose a model architecture based on representing individuals via Markov models. Individuals are coupled to populations — for which individuals are not explicitly represented — that are modeled by differential equations. The resulting models consisting of continuous-time finite-state Markov models coupled to systems of differential equations are examples of piecewise-deterministic Markov processes (PDMPs). We will demonstrate that PDMPs, also known as hybrid stochastic systems, allow us to design detailed state-based representations of individuals which, at the same time, can be systematically analyzed by taking advantage of the theory of PDMPs. We will illustrate design and analysis of IBMs using PDMPs via the example of a predator that intermittently feeds on a logistically growing prey by stochastically switching between a resting and a feeding state. This simple model shows a surprisingly rich dynamics which, nevertheless, can be comprehensively analyzed using the theory of PDMPs.

Stochastic modeling of a gene regulatory network driving B cell development in germinal centers.

Germinal centers (GCs) are the key histological structures of the adaptive immune system, responsible for the development and selection of B cells producing high-affinity antibodies against antigens. Due to their level of complexity, unexpected malfunctioning may lead to a range of pathologies, including various malignant formations. One promising way to improve the understanding of malignant transformation is to study the underlying gene regulatory networks (GRNs) associated with cell development and differentiation. Evaluation and inference of the GRN structure from gene expression data is a challenging task in systems biology: recent achievements in single-cell (SC) transcriptomics allow the generation of SC gene expression data, which can be used to sharpen the knowledge on GRN structure. In order to understand whether a particular network of three key gene regulators (BCL6, IRF4, BLIMP1), influenced by two external stimuli signals (surface receptors BCR and CD40), is able to describe GC B cell differentiation, we used a stochastic model to fit SC transcriptomic data from a human lymphoid organ dataset. The model is defined mathematically as a piecewise-deterministic Markov process. We showed that after parameter tuning, the model qualitatively recapitulates mRNA distributions corresponding to GC and plasmablast stages of B cell differentiation. Thus, the model can assist in validating the GRN structure and, in the future, could lead to better understanding of the different types of dysfunction of the regulatory mechanisms.

PDMP Monte Carlo methods for piecewise smooth densities

Abstract There has been substantial interest in developing Markov chain Monte Carlo algorithms based on piecewise deterministic Markov processes. However, existing algorithms can only be used if the target distribution of interest is differentiable everywhere. The key to adapting these algorithms so that they can sample from densities with discontinuities is to define appropriate dynamics for the process when it hits a discontinuity. We present a simple condition for the transition of the process at a discontinuity which can be used to extend any existing sampler for smooth densities, and give specific choices for this transition which work with popular algorithms such as the bouncy particle sampler, the coordinate sampler, and the zigzag process. Our theoretical results extend and make rigorous arguments that have been presented previously, for instance constructing samplers for continuous densities restricted to a bounded domain, and we present a version of the zigzag process that can work in such a scenario. Our novel approach to deriving the invariant distribution of a piecewise deterministic Markov process with boundaries may be of independent interest.

Adaptive Importance Sampling Based on Fault Tree Analysis for Piecewise Deterministic Markov Process

Adaptive Importance Sampling Based on Fault Tree Analysis for Piecewise Deterministic Markov Process

Necessary and sufficient symmetries in Event-Chain Monte Carlo with generalized flows and application to hard dimers.

Event-Chain Monte Carlo (ECMC) methods generate continuous-time and non-reversible Markov processes, which often display significant accelerations compared to their reversible counterparts. However, their generalization to any system may appear less straightforward. In this work, our aim is to distinctly define the essential symmetries that such ECMC algorithms must adhere to, differentiating between necessary and sufficient conditions. This exploration intends to delineate the balance between requirements that could be overly limiting in broad applications and those that are fundamentally essential. To do so, we build on the recent analytical description of such methods as generating piecewise deterministic Markov processes. Therefore, starting with translational flows, we establish the necessary rotational invariance of the probability flows, along with determining the minimum event rate. This rate is identified with the corresponding infinitesimal Metropolis rejection rate. Obeying such conditions ensures the correct invariance for any ECMC scheme. Subsequently, we extend these findings to encompass schemes involving deterministic flows that are more general than mere translational ones. Specifically, we define two classes of interest of general flows: the ideal and uniform-ideal ones. They, respectively, suppress or reduce the event rates. From there, we implement a comprehensive non-reversible sampling of a system of hard dimers by introducing rotational flows, which are uniform-ideal. This implementation results in a speed-up of up to ∼3 compared to the state-of-the-art ECMC/Metropolis hybrid scheme.

NuZZ: Numerical Zig-Zag for general models

Markov chain Monte Carlo (MCMC) is a key algorithm in computational statistics, and as datasets grow larger and models grow more complex, many popular MCMC algorithms become too computationally expensive to be practical. Recent progress has been made on this problem through development of MCMC algorithms based on Piecewise Deterministic Markov Processes (PDMPs), irreversible processes which can be engineered to converge at a rate which is independent of the size of the dataset. While there has understandably been a surge of theoretical studies following these results, PDMPs have so far only been implemented for models where certain gradients can be bounded in closed form, which is not possible in many relevant statistical problems. Furthermore, there has been substantionally less focus on practical implementation, or the efficiency of PDMP dynamics in exploring challenging densities. Focusing on the Zig-Zag process, we present the Numerical Zig-Zag (NuZZ) algorithm, which is applicable to general statistical models without the need for bounds on the gradient of the log posterior. This allows us to perform numerical experiments on: (i) how the Zig-Zag dynamics behaves on some test problems with common challenging features; and (ii) how the error between the target and sampled distributions evolves as a function of computational effort for different MCMC algorithms including NuZZ. Moreover, due to the specifics of the NuZZ algorithms, we are able to give an explicit bound on the Wasserstein distance between the exact posterior and its numerically perturbed counterpart in terms of the user-specified numerical tolerances of NuZZ.

A decentralized algorithm for a mean field control problem of piecewise deterministic Markov processes

This paper provides a decentralized approach for the control of a population of N agents to minimize an aggregate cost. Each agent evolves independently according to a Piecewise Deterministic Markov dynamics controlled via unbounded jumps intensities. The N-agent high dimensional stochastic control problem is approximated by the limiting mean field control problem. A Lagrangian approach is proposed. Although the mean field control problem is not convex, it is proved to achieve zero duality gap. A stochastic version of the Uzawa algorithm is shown to converge to the primal solution. At each dual iteration of the algorithm, each agent solves its own small dimensional sub problem by means of the Dynamic Programming Principal, while the dual multiplier is updated according to the aggregate response of the agents. Finally, this algorithm is used in a numerical simulation to coordinate the charging of a large fleet of electric vehicles in order to track a target consumption profile.

The Central Limit Theorem for Markov Processes that are Exponentially Ergodic in the Bounded-Lipschitz Norm

In this paper, we establish a version of the central limit theorem for Markov–Feller continuous time processes (with a Polish state space) that are exponentially ergodic in the bounded-Lipschitz distance and enjoy a continuous form of the Foster–Lyapunov condition. As an example, we verify the assumptions of our main result for a specific piecewise-deterministic Markov process, whose deterministic component evolves according to continuous semiflows, switched randomly at the jump times of a Poisson process.

Infinite dimensional Piecewise Deterministic Markov Processes

In this paper we aim to construct infinite dimensional versions of well established Piecewise Deterministic Monte Carlo methods, such as the Bouncy Particle Sampler, the Zig-Zag Sampler and the Boomerang Sampler. In order to do so we provide an abstract infinite dimensional framework for Piecewise Deterministic Markov Processes (PDMPs) with unbounded event intensities. We further develop exponential convergence to equilibrium of the infinite dimensional Boomerang Sampler, using hypocoercivity techniques. Furthermore we establish how the infinite dimensional Boomerang Sampler admits a finite dimensional approximation, rendering it suitable for computer simulation.

Posterior Computation with the Gibbs Zig-Zag Sampler

An intriguing new class of piecewise deterministic Markov processes (PDMPs) has recently been proposed as an alternative to Markov chain Monte Carlo (MCMC). We propose a new class of PDMPs termed Gibbs zig-zag samplers, which allow parameters to be updated in blocks with a zig-zag sampler applied to certain parameters and traditional MCMC-style updates to others. We demonstrate the flexibility of this framework on posterior sampling for logistic models with shrinkage priors for high-dimensional regression and random effects, and provide conditions for geometric ergodicity and the validity of a central limit theorem.

Pricing of Credit Risk Derivatives with Stochastic Interest Rate

This paper deals with a credit derivative pricing problem using the martingale approach. We generalize the conventional reduced-form credit risk model for a credit default swap market, assuming that the firms’ default intensities depend on the default states of counterparty firms and that the stochastic interest rate follows a jump-diffusion Cox–Ingersoll–Ross process. First, we derive the joint Laplace transform of the distribution of the vector process (rt,Rt) by applying piecewise deterministic Markov process theory and martingale theory. Then, using the joint Laplace transform, we obtain the explicit pricing of defaultable bonds and a credit default swap. Lastly, numerical examples are presented to illustrate the dynamic relationships between defaultable securities (defaultable bonds, credit default swap) and the maturity date.

Exponential ergodicity for damping Hamiltonian dynamics with state-dependent and non-local collisions

In this paper, we investigate the exponential ergodicity in a Wasserstein-type distance for a damping Hamiltonian dynamics with state-dependent and non-local collisions, which indeed is a special case of piecewise deterministic Markov processes that is very popular in numerous modelling situations including stochastic algorithms. The approach adopted in this work is based on a combination of the refined basic coupling and the refined reflection coupling for non-local operators. In a certain sense, the main result developed in the present paper is a continuation of the counterpart in (Stochastic Process. Appl. (2022) 146 114–142) on exponential ergodicity of stochastic Hamiltonian systems with Lévy noises and a complement of (Ann. Inst. Henri Poincaré Probab. Stat. 58 (2022a) 916–944) upon exponential ergodicity for Andersen dynamics with constant jump rate functions.

Piecewise deterministic Markov process for condition-based imperfect maintenance models

Piecewise deterministic Markov process for condition-based imperfect maintenance models

Output regulation of stochastic sampled-data systems with post-processing internal model

This paper deals with the output regulation problem (ORP) of a linear time-invariant (LTI) system in the presence of sporadically sampled measurement streams with the inter-sampling intervals following a stochastic process. Under such sporadically available measurement streams, a regulator consisting of a hybrid observer, continuous-time post-processing internal model, and stabilizer are proposed, which resets with the arrival of new measurements. The resulting system exhibits a deterministic behavior except for the jumps that occur at random sampling times and therefore the overall closed-loop system can be categorized as a piecewise deterministic Markov process (PDMP). In existing works on ORPs with aperiodic sampling, the requirement of boundedness on inter-sampling intervals precludes extending the solution to the random sampling intervals with possibly unbounded support. Using the Lyapunov-like theorem for the stability analysis of stochastic systems, we offer sufficient conditions to ensure that the overall closed-loop system is mean exponentially stable (MES) and the objectives of the ORP are achieved under stochastic sampling of measurement streams. The resulting LMI conditions lead to a numerically tractable design of the hybrid regulator. Finally, with the help of an illustrative example, the effectiveness of the theoretical results are verified.

Adaptive discounted control for piecewise deterministic Markov processes

The main goal of this paper is to study the adaptive infinite-horizon discounted continuous-time optimal control problem of piecewise deterministic Markov processes (PDMPs) with the control acting continuously on the jump intensity λ and on the transition measure Q of the process. It is assumed that jump parameters (λ and Q), as well the continuous and boundary costs (Cg and Ci respectively), depend on an unknown parameter β⁎. It is shown that the principle of estimation and control holds, that is, the strategy consisting of choosing, at each stage n, an action according to an optimal stationary policy, where the true but unknown parameter β⁎ is replaced by its estimated value βˆn, is asymptotically discount optimal, provided that the sequence of estimators {βˆn} of β⁎ is strongly consistent, that is, βˆn converge to β⁎ almost surely. In the framework of PDMPs, the so-called discrepancy function depends on the derivative along the flow of the value function as well as on some boundary conditions, which brings new challenges in the analysis of this problem.

Concave-Convex PDMP-based Sampling

Recently nonreversible samplers based on simulating piecewise deterministic Markov processes (PDMPs) have shown potential for efficient sampling in Bayesian inference problems. However, there remains a lack of guidance on how to best implement these algorithms. If implemented poorly, the computational costs of simulating event times can outweigh the statistical efficiency of the nonreversible dynamics. Drawing on the adaptive rejection literature, we propose the concave-convex adaptive thinning approach for simulating a piecewise deterministic Markov process, which we call CC-PDMP. This approach provides a general guide for constructing bounds that may be used to facilitate PDMP-based sampling. A key advantage of this method is its additive structure—adding concave-convex decompositions yields a concave-convex decomposition. This makes the construction of bounds modular, as given a concave-convex decomposition for a class of likelihoods and a family of priors, they can be combined to construct bounds for the posterior. We show that constructing our bounds is simple and leads to computationally efficient thinning. Our approach is well suited to local PDMP simulation where conditional independence of the target can be exploited for potentially huge computational gains. We provide an R package and compare with existing approaches to simulating events in the PDMP literature. Supplementary materials for this article are available online.