Markov Process In Space Research Articles

Reduced Markovian models of dynamical systems

Leveraging recent work on data-driven methods for constructing a finite state space Markov process from dynamical systems, we address two problems for obtaining further reduced statistical representations. The first problem is to extract the most salient reduced-order dynamics for a given timescale by using a modified clustering algorithm from network theory. The second problem is to provide an alternative construction for the infinitesimal generator of a Markov process that respects statistical features over a large range of time scales. We demonstrate the methodology on three low-dimensional dynamical systems with stochastic and chaotic dynamics. We then apply the method to two high-dimensional dynamical systems, the Kuramoto–Sivashinky equations and data sampled from fluid-flow experiments via Particle Image Velocimetry. We show that the methodology presented herein provides a robust reduced-order statistical representation of the underlying system.

Large deviations for the empirical measure of the zig-zag process

The zig-zag process is a piecewise deterministic Markov process in position and velocity space. The process can be designed to have an arbitrary Gibbs type marginal probability density for its position coordinate, which makes it suitable for Monte Carlo simulation of continuous probability distributions. An important question in assessing the efficiency of this method is how fast the empirical measure converges to the stationary distribution of the process. In this paper we provide a partial answer to this question by characterizing the large deviations of the empirical measure from the stationary distribution. Based on the Feng–Kurtz approach, we develop an abstract framework aimed at encompassing piecewise deterministic Markov processes in position-velocity space. We derive explicit conditions for the zig-zag process to allow the Donsker–Varadhan variational formulation of the rate function, both for a compact setting (the torus) and one-dimensional Euclidean space. Finally we derive an explicit expression for the Donsker–Varadhan functional for the case of a compact state space and use this form of the rate function to address a key question concerning the optimal choice of the switching rate of the zig-zag process.

Hypotheses testing and posterior concentration rates for semi-Markov processes

In this paper, we adopt a nonparametric Bayesian approach and investigate the asymptotic behavior of the posterior distribution in continuous-time and general state space semi-Markov processes. In particular, we obtain posterior concentration rates for semi-Markov kernels. For the purposes of this study, we construct robust statistical tests between Hellinger balls around semi-Markov kernels and present some specifications to particular cases, including discrete-time semi-Markov processes and countable state space Markov processes. The objective of this paper is to provide sufficient conditions on priors and semi-Markov kernels that enable us to establish posterior concentration rates.

Baryon stopping as a relativistic Markov process in phase space

We reconsider baryon stopping in relativistic heavy-ion collisions in a nonequilibrium-statistical framework. The approach combines earlier formulations based on quantum chromodynamics with a relativistic diffusion model through a suitably derived fluctuation-dissipation relation, thus allowing for a fully time-dependent theory that is consistent with QCD. We use an existing framework for relativistic stochastic processes in spacetime that are Markovian in phase space, and adapt it to derive a Fokker-Planck equation in rapidity space, which is solved numerically. The time evolution of the net-proton distribution function in rapidity space agrees with stopping data from the CERN Super Proton Synchrotron and the BNL Relativistic Heavy Ion Collider.

Chemical Kinetics Roots and Methods to Obtain the Probability Distribution Function Evolution of Reactants and Products in Chemical Networks Governed by a Master Equation.

In this paper first, we review the physical root bases of chemical reaction networks as a Markov process in multidimensional vector space. Then we study the chemical reactions from a microscopic point of view, to obtain the expression for the propensities for the different reactions that can happen in the network. These chemical propensities, at a given time, depend on the system state at that time, and do not depend on the state at an earlier time indicating that we are dealing with Markov processes. Then the Chemical Master Equation (CME) is deduced for an arbitrary chemical network from a probability balance and it is expressed in terms of the reaction propensities. This CME governs the dynamics of the chemical system. Due to the difficulty to solve this equation two methods are studied, the first one is the probability generating function method or z-transform, which permits to obtain the evolution of the factorial moment of the system with time in an easiest way or after some manipulation the evolution of the polynomial moments. The second method studied is the expansion of the CME in terms of an order parameter (system volume). In this case we study first the expansion of the CME using the propensities obtained previously and splitting the molecular concentration into a deterministic part and a random part. An expression in terms of multinomial coefficients is obtained for the evolution of the probability of the random part. Then we study how to reconstruct the probability distribution from the moments using the maximum entropy principle. Finally, the previous methods are applied to simple chemical networks and the consistency of these methods is studied.

An extended transfer operator approach to identify separatrices in open flows.

Vortices of coherent fluid volume are considered to have a substantial impact on transport processes in turbulent media. Yet, due to their Lagrangian nature, detecting these structures is highly nontrivial. In this respect, transfer operator approaches have been proven to provide useful tools: Approximating a possibly time-dependent flow as a discrete Markov process in space and time, information about coherent structures is contained in the operator's eigenvectors, which is usually extracted by employing clustering methods. Here, we propose an extended approach that couples surrounding filaments using "mixing boundary conditions" and focuses on the separation of the inner coherent set and embedding outer flow. The approach refrains from using unsupervised machine learning techniques such as clustering and uses physical arguments by maximizing a coherence ratio instead. We show that this technique improves the reconstruction of separatrices in stationary open flows and succeeds in finding almost-invariant sets in periodically perturbed flows.

ANALYSIS AND DESIGN OF JUMP COEFFICIENTS IN DISCRETE STOCHASTIC DIFFUSION MODELS.

In computational systems biology, the mesoscopic model of reaction-diffusion kinetics is described by a continuous time, discrete space Markov process. To simulate diffusion stochastically, the jump coefficients are obtained by a discretization of the diffusion equation. Using unstructured meshes to represent complicated geometries may lead to negative coefficients when using piecewise linear finite elements. Several methods have been proposed to modify the coefficients to enforce the nonnegativity needed in the stochastic setting. In this paper, we present a method to quantify the error introduced by that change. We interpret the modified discretization matrix as the exact finite element discretization of a perturbed equation. The forward error, the error between the analytical solutions to the original and the perturbed equations, is bounded by the backward error, the error between the diffusion of the two equations. We present a backward analysis algorithm to compute the diffusion coefficient from a given discretization matrix. The analysis suggests a new way of deriving nonnegative jump coefficients that minimizes the backward error. The theory is tested in numerical experiments indicating that the new method is superior and also minimizes the forward error.

Jump Markov models and transition state theory: the quasi-stationary distribution approach.

We are interested in the connection between a metastable continuous state space Markov process (satisfying e.g. the Langevin or overdamped Langevin equation) and a jump Markov process in a discrete state space. More precisely, we use the notion of quasi-stationary distribution within a metastable state for the continuous state space Markov process to parametrize the exit event from the state. This approach is useful to analyze and justify methods which use the jump Markov process underlying a metastable dynamics as a support to efficiently sample the state-to-state dynamics (accelerated dynamics techniques). Moreover, it is possible by this approach to quantify the error on the exit event when the parametrization of the jump Markov model is based on the Eyring-Kramers formula. This therefore provides a mathematical framework to justify the use of transition state theory and the Eyring-Kramers formula to build kinetic Monte Carlo or Markov state models.

COUNTABLE STATE MARKOV PROCESSES: NON-EXPLOSIVENESS AND MOMENT FUNCTION

The existence of a moment function satisfying a drift function condition is well known to guarantee non-explosiveness of the associated minimal Markov process (cf. [1,9]), under standard technical conditions. Surprisingly, the reverse is true as well for a countable space Markov process. We prove this result by showing that recurrence of an associated jump process, that we call the α-jump process, is equivalent to non-explosiveness. Non-explosiveness corresponds in a natural way to the validity of the Kolmogorov integral relation for the function identically equal to 1. In particular, we show that positive recurrence of the α-jump chain implies that all bounded functions satisfy the Kolmogorov integral relation. We present a drift function criterion characterizing positive recurrence of this α-jump chain.Suppose that to a drift functionVthere corresponds another drift functionW, which is a moment with respect toV. Via a transformation argument, the above relations hold for the transformed process with respect toV. Transferring the results back to the original process, allows to characterize theV-bounded functions that satisfy the Kolmogorov forward equation.

Parametric sensitivity analysis for stochastic molecular systems using information theoretic metrics.

In this paper, we present a parametric sensitivity analysis (SA) methodology for continuous time and continuous space Markov processes represented by stochastic differential equations. Particularly, we focus on stochastic molecular dynamics as described by the Langevin equation. The utilized SA method is based on the computation of the information-theoretic (and thermodynamic) quantity of relative entropy rate (RER) and the associated Fisher information matrix (FIM) between path distributions, and it is an extension of the work proposed by Y. Pantazis and M. A. Katsoulakis [J. Chem. Phys. 138, 054115 (2013)]. A major advantage of the pathwise SA method is that both RER and pathwise FIM depend only on averages of the force field; therefore, they are tractable and computable as ergodic averages from a single run of the molecular dynamics simulation both in equilibrium and in non-equilibrium steady state regimes. We validate the performance of the extended SA method to two different molecular stochastic systems, a standard Lennard-Jones fluid and an all-atom methane liquid, and compare the obtained parameter sensitivities with parameter sensitivities on three popular and well-studied observable functions, namely, the radial distribution function, the mean squared displacement, and the pressure. Results show that the RER-based sensitivities are highly correlated with the observable-based sensitivities.

Effective pore‐scale dispersion upscaling with a correlated continuous time random walk approach

We investigate the upscaling of dispersion from a pore‐scale analysis of Lagrangian velocities. A key challenge in the upscaling procedure is to relate the temporal evolution of spreading to the pore‐scale velocity field properties. We test the hypothesis that one can represent Lagrangian velocities at the pore scale as a Markov process in space. The resulting effective transport model is a continuous time random walk (CTRW) characterized by a correlated random time increment, here denoted as correlated CTRW. We consider a simplified sinusoidal wavy channel model as well as a more complex heterogeneous pore space. For both systems, the predictions of the correlated CTRW model, with parameters defined from the velocity field properties (both distribution and correlation), are found to be in good agreement with results from direct pore‐scale simulations over preasymptotic and asymptotic times. In this framework, the nontrivial dependence of dispersion on the pore boundary fluctuations is shown to be related to the competition between distribution and correlation effects. In particular, explicit inclusion of spatial velocity correlation in the effective CTRW model is found to be important to represent incomplete mixing in the pore throats.

Spatial Markov Model of Anomalous Transport Through Random Lattice Networks

Flow through lattice networks with quenched disorder exhibits a strong correlation in the velocity field, even if the link transmissivities are uncorrelated. This feature, which is a consequence of the divergence-free constraint, induces anomalous transport of passive particles carried by the flow. We propose a Lagrangian statistical model that takes the form of a continuous time random walk with correlated velocities derived from a genuinely multidimensional Markov process in space. The model captures the anomalous (non-Fickian) longitudinal and transverse spreading, and the tail of the mean first-passage time observed in the Monte Carlo simulations of particle transport. We show that reproducing these fundamental aspects of transport in disordered systems requires honoring the correlation in the Lagrangian velocity.

New optimality conditions for average-payoff continuous-time Markov games in Polish spaces

This paper concerns two-person zero-sum games for a class of average-payoff continuous-time Markov processes in Polish spaces. The underlying processes are determined by transition rates that are allowed to be unbounded, and the payoff function may have neither upper nor lower bounds. We use two optimality inequalities to replace the so-called optimality equation in the previous literature. Under more general conditions, these optimality inequalities yield the existence of the value of the game and of a pair of optimal stationary strategies. Under some additional conditions we further establish the optimality equation itself. Finally, we use several examples to illustrate our results, and also to show the difference between the conditions in this paper and those in the literature. In particular, one of these examples shows that our approach is more general than all of the existing ones because it allows nonergodic Markov processes.

Estimation using plug-in of the stationary distribution and Shannon entropy of continuous time Markov processes

A natural way to deal with the uncertainty of an ergodic finite state space Markov process is to investigate the entropy of its stationary distribution. When the process is observed, it becomes necessary to estimate this entropy. We estimate both the stationary distribution and its entropy by plug-in of the estimators of the infinitesimal generator. Three situations of observation are discussed: one long trajectory is observed, several independent short trajectories are observed, or the process is observed at discrete times. The good asymptotic behavior of the plug-in estimators is established. We also illustrate the behavior of the estimators through simulation.

Lumpable Markov chains in risk management

We explore the use of the concept of lumpability of continuous time discrete state space Markov processes in the context of risk management and propose an approximate lumpability procedure that may be useful when exact lumpability does not hold.

A Linear Programming Approach to Parameter Fitting for the Master Equation

This technical note proposes a new framework for the design of continuous time, finite state space Markov processes. In particular, we propose a paradigm for selecting an optimal matrix within a pre-specified pencil of transition rate matrices. Given any transition rate matrix specifying the time-evolution of the Markov process, we propose a class of figures of merit that upper-bounds the long-term evolution of any statistical moment. We show that optimization with respect to the aforementioned class of cost functions is tractable via dualization and linear programming methods. In addition, we suggest how this approach can be used as a tool for the sub-optimal design of the master equation, with performance guarantees. Our results are applied to illustrative examples.

Fluid queues with level dependent evolution

Abstract A fluid queue is a two-dimensional Markov process, of which the first component, or level , varies linearly according to the second component, the phase , which is the state of a finite state space Markov process evolving in the background. In this paper, we construct various models of fluid queues, with a level dependency component: the behavior of the phase process changes when the level crosses certain thresholds, as well as the rate at which fluid increases or decreases; this adds the possibility of having attractive and repellent states at the threshold levels. We derive expressions for the stationary distribution of such processes.

A Model for Ischaemic Heart Disease and Stroke I: The Model

ABSTRACTWe construct a stochastic model of an individual's lifetime that includes diagnosis with ischaemic heart disease and stroke and also the development of the major risk factors for these conditions: hypercholesterolaemia, hypertension, diabetes and obesity. Smoking, another major risk factor, is treated deterministically. Mathematically, the model is a continuous time, finite state space Markov process, with the individual's age playing the rôle of time. The model is parameterised using data from the Framingham Heart Study, with parameter values adjusted so that the model is appropriate for UK conditions in the early 21st century. The model has been designed so that it can be used to quantify the effects of:(i) trends, in particular increasing prevalence of obesity.(ii) changes in behaviour, in particular smoking patterns, and(iii) treatments, in particular statins for hypercholesterolaemia.These applications are covered in two accompanying papers.

On a Comparison Result for Markov Processes

A comparison theorem is stated for Markov processes in Polish state spaces. We consider a general class of stochastic orderings induced by a cone of real functions. The main result states that stochastic monotonicity of one process and comparability of the infinitesimal generators imply ordering of the processes. Several applications to convex type and to dependence orderings are given. In particular, Liggett's theorem on the association of Markov processes is a consequence of this comparison result.

On a Comparison Result for Markov Processes

A comparison theorem is stated for Markov processes in Polish state spaces. We consider a general class of stochastic orderings induced by a cone of real functions. The main result states that stochastic monotonicity of one process and comparability of the infinitesimal generators imply ordering of the processes. Several applications to convex type and to dependence orderings are given. In particular, Liggett's theorem on the association of Markov processes is a consequence of this comparison result.