Finite State Projection Algorithm Research Articles

BAYESIAN INFERENCE OF STOCHASTIC REACTION NETWORKS USING MULTIFIDELITY SEQUENTIAL TEMPERED MARKOV CHAIN MONTE CARLO.

Stochastic reaction network models are often used to explain and predict the dynamics of gene regulation in single cells. These models usually involve several parameters, such as the kinetic rates of chemical reactions, that are not directly measurable and must be inferred from experimental data. Bayesian inference provides a rigorous probabilistic framework for identifying these parameters by finding a posterior parameter distribution that captures their uncertainty. Traditional computational methods for solving inference problems such as Markov Chain Monte Carlo methods based on classical Metropolis-Hastings algorithm involve numerous serial evaluations of the likelihood function, which in turn requires expensive forward solutions of the chemical master equation (CME). We propose an alternate approach based on a multifidelity extension of the Sequential Tempered Markov Chain Monte Carlo (ST-MCMC) sampler. This algorithm is built upon Sequential Monte Carlo and solves the Bayesian inference problem by decomposing it into a sequence of efficiently solved subproblems that gradually increase both model fidelity and the influence of the observed data. We reformulate the finite state projection (FSP) algorithm, a well-known method for solving the CME, to produce a hierarchy of surrogate master equations to be used in this multifidelity scheme. To determine the appropriate fidelity, we introduce a novel information-theoretic criteria that seeks to extract the most information about the ultimate Bayesian posterior from each model in the hierarchy without inducing significant bias. This novel sampling scheme is tested with high performance computing resources using biologically relevant problems.

A finite state projection method for steady-state sensitivity analysis of stochastic reaction networks.

Consider the standard stochastic reaction network model where the dynamics is given by a continuous-time Markov chain over a discrete lattice. For such models, estimation of parameter sensitivities is an important problem, but the existing computational approaches to solve this problem usually require time-consuming Monte Carlo simulations of the reaction dynamics. Therefore, these simulation-based approaches can only be expected to work over finite time-intervals, while it is often of interest in applications to examine the sensitivity values at the steady-state after the Markov chain has relaxed to its stationary distribution. The aim of this paper is to present a computational method for the estimation of steady-state parameter sensitivities, which instead of using simulations relies on the recently developed stationary finite state projection algorithm [Gupta et al., J. Chem. Phys. 147, 154101 (2017)] that provides an accurate estimate of the stationary distribution at a fixed set of parameters. We show that sensitivity values at these parameters can be estimated from the solution of a Poisson equation associated with the infinitesimal generator of the Markov chain. We develop an approach to numerically solve the Poisson equation, and this yields an efficient estimator for steady-state parameter sensitivities. We illustrate this method using several examples.

A finite state projection algorithm for the stationary solution of the chemical master equation.

The chemical master equation (CME) is frequently used in systems biology to quantify the effects of stochastic fluctuations that arise due to biomolecular species with low copy numbers. The CME is a system of ordinary differential equations that describes the evolution of probability density for each population vector in the state-space of the stochastic reaction dynamics. For many examples of interest, this state-space is infinite, making it difficult to obtain exact solutions of the CME. To deal with this problem, the Finite State Projection (FSP) algorithm was developed by Munsky and Khammash [J. Chem. Phys. 124(4), 044104 (2006)], to provide approximate solutions to the CME by truncating the state-space. The FSP works well for finite time-periods but it cannot be used for estimating the stationary solutions of CMEs, which are often of interest in systems biology. The aim of this paper is to develop a version of FSP which we refer to as the stationary FSP (sFSP) that allows one to obtain accurate approximations of the stationary solutions of a CME by solving a finite linear-algebraic system that yields the stationary distribution of a continuous-time Markov chain over the truncated state-space. We derive bounds for the approximation error incurred by sFSP and we establish that under certain stability conditions, these errors can be made arbitrarily small by appropriately expanding the truncated state-space. We provide several examples to illustrate our sFSP method and demonstrate its efficiency in estimating the stationary distributions. In particular, we show that using a quantized tensor-train implementation of our sFSP method, problems admitting more than 100 × 106 states can be efficiently solved.

Implementation of variable parameters in the Krylov-based finite state projection for solving the chemical master equation

The finite state projection (FSP) algorithm is a reduction method for solving the chemical master equation (CME). The Krylov-FSP improved on the original FSP by using an embedded scheme where the action of the matrix exponential is evaluated by the Krylov subspace method of Expokit for greater efficiency. There are parameters that impact the method, such as the stepsize that must be controlled to ensure the accuracy of the computed matrix exponentials, or to ensure the accuracy of the FSP. Other parameters include the dimension of the Krylov basis, or even the extent of reachability when expanding the FSP. In this work, we incorporate adaptive strategies to automatically vary these parameters. Numerical experiments comparing the resulting variants are reported, showing how certain choices perform better than others.

Order reduction of the chemical master equation via balanced realisation.

We consider a Markov process in continuous time with a finite number of discrete states. The time-dependent probabilities of being in any state of the Markov chain are governed by a set of ordinary differential equations, whose dimension might be large even for trivial systems. Here, we derive a reduced ODE set that accurately approximates the probabilities of subspaces of interest with a known error bound. Our methodology is based on model reduction by balanced truncation and can be considerably more computationally efficient than solving the chemical master equation directly. We show the applicability of our method by analysing stochastic chemical reactions. First, we obtain a reduced order model for the infinitesimal generator of a Markov chain that models a reversible, monomolecular reaction. Later, we obtain a reduced order model for a catalytic conversion of substrate to a product (a so-called Michaelis-Menten mechanism), and compare its dynamics with a rapid equilibrium approximation method. For this example, we highlight the savings on the computational load obtained by means of the reduced-order model. Furthermore, we revisit the substrate catalytic conversion by obtaining a lower-order model that approximates the probability of having predefined ranges of product molecules. In such an example, we obtain an approximation of the output of a model with 5151 states by a reduced model with 16 states. Finally, we obtain a reduced-order model of the Brusselator.

Editorial: Special Issue on Stochastic Modelling of Reaction–Diffusion Processes in Biology

Editorial: Special Issue on Stochastic Modelling of Reaction–Diffusion Processes in Biology

Parallelising the finite state projection method

Many realistic mathematical models of biological and chemical systems, such as enzyme cascades and gene regulatory networks, need to include stochasticity. These systems are described as Markov processes and are modelled using the Chemical Master Equation. The Chemical Master Equation is a differential-difference equation (continuous in time and discrete in the state space) for the probability of a certain state at a given time. The state space is the population count of species in the system. A successful method for computing the Chemical Master Equation is the Finite State Projection Method. We give a new algorithm to distribute the Finite State Projection Method method onto multi-core systems. This method is called the Parallel Finite State Projection method. This article also analyses the theory needed for parallelisation of the Chemical Master Equation. References cmepy, Jan 2011. https://github.com/fcostin/cmepy K. Burrage, M. Hegland, S. MacNamara, and R.B. Sidje. A Krylov-based finite state projection algorithm for solving the chemical master equation arising in the discrete modeling of biological systems. In: Langville, A. N., Stewart, W. J. (Eds.), Proceedings of the 150th Markov Anniversary Meeting, Boson Books, pp. 21--38, 2006. S. Engblom. Numerical Solution Methods in Stochastic Chemical Kenetics. PhD thesis, Uppsala University, 2008. M. Hegland, A. Hellander, and P. Lotstedt. Sparse grid and hybrid methods for the chemical master equation. BIT Numerical Mathematics, 48(2):265--283, 2008. doi:10.1007/s10543-008-0174-z M. Khammash and B. Munsky. The finite state projection algorithm for the solution of the chemical master equation. Journal of Chemical Physics, 124(044104):1--12, 2006. doi:10.1063/1.2145882 S. MacNamara, A.M. Bersani, K. Burrage, and R.B. Sidje. Stochastic chemical kinetics and the total quasi-steady-state assumption: application to the stochastic simulation algorithm and chemical master equation. Journal of Chemical Physics, 129(095105):1--13, 2008. doi:10.1063/1.2971036 V. Sunkara and M. Hegland. An optimal finite state projection method. Procedia Computer Science, 1(1):1579--1586, 2010. ICCS 2010. http://www.sciencedirect.com/science/article/pii/S187705091000178X, doi:10.1016/j.procs.2010.04.177

The diffusive finite state projection algorithm for efficient simulation of the stochastic reaction-diffusion master equation.

We have developed a computational framework for accurate and efficient simulation of stochastic spatially inhomogeneous biochemical systems. The new computational method employs a fractional step hybrid strategy. A novel formulation of the finite state projection (FSP) method, called the diffusive FSP method, is introduced for the efficient and accurate simulation of diffusive transport. Reactions are handled by the stochastic simulation algorithm.

Listening to the noise: random fluctuations reveal gene network parameters

The cellular environment is abuzz with noise originating from the inherent random motion of reacting molecules in the living cell. In this noisy environment, clonal cell populations show cell-to-cell variability that can manifest significant phenotypic differences. Noise-induced stochastic fluctuations in cellular constituents can be measured and their statistics quantified. We show that these random fluctuations carry within them valuable information about the underlying genetic network. Far from being a nuisance, the ever-present cellular noise acts as a rich source of excitation that, when processed through a gene network, carries its distinctive fingerprint that encodes a wealth of information about that network. We show that in some cases the analysis of these random fluctuations enables the full identification of network parameters, including those that may otherwise be difficult to measure. This establishes a potentially powerful approach for the identification of gene networks and offers a new window into the workings of these networks.

Approximating the solution of the chemical master equation by aggregation

The chemical master equation is a continuous time discrete space Markov model of chemical reactions. The chemical master equation is derived mathematically and it is shown that the corresponding initial value problem has a unique solution. Conditions are given under which this solution is a probability distribution. We present finite state and aggregation-disaggregation approximations and provide error bounds for the case of piecewise constant disaggregation. The aggregation-disaggregation approximation allows the solution of the chemical master equation for larger state spaces and is also an important tool for the solution of multidimensional problems. References Adam Arkin, John Ross, and Harley H. McAdams. {Stochastic Kinetic Analysis of Developmental Pathway Bifurcation in Ph age lambda-Infected Escherichia coli Cells}. Genetics, 149(4):1633--1648, 1998. http://www.genetics.org/cgi/content/abstract/149/4/1633 P. Deuflhard, W. Huisinga, T. Jahnke, and M. Wulkow. Adaptive discrete Galerkin methods applied to the chemical master equation. Technical Report ZIB-Report 07-04, Konrad--Zuse Zentrum fuer Informationstechnik Berlin, 2007. http://www.zib.de/Publications/Reports/ZR-07-04.pdf Stefan Engblom. Computing the moments of high dimensional solutions of the master equation. Appl. Math. Comput., 180(2):498--515, 2006. doi:10.1016/j.amc.2005.12.032 Lars Ferm and Per Loetstedt. Numerical method for coupling the macro and meso scales in stochastic chemical kinetics. BIT Numerical Mathematics, 47:735--762, 2007. doi:10.1007/s10543-007-0150-z Daniel T. Gillespie. Exact stochastic simulation of coupled chemical reactions. Journal of Physical Chemistry, 81(25):2340--2361, December 1977. Markus Hegland, Conrad Burden, Lucia Santoso, Shev MacNamara, and Hilary Booth. A solver for the stochastic master equation applied to gene regulatory networks. J. Comput. Appl. Math., 205(2):708--724, 2007. doi:10.1016/j.cam.2006.02.053 Leo Huber and Markus Hegland. Dimension adaptive sparse grids for the chemical master equations. submitted, 2008. B. Munsky and M. Khammash. The finite state projection algorithm for the solution of the chemical master equation. The Journal of Chemical Physics, 124:044104--1--044104--1, 2006. doi:10.1063/1.2145882 Roger B. Sidje. EXPOKIT: Software package for computing matrix exponentials. ACM Transactions on Mathematical Software, 24(1):130--156, March 1998. doi:10.1145/285861.285868 T. Tian and K. Burrage. Bistability and switching in the lysis/lysogeny genetic regulatory network of bacteriophage-$\lambda $. Journal of Theoretical Biology, 227:229--237, 2004. doi:10.1016/j.jtbi.2003.11.003 N. G. van Kampen. Stochastic Processes in Physics and Chemistry. North Holland, Amsterdam, the Netherlands, 1981.

Stochastic chemical kinetics and the total quasi-steady-state assumption: Application to the stochastic simulation algorithm and chemical master equation

Recently the application of the quasi-steady-state approximation (QSSA) to the stochastic simulation algorithm (SSA) was suggested for the purpose of speeding up stochastic simulations of chemical systems that involve both relatively fast and slow chemical reactions [Rao and Arkin, J. Chem. Phys. 118, 4999 (2003)] and further work has led to the nested and slow-scale SSA. Improved numerical efficiency is obtained by respecting the vastly different time scales characterizing the system and then by advancing only the slow reactions exactly, based on a suitable approximation to the fast reactions. We considerably extend these works by applying the QSSA to numerical methods for the direct solution of the chemical master equation (CME) and, in particular, to the finite state projection algorithm [Munsky and Khammash, J. Chem. Phys. 124, 044104 (2006)], in conjunction with Krylov methods. In addition, we point out some important connections to the literature on the (deterministic) total QSSA (tQSSA) and place the stochastic analogue of the QSSA within the more general framework of aggregation of Markov processes. We demonstrate the new methods on four examples: Michaelis-Menten enzyme kinetics, double phosphorylation, the Goldbeter-Koshland switch, and the mitogen activated protein kinase cascade. Overall, we report dramatic improvements by applying the tQSSA to the CME solver.

Multiscale Modeling of Chemical Kinetics via the Master Equation

We present numerical methods for both the direct solution and simulation of the chemical master equation (CME), and, compared to popular methods in current use, such as the Gillespie stochastic simulation algorithm (SSA) and τ-Leap approximations, this new approach has the advantage of being able to detect when the system has settled down to equilibrium. This improved performance is due to the incorporation of information from the associated CME, a valuable complementary approach to the SSA that has often been felt to be too computationally inefficient. Hybrid methods, that combine these complementary approaches and so are able to detect equilibrium while maintaining the efficiency of the leap methods, are also presented. Amongst CME-solvers the recently suggested finite state projection algorithm is especially well suited to this purpose and has been adapted here for the task, leading to a type of “exact τ-Leap.” It is also observed that a CMEsolver is often more efficient than an SSA or even a τ-Leap approach for computing moments of the solution such as the mean and variance. These techniques are demonstrated on a test suite of five biologically inspired models, namely, stochastic models of the genetic toggle, receptor oligomerization, the Schl¨ogl reactions, Goutsias’ model of regulated gene transcription, and a decaying-dimerizing reaction set. For the gene toggle it is observed that important experimentally measurable traits such as the percentage of cells that undergo so-called switching may also be more efficiently approximated via a CME-based approach.

The Finite State Projection Approach for the Analysis of Stochastic Noise in Gene Networks

In order to capture important subcellular dynamics, researchers in computational biology have begun to turn to mesoscopic models in which molecular interactions at the gene level behave as discrete stochastic events. While the trajectories of such models cannot be described with deterministic expressions, the probability distributions of these trajectories can be described by the set of linear ordinary differential equations known as the chemical master equation (CME). Until recently, it has been believed that the CME could only be solved analytically in the most trivial of problems, and the CME has been analyzed almost exclusively with kinetic Monte Carlo (KMC) algorithms. However, concepts from linear systems theory have enabled the finite state projection (FSP) approach and have significantly enhanced our ability to solve the CME without resorting to KMC simulations. In this paper, we review the FSP approach and introduce a variety of systems-theory-based modifications and enhancements to the FSP algorithm. Notions such as observability, controllability, and minimal realizations enable large reductions and increase efficiency with little to no loss in accuracy. Model reduction techniques based upon linear perturbation theory allow for the systematic projection of multiple time-scale dynamics onto a slowly varying manifold of much smaller dimension. We also present a powerful new reduction approach, in which we perform computations on a small subset of configuration grid points and then interpolate to find the distribution on the full set. The power of the FSP and its various reduction approaches is illustrated on few important models of genetic regulatory networks.

An improved dynamic Finite State Projection algorithm for the numerical solution of the chemical master equation with applications

Recently, Munsky and Khammash suggested the Finite State Projection (FSP) algorithm for the numerical solution of the Chemical Master Equation, which provides a discrete and stochastic modelling framework for chemical kinetics. The important question of whether or not the algorithm is guaranteed to terminate is not addressed in the original work. We show that the well-known explosive birth process provides a counter example. We also give sufficient criteria for a model to be suitable for the FSP technique. We demonstrate the FSP technique on three novel applications. Results are presented for: (i) the Schlogl reactions; (ii) another example from Gillespie's celebrated book; and (iii) models for the role that dimerization plays in reducing noise in simple gene regulatory networks. Finally, we augment the dimerization model to include tetramers and show that this enhances the noise reduction properties of the network.

A multiple time interval finite state projection algorithm for the solution to the chemical master equation

At the mesoscopic scale, chemical processes have probability distributions that evolve according to an infinite set of linear ordinary differential equations known as the chemical master equation (CME). Although only a few classes of CME problems are known to have exact and computationally tractable analytical solutions, the recently proposed finite state projection (FSP) technique provides a systematic reduction of the CME with guaranteed accuracy bounds. For many non-trivial systems, the original FSP technique has been shown to yield accurate approximations to the CME solution. Other systems may require a projection that is still too large to be solved efficiently; for these, the linearity of the FSP allows for many model reductions and computational techniques, which can increase the efficiency of the FSP method with little or no loss in accuracy. In this paper, we present a new approach for choosing and expanding the projection for the original FSP algorithm. Based upon this approach, we develop a new algorithm that exploits the linearity property of super-position. The new algorithm retains the full accuracy guarantees of the original FSP approach, but with significantly increased efficiency for some problems and a greater range of applicability. We illustrate the benefits of this algorithm on a simplified model of the heat shock mechanism in Escherichia coli.

The Finite State Projection Approach for the Analysis of Stochastic Noise in Gene Networks

In order to capture important subcellular dynamics, researchers in computational biology have begun to turn to mesoscopic models in which molecular interactions at the gene level behave as discrete stochastic events. While the trajectories of such models cannot be described with deterministic expressions, the probability distributions of these trajectories can be described by the set of linear ordinary differential equations known as the chemical master equation (CME). Until recently, it has been believed that the CME could only be solved analytically in the most trivial of problems, and the CME has been analyzed almost exclusively with kinetic Monte Carlo (KMC) algorithms. However, concepts from linear systems theory have enabled the finite state projection (FSP) approach and have significantly enhanced our ability to solve the CME without resorting to KMC simulations. In this paper, we review the FSP approach and introduce a variety of systems-theory-based modifications and enhancements to the FSP algorithm. Notions such as observability, controllability, and minimal realizations enable large reductions and increase efficiency with little to no loss in accuracy. Model reduction techniques based upon linear perturbation theory allow for the systematic projection of multiple time-scale dynamics onto a slowly varying manifold of much smaller dimension. We also present a powerful new reduction approach, in which we perform computations on a small subset of configuration grid points and then interpolate to find the distribution on the full set. The power of the FSP and its various reduction approaches is illustrated on few important models of genetic regulatory networks.

The finite state projection algorithm for the solution of the chemical master equation

This article introduces the finite state projection (FSP) method for use in the stochastic analysis of chemically reacting systems. One can describe the chemical populations of such systems with probability density vectors that evolve according to a set of linear ordinary differential equations known as the chemical master equation (CME). Unlike Monte Carlo methods such as the stochastic simulation algorithm (SSA) or tau leaping, the FSP directly solves or approximates the solution of the CME. If the CME describes a system that has a finite number of distinct population vectors, the FSP method provides an exact analytical solution. When an infinite or extremely large number of population variations is possible, the state space can be truncated, and the FSP method provides a certificate of accuracy for how closely the truncated space approximation matches the true solution. The proposed FSP algorithm systematically increases the projection space in order to meet prespecified tolerance in the total probability density error. For any system in which a sufficiently accurate FSP exists, the FSP algorithm is shown to converge in a finite number of steps. The FSP is utilized to solve two examples taken from the field of systems biology, and comparisons are made between the FSP, the SSA, and tau leaping algorithms. In both examples, the FSP outperforms the SSA in terms of accuracy as well as computational efficiency. Furthermore, due to very small molecular counts in these particular examples, the FSP also performs far more effectively than tau leaping methods.

Practice Expenses and Ancillary Help

<h3>Abstract</h3> Stochastic reaction networks are a popular modeling framework for biochemical processes that treat the molecular copy numbers within a single cell as a continuous time Markov chain, whose forward Chapman-Kolmogorov equation is known in biochemistry literature as the chemical master equation (CME). The solution of the CME contains extremely useful information that can be compared to experimental data in order to improve the quantitative understanding of biochemical reaction networks within the cell. However, this solution is costly to compute as it requires integrating an enormous system of differential equations that grows exponentially with the number of chemical species. To address this issue, we introduce a novel multiple-sinks Finite State Projection algorithm that approximates the CME with an adaptive sequence of reduced-order models with an effecient parallelization based on MPI. The implementation is tested on models of sizable state spaces using a high-performance computing node on Amazon Web Services, showing favorable scalability.