Nonequilibrium Stochastic Systems Research Articles

How particular is the physics of the free energy principle?

The free energy principle (FEP) states that any dynamical system can be interpreted as performing Bayesian inference upon its surrounding environment. Although, in theory, the FEP applies to a wide variety of systems, there has been almost no direct exploration or demonstration of the principle in concrete systems. In this work, we examine in depth the assumptions required to derive the FEP in the simplest possible set of systems – weakly-coupled non-equilibrium linear stochastic systems. Specifically, we explore (i) how general the requirements imposed on the statistical structure of a system are and (ii) how informative the FEP is about the behaviour of such systems. We discover that two requirements of the FEP – the Markov blanket condition (i.e. a statistical boundary precluding direct coupling between internal and external states) and stringent restrictions on its solenoidal flows (i.e. tendencies driving a system out of equilibrium) – are only valid for a very narrow space of parameters. Suitable systems require an absence of perception-action asymmetries that is highly unusual for living systems interacting with an environment. More importantly, we observe that a mathematically central step in the argument, connecting the behaviour of a system to variational inference, relies on an implicit equivalence between the dynamics of the average states of a system with the average of the dynamics of those states. This equivalence does not hold in general even for linear stochastic systems, since it requires an effective decoupling from the system's history of interactions. These observations are critical for evaluating the generality and applicability of the FEP and indicate the existence of significant problems of the theory in its current form. These issues make the FEP, as it stands, not straightforwardly applicable to the simple linear systems studied here and suggest that more development is needed before the theory could be applied to the kind of complex systems that describe living and cognitive processes.

Entropy Production in Exactly Solvable Systems.

The rate of entropy production by a stochastic process quantifies how far it is from thermodynamic equilibrium. Equivalently, entropy production captures the degree to which global detailed balance and time-reversal symmetry are broken. Despite abundant references to entropy production in the literature and its many applications in the study of non-equilibrium stochastic particle systems, a comprehensive list of typical examples illustrating the fundamentals of entropy production is lacking. Here, we present a brief, self-contained review of entropy production and calculate it from first principles in a catalogue of exactly solvable setups, encompassing both discrete- and continuous-state Markov processes, as well as single- and multiple-particle systems. The examples covered in this work provide a stepping stone for further studies on entropy production of more complex systems, such as many-particle active matter, as well as a benchmark for the development of alternative mathematical formalisms.

Generalized Fluctuation-Dissipation Theorem for Non-equilibrium Spatially Extended Systems

The fluctuation-dissipation theorem (FDT) connecting the response of the system to external perturbations with the fluctuations at thermodynamic equilibrium is a central result in statistical physics. There has been effort devoted to extending the FDT in several different directions since its original formulation. In this work we establish a generalized form of the FDT for spatially extended nonequilibrium stochastic systems described by continuous fields. The generalized FDT is formulated with the aid of the nonequilibrium force decomposition in the potential landscape and flux field theoretical framework. The general results are substantiated in the setting of the Ornstein-Uhlenbeck (OU) process and further illustrated by a more specific example worked out in detail. The key feature of this generalized FDT for nonequilibrium spatially extended systems is that it represents a ternary relation rather than a binary relation as the FDT for equilibrium systems does. In addition to the response function and the time derivative of the field-field correlation function that are present in the equilibrium FDT, the field-flux correlation function also enters the generalized FDT. This additional contribution originates from detailed balance breaking that signifies the nonequilibrium irreversible nature of the steady state. In the special case when the steady state is an equilibrium state obeying detailed balance, the field-flux correlation function vanishes and the ternary relation in the generalized FDT reduces to the binary relation in the equilibrium FDT.

Modules or Mean-Fields?

The segregation of neural processing into distinct streams has been interpreted by some as evidence in favour of a modular view of brain function. This implies a set of specialised ‘modules’, each of which performs a specific kind of computation in isolation of other brain systems, before sharing the result of this operation with other modules. In light of a modern understanding of stochastic non-equilibrium systems, like the brain, a simpler and more parsimonious explanation presents itself. Formulating the evolution of a non-equilibrium steady state system in terms of its density dynamics reveals that such systems appear on average to perform a gradient ascent on their steady state density. If this steady state implies a sufficiently sparse conditional independency structure, this endorses a mean-field dynamical formulation. This decomposes the density over all states in a system into the product of marginal probabilities for those states. This factorisation lends the system a modular appearance, in the sense that we can interpret the dynamics of each factor independently. However, the argument here is that it is factorisation, as opposed to modularisation, that gives rise to the functional anatomy of the brain or, indeed, any sentient system. In the following, we briefly overview mean-field theory and its applications to stochastic dynamical systems. We then unpack the consequences of this factorisation through simple numerical simulations and highlight the implications for neuronal message passing and the computational architecture of sentience.

Stationary distribution simulation of rare events under colored Gaussian noise

Forward flux sampling (FFS) has provided a convenient and efficient way to simulate rare events in equilibrium as well as non-equilibrium stochastic systems. In the present paper, the FFS scheme is applied to systems driven by colored Gaussian noise through enlarging the dimension to deal with the non-Markovian property. Besides, the parameters of the FFS scheme have to be reconsidered. Interestingly, by analyzing the effect of colored Gaussian noise on stationary distributions, some results are found which are clearly different from the case of Gaussian white noise excitation. We mainly found that the probability of the occurrence of rare events is inversely proportional to the correlation time. Comparing to the case of Gaussian white noise with the same intensity, the presence of colored Gaussian noise exerts a hindrance to the occurrence of rare events. Meanwhile, the FFS results show a good agreement with those from Monte Carlo simulations, even for the colored Gaussian noise case. This provides a potential insight into rare events of systems under non-white Gaussian noise via the FFS scheme.

A study of void size growth in nonequilibrium stochastic systems of point defects

We study properties of voids growth dynamics in a stochastic system of point defects in solids under nonequilibrium conditions (sustained irradiation). It is shown that fluctuations of defect production rate (external noise) increase the critical void radius comparing to a deterministic system. An automodel regime of void size growth in a stochastic system is studied in detail. Considering a homogeneous system, it is found that external noise does not change the universality of the void size distribution function; the mean void size evolves according to classical nucleation theory. The noise increases the mean void size and spreads the void size distribution. Studying dynamics of spatially extended systems it was shown that vacancies remaining in a matrix phase are able to organize into vacancy enriched domains due to an instability caused by an elastic lattice deformation. It is shown that dynamics of voids growth is defined by void sinks strength with void size growth exponent varying from 1/3 up to 1/2.

Conditional reversibility in nonequilibrium stochastic systems.

For discrete-state stochastic systems obeying Markovian dynamics, we establish the counterpart of the conditional reversibility theorem obtained by Gallavotti for deterministic systems [Ann. de l'Institut Henri Poincaré (A) 70, 429 (1999)]. Our result states that stochastic trajectories conditioned on opposite values of entropy production are related by time reversal, in the long-time limit. In other words, the probability of observing a particular sequence of events, given a long trajectory with a specified entropy production rate σ, is the same as the probability of observing the time-reversed sequence of events, given a trajectory conditioned on the opposite entropy production, -σ, where both trajectories are sampled from the same underlying Markov process. To obtain our result, we use an equivalence between conditioned ("microcanonical") and biased ("canonical") ensembles of nonequilibrium trajectories. We provide an example to illustrate our findings.

Temperature response in nonequilibrium stochastic systems

The linear response to temperature changes is derived for systems with overdamped stochastic dynamics. Holding both in transient and steady-state conditions, the results allow to compute nonequilibrium thermal susceptibilities from unperturbed correlation functions. These correlations contain a novel form of entropy flow due to temperature unbalances, next to the standard entropy flow of stochastic energetics and to complementary time-symmetric dynamical aspects. Our derivation hinges on a time rescaling, which is a key procedure for comparing apparently incommensurable path weights. An interesting notion of thermal time emerges from this approach.

On the steady-state probability distribution of nonequilibrium stochastic systems

A driven stochastic system in a constant temperature heat bath relaxes into a steady state which is characterized by the steady state probability distribution. We investigate the relationship between the driving force and the steady state probability distribution. We adopt the force decomposition method in which the force is decomposed as the sum of a gradient of a steady state potential and the remaining part. The decomposition method allows one to find a set of force fields each of which is compatible to a given steady state. Such a knowledge provides a useful insight on stochastic systems especially in the nonequilibrium situation. We demonstrate the decomposition method in stochastic systems under overdamped and underdamped dynamics and discuss the connection between them.

Perturbative Large Deviation Analysis of Non-Equilibrium Dynamics

Macroscopic fluctuation theory has shown that a wide class of non-equilibrium stochastic dynamical systems obey a large deviation principle, but except for a few one-dimensional examples these large deviation principles are in general not known in closed form. We consider the problem of constructing successive approximations to an (unknown) large deviation functional and show that the non-equilibrium probability distribution the takes a Gibbs-Boltzmann form with a set of auxiliary (non-physical) energy functions. The expectation values of these auxiliary energy functions and their conjugate quantities satisfy a closed system of equations which can imply a considerable reduction of dimensionality of the dynamics. We show that the accuracy of the approximations can be tested self-consistently without solving the full non- equilibrium equations. We test the general procedure on the simple model problem of a relaxing 1D Ising chain.

On polymer expansions for generalized Gibbs lattice systems of oscillators with ternary interaction

We propose a new short proof of the convergence of high-temperature polymer expansions in the thermodynamic limit of canonical correlation functions for classical and quantum Gibbs lattice systems of oscillators interacting via pair and ternary potentials and nonequilibrium stochastic systems of oscillators interacting via a pair potential with Gibbsian initial correlation functions.

Potential and flux field landscape theory. I. Global stability and dynamics of spatially dependent non-equilibrium systems

We established a potential and flux field landscape theory to quantify the global stability and dynamics of general spatially dependent non-equilibrium deterministic and stochastic systems. We extended our potential and flux landscape theory for spatially independent non-equilibrium stochastic systems described by Fokker-Planck equations to spatially dependent stochastic systems governed by general functional Fokker-Planck equations as well as functional Kramers-Moyal equations derived from master equations. Our general theory is applied to reaction-diffusion systems. For equilibrium spatially dependent systems with detailed balance, the potential field landscape alone, defined in terms of the steady state probability distribution functional, determines the global stability and dynamics of the system. The global stability of the system is closely related to the topography of the potential field landscape in terms of the basins of attraction and barrier heights in the field configuration state space. The effective driving force of the system is generated by the functional gradient of the potential field alone. For non-equilibrium spatially dependent systems, the curl probability flux field is indispensable in breaking detailed balance and creating non-equilibrium condition for the system. A complete characterization of the non-equilibrium dynamics of the spatially dependent system requires both the potential field and the curl probability flux field. While the non-equilibrium potential field landscape attracts the system down along the functional gradient similar to an electron moving in an electric field, the non-equilibrium flux field drives the system in a curly way similar to an electron moving in a magnetic field. In the small fluctuation limit, the intrinsic potential field as the small fluctuation limit of the potential field for spatially dependent non-equilibrium systems, which is closely related to the steady state probability distribution functional, is found to be a Lyapunov functional of the deterministic spatially dependent system. Therefore, the intrinsic potential landscape can characterize the global stability of the deterministic system. The relative entropy functional of the stochastic spatially dependent non-equilibrium system is found to be the Lyapunov functional of the stochastic dynamics of the system. Therefore, the relative entropy functional quantifies the global stability of the stochastic system with finite fluctuations. Our theory offers an alternative general approach to other field-theoretic techniques, to study the global stability and dynamics of spatially dependent non-equilibrium field systems. It can be applied to many physical, chemical, and biological spatially dependent non-equilibrium systems.

Kinetic theory of nonequilibrium stochastic long-range systems: phase transition and bistability

We study long-range interacting systems driven by external stochastic forces that act collectively on all the particles constituting the system. Such a scenario is frequently encountered in the context of plasmas, self-gravitating systems, two-dimensional turbulence, and also in a broad class of other systems. Under the effect of stochastic driving, the system reaches a stationary state where external forces balance dissipation on average. These states have an invariant probability that does not respect detailed balance, and are characterized by non-vanishing currents of conserved quantities. In order to analyze spatially homogeneous stationary states, we develop a kinetic approach that generalizes the one known for deterministic long-range systems; we obtain a very good agreement between predictions from kinetic theory and extensive numerical simulations. Our approach may also be generalized to describe spatially inhomogeneous stationary states. We also report on numerical simulations exhibiting a first-order nonequilibrium phase transition from homogeneous to inhomogeneous states. Close to the phase transition, the system shows bistable behavior between the two states, with a mean residence time that diverges as an exponential in the inverse of the strength of the external stochastic forces, in the limit of low values of such forces.

Dynamical simulations of classical stochastic systems using matrix product states

We adapt the time-evolving block decimation (TEBD) algorithm, originally devised to simulate the dynamics of one-dimensional quantum systems, to simulate the time evolution of nonequilibrium stochastic systems. We describe this method in detail; a system's probability distribution is represented by a matrix product state (MPS) of finite dimension and then its time evolution is efficiently simulated by repeatedly updating and approximately refactorizing this representation. We examine the use of MPS as an approximation method, looking at parallels between the interpretations of applying it to quantum state vectors and probability distributions. In the context of stochastic systems we consider two types of factorization for use in the TEBD algorithm: non-negative matrix factorization (NMF), which ensures that the approximate probability distribution is manifestly non-negative, and the singular value decomposition (SVD). Comparing these factorizations, we find the accuracy of the SVD to be substantially greater than current NMF algorithms. We then apply TEBD to simulate the totally asymmetric simple exclusion process (TASEP) for systems of up to hundreds of lattice sites in size. Using exact analytic results for the TASEP steady state, we find that TEBD reproduces this state such that the error in calculating expectation values can be made negligible even when severely compressing the description of the system by restricting the dimension of the MPS to be very small. Out of the steady state we show for specific observables that expectation values converge as the dimension of the MPS is increased to a moderate size.

Exact solvability of interacting many body lattice systems

We address the problem of exactly describing stochastic nonequilibrium systems that are widely used to model one-dimensional transport, in biology, traffic flow and others. We review the matrix product states ansatz to interacting multiparticle systems and its extension to a tridiagonal (generalized Onsager) algebra approach. The stationary probability distribution is expressed as a matrix product state with respect to a quadratic algebra defined by the dynamics of the process. The states involved in the matrix dements are determined by the boundary conditions. This reflects the intriguing feature of open systems that the bulk behaviour in the steady state strongly depends on the boundary rates. The importance of the boundary conditions manifests itself in the fact that the boundary operators are generators of a tridiagonal algebra whose irreducible modules are the Askey-Wilson polynomials. The matrices of the matrix product ansatz obey the tridiagonal algebraic relations for particular values of the structure, constants. Previously known representations, both infinite dimensional and finite dimensional ones, are recovered within the tridiagonal framework. The boundary Askey-Wilson and. tridiagonal symmetry is the deep algebraic property of driven diffusive systems allowing for the exact solvability in the steady state and the exact description of the stochastic dynamics.

Computation of Current Cumulants for Small Nonequilibrium Systems

We analyze a systematic algorithm for the exact computation of the current cumulants in stochastic nonequilibrium systems, recently discussed in the framework of full counting statistics for mesoscopic systems. This method is based on identifying the current cumulants from a Rayleigh-Schrödinger perturbation expansion for the generating function. Here it is derived from a simple path-distribution identity and extended to the joint statistics of multiple currents. For a possible thermodynamical interpretation, we compare this approach to a generalized Onsager-Machlup formalism. We present calculations for a boundary driven Kawasaki dynamics on a one-dimensional chain, both for attractive and repulsive particle interactions.

Generalized Clausius relation and power dissipation in nonequilibrium stochastic systems

In the framework of the stochastic dynamics of open Markov systems, we derive an extension of the Clausius inequality for transitions between states of the system. We give a formula for the power produced when the system is in its stationary state and relate it to the dissipation of energy needed to maintain the system out of equilibrium. We deduce that, near equilibrium, maximal power production requires an energy dissipation of the same order of magnitude as the power production.

Extinction of an infectious disease: A large fluctuation in a nonequilibrium system

We develop a theory of first passage processes in stochastic nonequilibrium systems of birth-death type using two closely related epidemiological models as examples. Our method employs the probability generating function technique in conjunction with the eikonal approximation. In this way the problem is reduced to finding the optimal path to extinction: a heteroclinic trajectory of an effective multidimensional classical Hamiltonian system. We compute this trajectory and mean extinction time of the disease numerically and uncover a nonmonotone, spiral path to extinction of a disease. We also obtain analytical results close to a bifurcation point, where the problem is described by a Hamiltonian previously identified in one-species population models.

Path ensembles and path sampling in nonequilibrium stochastic systems

Markovian models based on the stochastic master equation are often encountered in single molecule dynamics, reaction networks, and nonequilibrium problems in chemistry, physics, and biology. An efficient and convenient method to simulate these systems is the kinetic Monte Carlo algorithm which generates continuous-time stochastic trajectories. We discuss an alternative simulation method based on sampling of stochastic paths. Utilizing known probabilities of stochastic paths, it is possible to apply Metropolis Monte Carlo in path space to generate a desired ensemble of stochastic paths. The method is a generalization of the path sampling idea to stochastic dynamics, and is especially suited for the analysis of rare paths which are not often produced in the standard kinetic Monte Carlo procedure. Two generic examples are presented to illustrate the methodology.

Simulating rare events in equilibrium or nonequilibrium stochastic systems

We present three algorithms for calculating rate constants and sampling transition paths for rare events in simulations with stochastic dynamics. The methods do not require a priori knowledge of the phase-space density and are suitable for equilibrium or nonequilibrium systems in stationary state. All the methods use a series of interfaces in phase space, between the initial and final states, to generate transition paths as chains of connected partial paths, in a ratchetlike manner. No assumptions are made about the distribution of paths at the interfaces. The three methods differ in the way that the transition path ensemble is generated. We apply the algorithms to kinetic Monte Carlo simulations of a genetic switch and to Langevin dynamics simulations of intermittently driven polymer translocation through a pore. We find that the three methods are all of comparable efficiency, and that all the methods are much more efficient than brute-force simulation.