Dynamic Cavity Method Research Articles

Nature of the Volcano Transition in the Fully Disordered Kuramoto Model.

Randomly coupled phase oscillators may synchronize into disordered patterns of collective motion. We analyze this transition in a large, fully connected Kuramoto model with symmetric but otherwise independent random interactions. Using the dynamical cavity method, we reduce the dynamics to a stochastic single-oscillator problem with self-consistent correlation and response functions that we study analytically and numerically. We clarify the nature of the volcano transition and elucidate its relation to the existence of an oscillator glass phase.

Dynamical phase transitions in graph cellular automata.

Discrete dynamical systems can exhibit complex behavior from the iterative application of straightforward local rules. A famous class of examples comes from cellular automata whose global dynamics are notoriously challenging to analyze. To address this, we relax the regular connectivity grid of cellular automata to a random graph, which gives the class of graph cellular automata. Using the dynamical cavity method and its backtracking version, we show that this relaxation allows us to derive asymptotically exact analytical results on the global dynamics of these systems on sparse random graphs. Concretely, we showcase the results on a specific subclass of graph cellular automata with "conforming nonconformist" update rules, which exhibit dynamics akin to opinion formation. Such rules update a node's state according to the majority within their own neighborhood. In cases where the majority leads only by a small margin over the minority, nodes may exhibit nonconformist behavior. Instead of following the majority, they either maintain their own state, switch it, or follow the minority. For configurations with different initial biases towards one state we identify sharp dynamical phase transitions in terms of the convergence speed and attractor types. From the perspective of opinion dynamics this answers when consensus will emerge and when two opinions coexist almost indefinitely.

Introduction to dynamical mean-field theory of randomly connected neural networks with bidirectionally correlated couplings

Dynamical mean-field theory is a powerful physics tool used to analyze the typical behavior of neural networks, where neurons can be recurrently connected, or multiple layers of neurons can be stacked. However, it is not easy for beginners to access the essence of this tool and the underlying physics. Here, we give a pedagogical introduction of this method in a particular example of random neural networks, where neurons are randomly and fully connected by correlated synapses and therefore the network exhibits rich emergent collective dynamics. We also review related past and recent important works applying this tool. In addition, a physically transparent and alternative method, namely the dynamical cavity method, is also introduced to derive exactly the same results. The numerical implementation of solving the integro-differential mean-field equations is also detailed, with an illustration of exploring the fluctuation dissipation theorem.

A closure for the master equation starting from the dynamic cavity method

We consider classical spin systems evolving in continuous time with interactions given by a locally tree-like graph. Several approximate analysis methods have earlier been reported based on the idea of Belief Propagation / cavity method. We introduce a new such method which can be derived in a more systematic manner using the theory of Random Point Processes. Within this approach, the master equation governing the system’s dynamics is closed via a set of differential equations for the auxiliary cavity probabilities. The numerical results improve on the earlier versions of the closure on several important classes of problems. We re-visit here the cases of the Ising ferromagnet and the Viana–Bray spin-glass model.

Dynamics of sparse Boolean networks with multi-node and self-interactions

We analyse the equilibrium behaviour and non-equilibrium dynamics of sparse Boolean networks with self-interactions that evolve according to synchronous Glauber dynamics. Equilibrium analysis is achieved via a novel application of the cavity method to the temperature-dependent pseudo-Hamiltonian that characterizes the equilibrium state of systems with parallel dynamics. Similarly, the non-equilibrium dynamics can be analysed by using the dynamical version of the cavity method. It is well known, however, that when self-interactions are present, direct application of the dynamical cavity method is cumbersome, due to the presence of strong memory effects, which prevent explicit analysis of the dynamics beyond a few time steps. To overcome this difficulty, we show that it is possible to map a system of N variables to an equivalent bipartite system of 2N variables, for which the dynamical cavity method can be used under the usual one time approximation scheme. This substantial technical advancement allows for the study of transient and long-time behaviour of systems with self-interactions. Finally, we study the dynamics of systems with multi-node interactions, recently used to model gene-regulatory networks (GRNs), by mapping this to a bipartite system of Boolean variables with two-body interactions. We show that when interactions have a degree of bidirectionality such systems are able to support a multiplicity of diverse attractors, an important requirement for a GRN to sustain multi-cellular life.

Uncovering the non-equilibrium stationary properties in sparse Boolean networks

Dynamic processes of interacting units on a network are out of equilibrium in general. In the case of a directed tree, the dynamic cavity method provides an efficient tool that characterises the dynamic trajectory of the process for the linear threshold model. However, because of the computational complexity of the method, the analysis has been limited to systems where the largest number of neighbours is small. We devise an efficient implementation of the dynamic cavity method which substantially reduces the computational complexity of the method for systems with discrete couplings. Our approach opens up the possibility to investigate the dynamic properties of networks with fat-tailed degree distribution. We exploit this new implementation to study properties of the non-equilibrium steady-state. We extend the dynamic cavity approach to calculate the pairwise correlations induced by different motifs in the network. Our results suggest that just two basic motifs of the network are able to accurately describe the entire statistics of observed correlations. Finally, we investigate models defined on networks containing bi-directional interactions. We observe that the stationary state associated with networks with symmetric or anti-symmetric interactions is biased towards the active or inactive state respectively, even if independent interaction entries are drawn from a symmetric distribution. This phenomenon, which can be regarded as a form of spontaneous symmetry-breaking, is peculiar to systems formulated in terms of Boolean variables, as opposed to Ising spins.

Overcoming the complexity barrier of the dynamic message-passing method in networks with fat-tailed degree distributions.

The dynamic cavity method provides the most efficient way to evaluate probabilities of dynamic trajectories in systems of stochastic units with unidirectional sparse interactions. It is closely related to sum-product algorithms widely used to compute marginal functions from complicated global functions of many variables, with applications in disordered systems, combinatorial optimization, and computer science. However, the complexity of the cavity approach grows exponentially with the in-degrees of the interacting units, which creates a defacto barrier for the successful analysis of systems with fat-tailed in-degree distributions. In this paper, we present a dynamic programming algorithm that overcomes this barrier by reducing the computational complexity in the in-degrees from exponential to quadratic, whenever couplings are chosen randomly from (or can be approximated in terms of) discrete, possibly unit-dependent, sets of equidistant values. As a case study, we analyze the dynamics of a random Boolean network with a fat-tailed degree distribution and fully asymmetric binary ±J couplings, and we use the power of the algorithm to unlock the noise-dependent heterogeneity of stationary node activation patterns in such a system.

The matrix product approximation for the dynamic cavity method

Stochastic dynamics of classical degrees of freedom, defined on vertices of locally tree-like graphs, can be studied in the framework of the dynamic cavity method which is exact for tree graphs. Such models correspond for example to spin-glass systems, Boolean networks, neural networks, and other technical, biological, and social networks. The central objects in the cavity method are edge messages—conditional probabilities of two vertex variable trajectories. In this paper, we discuss a rather pedagogical derivation for the dynamic cavity method, give a detailed account of the novel matrix product edge message (MPEM) algorithm for the solution of the dynamic cavity equation as introduced in Barthel et al (2018 Phys. Rev. E 97 010104 (R)), and present optimizations and extensions. Matrix product approximations of the edge messages are constructed recursively in an iteration over time. Computation costs and precision can be tuned by controlling the matrix dimensions of the MPEM in truncations. Without truncations, the dynamics is exact. Data for Glauber–Ising dynamics shows a linear growth of computation costs in time. In contrast to Monte Carlo simulations, the approach has a much better error scaling. Hence, it gives for example access to low probability events and decaying observables like temporal correlations. We discuss optimized truncation schemes and an extension that allows to capture models which have a continuum time limit.

Numerical implementation of dynamical mean field theory for disordered systems: application to the Lotka–Volterra model of ecosystems

Dynamical mean field theory (DMFT) is a tool that allows one to analyze the stochastic dynamics of N interacting degrees of freedom in terms of a self-consistent 1-body problem. In this work, focusing on models of ecosystems, we present the derivation of DMFT through the dynamical cavity method, and we develop a method for solving it numerically. Our numerical procedure can be applied to a large variety of systems for which DMFT holds. We implement and test it for the generalized random Lotka–Volterra model, and show that complex dynamical regimes characterized by chaos and aging can be captured and studied by this framework.

Out-of-equilibrium dynamical equations of infinite-dimensional particle systems I. The isotropic case

We consider the Langevin dynamics of a many-body system of interacting particles in d dimensions, in a very general setting suitable to model several out-of-equilibrium situations, such as liquid and glass rheology, active self-propelled particles, and glassy aging dynamics. The pair interaction potential is generic, and can be chosen to model colloids, atomic liquids, and granular materials. In the limit , we show that the dynamics can be exactly reduced to a single one-dimensional effective stochastic equation, with an effective thermal bath described by kernels that have to be determined self-consistently. We present two complementary derivations, via a dynamical cavity method and via a path-integral approach. From the effective stochastic equation, one can compute dynamical observables such as pressure, shear stress, particle mean-square displacement, and the associated response function. As an application of our results, we derive dynamically the ‘state-following’ equations that describe the response of a glass to quasistatic perturbations, thus bypassing the use of replicas. The article is written in a modular way, that allows the reader to skip the details of the derivations and focus on the physical setting and the main results.

Exploring the diluted ferromagnetic p-spin model with a cavity master equation.

We introduce an alternative solution to Glauber multispin dynamics on random graphs. The solution is based on the recently introduced cavity master equation (CME), a time-closure turning the, in principle, exact dynamic cavity method into a practical method of analysis and of fast simulation. Running CME once is of comparable computational complexity as one Monte Carlo run on the same problem. We show that CME correctly models the ferromagnetic p-spin Glauber dynamics from high temperatures down to and below the spinoidal transition. We also show that CME allows an alternative exploration of the low-temperature spin-glass phase of the model.

Out-of-equilibrium dynamical mean-field equations for the perceptron model

Perceptrons are the building blocks of many theoretical approaches to a wide range of complex systems, ranging from neural networks and deep learning machines, to constraint satisfaction problems, glasses and ecosystems. Despite their applicability and importance, a detailed study of their Langevin dynamics has never been performed yet. Here we derive the mean-field dynamical equations that describe the continuous random perceptron in the thermodynamic limit, in a very general setting with arbitrary noise and friction kernels, not necessarily related by equilibrium relations. We derive the equations in two ways: via a dynamical cavity method, and via a path-integral approach in its supersymmetric formulation. The end point of both approaches is the reduction of the dynamics of the system to an effective stochastic process for a representative dynamical variable. Because the perceptron is formally very close to a system of interacting particles in a high dimensional space, the methods we develop here can be transferred to the study of liquid and glasses in high dimensions. Potentially interesting applications are thus the study of the glass transition in active matter, the study of the dynamics around the jamming transition, and the calculation of rheological properties in driven systems.

Dynamic message-passing equations for models with unidirectional dynamics.

Understanding and quantifying the dynamics of disordered out-of-equilibrium models is an important problem in many branches of science. Using the dynamic cavity method on time trajectories, we construct a general procedure for deriving the dynamic message-passing equations for a large class of models with unidirectional dynamics, which includes the zero-temperature random-field Ising model, the susceptible-infected-recovered model, and rumor spreading models. We show that unidirectionality of the dynamics is the key ingredient that makes the problem solvable. These equations are applicable to single instances of the corresponding problems with arbitrary initial conditions and are asymptotically exact for problems defined on locally treelike graphs. When applied to real-world networks, they generically provide a good analytic approximation of the real dynamics.

Inference of Kinetic Ising Model on Sparse Graphs

Based on dynamical cavity method, we propose an approach to the inference of kinetic Ising model, which asks to reconstruct couplings and external fields from given time-dependent output of original system. Our approach gives an exact result on tree graphs and a good approximation on sparse graphs, it can be seen as an extension of Belief Propagation inference of static Ising model to kinetic Ising model. While existing mean field methods to the kinetic Ising inference e.g., na\" ive mean-field, TAP equation and simply mean-field, use approximations which calculate magnetizations and correlations at time $t$ from statistics of data at time $t-1$, dynamical cavity method can use statistics of data at times earlier than $t-1$ to capture more correlations at different time steps. Extensive numerical experiments show that our inference method is superior to existing mean-field approaches on diluted networks.

Dynamic mean-field and cavity methods for diluted Ising systems

We compare dynamic mean-field and dynamic cavity methods to describe the stationary states of dilute kinetic Ising models. We compute dynamic mean-field theory by expanding in interaction strength to third order, and we compare to the exact dynamic mean-field theory for fully asymmetric networks. We show that in diluted networks, the dynamic cavity method generally predicts magnetizations of individual spins better than both first-order ("naive") and second-order ("TAP") dynamic mean-field theory.

Majority dynamics on trees and the dynamic cavity method

A voter sits on each vertex of an infinite tree of degree $k$, and has to decide between two alternative opinions. At each time step, each voter switches to the opinion of the majority of her neighbors. We analyze this majority process when opinions are initialized to independent and identically distributed random variables. In particular, we bound the threshold value of the initial bias such that the process converges to consensus. In order to prove an upper bound, we characterize the process of a single node in the large $k$-limit. This approach is inspired by the theory of mean field spin-glass and can potentially be generalized to a wider class of models. We also derive a lower bound that is nontrivial for small, odd values of $k$.

Three Lemmas on Dynamic Cavity Method

We study the dynamic cavity method for dilute kinetic Ising models with synchronous update rules. For the parallel update rule we find for fully asymmetric models that the dynamic cavity equations reduce to a Markovian dynamics of the (time-dependent) marginal probabilities. For the random sequential update rule, also an instantiation of a synchronous update rule, we find on the other hand that the dynamic cavity equations do not reduce to a Markovian dynamics, unless an additional assumption of time factorization is introduced. For symmetric models we show that a fixed point of ordinary Belief propagation is also a fixed point of the dynamic cavity equations in the time factorized approximation. For clarity, the conclusions of the paper are formulated as three lemmas.

A message-passing scheme for non-equilibrium stationary states

We study stationary states in a diluted asymmetric (kinetic) Ising model. We apply therecently introduced dynamic cavity method to compute magnetizations of these stationarystates. Depending on the update rule, different versions of the dynamic cavity methodapply. We here study synchronous updates and random sequential updates, and comparelocal properties computed by the dynamic cavity method to numerical simulations.Using both types of updates, the dynamic cavity method is highly accurate athigh enough temperatures. At low enough temperatures, for sequential updatesthe dynamic cavity method tends to a fixed point, but this does not agree withnumerical simulations, while for parallel updates, the dynamic cavity method maydisplay oscillatory behavior. When it converges and is accurate, the dynamic cavitymethod offers a huge speed-up compared to Monte Carlo, particularly for largesystems.