Asynchronous Irregular Regime Research Articles

Metamodelling of a two-population spiking neural network.

In computational neuroscience, hypotheses are often formulated as bottom-up mechanistic models of the systems in question, consisting of differential equations that can be numerically integrated forward in time. Candidate models can then be validated by comparison against experimental data. The model outputs of neural network models depend on both neuron parameters, connectivity parameters and other model inputs. Successful model fitting requires sufficient exploration of the model parameter space, which can be computationally demanding. Additionally, identifying degeneracy in the parameters, i.e. different combinations of parameter values that produce similar outputs, is of interest, as they define the subset of parameter values consistent with the data. In this computational study, we apply metamodels to a two-population recurrent spiking network of point-neurons, the so-called Brunel network. Metamodels are data-driven approximations to more complex models with more desirable computational properties, which can be run considerably faster than the original model. Specifically, we apply and compare two different metamodelling techniques, masked autoregressive flows (MAF) and deep Gaussian process regression (DGPR), to estimate the power spectra of two different signals; the population spiking activities and the local field potential. We find that the metamodels are able to accurately model the power spectra in the asynchronous irregular regime, and that the DGPR metamodel provides a more accurate representation of the simulator compared to the MAF metamodel. Using the metamodels, we estimate the posterior probability distributions over parameters given observed simulator outputs separately for both LFP and population spiking activities. We find that these distributions correctly identify parameter combinations that give similar model outputs, and that some parameters are significantly more constrained by observing the LFP than by observing the population spiking activities.

Online Learning and Memory of Neural Trajectory Replays for Prefrontal Persistent and Dynamic Representations in the Irregular Asynchronous State.

In the prefrontal cortex (PFC), higher-order cognitive functions and adaptive flexible behaviors rely on continuous dynamical sequences of spiking activity that constitute neural trajectories in the state space of activity. Neural trajectories subserve diverse representations, from explicit mappings in physical spaces to generalized mappings in the task space, and up to complex abstract transformations such as working memory, decision-making and behavioral planning. Computational models have separately assessed learning and replay of neural trajectories, often using unrealistic learning rules or decoupling simulations for learning from replay. Hence, the question remains open of how neural trajectories are learned, memorized and replayed online, with permanently acting biological plasticity rules. The asynchronous irregular regime characterizing cortical dynamics in awake conditions exerts a major source of disorder that may jeopardize plasticity and replay of locally ordered activity. Here, we show that a recurrent model of local PFC circuitry endowed with realistic synaptic spike timing-dependent plasticity and scaling processes can learn, memorize and replay large-size neural trajectories online under asynchronous irregular dynamics, at regular or fast (sped-up) timescale. Presented trajectories are quickly learned (within seconds) as synaptic engrams in the network, and the model is able to chunk overlapping trajectories presented separately. These trajectory engrams last long-term (dozen hours) and trajectory replays can be triggered over an hour. In turn, we show the conditions under which trajectory engrams and replays preserve asynchronous irregular dynamics in the network. Functionally, spiking activity during trajectory replays at regular timescale accounts for both dynamical coding with temporal tuning in individual neurons, persistent activity at the population level, and large levels of variability consistent with observed cognitive-related PFC dynamics. Together, these results offer a consistent theoretical framework accounting for how neural trajectories can be learned, memorized and replayed in PFC networks circuits to subserve flexible dynamic representations and adaptive behaviors.

Biologically Realistic Mean-Field Models of Conductance-Based Networks of Spiking Neurons with Adaptation.

Accurate population models are needed to build very large-scale neural models, but their derivation is difficult for realistic networks of neurons, in particular when nonlinear properties are involved, such as conductance-based interactions and spike-frequency adaptation. Here, we consider such models based on networks of adaptive exponential integrate-and-fire excitatory and inhibitory neurons. Using a master equation formalism, we derive a mean-field model of such networks and compare it to the full network dynamics. The mean-field model is capable of correctly predicting the average spontaneous activity levels in asynchronous irregular regimes similar to in vivo activity. It also captures the transient temporal response of the network to complex external inputs. Finally, the mean-field model is also able to quantitatively describe regimes where high- and low-activity states alternate (up-down state dynamics), leading to slow oscillations. We conclude that such mean-field models are biologically realistic in the sense that they can capture both spontaneous and evoked activity, and they naturally appear as candidates to build very large-scale models involving multiple brain areas.

Effective connectivity analysis explains metastable states of ongoing activity in cortically embedded systems of coupled synfire chains

In models of the cortex, synaptic connectivity is often assumed to be random within broad constraints such as interlayer connection densities. Alternatively, the connectivity could include richly structured circuitry. A recent study [1] demonstrated this in a model of local cortex of order 1 mm3 in size: a large number of synfire chains (small pools of neurons sequentially linked by feedforward connections) activated by waves (sequences of propagating spike packets) were embedded in a recurrent network of excitatory and inhibitory neurons. The model exhibits stable global dynamics in the asynchronous irregular regime and stable propagation of multiple synfire waves. Background noise generated by waves destabilizes wave propagation, providing a negative feedback signal limiting their number. Here we add inter-chain couplings (each chain branches to two successors) and variability in chain strengths to obtain a recurrent system with a topography. We study emergent patterns of activity propagation in such a system. Ongoing endogenous activity due to branching of waves and regulation of wave activity by noise feedback is typically found. Across model realizations and runs, ongoing activity manifests diverse steady-state patterns and transitions. We argue that steady states arise jointly and consistently with an effective connectivity: strength-dependent chain traversal probabilities averaged over noise fluctuations. By excluding chains with sub-threshold traversal probability we derive a family of effective coupling graphs (ECGs) parameterized by the global activity level. The distribution of wave activity in steady states is largely confined to the islands of circulation: strongly connected components (SCCs) and their associated out-components (OCs) in optimally chosen ECGs. A condensed ECG (cECG) allows the relationship between activity and effective connectivity to be visualized (Figure ​(Figure11). Figure 1 A The cECG with SCCs and OCs partitioned by their overlaps as nodes. B Chain indices of traversal events versus time (ms), permuted so chains in each cECG node are adjacent and colored accordingly. The system is implemented both as a large-scale network of integrate-and-fire neurons and as a reduced model with binary-state pools as basic units. The reduced model exhibits activity patterns very similar to those of the full model and provides a valuable tool for studying the latter. We propose that the principle whereby activity patterns arise in concert with dynamically tuned effective connectivity applies to a broad class of networks with complex topologies.

On the influence of inhibitory STDP on balanced state random networks

The distribution of synaptic efficacies in neural networks takes fundamental influence on their dynamics and the modification of synaptic strengths forms the foundation of learning and memory. A prominent plasticity rule that has been observed in vitro is spike-timing-dependent plasticity (STDP). While first studied in glutamatergic synapses, recently also STDP of GABAergic synapses came into the focus of experimental and theoretical research [1]. We study random balanced state networks of leaky integrate-and-fire neurons in the asynchronous irregular (AI) regime [2] that is believed to be a good theoretical fit to the activity of cortical networks in vivo. We consider driven networks that receive Poisson input as well as networks in a self-sustained state of activity. In order to assess the influence of excitatory and inhibitory STDP on the network dynamics, we introduce these two plasticity rules independently, observing network dynamics and weight distributions after a transient phase. Note that both additive and multiplicative STDP rules yield the same network dynamics as described below. When introducing excitatory STDP alone, parameters involving the maximal weight have to be fine-tuned in order to keep the network activity stably in the AI regime [3]. For almost all parameter values the network activity becomes unstable, leaving the AI regime and settling in a pathological, highly synchronized state with saturated firing rates of most cells, see Figure ​Figure1A.1A. We also observed that even without STDP, few strong excitatory connections can substantially destabilize network dynamics yielding pathological states. Interestingly, this destabilization does not happen when in addition to excitatory STDP we also introduce STDP for inhibitory synapses projecting onto excitatory cells. The latter setup results in a network that stably rests in the AI regime, see Figure ​Figure1A.1A. Both STDP rules yield near-Gaussian distributions of synaptic weights, see Figure ​Figure1B.1B. Inhibitory STDP even manages to stabilize a network that was brought to a pathological state by excitatory STDP, see Figure ​Figure1A.1A. This clearly shows that inhibitory STDP has a stabilizing effect on network dynamics and we expect that especially in combination with synaptic scaling and in the context of clustered networks [4] other non-trivial dynamical effects will become visible. Figure 1 A. Raster plot of 30 randomly sampled cells showing network activity. Red line: activation of excitatory STDP, green line: activation of inhibitory STDP. B. Weight distributions of plastic synapses converging onto 100 randomly sampled excitatory neurons. ...

Influence of different types of downscaling on a cortical microcircuit model

Neural network models are routinely downscaled in terms of numbers of neurons or synapses because of a lack of computational resources or the limited capacity of a given neuromorphic hardware. Unfortunately the downscaling is often performed without explicit mention of the limitations this entails [1]. This is relevant since downscaling can substantially affect the dynamics. For instance, reducing the number of neurons N while preserving in-degrees K increases shared inputs and hence correlations. Theoretical results on scaling are derived using simplifying assumptions. Therefore we use simulations to systematically investigate the effects of downscaling on the dynamics of a layered microcircuit model of early sensory cortex [2]. The model consists of eight excitatory and inhibitory populations of leaky integrate-and-fire point neurons with current-based synapses and homogeneous Poisson input. The full-scale network, comprising approximately 80,000 neurons and 0.3 billion synapses, is in the balanced state [3,4], displaying asynchronous irregular activity. For networks of binary or integrate-and-fire neurons in this state, the total current-based synaptic input to a given neuron is well approximated by a Gaussian noise, of which the mean and variance determine the firing rate. One way of preserving this mean and variance has been touched upon: reducing N but keeping K constant. For reduced K, firing rates can be preserved by an appropriate choice of synaptic weights, the ratio between excitatory and inhibitory weights, and mean external input [3,4]. However, these scaling methods are based on single-neuron dynamics and ignore network effects. For instance, scaling synaptic weights J and mean external input to preserve rates implies constant J2 K [3], which changes the feedback on the population level and hence qualitatively changes the network state [5]. An alternative that takes into account both single-neuron and network effects is to choose J so that J K is constant and adjusting the external drive to maintain the input variance. This preserves the shape of cross-correlation functions in balanced E-I networks [5]. However, the magnitude of the correlations increases as K decreases, and a restriction to positive external input variance limits the compensation for increased intrinsic variance. We consider the effects of the above scaling options on correlations, firing rates, synchrony, and irregularity of spiking activity. Although each method achieves near-constancy of one or more dynamical features, none preserves all of them. These results can be used to downscale networks of neurons with current-based synapses in the asynchronous irregular regime in a controlled manner. However, since it is not yet possible to analytically predict all the effects of scaling, it remains essential to compare down-scaled simulations with their full-scale counterparts.

Taming the model zoo: a unified view on correlations in recurrent networks

The meaning of correlated neural activity for the processing and representation of information in cortical networks is still not understood, but evidence for a pivotal role of correlations increases [1]. Recent theoretical work has shown [2-4] that balanced recurrent networks of binary model neurons [3] and spiking integrate-and-fire models [2-4] are able to produce weak correlations despite common input to pairs of cells. For binary model neurons, the theory of correlations in recurrent networks is well established [5]. Investigating learning in recurrent networks with spike-timing dependent plasticity requires spiking neuron models. Theoretical work often employs linear stochastic point process models [6] for their analytic tractability [7]. The diversity of neuron models used in contemporary theoretical neuroscience brings up the question, which features of correlations are generic properties of recurrent networks and which are peculiarities of the often abstracted neuronal dynamics. Moreover, the variety of different theories employed to describe pairwise correlations in neural networks is confusing at times, even for experts in the field. Currently it is unclear how different neuron models relate to each other and whether and how results obtained with one model carry over to another. In this work we present a unified theoretical view on pairwise correlations in recurrent random networks. We consider binary neuron models, leaky integrate-and-fire models, and linear point process models. For networks in the asynchronous irregular regime, we show that these models can be mapped to either of two definitions of an Ornstein-Uhlenbeck (OU) process [8]. The distinction between both classes is how the effective noise enters the model: Leaky integrate-and-fire models and spiking point process models belong to the class with noise on the output side, the binary neuron model is equivalent to an OU process with noise on the input side. The closed solution for the correlation structure of OU processes [8] holds for both classes. We extend this solution to the presence of synaptic conduction delays. The presented theory recovers and unifies the theories of correlations for binary neurons [5] and linear point processes [7] and generalizes both models to the case of finite conduction delays. Moreover we obtain a good approximation for the temporal structure of correlations for the spiking leaky integrate-and-fire model in the asynchronous regime [9]. Finally we show that the oscillatory instability known for networks of integrate-and-fire models [9] is a model-invariant feature of any of the studied dynamics and we explain the class dependent differences in the temporal shape of correlation functions.

Towards a unified theory of correlations in recurrent neural networks

Recent theoretical progress elucidated, why in recurrent networks the magnitude of correlations is much smaller than expected considering the amount of common input received by pairs of neurons [1,2]. The net inhibitory feedback is the underlying reason for the suppression of correlated activity in inhibition dominated networks [2], explaining the earlier observation of the cancellation of excitatory and inhibitory fluctuations in the synaptic afferents to each neuron in the network [1]. In [2] Fokker-Planck theory is used to reduce the integrate-and-fire dynamics to an effective linear rate model. Averaging over neuronal populations leads to closed form expressions for the time integrated pairwise correlation. A different line of theoretical work uses Hawkes theory to study correlations [3]. Recently this theory was extended to capture inhibitory coupling and delayed interaction [4]. The latter work exposes that the asymmetry of the cross-correlation function relating the spiking of excitatory and inhibitory neurons is due to the reverberation of the network caused by the spikes of the two neurons. In networks without time delay the asymmetry appears as a time lag of inhibition with respect to excitation [1]. The predicted correlation functions, however, deviate quantitatively from direct simulations of integrate-and-fire networks. In the current work we present a unification of both theories [2,4] aiming at a self-consistent description of time-lagged correlations in recurrent networks. The new theory is governed by a convolution equation for the cross-correlation resulting from a linear perturbation expansion of the neuronal dynamics. The perturbative approach is similar to [2] and the convolution equation is structurally the same as in [3]. For the example of a balanced recurrent network we obtain self-sonsistent expressions of the population-averaged correlation structure in the asynchronous irregular regime [5]. Our results elucidate the interplay between the single spike train auto-covariance and the cross-covariance in the network. The unification of firing rate approximations [2], point process models [3,6], and integrate-and-fire dynamics [5] is a step towards a coherent picture of the correlation structure of recurrent networks enabling the mapping of results from abstract models [1,6,7] to biologically more realistic ones.