Networks Of Spiking Neurons Research Articles

Reconciliation of weak pairwise spike-train correlations and highly coherent local field potentials across space.

Multi-electrode arrays covering several square millimeters of neural tissue provide simultaneous access to population signals such as extracellular potentials and spiking activity of one hundred or more individual neurons. The interpretation of the recorded data calls for multiscale computational models with corresponding spatial dimensions and signal predictions. Multi-layer spiking neuron network models of local cortical circuits covering about $1\,{\text{mm}^{2}}$ have been developed, integrating experimentally obtained neuron-type-specific connectivity data and reproducing features of observed in-vivo spiking statistics. Local field potentials can be computed from the simulated spiking activity. We here extend a local network and local field potential model to an area of $4\times 4\,{\text{mm}^{2}}$, preserving the neuron density and introducing distance-dependent connection probabilities and conduction delays. We find that the upscaling procedure preserves the overall spiking statistics of the original model and reproduces asynchronous irregular spiking across populations and weak pairwise spike-train correlations in agreement with experimental recordings from sensory cortex. Also compatible with experimental observations, the correlation of local field potential signals is strong and decays over a distance of several hundred micrometers. Enhanced spatial coherence in the low-gamma band around $50\,\text{Hz}$ may explain the recent report of an apparent band-pass filter effect in the spatial reach of the local field potential.

Novel Two-Terminal Synapse/Neuron Based on an Antiferroelectric Hafnium Zirconium Oxide Device for Neuromorphic Computing.

Functionally diverse devices with artificial neuron and synapse properties are critical for neuromorphic systems. We present a two-terminal artificial leaky-integrate-fire (LIF) neuron based on 6 nm Hf0.1Zr0.9O2 (HZO) antiferroelectric (AFE) thin films and develop a synaptic device through work function (WF) engineering. LIF neuron characteristics, including integration, firing, and leakage, are achieved in W/HZO/W devices due to the accumulated polarization and spontaneous depolarization of AFE HZO films. By engineering the top electrode with asymmetric WFs, we found that Au/Ti/HZO/W devices exhibit synaptic weight plasticity, such as paired-pulse facilitation and long-term potentiation/depression, achieving >90% accuracy in digit recognition within constructed artificial neural network systems. These findings suggest that AFE HZO capacitor-based neurons and WF-engineered artificial synapses hold promise for constructing efficient spiking neuron networks and artificial neural networks, thereby advancing neuromorphic computing applications based on emerging AFE HZO devices.

Rosetta stone for the population dynamics of spiking neuron networks.

Populations of spiking neuron models have densities of their microscopic variables (e.g., single-cell membrane potentials) whose evolution fully capture the collective dynamics of biological networks, even outside equilibrium. Despite its general applicability, the Fokker-Planck equationgoverning such evolution is mainly studied within the borders of the linear response theory, although alternative spectral expansion approaches offer some advantages in the study of the out-of-equilibrium dynamics. This is mainly due to the difficulty in computing the state-dependent coefficients of the expanded system of differential equations. Here, we address this issue by deriving analytic expressions for such coefficients by pairing perturbative solutions of the Fokker-Planck approach with their counterparts from the spectral expansion. A tight relationship emerges between several of these coefficients and the Laplace transform of the interspike interval density (i.e., the distribution of first-passage times). "Coefficients" like the current-to-rate gain function, the eigenvalues of the Fokker-Planck operator and its eigenfunctions at the boundaries are derived without resorting to integral expressions. For the leaky integrate-and-fire neurons, the coupling terms between stationary and nonstationary modes are also worked out paving the way to accurately characterize the critical points and the relaxation timescales in networks of interacting populations.

Audio Signal-Stimulated Multilayered HfOx/TiOy Spiking Neuron Network for Neuromorphic Computing.

As the key hardware of a brain-like chip based on a spiking neuron network (SNN), memristor has attracted more attention due to its similarity with biological neurons and synapses to deal with the audio signal. However, designing stable artificial neurons and synapse devices with a controllable switching pathway to form a hardware network is a challenge. For the first time, we report that artificial neurons and synapses based on multilayered HfOx/TiOy memristor crossbar arrays can be used for the SNN training of audio signals, which display the tunable threshold switching and memory switching characteristics. It is found that tunable volatile and nonvolatile switching from the multilayered HfOx/TiOy memristor is induced by the size-controlled atomic oxygen vacancy pathway, which depends on the atomic sublayer in the multilayered structure. The successful emulation of the biological neuron's integrate-and-fire function can be achieved through the utilization of the tunable threshold switching characteristic. Based on the stable performance of the multilayered HfOx/TiOy neuron and synapse, we constructed a hardware SNN architecture for processing audio signals, which provides a base for the recognition of audio signals through the function of integration and firing. Our design of an atomic conductive pathway by using a multilayered TiOy/HfOx memristor supplies a new method for the construction of an artificial neuron and synapse in the same matrix, which can reduce the cost of integration in an AI chip. The implementation of synaptic functionalities by the hardware of SNNs paves the way for novel neuromorphic computing paradigms in the AI era.

Dynamical manifold dimensionality as characterization measure of chimera states in bursting neuronal networks

Methods that distinguish dynamical regimes in networks of active elements make it possible to design the dynamics of models of realistic networks. A particularly salient example of such dynamics is partial synchronization, which may play a pivotal role in emergent behaviors of biological neural networks. Such emergent partial synchronization in structurally homogeneous networks is commonly denoted as chimera states. While several methods for detecting chimeras in networks of spiking neurons have been proposed, these are less effective when applied to networks of bursting neurons. In this study, we propose the correlation dimension as a novel approach that can be employed to identify dynamic network states. To assess the viability of this new method, we study networks of intrinsically bursting Hindmarsh–Rose neurons with non-local connections. In comparison to other measures of chimera states, the correlation dimension effectively characterizes chimeras in burst neurons, whether the incoherence arises in spikes or bursts. The generality of dimensionality measures inherent in the correlation dimension renders this approach applicable to a wide range of dynamic systems, thereby facilitating the comparison of simulated and experimental data. This methodology enhances our ability to tune and simulate intricate network processes, ultimately contributing to a deeper understanding of neural dynamics.

State modulation in spatial networks with three interneuron subtypes.

Several inhibitory interneuron subtypes have been identified as critical in regulating sensory responses. However, the specific contribution of each interneuron subtype remains uncertain. In this work, we explore the contributions of cell-type specific activity and synaptic connections to dynamics of a spatially organized spiking neuron network. We find that the firing rates of the somatostatin (SOM) interneurons align closely with the level of network synchrony irrespective of the target of modulatory input. Further analysis reveals that inhibition from SOM to parvalbumin (PV) interneurons must be limited to allow gradual transitions from asynchrony to synchrony and that the strength of recurrent excitation onto SOM neurons determines the level of synchrony achievable in the network. Our results are consistent with recent experimental findings on cell-type specific manipulations. Overall, our results highlight common dynamic regimes achieved across modulations of different cell populations and identify SOM cells as the main driver of network synchrony.

STCSNN: High energy efficiency spike-train level spiking neural networks with spatio-temporal conversion

Brain-inspired spiking neuron networks (SNNs) have attracted widespread research interest due to their low power features, high biological plausibility, and strong spatiotemporal information processing capability. Although adopting a surrogate gradient (SG) makes the non-differentiability SNN trainable, achieving comparable accuracy for ANNs and keeping low-power features simultaneously is still tricky. In this paper, we proposed an energy-efficient spike-train level spiking neural network with spatio-temporal conversion, which has low computational cost and high accuracy. In the STCSNN, spatio-temporal conversion blocks (STCBs) are proposed to keep the low power features of SNNs and improve accuracy. However, STCSNN cannot adopt backpropagation algorithms directly due to the non-differentiability nature of spike trains. We proposed a suitable learning rule for STCSNNs by deducing the equivalent gradient of STCB. We evaluate the proposed STCSNN on static and neuromorphic datasets, including Fashion-Mnist, Cifar10, Cifar100, TinyImageNet, and DVS-Cifar10. The experiment results show that our proposed STCSNN outperforms the state-of-the-art accuracy on nearly all datasets, using fewer time steps and being highly energy-efficient.

Inference on the Macroscopic Dynamics of Spiking Neurons.

The process of inference on networks of spiking neurons is essential to decipher the underlying mechanisms of brain computation and function. In this study, we conduct inference on parameters and dynamics of a mean-field approximation, simplifying the interactions of neurons. Estimating parameters of this class of generative model allows one to predict the system's dynamics and responses under changing inputs and, indeed, changing parameters. We first assume a set of known state-space equationsand address the problem of inferring the lumped parameters from observed time series. Crucially, we consider this problem in the setting of bistability, random fluctuations in system dynamics, and partial observations, in which some states are hidden. To identify the most efficient estimation or inversion scheme in this particular system identification, we benchmark against state-of-the-art optimization and Bayesian estimation algorithms, highlighting their strengths and weaknesses. Additionally, we explore how well the statistical relationships between parameters are maintained across different scales. We found that deep neural density estimators outperform other algorithms in the inversion scheme, despite potentially resulting in overestimated uncertainty and correlation between parameters. Nevertheless, this issue can be improved by incorporating time-delay embedding. We then eschew the mean-field approximation and employ deep neural ODEs on spiking neurons, illustrating prediction of system dynamics and vector fields from microscopic states. Overall, this study affords an opportunity to predict brain dynamics and responses to various perturbations or pharmacological interventions using deep neural networks.

Investigating the Effects of Transcranial Alternating Current Stimulation on Cortical Oscillations and Network Dynamics.

Transcranial electrical brain stimulation techniques like transcranial direct current (tDCS) and transcranial alternating current (tACS) have emerged as potential tools for treating neurological diseases by modulating cortical excitability. These techniques deliver small electric currents to the brain non-invasively through electrodes on the scalp. tDCS uses constant direct current which weakly alters the membrane voltage of cortical neurons, while tACS utilizes alternating current to target and enhance cortical oscillations, though the underlying mechanisms are not fully understood more specifically. To elucidate how tACS perturbs endogenous network dynamics, we simulated spiking neuron network models. We identified distinct roles of the depolarizing and hyperpolarizing phases in driving network activity towards and away from the strong nonlinearity provided by pyramidal neurons. Exploring resonance effects, we found matching tACS frequency to the network's endogenous resonance frequency creates greater entrainment. Based on this, we developed an algorithm to determine the network's endogenous frequency, phase, and amplitude, then deliver optimized tACS to entrain network oscillations. Together, these computational results provide mechanistic insight into the effects of tACS on network dynamics and could inform future closed-loop tACS systems that dynamically tune stimulation parameters to ongoing brain activity.

Pulse Shape and Voltage-Dependent Synchronization in Spiking Neuron Networks.

Pulse-coupled spiking neural networks are a powerful tool to gain mechanistic insights into how neurons self-organize to produce coherent collective behavior. These networks use simple spiking neuron models, such as the θ-neuron or the quadratic integrate-and-fire (QIF) neuron, that replicate the essential features of real neural dynamics. Interactions between neurons are modeled with infinitely narrow pulses, or spikes, rather than the more complex dynamics of real synapses. To make these networks biologically more plausible, it has been proposed that they must also account for the finite width of the pulses, which can have a significant impact on the network dynamics. However, the derivation and interpretation of these pulses are contradictory, and the impact of the pulse shape on the network dynamics is largely unexplored. Here, I take a comprehensive approach to pulse coupling in networks of QIF and θ-neurons. I argue that narrow pulses activate voltage-dependent synaptic conductances and show how to implement them in QIF neurons such that their effect can last through the phase after the spike. Using an exact low-dimensional description for networks of globally coupled spiking neurons, I prove for instantaneous interactions that collective oscillations emerge due to an effective coupling through the mean voltage. I analyze the impact of the pulse shape by means of a family of smooth pulse functions with arbitrary finite width and symmetric or asymmetric shapes. For symmetric pulses, the resulting voltage coupling is not very effective in synchronizing neurons, but pulses that are slightly skewed to the phase after the spike readily generate collective oscillations. The results unveil a voltage-dependent spike synchronization mechanism at the heart of emergent collective behavior, which is facilitated by pulses of finite width and complementary to traditional synaptic transmission in spiking neuron networks.

Co-existence of synaptic plasticity and metastable dynamics in a spiking model of cortical circuits.

Evidence for metastable dynamics and its role in brain function is emerging at a fast pace and is changing our understanding of neural coding by putting an emphasis on hidden states of transient activity. Clustered networks of spiking neurons have enhanced synaptic connections among groups of neurons forming structures called cell assemblies; such networks are capable of producing metastable dynamics that is in agreement with many experimental results. However, it is unclear how a clustered network structure producing metastable dynamics may emerge from a fully local plasticity rule, i.e., a plasticity rule where each synapse has only access to the activity of the neurons it connects (as opposed to the activity of other neurons or other synapses). Here, we propose a local plasticity rule producing ongoing metastable dynamics in a deterministic, recurrent network of spiking neurons. The metastable dynamics co-exists with ongoing plasticity and is the consequence of a self-tuning mechanism that keeps the synaptic weights close to the instability line where memories are spontaneously reactivated. In turn, the synaptic structure is stable to ongoing dynamics and random perturbations, yet it remains sufficiently plastic to remap sensory representations to encode new sets of stimuli. Both the plasticity rule and the metastable dynamics scale well with network size, with synaptic stability increasing with the number of neurons. Overall, our results show that it is possible to generate metastable dynamics over meaningful hidden states using a simple but biologically plausible plasticity rule which co-exists with ongoing neural dynamics.

The vitals for steady nucleation maps of spontaneous spiking coherence in autonomous two-dimensional neuronal networks

Thin pancake-like neuronal networks cultured on top of a planar microelectrode array have been extensively tried out in neuroengineering, as a substrate for the mobile robot’s control unit, i.e., as a cyborg’s brain. Most of these attempts failed due to intricate self-organizing dynamics in the neuronal systems. In particular, the networks may exhibit an emergent spatial map of steady nucleation sites (“n-sites”) of spontaneous population spikes. Being unpredictable and independent of the surface electrode locations, the n-sites drastically change local ability of the network to generate spikes. Here, using a spiking neuronal network model with generative spatially-embedded connectome, we systematically show in simulations that the number, location, and relative activity of spontaneously formed n-sites (“the vitals”) crucially depend on the samplings of three distributions: (1) the network distribution of neuronal excitability, (2) the distribution of connections between neurons of the network, and (3) the distribution of maximal amplitudes of a single synaptic current pulse. Moreover, blocking the dynamics of a small fraction (about 4%) of non-pacemaker neurons having the highest excitability was enough to completely suppress the occurrence of population spikes and their n-sites. This key result is explained theoretically. Remarkably, the n-sites occur taking into account only short-term synaptic plasticity, i.e., without a Hebbian-type plasticity. As the spiking network model used in this study is strictly deterministic, all simulation results can be accurately reproduced. The model, which has already demonstrated a very high richness-to-complexity ratio, can also be directly extended into the three-dimensional case, e.g., for targeting peculiarities of spiking dynamics in cerebral (or brain) organoids. We recommend the model as an excellent illustrative tool for teaching network-level computational neuroscience, complementing a few benchmark models.

A lightweight data-driven spiking neuronal network model of Drosophila olfactory nervous system with dedicated hardware support.

Data-driven spiking neuronal network (SNN) models enable in-silico analysis of the nervous system at the cellular and synaptic level. Therefore, they are a key tool for elucidating the information processing principles of the brain. While extensive research has focused on developing data-driven SNN models for mammalian brains, their complexity poses challenges in achieving precision. Network topology often relies on statistical inference, and the functions of specific brain regions and supporting neuronal activities remain unclear. Additionally, these models demand huge computing facilities and their simulation speed is considerably slower than real-time. Here, we propose a lightweight data-driven SNN model that strikes a balance between simplicity and reproducibility. The model is built using a qualitative modeling approach that can reproduce key dynamics of neuronal activity. We target the Drosophila olfactory nervous system, extracting its network topology from connectome data. The model was successfully implemented on a small entry-level field-programmable gate array and simulated the activity of a network in real-time. In addition, the model reproduced olfactory associative learning, the primary function of the olfactory system, and characteristic spiking activities of different neuron types. In sum, this paper propose a method for building data-driven SNN models from biological data. Our approach reproduces the function and neuronal activities of the nervous system and is lightweight, acceleratable with dedicated hardware, making it scalable to large-scale networks. Therefore, our approach is expected to play an important role in elucidating the brain's information processing at the cellular and synaptic level through an analysis-by-construction approach. In addition, it may be applicable to edge artificial intelligence systems in the future.

Co-existence of synaptic plasticity and metastable dynamics in a spiking model of cortical circuits.

Evidence for metastable dynamics and its role in brain function is emerging at a fast pace and is changing our understanding of neural coding by putting an emphasis on hidden states of transient activity. Clustered networks of spiking neurons have enhanced synaptic connections among groups of neurons forming structures called cell assemblies; such networks are capable of producing metastable dynamics that is in agreement with many experimental results. However, it is unclear how a clustered network structure producing metastable dynamics may emerge from a fully local plasticity rule, i.e., a plasticity rule where each synapse has only access to the activity of the neurons it connects (as opposed to the activity of other neurons or other synapses). Here, we propose a local plasticity rule producing ongoing metastable dynamics in a deterministic, recurrent network of spiking neurons. The metastable dynamics co-exists with ongoing plasticity and is the consequence of a self-tuning mechanism that keeps the synaptic weights close to the instability line where memories are spontaneously reactivated. In turn, the synaptic structure is stable to ongoing dynamics and random perturbations, yet it remains sufficiently plastic to remap sensory representations to encode new sets of stimuli. Both the plasticity rule and the metastable dynamics scale well with network size, with synaptic stability increasing with the number of neurons. Overall, our results show that it is possible to generate metastable dynamics over meaningful hidden states using a simple but biologically plausible plasticity rule which co-exists with ongoing neural dynamics.

Spiking networks that efficiently process dynamic sensory features explain receptor information mixing in somatosensory cortex.

How do biological neural systems efficiently encode, transform and propagate information between the sensory periphery and the sensory cortex about sensory features evolving at different time scales? Are these computations efficient in normative information processing terms? While previous work has suggested that biologically plausible models of of such neural information processing may be implemented efficiently within a single processing layer, how such computations extend across several processing layers is less clear. Here, we model propagation of multiple time-varying sensory features across a sensory pathway, by extending the theory of efficient coding with spikes to efficient encoding, transformation and transmission of sensory signals. These computations are optimally realized by a multilayer spiking network with feedforward networks of spiking neurons (receptor layer) and recurrent excitatory-inhibitory networks of generalized leaky integrate-and-fire neurons (recurrent layers). Our model efficiently realizes a broad class of feature transformations, including positive and negative interaction across features, through specific and biologically plausible structures of feedforward connectivity. We find that mixing of sensory features in the activity of single neurons is beneficial because it lowers the metabolic cost at the network level. We apply the model to the somatosensory pathway by constraining it with parameters measured empirically and include in its last node, analogous to the primary somatosensory cortex (S1), two types of inhibitory neurons: parvalbumin-positive neurons realizing lateral inhibition, and somatostatin-positive neurons realizing winner-take-all inhibition. By implementing a negative interaction across stimulus features, this model captures several intriguing empirical observations from the somatosensory system of the mouse, including a decrease of sustained responses from subcortical networks to S1, a non-linear effect of the knock-out of receptor neuron types on the activity in S1, and amplification of weak signals from sensory neurons across the pathway.

Correlations reveal the hierarchical organization of biological networks with latent variables

Deciphering the functional organization of large biological networks is a major challenge for current mathematical methods. A common approach is to decompose networks into largely independent functional modules, but inferring these modules and their organization from network activity is difficult, given the uncertainties and incompleteness of measurements. Typically, some parts of the overall functional organization, such as intermediate processing steps, are latent. We show that the hidden structure can be determined from the statistical moments of observable network components alone, as long as the functional relevance of the network components lies in their mean values and the mean of each latent variable maps onto a scaled expectation of a binary variable. Whether the function of biological networks permits a hierarchical modularization can be falsified by a correlation-based statistical test that we derive. We apply the test to gene regulatory networks, dendrites of pyramidal neurons, and networks of spiking neurons.

Persistent spiking activity in neuromorphic circuits incorporating post-inhibitory rebound excitation

Objective. This study introduces a novel approach for integrating the post-inhibitory rebound excitation (PIRE) phenomenon into a neuronal circuit. Excitatory and inhibitory synapses are designed to establish a connection between two hardware neurons, effectively forming a network. The model demonstrates the occurrence of PIRE under strong inhibitory input. Emphasizing the significance of incorporating PIRE in neuromorphic circuits, the study showcases generation of persistent activity within cyclic and recurrent spiking neuronal networks. Approach. The neuronal and synaptic circuits are designed and simulated in Cadence Virtuoso using TSMC 180 nm technology. The operating mechanism of the PIRE phenomenon integrated into a hardware neuron is discussed. The proposed circuit encompasses several parameters for effectively controlling multiple electrophysiological features of a neuron. Main results. The neuronal circuit has been tuned to match the response of a biological neuron. The efficiency of this circuit is evaluated by computing the average power dissipation and energy consumption per spike through simulation. The sustained firing of neural spikes is observed till 1.7 s using the two neuronal networks. Significance. Persistent activity has significant implications for various cognitive functions such as working memory, decision-making, and attention. Therefore, hardware implementation of these functions will require our PIRE-integrated model. Energy-efficient neuromorphic systems are useful in many artificial intelligence applications, including human-machine interaction, IoT devices, autonomous systems, and brain-computer interfaces.

Learning spiking neuronal networks with artificial neural networks: neural oscillations.

First-principles-based modelings have been extremely successful in providing crucial insights and predictions for complex biological functions and phenomena. However, they can be hard to build and expensive to simulate for complex living systems. On the other hand, modern data-driven methods thrive at modeling many types of high-dimensional and noisy data. Still, the training and interpretation of these data-driven models remain challenging. Here, we combine the two types of methods to model stochastic neuronal network oscillations. Specifically, we develop a class of artificial neural networks to provide faithful surrogates to the high-dimensional, nonlinear oscillatory dynamics produced by a spiking neuronal network model. Furthermore, when the training data set is enlarged within a range of parameter choices, the artificial neural networks become generalizable to these parameters, covering cases in distinctly different dynamical regimes. In all, our work opens a new avenue for modeling complex neuronal network dynamics with artificial neural networks.

Fast learning without synaptic plasticity in spiking neural networks

Spiking neural networks are of high current interest, both from the perspective of modelling neural networks of the brain and for porting their fast learning capability and energy efficiency into neuromorphic hardware. But so far we have not been able to reproduce fast learning capabilities of the brain in spiking neural networks. Biological data suggest that a synergy of synaptic plasticity on a slow time scale with network dynamics on a faster time scale is responsible for fast learning capabilities of the brain. We show here that a suitable orchestration of this synergy between synaptic plasticity and network dynamics does in fact reproduce fast learning capabilities of generic recurrent networks of spiking neurons. This points to the important role of recurrent connections in spiking networks, since these are necessary for enabling salient network dynamics. We show more specifically that the proposed synergy enables synaptic weights to encode more general information such as priors and task structures, since moment-to-moment processing of new information can be delegated to the network dynamics.

A robust balancing mechanism for spiking neural networks.

Dynamical balance of excitation and inhibition is usually invoked to explain the irregular low firing activity observed in the cortex. We propose a robust nonlinear balancing mechanism for a random network of spiking neurons, which works also in the absence of strong external currents. Biologically, the mechanism exploits the plasticity of excitatory-excitatory synapses induced by short-term depression. Mathematically, the nonlinear response of the synaptic activity is the key ingredient responsible for the emergence of a stable balanced regime. Our claim is supported by a simple self-consistent analysis accompanied by extensive simulations performed for increasing network sizes. The observed regime is essentially fluctuation driven and characterized by highly irregular spiking dynamics of all neurons.