Simulation Of Spiking Neural Networks Research Articles

Commentary: Accelerating spiking neural network simulations with PymoNNto and PymoNNtorch.

Commentary: Accelerating spiking neural network simulations with PymoNNto and PymoNNtorch.

Compact Physical Implementation of Spiking Neural Network Using Ambipolar WSe2 n-Type/p-Type Ferroelectric Field-Effect Transistor.

Spiking neural networks (SNNs) are attracting increasing interests for their ability to emulate biological processes, offering energy-efficient computation and event-driven processing. Currently, no devices are known to combine both neuronal and synaptic functions. This study presents an experimental demonstration of an ambipolar WSe2 n-type/p-type ferroelectric field-effect transistor (n/p-FeFET) integrated with ferroelectric Hf0.5Zr0.5O2 (HZO) to achieve both volatile and nonvolatile properties in a single device. The nonvolatile n-FeFET, driven by the stable ferroelectric properties of HZO, exhibits highly linear synaptic behavior. In contrast, the volatile p-FeFET, influenced by electron self-compensation in the ambipolar WSe2, enables self-resetting leaky-integrate-and-fire neurons. Integrating neuronal and synaptic functions in the same device allows for compact neuromorphic computing applications. Additionally, simulations of SNNs using experimentally calibrated synaptic and neuronal models achieved a 93.8% accuracy in MNIST digit recognition. This innovative approach advances the development of SNNs with high biomimetic fidelity and reduced hardware costs.

A neural basis for learning sequential memory in brain loop structures.

Behaviors often involve a sequence of events, and learning and reproducing it is essential for sequential memory. Brain loop structures refer to loop-shaped inter-regional connection structures in the brain such as cortico-basal ganglia-thalamic and cortico-cerebellar loops. They are thought to play a crucial role in supporting sequential memory, but it is unclear what properties of the loop structure are important and why. In this study, we investigated conditions necessary for the learning of sequential memory in brain loop structures via computational modeling. We assumed that sequential memory emerges due to delayed information transmission in loop structures and presented a basic neural activity model and validated our theoretical considerations with spiking neural network simulations. Based on this model, we described the factors for the learning of sequential memory: first, the information transmission delay should decrease as the size of the loop structure increases; and second, the likelihood of the learning of sequential memory increases as the size of the loop structure increases and soon saturates. Combining these factors, we showed that moderate-sized brain loop structures are advantageous for the learning of sequential memory due to the physiological restrictions of information transmission delay. Our results will help us better understand the relationship between sequential memory and brain loop structures.

Spike-Based Neuromorphic Hardware for Dynamic Tactile Perception with a Self-Powered Mechanoreceptor Array.

A self-powered mechanoreceptor array is demonstrated using four mechanoreceptor cells for recognition of dynamic touch gestures. Each cell consists of a triboelectric nanogenerator (TENG) for touch sensing and a bi-stable resistor (biristor) for spike encoding. It produces informative spike signals by sensing a force of an external touch and encoding the force into the number of spikes. An array of the mechanoreceptor cells is utilized to monitor various touch gestures and it successfully generated spike signals corresponding to all the gestures. To validate the practicality of the mechanoreceptor array, a spiking neural network (SNN), highly attractive for power consumption compared to the conventional von Neumann architecture, is used for the identification of touch gestures. The measured spiking signals are reflected as inputs for the SNN simulations. Consequently, touch gestures are classified with a high accuracy rate of 92.5%. The proposed mechanoreceptor array emerges as a promising candidate for a building block of tactile in-sensor computing in the era of the Internet of Things (IoT), due to the low cost and high manufacturability of the TENG. This eliminates the need for a power supply, coupled with the intrinsic high throughput of the Si-based biristor employing complementary metal-oxide-semiconductor (CMOS) technology.

Graphene-Based Artificial Dendrites for Bio-Inspired Learning in Spiking Neuromorphic Systems.

Analog neuromorphic computing systems emulate the parallelism and connectivity of the human brain, promising greater expressivity and energy efficiency compared to those of digital systems. Though many devices have emerged as candidates for artificial neurons and artificial synapses, there have been few device candidates for artificial dendrites. In this work, we report on biocompatible graphene-based artificial dendrites (GrADs) that can implement dendritic processing. By using a dual side-gate configuration, current applied through a Nafion membrane can be used to control device conductance across a trilayer graphene channel, showing spatiotemporal responses of leaky recurrent, alpha, and Gaussian dendritic potentials. The devices can be variably connected to enable higher-order neuronal responses, and we show through data-driven spiking neural network simulations that spiking activity is reduced by ≤15% without accuracy loss while low-frequency operation is stabilized. This positions the GrADs as strong candidates for energy efficient bio-interfaced spiking neural networks.

Scaling neural simulations in STACS

As modern neuroscience tools acquire more details about the brain, the need to move towards biological-scale neural simulations continues to grow. However, effective simulations at scale remain a challenge. Beyond just the tooling required to enable parallel execution, there is also the unique structure of the synaptic interconnectivity, which is globally sparse but has relatively high connection density and non-local interactions per neuron. There are also various practicalities to consider in high performance computing applications, such as the need for serializing neural networks to support potentially long-running simulations that require checkpoint-restart. Although acceleration on neuromorphic hardware is also a possibility, development in this space can be difficult as hardware support tends to vary between platforms and software support for larger scale models also tends to be limited.In this paper, we focus our attention on Simulation Tool for Asynchronous Cortical Streams (STACS), a spiking neural network simulator that leverages the Charm++ parallel programming framework, with the goal of supporting biological-scale simulations as well as interoperability between platforms. Central to these goals is the implementation of scalable data structures suitable for efficiently distributing a network across parallel partitions. Here, we discuss a straightforward extension of a parallel data format with a history of use in graph partitioners, which also serves as a portable intermediate representation for different neuromorphic backends.We perform scaling studies on the Summit supercomputer, examining the capabilities of STACS in terms of network build and storage, partitioning, and execution. We highlight how a suitably partitioned, spatially dependent synaptic structure introduces a communication workload well-suited to the multicast communication supported by Charm++. We evaluate the strong and weak scaling behavior for networks on the order of millions of neurons and billions of synapses, and show that STACS achieves competitive levels of parallel efficiency.

Release 2.0 — NEMSIM-RT: A real-time distributed spiking neural network simulator

NESIM-RT is a specialized tool designed for simulating neuromorphic systems. In this new release we extend its capabilities to include state-of-the art models like the AdexLIF and Izhikevich, and to incorporate dynamic synaptic mechanisms such as Spike-Timing Dependent Plasticity (STDP). With these new features, researchers can now observe in real-time how different parameters influence these models and learning rules, thereby gaining deeper insights into neuronal function and network dynamics.

Accelerating spiking neural network simulations with PymoNNto and PymoNNtorch.

Spiking neural network simulations are a central tool in Computational Neuroscience, Artificial Intelligence, and Neuromorphic Engineering research. A broad range of simulators and software frameworks for such simulations exist with different target application areas. Among these, PymoNNto is a recent Python-based toolbox for spiking neural network simulations that emphasizes the embedding of custom code in a modular and flexible way. While PymoNNto already supports GPU implementations, its backend relies on NumPy operations. Here we introduce PymoNNtorch, which is natively implemented with PyTorch while retaining PymoNNto's modular design. Furthermore, we demonstrate how changes to the implementations of common network operations in combination with PymoNNtorch's native GPU support can offer speed-up over conventional simulators like NEST, ANNarchy, and Brian 2 in certain situations. Overall, we show how PymoNNto's modular and flexible design in combination with PymoNNtorch's GPU acceleration and optimized indexing operations facilitate research and development of spiking neural networks in the Python programming language.

Hippocampome.org 2.0 is a knowledge base enabling data-driven spiking neural network simulations of rodent hippocampal circuits.

Hippocampome.org is a mature open-access knowledge base of the rodent hippocampal formation focusing on neuron types and their properties. Previously, Hippocampome.org v1.0 established a foundational classification system identifying 122 hippocampal neuron types based on their axonal and dendritic morphologies, main neurotransmitter, membrane biophysics, and molecular expression (Wheeler et al., 2015). Releases v1.1 through v1.12 furthered the aggregation of literature-mined data, including among others neuron counts, spiking patterns, synaptic physiology, in vivo firing phases, and connection probabilities. Those additional properties increased the online information content of this public resource over 100-fold, enabling numerous independent discoveries by the scientific community. Hippocampome.org v2.0, introduced here, besides incorporating over 50 new neuron types, now recenters its focus on extending the functionality to build real-scale, biologically detailed, data-driven computational simulations. In all cases, the freely downloadable model parameters are directly linked to the specific peer-reviewed empirical evidence from which they were derived. Possible research applications include quantitative, multiscale analyses of circuit connectivity and spiking neural network simulations of activity dynamics. These advances can help generate precise, experimentally testable hypotheses and shed light on the neural mechanisms underlying associative memory and spatial navigation.

Hippocampome.org 2.0 is a knowledge base enabling data-driven spiking neural network simulations of rodent hippocampal circuits

Hippocampome.org is a mature open-access knowledge base of the rodent hippocampal formation focusing on neuron types and their properties. Previously, Hippocampome.org v1.0 established a foundational classification system identifying 122 hippocampal neuron types based on their axonal and dendritic morphologies, main neurotransmitter, membrane biophysics, and molecular expression (Wheeler et al., 2015). Releases v1.1 through v1.12 furthered the aggregation of literature-mined data, including among others neuron counts, spiking patterns, synaptic physiology, in vivo firing phases, and connection probabilities. Those additional properties increased the online information content of this public resource over 100-fold, enabling numerous independent discoveries by the scientific community. Hippocampome.org v2.0, introduced here, besides incorporating over 50 new neuron types, now recenters its focus on extending the functionality to build real-scale, biologically detailed, data-driven computational simulations. In all cases, the freely downloadable model parameters are directly linked to the specific peer-reviewed empirical evidence from which they were derived. Possible research applications include quantitative, multiscale analyses of circuit connectivity and spiking neural network simulations of activity dynamics. These advances can help generate precise, experimentally testable hypotheses and shed light on the neural mechanisms underlying associative memory and spatial navigation.

Review of common spiking neural network simulation tools

As a recognized third-generation neural network, the spiking neural network has high concurrency and complexity. Although the degree of research is still far from the previous generation of neural networks, spiking neural networks are excellent in performance and energy consumption. In this paper, the common spiking neural network simulation tools are reviewed. The most frequently used and mentioned tools are NEURON, NEST, and BRAIN. NEURON is more suitable for simulation based on biological applications, pays more attention to biological characteristics, and can support large-scale network simulation. Examples used in the official documentation are neural simulations of invertebrates and mammals. Large heterogeneous networks of point neurons or neurons with a few compartments are frequently simulated using NEST. In contrast to models that concentrate on the specific morphological and biophysical characteristics of individual neurons, NEST is appropriate for those that emphasize the dynamics, size, and structure of the nervous system. Brian was originally designed for research and teaching and is well suited as a teaching and presentation tool for simulating and observing the effects of different parameters for classical neural network projects such as picture classification. In addition, CSIM, SPLIT, SPINNAKER, and other tools also have their merits, but due to the low frequency of relevant references and lack of universality, this study will not give a detailed introduction.

SHIP: a computational framework for simulating and validating novel technologies in hardware spiking neural networks.

Investigations in the field of spiking neural networks (SNNs) encompass diverse, yet overlapping, scientific disciplines. Examples range from purely neuroscientific investigations, researches on computational aspects of neuroscience, or applicative-oriented studies aiming to improve SNNs performance or to develop artificial hardware counterparts. However, the simulation of SNNs is a complex task that can not be adequately addressed with a single platform applicable to all scenarios. The optimization of a simulation environment to meet specific metrics often entails compromises in other aspects. This computational challenge has led to an apparent dichotomy of approaches, with model-driven algorithms dedicated to the detailed simulation of biological networks, and data-driven algorithms designed for efficient processing of large input datasets. Nevertheless, material scientists, device physicists, and neuromorphic engineers who develop new technologies for spiking neuromorphic hardware solutions would find benefit in a simulation environment that borrows aspects from both approaches, thus facilitating modeling, analysis, and training of prospective SNN systems. This manuscript explores the numerical challenges deriving from the simulation of spiking neural networks, and introduces SHIP, Spiking (neural network) Hardware In PyTorch, a numerical tool that supports the investigation and/or validation of materials, devices, small circuit blocks within SNN architectures. SHIP facilitates the algorithmic definition of the models for the components of a network, the monitoring of states and output of the modeled systems, and the training of the synaptic weights of the network, by way of user-defined unsupervised learning rules or supervised training techniques derived from conventional machine learning. SHIP offers a valuable tool for researchers and developers in the field of hardware-based spiking neural networks, enabling efficient simulation and validation of novel technologies.

Chip-In-Loop SNN Proxy Learning: a new method for efficient training of spiking neural networks.

The primary approaches used to train spiking neural networks (SNNs) involve either training artificial neural networks (ANNs) first and then transforming them into SNNs, or directly training SNNs using surrogate gradient techniques. Nevertheless, both of these methods encounter a shared challenge: they rely on frame-based methodologies, where asynchronous events are gathered into synchronous frames for computation. This strays from the authentic asynchronous, event-driven nature of SNNs, resulting in notable performance degradation when deploying the trained models on SNN simulators or hardware chips for real-time asynchronous computation. To eliminate this performance degradation, we propose a hardware-based SNN proxy learning method that is called Chip-In-Loop SNN Proxy Learning (CIL-SPL). This approach effectively eliminates the performance degradation caused by the mismatch between synchronous and asynchronous computations. To demonstrate the effectiveness of our method, we trained models using public datasets such as N-MNIST and tested them on the SNN simulator or hardware chip, comparing our results to those classical training methods.

Memory-efficient neurons and synapses for spike-timing-dependent-plasticity in large-scale spiking networks.

This paper addresses the challenges posed by frequent memory access during simulations of large-scale spiking neural networks involving synaptic plasticity. We focus on the memory accesses performed during a common synaptic plasticity rule since this can be a significant factor limiting the efficiency of the simulations. We propose neuron models that are represented by only three state variables, which are engineered to enforce the appropriate neuronal dynamics. Additionally, memory retrieval is executed solely by fetching postsynaptic variables, promoting a contiguous memory storage and leveraging the capabilities of burst mode operations to reduce the overhead associated with each access. Different plasticity rules could be implemented despite the adopted simplifications, each leading to a distinct synaptic weight distribution (i.e., unimodal and bimodal). Moreover, our method requires fewer average memory accesses compared to a naive approach. We argue that the strategy described can speed up memory transactions and reduce latencies while maintaining a small memory footprint.

Algorithms for Fast Spiking Neural Network Simulation on FPGAs

Algorithms for Fast Spiking Neural Network Simulation on FPGAs

Circuit implementation of on-chip trainable spiking neural network using CMOS based memristive STDP synapses and LIF neurons

Computation on a large volume of data at high speed and low power requires energy-efficient architectures for edge computing applications. As a result, scientists focus on memristive circuits and systems for area and energy efficiency. Spiking neural network (SNN) with bio-inspired spike-timing-dependent plasticity learning (STDP) is a promising solution for energy-efficient neuromorphic systems than conventional artificial neural network (ANN). Previous works on SNN with STDP learning primarily use memristor macro models, which are software-based and cannot give complete insight into circuit implementation challenges. Some reported works on SNN use memristive devices, which require additional fabrication steps. This article presents a full circuit-level implementation of the SNN system featuring on-chip training and classification using memristive STDP synapse in standard CMOS technology. A new learning rule using the modified STDP is implemented to simplify the weight modification process. It does not involve FPGAs, CPUs, or GPUs to train the neural network. The approach used in this paper to modify the weights does not require any additional combinational or digital circuits attached to the memristive synapse resulting in less consumption of area, energy and time. We demonstrated the complete circuit-level design, implementation and simulation of SNN with on-chip training and pattern classification using 180 nm CMOS technology. A comprehensive comparison of the proposed SNN circuit with the previous related work is also presented. To demonstrate the versatility of the CMOS synapse circuit for application scenarios requiring rate-based learning, we have tuned the pair-based STDP circuit to obtain Bienenstock–Cooper–Munro (BCM) characteristics and applied it to heart rate classification.

Optimization of post-annealing temperature for high-performance synaptic transistors based on In–Ga–Zn–O channel and trap layers

Synaptic transistors using an indium–gallium–zinc oxide (IGZO) charge trap layer (CTL) have great potential for high-performance neuromorphic applications due to their excellent stability and high linearity in conductance modulation. Since the high conductivity of the IGZO CTL enhances charge de-trapping performance, the post-annealing process, which directly affects the conductivity of the film, should be carefully conducted. In this study, we investigated the effect of post-annealing temperature (TPA) on the charge de-trapping performance of synaptic transistors. The synaptic transistor showed superior charge de-trapping performance at a TPA of 250 °C, resulting from the increased conductivity of the CTL. Through various electrical and chemical analysis methods, it was revealed that the hydrogen decrease within the CTL and the oxidation–reduction reaction at the interface between the tunneling layer and the CTL contributed to the significant increase in the conductivity of the CTL. The synaptic transistor with the optimal TPA exhibited high linearity of potentiation (αp: 0.56) and depression (αd: −0.65) with 64 conductance levels, enabling the spiking neural network simulation to achieve a high accuracy of 88.75% using a Fashion-MNIST dataset. This work shows that TPA is one of the critical process strategies for the high charge de-trapping performance of the synaptic transistor by inducing the chemical reaction at the CTL interface and hydrogen content change within the CTL.

Bridging Neuroscience and Robotics: Spiking Neural Networks in Action.

Robots are becoming increasingly sophisticated in the execution of complex tasks. However, an area that requires development is the ability to act in dynamically changing environments. To advance this, developments have turned towards understanding the human brain and applying this to improve robotics. The present study used electroencephalogram (EEG) data recorded from 54 human participants whilst they performed a two-choice task. A build-up of motor activity starting around 400 ms before response onset, also known as the lateralized readiness potential (LRP), was observed. This indicates that actions are not simply binary processes but rather, response-preparation is gradual and occurs in a temporal window that can interact with the environment. In parallel, a robot arm executing a pick-and-place task was developed. The understanding from the EEG data and the robot arm were integrated into the final system, which included cell assemblies (CAs)-a simulated spiking neural network-to inform the robot to place the object left or right. Results showed that the neural data from the robot simulation were largely consistent with the human data. This neurorobotics study provides an example of how to integrate human brain recordings with simulated neural networks in order to drive a robot.

Reconfigurable Mott electronics for homogeneous neuromorphic platform

Abstract To simplify the fabrication process and increase the versatility of neuromorphic systems, the reconfiguration concept has attracted much attention. Here, we developed a novel electrochemical VO2 (EC-VO2) device, which can be reconfigured as synapses or LIF neurons. The ionic dynamic doping contributes to the resistance changes of VO2, which enables the reversible modulation of device states. The analog resistance switching and tunable LIF functions were both measured based on the same device to demonstrate the capacity of reconfiguration. Based on the reconfigurable EC-VO2, the simulated spiking neural network model exhibited excellent performances by using low-precision weights and tunable output neurons, whose final accuracy reached 91.92%.

Fast Simulation of Analog Spiking Neural Network With Device Non-Idealites

We present a method for spiking neural network simulation with hardware realistic device non-idealities present in the neuron and synapse circuits as an alternative to time-consumingexpensive spice simulation. Neuromorphic machine learning algorithms for spiking neural networks are often simulatedtested with an ideal mathematical description of spiking neurons and synapses. However, silicon implementations of spiking neurons differ significantly from their ideal mathematical models because of device non-idealities and restrictions in the range of device operation. These non-idealities affect the performance of chip implementations of spiking networks. The dynamical system phase plane of the neuron and synapse circuit is used to create a compact representation of the neuron dynamics that captures device non-idealities for simulation of spiking neural networks. The proposed method would allow hardware-aware optimization of neuromorphic algorithms using standard machine learning tools and provide simulated network prediction close to what would be expected from a chip implementation.