Design Of Spiking Neural Networks Research Articles

Sampling complex topology structures for spiking neural networks.

Spiking Neural Networks (SNNs) have been considered a potential competitor to Artificial Neural Networks (ANNs) due to their high biological plausibility and energy efficiency. However, the architecture design of SNN has not been well studied. Previous studies either use ANN architectures or directly search for SNN architectures under a highly constrained search space. In this paper, we aim to introduce much more complex connection topologies to SNNs to further exploit the potential of SNN architectures. To this end, we propose the topology-aware search space, which is the first search space that enables a more diverse and flexible design for both the spatial and temporal topology of the SNN architecture. Then, to efficiently obtain architecture from our search space, we propose the spatio-temporal topology sampling (STTS) algorithm. By leveraging the benefits of random sampling, STTS can yield powerful architecture without the need for an exhaustive search process, making it significantly more efficient than alternative search strategies. Extensive experiments on CIFAR-10, CIFAR-100, and ImageNet demonstrate the effectiveness of our method. Notably, we obtain 70.79% top-1 accuracy on ImageNet with only 4 time steps, 1.79% higher than the second best model. Our code is available under https://github.com/stiger1000/Random-Sampling-SNN.

The High-Performance Design of a General Spiking Convolution Computation Unit for Supporting Neuromorphic Hardware Acceleration

Recently, the design of spiking neural networks (SNNs) processors based on FPGA have become a hot research topic for their low power and high efficiency. However, most existing works lack flexibility in configuration to accomplish efficient computation of deep SNNs with different network structures (e.g., residuals). In this brief, a high-performance general spiking convolution processing unit and the corresponding hardware architecture (named SCPU) is designed to support both standard and residual convolution; then, a neuromorphic processor containing SCPU (called FPGA-SCPP) is implemented on Xilinx Virtex-7 FPGA. Specifically, the model parameters are fused and quantified to reduce the computational amounts and simplify its hardware implementation. And a spike event processing unit is designed based on event computation and direct data flow to complete the standard and residual convolution computations. The experimental results show the FPGA-SCPP can effectively deploy different large-scale SNNs models such as VGG and ResNet, with the highest recognition accuracies of 92.45% and 68.55% on CIFAR-10/CIFAR-100, respectively. Our FPGA-SCPP has a computational power of only 1.7W and a frame rate of 40fps, which improves inference speed by about 6 times and reduces power consumption by 3 times compared to existing processors. Therefore, FPGA-SCPP can effectively accomplish the acceleration of deep SNNs, especially for models with spiking residual blocks.

Neuromorphic Functional Modules of a Spiking Neural Network

In the current era, the design and development of artificial neural networks exploiting the architecture of the human brain have evolved rapidly. Artificial neural networks (ANN) effectively solve a wide range of common artificial-intelligence tasks involving data classification and recognition, prediction, forecasting and adaptive control of the behavior of an object. The biologically inspired underlying principles of ANN operation have certain advantages over the conventional von Neumann architecture including unsupervised learning, architectural flexibility and adaptability to environmental change and high performance under significantly reduced power consumption due to large parallel and asynchronous data processing. In this paper, we present a circuit design of main functional blocks (neurons and synapses) intended for the hardware implementation of a perceptron-based feedforward spiking neural network. As the third generation of artificial neural networks, spiking neural networks perform data processing utilizing spikes, which are discrete events (or functions) that take place at points in time. Neurons in spiking neural networks initiate precisely timed spikes and communicate with each other via spikes transmitted through synaptic connections or synapses with adaptable scalable weight. One of the prospective approaches to emulating synaptic behavior in hardware implemented spiking neural networks is to use nonvolatile-memory devices with analog conduction modulation (or memristive structures). Here we propose a circuit design for functional analogues of memristive structures to mimic synaptic plasticity, pre- and postsynaptic neurons which could be used for developing a circuit design of spiking-neural-network architectures with different training algorithms including the spike-timing-dependent-plasticity learning rule. Two different circuits of an electrical synapse are developed. The first one is an analog synapse with a photoresistor optocoupler used to ensure the tunable conductivity for synaptic-plasticity emulation. While the second one is a digital synapse, in which the synaptic weight is stored in a digital code with its direct conversion into conductivity (without a digital-to-analog converter and photoresistor optocoupler). The results of prototyping the developed circuits for electronic analogs of synapses, pre- and postsynaptic neurons and the study of transient processes are presented. The developed approach could provide a basis for the ASIC (application-specific integrated circuit) design of spiking neural networks based on CMOS (complementary metal—oxide–semiconductor) design technology.

Neuromorphic Functional Modules of Spiking Neural Network

In the current era, design and development of artificial neural networks exploiting the architecture of the human brain have evolved rapidly. Artificial neural networks effectively solve a wide range of common for artificial intelligence tasks involving data classification and recognition, prediction, forecasting and adaptive control of object behavior. Biologically inspired underlying principles of ANN operation have certain advantages over the conventional von Neumann architecture including unsupervised learning, architectural flexibility and adaptability to environmental change and high performance under significantly reduced power consumption due to heavy parallel and asynchronous data processing. In this paper, we present the circuit design of main functional blocks (neurons and synapses) intended for hardware implementation of a perceptron-based feedforward spiking neural network. As the third generation of artificial neural networks, spiking neural networks perform data processing utilizing spikes, which are discrete events (or functions) that take place at points in time. Neurons in spiking neural networks initiate precisely timing spikes and communicate with each other via spikes transmitted through synaptic connections or synapses with adaptable scalable weight. One of the prospective approach to emulate the synaptic behavior in hardware implemented spiking neural networks is to use non-volatile memory devices with analog conduction modulation (or memristive structures). Here we propose a circuit design for functional analogues of memristive structure to mimic a synaptic plasticity, pre- and postsynaptic neurons which could be used for developing circuit design of spiking neural network architectures with different training algorithms including spike-timing dependent plasticity learning rule. Two different circuits of electronic synapse were developed. The first one is an analog synapse with photoresistive optocoupler used to ensure the tunable conductivity for synaptic plasticity emulation. While the second one is a digital synapse, in which the synaptic weight is stored in a digital code with its direct conversion into conductivity (without digital-to-analog converter andphotoresistive optocoupler). The results of the prototyping of developed circuits for electronic analogues of synapses, pre- and postsynaptic neurons and the study of transient processes are presented. The developed approach could provide a basis for ASIC design of spiking neural networks based on CMOS (complementary metal oxide semiconductor) design technology.

Spiking pattern recognition using informative signal of image and unsupervised biologically plausible learning

The recent progress of low-power neuromorphic hardware provides exceptional conditions for applications where their focus is more on saving power. However, the design of spiking neural networks (SNN) to recognize real-world patterns on such hardware remains a major challenge ahead of the researchers. In this paper, SNN inspired by the model of local cortical population as a biological neuro-computing resource for digit recognition was presented. SNN was equipped with spike-based unsupervised weight optimization based on the dynamical behavior of the excitatory (AMPA) and inhibitory (GABA) synapses using Spike Timing Dependent Plasticity (STDP). This biologically plausible learning enables neurons to make decisions and learns the structure of the input examples. There are two main reasons why this structure is state of the art compared to previous works: learning process is compatible with many experimental observations on induction of long-term potentiation and long-term depression, image to signal mapping created an informative signal of the image based on sequences of prolate spheroidal wave functions (PSWFs). The proposed image mapping translates the pixels attributes to the frequency, phase, and amplitude of a sinusoidal signal. This mapping enables the SNN to generalize better to the realistic sized images and significantly decreases the size of the input layer. Cortical SNN compared to earlier related studies recognized MNIST digits more accurate and achieved 96.1% classification accuracy with unsupervised learning based on sparse spike activity.

Design of spiking neural networks for blood pressure prediction during general anesthesia: considerations for optimizing results

The ability to predict blood pressure changes during general anesthesia would assist anesthetists minimize the risk of complications due to hypotensive events. However, such prediction is not trivial. Evolving spiking neural networks are a relatively new computational method that may have application to this problem. NeuCubeST consists of a 3-dimensional network of locally connected neurons called a Spiking Neural Network reservoir (SNNr) and can be used to classify time series data for prediction. There are a number of design considerations when using NeuCubeST as a classifier of time-series data: what pre-processing of the raw data is required (pre-processing), how to convert the time-series data into a spike train (input-encoding), which neurons the data are connected to (input-mapping), and how many nearest neighbours to use in classification (classification). However, it is still unclear how sensitive NeuCubeST-based systems are to perturbations of any of the above. In this paper we evaluate the contribution of these design factors to blood pressure prediction using NeuCubeST. 6000 possible combinations of those NeuCubeST options were tested for each of 100 patients and for each a Signal to Noise Ratio was obtained. All four investigated design factors showed significant contribution to SNR. Intra-operative MAP prediction using NeuCubeST can be effective but performance is sensitive to the design choices.

CHAOS-BASED MIXED SIGNAL IMPLEMENTATION OF SPIKING NEURONS

A new design of Spiking Neural Networks is proposed and fabricated using a 0.35 microm CMOS technology. The architecture is based on the use of both digital and analog circuitry. The digital circuitry is dedicated to the inter-neuron communication while the analog part implements the internal non-linear behavior associated to spiking neurons. The main advantages of the proposed system are the small area of integration with respect to digital solutions, its implementation using a standard CMOS process only and the reliability of the inter-neuron communication.