LIF Neurons Research Articles

Adjusted SpikeProp algorithm for recurrent spiking neural networks with LIF neurons

A problem related to the development of a supervised learning method for recurrent spiking neural networks is addressed in the paper. The widely used Leaky-Integrate-and-Fire model has been adopted as a spike neuron model. The proposed method is based on a known SpikeProp algorithm. In detail, the developed method enables gradient descent learning of recurrent or multi-layer feedforward spiking neural networks. The research included an extended verification study for the classical XOR classification problem. In addition, the developed learning method has been used to provide a spiking neural black-box model of fast processes occurring in a pressurised water nuclear reactor. The obtained simulation results demonstrate satisfactory effectiveness of the proposed approach.

Highly Uniform Silver Ion Memristors With Ultralow Leakage Current for Constructing Homogeneously Spiking LIF Neurons

Highly Uniform Silver Ion Memristors With Ultralow Leakage Current for Constructing Homogeneously Spiking LIF Neurons

Circuit implementation of on-chip trainable spiking neural network using CMOS based memristive STDP synapses and LIF neurons

Computation on a large volume of data at high speed and low power requires energy-efficient architectures for edge computing applications. As a result, scientists focus on memristive circuits and systems for area and energy efficiency. Spiking neural network (SNN) with bio-inspired spike-timing-dependent plasticity learning (STDP) is a promising solution for energy-efficient neuromorphic systems than conventional artificial neural network (ANN). Previous works on SNN with STDP learning primarily use memristor macro models, which are software-based and cannot give complete insight into circuit implementation challenges. Some reported works on SNN use memristive devices, which require additional fabrication steps. This article presents a full circuit-level implementation of the SNN system featuring on-chip training and classification using memristive STDP synapse in standard CMOS technology. A new learning rule using the modified STDP is implemented to simplify the weight modification process. It does not involve FPGAs, CPUs, or GPUs to train the neural network. The approach used in this paper to modify the weights does not require any additional combinational or digital circuits attached to the memristive synapse resulting in less consumption of area, energy and time. We demonstrated the complete circuit-level design, implementation and simulation of SNN with on-chip training and pattern classification using 180 nm CMOS technology. A comprehensive comparison of the proposed SNN circuit with the previous related work is also presented. To demonstrate the versatility of the CMOS synapse circuit for application scenarios requiring rate-based learning, we have tuned the pair-based STDP circuit to obtain Bienenstock–Cooper–Munro (BCM) characteristics and applied it to heart rate classification.

FPGA Realization of a High-Speed Spiking Neural Network with Modified LIF Neurons for Pattern Recognition

This paper presents the multiplier-less Modified Leaky Integrate and Fire Neuron Unit (MLIFNU) and proposed a modified Spiking Neural Network (SNN) architecture for Pattern Recognition application. Synthesis results of implementation on Field-Programmable Gate Array are presented. Accordingly, the maximum frequency of the MLIFNU and SNN are 361.991 MHz and 281.442 MHz, respectively.

Reconfigurable Mott electronics for homogeneous neuromorphic platform

Abstract To simplify the fabrication process and increase the versatility of neuromorphic systems, the reconfiguration concept has attracted much attention. Here, we developed a novel electrochemical VO2 (EC-VO2) device, which can be reconfigured as synapses or LIF neurons. The ionic dynamic doping contributes to the resistance changes of VO2, which enables the reversible modulation of device states. The analog resistance switching and tunable LIF functions were both measured based on the same device to demonstrate the capacity of reconfiguration. Based on the reconfigurable EC-VO2, the simulated spiking neural network model exhibited excellent performances by using low-precision weights and tunable output neurons, whose final accuracy reached 91.92%.

Receptive field sizes and neuronal encoding bandwidth are constrained by axonal conduction delays.

Studies on population coding implicitly assume that spikes from the presynaptic cells arrive simultaneously at the integrating neuron. In natural neuronal populations, this is usually not the case-neuronal signaling takes time and populations cover a certain space. The spread of spike arrival times depends on population size, cell density and axonal conduction velocity. Here we analyze the consequences of population size and axonal conduction delays on the stimulus encoding performance in the electrosensory system of the electric fish Apteronotus leptorhynchus. We experimentally locate p-type electroreceptor afferents along the rostro-caudal body axis and relate locations to neurophysiological response properties. In an information-theoretical approach we analyze the coding performance in homogeneous and heterogeneous populations. As expected, the amount of information increases with population size and, on average, heterogeneous populations encode better than the average same-size homogeneous population, if conduction delays are compensated for. The spread of neuronal conduction delays within a receptive field strongly degrades encoding of high-frequency stimulus components. Receptive field sizes typically found in the electrosensory lateral line lobe of A. leptorhynchus appear to be a good compromise between the spread of conduction delays and encoding performance. The limitations imposed by finite axonal conduction velocity are relevant for any converging network as is shown by model populations of LIF neurons. The bandwidth of natural stimuli and the maximum meaningful population sizes are constrained by conduction delays and may thus impact the optimal design of nervous systems.

A binarized spiking neural network based on auto-reset LIF neurons and large signal synapses using STT-MTJs

A binarized spiking neural network using auto-reset leaky integrate-and-fire neurons with a two-transistor and three-magnetic tunnel junction core and large signal synapses with two-transistor and two-magnetic tunnel junctions is designed. The network is applied to a classifier of the MNIST handwritten digit dataset with a 784 × 400 synapse crossbar array. The weights are trained offline using the spike-timing-dependent plasticity learning algorithm and deployed to the spin-transfer torque magnetic tunnel junction (STT-MTJ) resistances in the synapses after being binarized. Its performance is evaluated by HSPICE using the STT-MTJ device model, which takes the stochastic change in the angle between the two magnetic moments in the free and pinned layers into consideration. 75% test accuracy is achieved for 1200 patterns with 1 ns read and 1 ns write operations and 0.23 pJ/SOP energy consumption.

A 5.3 pJ/Spike CMOS Neural Array Employing Time-Modulated Axon-Sharing and Background Mismatch Calibration Techniques

Inspired by the human brain, spiking neuron networks are promising to realize energy-efficient and low-latency neuromorphic computing. However, even state-of-the-art silicon neurons are orders of magnitude worse than biological neurons in terms of area and power consumption due to the limitations. Moreover, limited routing in typical CMOS processes is another challenge for realizing the fully-parallel high-throughput synapse connections compared to biological synapses. This paper presents an SNN circuit that utilizes resource-sharing techniques to address the two challenges. Firstly, a comparator sharing neuron circuit with a background calibration technique is proposed to shrink the size of a single neuron without performance degradation. Secondly, a time-modulated axon-sharing synapse system is proposed to realize a fully-parallel connection with limited hardware overhead. To validate the proposed approaches, a CMOS neuron array is designed and fabricated under a 55-nm process. It consists of 48 LIF neurons with 3125 neurons/mm 2 area density, power consumption of 5.3 pJ/spike, and equivalent 2304 fully parallel synapses providing a unit throughput of 5500 events/s/neuron. It proves the proposed approaches are promising to realize a high-throughput high-efficiency SNN with CMOS technology.

Design of Leaky Integrate and Fire Neuron for Spiking Neural Networks Using Trench Bipolar I-MOS

In this paper, using calibrated 2-D TCAD simulations we report a trench Bipolar I-MOS for the realization of a spiking neural network. We demonstrated that the proposed trench Bipolar I-MOS LIF neuron can emulate the biological neuronal nature and exhibits a low threshold voltage (-0.16 V), which is ∼ |400 mV| lower than the past reported LBIMOS LIF neuron. Moreover, the trench Bipolar I-MOS neuron consumes 0.35 pJ energy per spike, which is ∼ 100x lower in comparison to the PDSOI LIF neuron. Further, the proposed LIF neuron shows ∼ 10x reduction in energy per spike than the recently published Ge MOSFET and JLFET based LIF neurons. In addition, the proposed trench Bipolar I-MOS LIF neuron exhibits ∼ 6 orders high spiking frequency than the biological neuron. Also, the proposed device shows a ∼ 1.1x reduction in the breakdown voltage as compared to the conventional Bipolar I-MOS. This is due to the crowding of the electric field near the gate edges.

An Extended Spatiotemporal Contextual Learning and Memory Network Model for Hardware Implementation

A spatiotemporal contextual learning and memory network (STCLMN) model was proposed based on the hippocampal CA1 network using the difference equations suitable for digital computer simulations. However, the inherent capability of handling and processing of continuous-time asynchronous signals (spikes) cannot be fully employed in the STCLMN model. Furthermore, digital circuit implementation of the STCLMN is not suitable for real-time edge applications, wherein small size, low power consumption, and real-time learning capability are necessary. Therefore, this study extends the original STCLMN model using a set of differential equations to overcome the above problems encountered by STCLMN for asynchronous circuit implementation. The minimum accuracy required for a compact and low-power hardware implementation of the proposed model is then investigated. The validity and feasibility of the proposed model are confirmed through numerical simulations and experiments with a small mixed analog/digital circuit based on the investigated accuracy. In this study, LIF neurons are used because of their ease of circuit implementation. The proposed eSTCLMN model would enable an efficient circuit implementation of an asynchronous spiking neural network with spatiotemporal contextual one-shot learning and memory.

A SmNiO3 memristor with artificial synapse function properties and the implementation of Boolean logic circuits.

Recently, with the improvement of the requirements for fast and efficient data processing in the era of artificial intelligence, new forms of computing have come into being. Developing memristor devices that can simulate the brain's computing neutral network is particularly important for applications in the field of artificial intelligence. However, there are still some challenges in their biological function simulation and related circuit design. In this work, a memristor based on perovskite rare earth nickelates (RNiO3) is presented with excellent electrical performance, including three orders of magnitude higher current switching ratio and good repeatability, and can achieve bidirectional conductance regulation like weight modulation in bio-synapse. Furthermore, the synaptic like characteristics of the device have been mimicked successfully, such as excitatory postsynaptic current (EPSC), paired pulse facilitation (PPF), classical double pulse spike time-dependent plasticity (classical pair-STDP), triplet spike time-dependent plasticity (triplet-STDP), short-term plasticity (STP), long-term plasticity (LTP), the refractory period phenomenon and learning and forgetting rules. In particular, two synaptic devices and a leaky integrate-and-fire (LIF) neuron device are used to achieve a logic gate circuit to realize "AND", "OR", and "NOT" functions. The device paves the way for the application of high-density circuits in artificial intelligence.

Neuromorphic adaptive spiking CPG towards bio-inspired locomotion

In recent years, locomotion mechanisms exhibited by vertebrate animals have been the inspiration for the improvement in the performance of robotic systems. These mechanisms include the adaptability of their locomotion to any change registered in the environment through their biological sensors. In this regard, we aim to replicate such kind of adaptability through a sCPG. This sCPG generates different locomotion (rhythmic) patterns which are driven by an external stimulus, that is, the output of a FSR sensor to provide feedback. The sCPG consists of a network of five populations of LIF neurons designed with a specific topology in such a way that the rhythmic patterns can be generated and driven by the aforementioned external stimulus. Therefore, eventually, the locomotion of an end robotic platform could be adapted to the terrain by using any sensor as input. The sCPG with adaptation has been numerically validated at software and hardware level, using the Brian 2 simulator and the SpiNNaker neuromorphic platform for the latest. In particular, our experiments clearly show an adaptation in the oscillation frequencies between the spikes produced in the populations of the sCPG while the input stimulus varies. To validate the robustness and adaptability of the sCPG, we have performed several tests by variating the output of the sensor. These experiments were carried out in Brian 2 and SpiNNaker; both implementations showed a similar behavior with a Pearson correlation coefficient of 0.905.

Computational properties of multi-compartment LIF neurons with passive dendrites

Mixed-signal neuromorphic computers often emulate some variant of the LIF neuron model. While, in theory, two-layer networks of these neurons are universal function approximators, single-layer networks consisting of slightly more complex neurons can, at the cost of universality, be more efficient. In this paper, we discuss a family of LIF neurons with passive dendrites. We provide rules that describe how input channels targeting different dendritic compartments interact, and test in how far these interactions can be harnessed in a spiking neural network context. We find that a single layer of two-compartment neurons approximates some functions at smaller errors than similarly sized hidden-layer networks. Single-layer networks with with three compartment neurons can approximate functions such as XOR and four-quadrant multiplication well; adding more compartments only offers small improvements in accuracy. From the perspective of mixed-signal neuromorphic systems, our results suggest that only small modifications to the neuron circuit are necessary to construct more computationally powerful and energy efficient systems that move more computation into the dendritic, analogue domain.

FPGA-NHAP: A General FPGA-Based Neuromorphic Hardware Acceleration Platform With High Speed and Low Power

Spiking neural network (SNN) can process discrete spikes and offers a high degree of real-time performance and excellent energy efficiency ratio. However, most current neuromorphic hardware platforms lack efficient driven algorithms and only support a single type of neuron model, which has slow speed and poor scalability. This paper proposes a general FPGA-based neuromorphic hardware acceleration platform (FPGA-NHAP), supporting the effective inference and acceleration of SNN network with low power, high speed and good scalability. First, a neuron computing unit is designed to simulate the both LIF and Izhikevich (IZH) neurons with the parallel spike caching and scheduling technique. Second, a novel integrated driven update algorithm is proposed to complete the spike encoding of external data, reducing the waiting time of neuron state update effectively. Third, the proposed platform is implemented using a RISC-V processor and a Xilinx FPGA, simulating 16,384 neurons and 16.8 million synapses with a power consumption of 0.535 W. Finally, two different three-layer SNN networks are deployed on the proposed platform for recognition tasks on the MNIST and Fashion-MNIST datasets, achieving the accuracy of 97.70%, 85.14% (LIF) and 97.81%, 83.16% (IZH), frame rates of 208 frame/s, 128 frame/s (LIF) and 206 frame/s, 141 frame/s (IZH), respectively.

Simulating the Cortical Microcircuit Significantly Faster Than Real Time on the IBM INC-3000 Neural Supercomputer.

This article employs the new IBM INC-3000 prototype FPGA-based neural supercomputer to implement a widely used model of the cortical microcircuit. With approximately 80,000 neurons and 300 Million synapses this model has become a benchmark network for comparing simulation architectures with regard to performance. To the best of our knowledge, the achieved speed-up factor is 2.4 times larger than the highest speed-up factor reported in the literature and four times larger than biological real time demonstrating the potential of FPGA systems for neural modeling. The work was performed at Jülich Research Centre in Germany and the INC-3000 was built at the IBM Almaden Research Center in San Jose, CA, United States. For the simulation of the microcircuit only the programmable logic part of the FPGA nodes are used. All arithmetic is implemented with single-floating point precision. The original microcircuit network with linear LIF neurons and current-based exponential-decay-, alpha-function- as well as beta-function-shaped synapses was simulated using exact exponential integration as ODE solver method. In order to demonstrate the flexibility of the approach, additionally networks with non-linear neuron models (AdEx, Izhikevich) and conductance-based synapses were simulated, applying Runge–Kutta and Parker–Sochacki solver methods. In all cases, the simulation-time speed-up factor did not decrease by more than a very few percent. It finally turns out that the speed-up factor is essentially limited by the latency of the INC-3000 communication system.

Demonstration of an UltraLow Energy PD-SOI FinFET Based LIF Neuron for SNN

In this article, partially depleted silicon on insulator (PD-SOI) FinFET based LIF neuron is demonstrated to mimic biological neuronal behaviour with aid of well-calibrated 3D TCAD simulation. The floating body effect of PD-SOI FinFET is used to store the holes generated by the impact ionisation (&#x2018;II&#x2019;), which exhibits the integration phenomenon and recombination manifests the leaky function. It shows 5.4 fJ of energy per spike, which is significantly lower than other FinFET based neurons reported till date. Moreover, it needs only 1.8 V of supply voltage, which is lower as compared to its equivalent bulk FinFET and PD-SOI MOSFET based LIF neurons. The proposed PD-SOI FinFET LIF neuron shows megahertz range of spiking frequency, that is <inline-formula><tex-math notation="LaTeX">$\sim 10 ^{5}$</tex-math></inline-formula>&#x00D7; higher than biological neuron (<inline-formula><tex-math notation="LaTeX">$\sim$</tex-math></inline-formula>1-10 Hz). Furthermore, the effective area of the FinFET is optimized as 0.023 <inline-formula><tex-math notation="LaTeX">$\mu$</tex-math></inline-formula>m<inline-formula><tex-math notation="LaTeX">$ ^{2}$</tex-math></inline-formula>. Thus, the proposed PD-SOI FinFET based LIF neuron is more attractive for large scale hardware implementation of SNN, due to its energy and area efficiency comparable with biological neuron along with CMOS compatibility.

Passive Nonlinear Dendritic Interactions as a Computational Resource in Spiking Neural Networks.

Nonlinear interactions in the dendritic tree play a key role in neural computation. Nevertheless, modeling frameworks aimed at the construction of large-scale, functional spiking neural networks, such as the Neural Engineering Framework, tend to assume a linear superposition of postsynaptic currents. In this letter, we present a series of extensions to the Neural Engineering Framework that facilitate the construction of networks incorporating Dale's principle and nonlinear conductance-based synapses. We apply these extensions to a two-compartment LIF neuron that can be seen as a simple model of passive dendritic computation. We show that it is possible to incorporate neuron models with input-dependent nonlinearities into the Neural Engineering Framework without compromising high-level function and that nonlinear postsynaptic currents can be systematically exploited to compute a wide variety of multivariate, band-limited functions, including the Euclidean norm, controlled shunting, and nonnegative multiplication. By avoiding an additional source of spike noise, the function approximation accuracy of a single layer of two-compartment LIF neurons is on a par with or even surpasses that of two-layer spiking neural networks up to a certain target function bandwidth.

Discrete Langevin machine: Bridging the gap between thermodynamic and neuromorphic systems.

A formulation of Langevin dynamics for discrete systems is derived as a class of generic stochastic processes. The dynamics simplify for a two-state system and suggest a network architecture which is implemented by the Langevin machine. The Langevin machine represents a promising approach to compute successfully quantitative exact results of Boltzmann distributed systems by LIF neurons. Besides a detailed introduction of the dynamics, different simplified models of a neuromorphic hardware system are studied with respect to a control of emerging sources of errors.

Enabling Spike-Based Backpropagation for Training Deep Neural Network Architectures.

Spiking Neural Networks (SNNs) have recently emerged as a prominent neural computing paradigm. However, the typical shallow SNN architectures have limited capacity for expressing complex representations while training deep SNNs using input spikes has not been successful so far. Diverse methods have been proposed to get around this issue such as converting off-the-shelf trained deep Artificial Neural Networks (ANNs) to SNNs. However, the ANN-SNN conversion scheme fails to capture the temporal dynamics of a spiking system. On the other hand, it is still a difficult problem to directly train deep SNNs using input spike events due to the discontinuous, non-differentiable nature of the spike generation function. To overcome this problem, we propose an approximate derivative method that accounts for the leaky behavior of LIF neurons. This method enables training deep convolutional SNNs directly (with input spike events) using spike-based backpropagation. Our experiments show the effectiveness of the proposed spike-based learning on deep networks (VGG and Residual architectures) by achieving the best classification accuracies in MNIST, SVHN, and CIFAR-10 datasets compared to other SNNs trained with a spike-based learning. Moreover, we analyze sparse event-based computations to demonstrate the efficacy of the proposed SNN training method for inference operation in the spiking domain.

Bistability and up/down state alternations in inhibition-dominated randomly connected networks of LIF neurons

Electrophysiological recordings in cortex in vivo have revealed a rich variety of dynamical regimes ranging from irregular asynchronous states to a diversity of synchronized states, depending on species, anesthesia, and external stimulation. The average population firing rate in these states is typically low. We study analytically and numerically a network of sparsely connected excitatory and inhibitory integrate-and-fire neurons in the inhibition-dominated, low firing rate regime. For sufficiently high values of the external input, the network exhibits an asynchronous low firing frequency state (L). Depending on synaptic time constants, we show that two scenarios may occur when external inputs are decreased: (1) the L state can destabilize through a Hopf bifucation as the external input is decreased, leading to synchronized oscillations spanning d δ to β frequencies; (2) the network can reach a bistable region, between the low firing frequency network state (L) and a quiescent one (Q). Adding an adaptation current to excitatory neurons leads to spontaneous alternations between L and Q states, similar to experimental observations on UP and DOWN states alternations.