Leaky Integrate-and-fire Research Articles

Solving Max-Cut Problem Using Spiking Boltzmann Machine Based on Neuromorphic Hardware with Phase Change Memory.

Efficiently solving combinatorial optimization problems (COPs) such as Max-Cut is challenging because the resources required increase exponentially with the problem size. This study proposes a hardware-friendly method for solving the Max-Cut problem by implementing a spiking neural network (SNN)-based Boltzmann machine (BM) in neuromorphic hardware systems. To implement the hardware-oriented version of the spiking Boltzmann machine (sBM), the stochastic dynamics of leaky integrate-and-fire (LIF) neurons with random walk noise are analyzed, and an innovative algorithm based on overlapping time windows is proposed. The simulation results demonstrate the effective convergence and high accuracy of the proposed method for large-scale Max-Cut problems. The proposed method is validated through successful hardware implementation on a 6-transistor/2-resistor (6T2R) neuromorphic chip with phase change memory (PCM) synapses. In addition, as an expansion of the algorithm, several annealing techniques and bias split methods are proposed to improve convergence, along with circuit design ideas for efficient evaluation of sampling convergence using cell arrays and spiking systems. Overall, the results of the proposed methods demonstrate the potential of energy-efficient and hardware-implementable approaches using SNNs to solve COPs. To the best of the author's knowledge, this is the first study to solve the Max-Cut problem using an SNN neuromorphic hardware chip.

High-speed Ta2O5-based threshold switching memristor for LIF neurons

Due to their high similarity to biological ion channels, low power consumption, small footprint, and the fact that they do not require reset circuits, threshold switching memristors have been intensively studied for simulating neurons in neuromorphic chips. Switching speed is one of the key challenges which limit the application of threshold switching memristors in chips. In this study, Ta2O5 threshold switching memristors with high switching speeds were prepared by doping with silver. The results show that 14 wt. % Ag doped Ta2O5 threshold switching memristors exhibit excellent bi-directional threshold switching performance, featuring fast switching speeds (<20 ns, <18 ns), low leakage currents (<10 pA), and high switching ratio (>107). According to the field nucleation theory, the rapid switching speed can be attributed to the low nucleation energy (0.26 eV) of silver within the Ta2O5 matrix, which is achieved by incorporating 14 wt. % Ag during the doping process. Based on Pspice, a LIF (leaky integrate-and-fire) neuron based on the silver nanoparticles doped Ta2O5 threshold switching memristors is built, and its firing function has been simulated. The results show that the LIF neuron with a short switching time is able to excite pulse spiking with high frequencies. These results demonstrated that the silver nanoparticles doped Ta2O5-based threshold switching memristors hold significant potential for constructing high-speed artificial neural networks.

All-Ferroelectric Spiking Neural Networks via Morphotropic Phase Boundary Neurons.

Artificial neurons and synapses are crucial for efficiently implementing spiking neural networks (SNNs) in hardware. The distinct functional requirements of artificial neurons and synapses present significant challenges in the implementation of area- and energy-efficient SNNs. This study reports an all-ferroelectric SNN system through co-optimization of material properties and device configurations using wafer-scale atomic layer deposition. For the first time, a double-gate (DG) morphotropic phase boundary-based thin-film transistor (MPBTFT) is utilized for a leaky integrate-and-fire (LIF) neuron. The DG MPBTFT-based LIF neuron eliminates the need for capacitors and reset circuits, thereby enhancing area and energy efficiency. The DG configuration demonstrates various neuronal functions with high reliability. Co-optimizing materials and devices significantly enhance the performance and functional versatility of artificial neurons and synapses. Meticulous material engineering facilitates the seamless co-integration of DG MPBTFT-based neurons, ferroelectric thin-film transistor (TFT)-based synapses, and normal TFTs on a single wafer. All-ferroelectric SNN systems achieved a high classification accuracy of 94.9%, thereby highlighting the potential of DG MPBTFT-based LIF neurons for advanced neuromorphic computing.

An energy efficient leaky integrate and fire neuron using Ge-source TFET for spiking neural network: simulation analysis

The basic building block of neural network is a device, which can mimic the neural behavior. The spiking neural network (SNN) is an efficient methodology in terms of power and area. Due to the excess energy consumption and larger area, various spintronic neural devices are unfit for neuron applications. In this article, we have implemented Ge source based Tunnel FET (TFET) for ultralow energy spike generation using TCAD simulator. It is seen that Ge source TFET has signature spiking frequency in THz range versus input voltage curve of an artificial biological neuron. The simulated device deploy the leaky integrate and fire (LIF) technique for generation of neurons. The simulation result highlights that the energy of device is 1.08 aJ/spike, which is several order less than existing neural based FET devices in literature.

Proposal and detailed theoretical analysis on a photonic neural network with optically pumped Spin-VCSEL spiking neurons

A dual-layer photonic spiking neural network (PSNN) was constructed, where multiple optically pumped spin vertical-cavity surface-emitting lasers (Spin-VCSELs) were proposed as spiking neurons. Based on a detailed theoretical analysis of leaky integrate-and-fire (LIF) and refractory period characteristics of Spin-VCSEL neurons, the training and testing performance for the studied PSNN was evaluated using two standard pattern classification tasks (Iris dataset, simple digit recognition). The results showed that, by selecting appropriate parameters such as frequency detuning and number of pre-synaptic neurons, etc., higher training/testing accuracies beyond 90% can be obtained. When compared with traditional electrically pumped VCSEL, a threshold reduction of up to 50% can be achieved under nanosecond scale spin relaxation time and circular polarization optical pumping, the feasibility of realizing high accuracy (88%) pattern classification near the reduced threshold was also verified. Therefore, optically pumped Spin-VCSEL neurons can become a valuable new choice for high-performance PSNN with reduced power consumption.

Memristive Circuit Implementation of Caenorhabditis Elegans Mechanism for Neuromorphic Computing.

To overcome the energy efficiency bottleneck of the von Neumann architecture and scaling limit of silicon transistors, an emerging but promising solution is neuromorphic computing, a new computing paradigm inspired by how biological neural networks handle the massive amount of information in a parallel and efficient way. Recently, there is a surge of interest in the nematode worm Caenorhabditis elegans (C. elegans), an ideal model organism to probe the mechanisms of biological neural networks. In this article, we propose a neuron model for C. elegans with leaky integrate-and-fire (LIF) dynamics and adjustable integration time. We utilize these neurons to build the C. elegans neural network according to their neural physiology, which comprises: 1) sensory modules; 2) interneuron modules; and 3) motoneuron modules. Leveraging these block designs, we develop a serpentine robot system, which mimics the locomotion behavior of C. elegans upon external stimulus. Moreover, experimental results of C. elegans neurons presented in this article reveals the robustness (1% error w.r.t. 10% random noise) and flexibility of our design in term of parameter setting. The work paves the way for future intelligent systems by mimicking the C. elegans neural system.

Gated parametric neuron for spike-based audio recognition

Spiking neural networks (SNNs) aim to simulate real neural networks in the human brain with biologically plausible neurons. The leaky integrate-and-fire (LIF) neuron is one of the most widely studied SNN architectures. However, it has the vanishing gradient problem when trained with backpropagation. Additionally, its neuronal parameters are often manually specified and fixed, in contrast to the heterogeneity of real neurons in the human brain. This paper proposes a gated parametric neuron (GPN) to process spatio-temporal information effectively with the gating mechanism. Compared with the LIF neuron, the GPN has two distinguishing advantages: (1) it copes well with the vanishing gradients by improving the flow of gradient propagation; and, (2) it learns spatio-temporal heterogeneous neuronal parameters automatically. Additionally, we use the same gate structure to eliminate initial neuronal parameter selection and design a hybrid recurrent neural network-SNN structure. Experiments on two spike-based audio datasets demonstrated that the GPN network outperformed several state-of-the-art SNNs, could mitigate vanishing gradients, and had spatio-temporal heterogeneous parameters. Our work shows the ability of SNNs to handle long-term dependencies and achieve high performance simultaneously.

Understanding the functional roles of modelling components in spiking neural networks

Abstract Spiking neural networks (SNNs), inspired by the neural circuits of the brain, are promising in achieving high computational efficiency with biological fidelity. Nevertheless, it is quite difficult to optimize SNNs because the functional roles of their modelling components remain unclear. By designing and evaluating several variants of the classic model, we systematically investigate the functional roles of key modelling components, leakage, reset, and recurrence, in leaky integrate-and-fire (LIF) based SNNs. Through extensive experiments, we demonstrate how these components influence the accuracy, generalization, and robustness of SNNs. Specifically, we find that the leakage plays a crucial role in balancing memory retention and robustness, the reset mechanism is essential for uninterrupted temporal processing and computational efficiency, and the recurrence enriches the capability to model complex dynamics at a cost of robustness degradation. With these interesting observations, we provide optimization suggestions for enhancing the performance of SNNs in different scenarios. This work deepens the understanding of how SNNs work, which offers valuable guidance for the development of more effective and robust neuromorphic models.

Low-energy and tunable LIF neuron using SiGe bandgap-engineered resistive switching transistor

We have proposed leaky integrate-and-fire (LIF) neuron having low-energy consumption and tunable functionality without external circuit components. Our LIF neuron has a simple configuration consisting of only three components: one bandgap-engineered resistive switching transistor (BE-RST), one capacitor, and one resistor. Here, the crucial point is that BE-RST with a silicon–germanium heterojunction possesses an amplified hysteric current switching with a low latch-up voltage due to improved hole storage capability and impact ionization coefficient. Therefore, the proposed neuron utilizing BE-RST requires an energy consumption of 0.36 pJ/spike, which is approximately six times lower than 2.08 pJ/spike of pure silicon-RST based neuron. In addition, the spiking properties can be tuned by modulating the leakage rate and threshold through gate bias, which contributes to energy-efficient sparse-activity and high learning accuracy. As a result, our proposed neuron can be a promising candidate for executing various spiking neural network applications.

An energy and area-efficient spike frequency adaptable LIF neuron for spiking neural networks

As Moore's law approaches its limit, conventional computing methods face energy efficiency challenges, prompting the exploration of alternative information processing methods like spiking neural networks (SNNs). Neurons, the fundamental unit of the nervous system, are crucial for efficiently implementing SNNs in hardware, requiring energy and area-efficient design. This paper presents novel circuit designs for the leaky integrate-and-fire (LIF) neuron model at 7 nm FinFET technology, including 8T FinFET LIF Neuron 1, 7T FinFET LIF Neuron 2, 7T FinFET LIF Neuron 3 and 6T FinFET LIF Neuron 4. In the proposed configurations of LIF neuron, the energy per spike is reduced by 95.80 %, 95.89 %, 95.89 %, 96.76 %, and 96.65 % in the 8T FinFET LIF neuron 1 (case 1), 8T FinFET LIF neuron 1 (case 2), 7T FinFET LIF Neuron 2, 7T FinFET LIF Neuron 3, and 6T FinFET LIF Neuron 4, respectively, compared to the lowest energy per spike reported in the literature, with the minimum known energy per spike being 0.67 fJ. The proposed FinFET neurons offer attractive hardware acceleration (spiking frequency in the MHz range), minimal energy per spike (fJ range), simplicity, and area efficiency, making them more biologically plausible due to their adaptable spiking frequency.

Network model with internal complexity bridges artificial intelligence and neuroscience.

Artificial intelligence (AI) researchers currently believe that the main approach to building more general model problems is the big AI model, where existing neural networks are becoming deeper, larger and wider. We term this the big model with external complexity approach. In this work we argue that there is another approach called small model with internal complexity, which can be used to find a suitable path of incorporating rich properties into neurons to construct larger and more efficient AI models. We uncover that one has to increase the scale of the network externally to stimulate the same dynamical properties. To illustrate this, we build a Hodgkin-Huxley (HH) network with rich internal complexity, where each neuron is an HH model, and prove that the dynamical properties and performance of the HH network can be equivalent to a bigger leaky integrate-and-fire (LIF) network, where each neuron is a LIF neuron with simple internal complexity.

Phase-Dependent Response to Electrical Stimulation of Cortical Networks during Recurrent Epileptiform Short Discharge Generation In Vitro.

The closed-loop control of pathological brain activity is a challenging task. In this study, we investigated the sensitivity of continuous epileptiform short discharge generation to electrical stimulation applied at different phases between the discharges using an in vitro 4-AP-based model of epilepsy in rat hippocampal slices. As a measure of stimulation effectiveness, we introduced a sensitivity function, which we then measured in experiments and analyzed with different biophysical and abstract mathematical models, namely, (i) the two-order subsystem of our previous Epileptor-2 model, describing short discharge generation governed by synaptic resource dynamics; (ii) a similar model governed by shunting conductance dynamics (Epileptor-2B); (iii) the stochastic leaky integrate-and-fire (LIF)-like model applied for the network; (iv) the LIF model with potassium M-channels (LIF+KM), belonging to Class II of excitability; and (v) the Epileptor-2B model with after-spike depolarization. A semi-analytic method was proposed for calculating the interspike interval (ISI) distribution and the sensitivity function in LIF and LIF+KM models, which provided parametric analysis. Sensitivity was found to increase with phase for all models except the last one. The Epileptor-2B model is favored over other models for subthreshold oscillations in the presence of large noise, based on the comparison of ISI statistics and sensitivity functions with experimental data. This study also emphasizes the stochastic nature of epileptiform discharge generation and the greater effectiveness of closed-loop stimulation in later phases of ISIs.

Radar Emitter Recognition Based on Spiking Neural Networks

Efficient and effective radar emitter recognition is critical for electronic support measurement (ESM) systems. However, in complex electromagnetic environments, intercepted pulse trains generally contain substantial data noise, including spurious and missing pulses. Currently, radar emitter recognition methods utilizing traditional artificial neural networks (ANNs) like CNNs and RNNs are susceptible to data noise and require intensive computations, posing challenges to meeting the performance demands of modern ESM systems. Spiking neural networks (SNNs) exhibit stronger representational capabilities compared to traditional ANNs due to the temporal dynamics of spiking neurons and richer information encoded in precise spike timing. Furthermore, SNNs achieve higher computational efficiency by performing event-driven sparse addition calculations. In this paper, a lightweight spiking neural network is proposed by combining direct coding, leaky integrate-and-fire (LIF) neurons, and surrogate gradients to recognize radar emitters. Additionally, an improved SNN for radar emitter recognition is proposed, leveraging the local timing structure of pulses to enhance adaptability to data noise. Simulation results demonstrate the superior performance of the proposed method over existing methods.

Spiking neural networks in the Alexiewicz topology: A new perspective on analysis and error bounds

In order to ease the analysis of error propagation in neuromorphic computing and to get a better understanding of spiking neural networks (SNN), we address the problem of mathematical analysis of SNNs as endomorphisms that map spike trains to spike trains. A central question is the adequate structure for a space of spike trains and its implication for the design of error measurements of SNNs including time delay, threshold deviations, and the design of the reinitialization mode of the leaky-integrate-and-fire (LIF) neuron model. First, we identify the underlying topology by analyzing the closure of all sub-threshold signals of a LIF model. For zero leakage this approach yields the Alexiewicz topology, which we adopt to LIF neurons with arbitrary positive leakage. As a result, LIF can be understood as spike train quantization in the corresponding norm. This way we obtain various error bounds and inequalities such as a quasi-isometry relation between incoming and outgoing spike trains. Another result is a Lipschitz-style global upper bound for the error propagation and a related resonance-type phenomenon.

Rethinking Pretraining as a Bridge From ANNs to SNNs.

Spiking neural networks (SNNs) are known as typical kinds of brain-inspired models with their unique features of rich neuronal dynamics, diverse coding schemes, and low power consumption properties. How to obtain a high-accuracy model has always been the main challenge in the field of SNN. Currently, there are two mainstream methods, i.e., obtaining a converted SNN through converting a well-trained artificial NN (ANN) to its SNN counterpart or training an SNN directly. However, the inference time of a converted SNN is too long, while SNN training is generally very costly and inefficient. In this work, a new SNN training paradigm is proposed by combining the concepts of the two different training methods with the help of the pretrain technique and BP-based deep SNN training mechanism. We believe that the proposed paradigm is a more efficient pipeline for training SNNs. The pipeline includes pipe-S for static data transfer tasks and pipe-D for dynamic data transfer tasks. State-of-the-art (SOTA) results are obtained in a large-scale event-driven dataset ES-ImageNet. For training acceleration, we achieve the same (or higher) best accuracy as similar leaky-integrate-and-fire (LIF)-SNNs using 1/8 training time on ImageNet-1K and 1/2 training time on ES-ImageNet and also provide a time-accuracy benchmark for a new dataset ES-UCF101. These experimental results reveal the similarity of the functions of parameters between ANNs and SNNs and also demonstrate various potential applications of this SNN training pipeline.

Analog Implementation of a Spiking Neuron with Memristive Synapses for Deep Learning Processing

Analog neuromorphic prototyping is essential for designing and testing spiking neuron models that use memristive devices as synapses. These prototypes can have various circuit configurations, implying different response behaviors that custom silicon designs lack. The prototype’s behavior results can be optimized for a specific foundry node, which can be used to produce a customized on-chip parallel deep neural network. Spiking neurons mimic how the biological neurons in the brain communicate through electrical potentials. Doing so enables more powerful and efficient functionality than traditional artificial neural networks that run on von Neumann computers or graphic processing unit-based platforms. Therefore, on-chip parallel deep neural network technology can accelerate deep learning processing, aiming to exploit the brain’s unique features of asynchronous and event-driven processing by leveraging the neuromorphic hardware’s inherent parallelism and analog computation capabilities. This paper presents the design and implementation of a leaky integrate-and-fire (LIF) neuron prototype implemented with commercially available components on a PCB board. The simulations conducted in LTSpice agree well with the electrical test measurements. The results demonstrate that this design can be used to interconnect many boards to build layers of physical spiking neurons, with spike-timing-dependent plasticity as the primary learning algorithm, contributing to the realization of experiments in the early stage of adopting analog neuromorphic computing.

Dynamics of Leaky Integrate‐and‐Fire Neurons Based on Oxyvanite Memristors for Spiking Neural Networks

Neuromorphic computing implemented with spiking neural networks (SNNs) based on volatile threshold switching is an energy‐efficient computing paradigm that may overcome future limitations of the von Neumann architecture. Herein, threshold switching in oxyvanite (V3O5) memristors and their application as a leaky integrate‐and‐fire (LIF) neuron are explored. The spiking response of individual neurons is examined as a function of circuit parameters, input pulse train, and temperature and reveals a pulse height‐dependent spike rate in which devices exhibit excitatory spiking behavior under low input voltages and protective inhibition spiking under high voltages. Resistively coupled LIF neurons are shown to exhibit additional neural functionalities (i.e., phasic, regular and adaptation, etc.) depending on the input voltage and circuit parameters. The behavior of both individual and coupled neurons is shown to be described by a physics‐based lumped element circuit model, which therefore provides a solid foundation for exploring more complex systems. Finally, the performance of a perceptron SNN employing these LIF neurons is assessed by simulating the classification of image recognition algorithm. These results advance the development of robust solid‐state neurons with low power consumption for neuromorphic computing.

Uncertain Leaky Integrate-and-Fire Neuron Equation

The Leaky Integrate-and-Fire (LIF) neuron model is a simplified neuronal model commonly used to describe voltage changes on the cell membrane. Considering the influence of real-world environmental factors on the total input current received by neurons, we propose an uncertain LIF neuron equation driven by the Liu process, which belongs to a class of uncertain differential equations. Furthermore, analytical solutions for the uncertain LIF neuron equation are derived. To explore the sensitivity of the uncertain LIF neuron equation to initial state perturbations, we discuss many kinds of stability of the uncertain LIF neuron equation, including stability in measure, stability in distribution and stability in mean. Additionally, a paradox for the stochastic LIF neuron equation is identified.

Electrocardiography Classification with Leaky Integrate-and-Fire Neurons in an Artificial Neural Network-Inspired Spiking Neural Network Framework.

Monitoring heart conditions through electrocardiography (ECG) has been the cornerstone of identifying cardiac irregularities. Cardiologists often rely on a detailed analysis of ECG recordings to pinpoint deviations that are indicative of heart anomalies. This traditional method, while effective, demands significant expertise and is susceptible to inaccuracies due to its manual nature. In the realm of computational analysis, Artificial Neural Networks (ANNs) have gained prominence across various domains, which can be attributed to their superior analytical capabilities. Conversely, Spiking Neural Networks (SNNs), which mimic the neural activity of the brain more closely through impulse-based processing, have not seen widespread adoption. The challenge lies primarily in the complexity of their training methodologies. Despite this, SNNs offer a promising avenue for energy-efficient computational models capable of displaying a high-level performance. This paper introduces an innovative approach employing SNNs augmented with an attention mechanism to enhance feature recognition in ECG signals. By leveraging the inherent efficiency of SNNs, coupled with the precision of attention modules, this model aims to refine the analysis of cardiac signals. The novel aspect of our methodology involves adapting the learned parameters from ANNs to SNNs using leaky integrate-and-fire (LIF) neurons. This transfer learning strategy not only capitalizes on the strengths of both neural network models but also addresses the training challenges associated with SNNs. The proposed method is evaluated through extensive experiments on two publicly available benchmark ECG datasets. The results show that our model achieves an overall accuracy of 93.8% on the MIT-BIH Arrhythmia dataset and 85.8% on the 2017 PhysioNet Challenge dataset. This advancement underscores the potential of SNNs in the field of medical diagnostics, offering a path towards more accurate, efficient, and less resource-intensive analyses of heart diseases.

Artificial Neuron Based on the Bloch-Point Domain Wall in Ferromagnetic Nanowires.

Nanomagnetism and spintronics are currently active areas of research, with one of the main goals being the creation of low-energy-consuming magnetic memories based on nanomagnet switching. These types of devices could also be implemented in neuromorphic computing by crafting artificial neurons (ANs) that emulate the characteristics of biological neurons through the implementation of neuron models such as the widely used leaky integrate-and-fire (LIF) with a refractory period. In this study, we have carried out numerical simulations of a 120 nm diameter, 250 nm length ferromagnetic nanowire (NW) with the aim of exploring the design of an artificial neuron based on the creation and destruction of a Bloch-point domain wall. To replicate signal integration, we applied pulsed trains of spin currents to the opposite faces of the ferromagnetic NW. These pulsed currents (previously studied only in the continuous form) are responsible for inducing transitions between the stable single vortex (SV) state and the metastable Bloch point domain wall (BP-DW) state. To ensure the system exhibits leak and refractory properties, the NW was placed in a homogeneous magnetic field of the order of mT in the axial direction. The suggested configuration fulfills the requirements and characteristics of a biological neuron, potentially leading to the future creation of artificial neural networks (ANNs) based on reversible changes in the topology of magnetic NWs.