Fire Neuron Research Articles

Energy-efficient neural network using an anisotropy field gradient-based self-resetting neuron and meander synapse

Neuromorphic computing (NC) is considered a potential solution for energy-efficient artificial intelligence applications. The development of reliable neural network (NN) hardware with low energy and area footprints plays a crucial role in realizing NC. Even though neurons and synapses have already been investigated using a variety of spintronic devices, the research is still in the primitive stages. Particularly, there is not much experimental research on the self-reset (and leaky) aspect(s) of domain wall (DW) device-based neurons. Here, we have demonstrated an energy-efficient NN using a spintronic DW device-based neuron with self-reset (leaky) and integrate-and-fire functions. An “anisotropy field gradient” provides the self-resetting behavior of auto-leaky, integrate, and fire neurons. The leaky property of the neuron was experimentally demonstrated using a voltage-assisted modification of the anisotropy field. A synapse with a meander wire configuration was used to achieve multiple-resistance states corresponding to the DW position and controlled pinning of the DW. The NN showed an energy efficiency of 0.189 nJ/image/epoch while achieving an accuracy of 92.4%. This study provides a fresh path for developing more energy-efficient DW-based NN systems.

Brain topology improved spiking neural network for efficient reinforcement learning of continuous control.

The brain topology highly reflects the complex cognitive functions of the biological brain after million-years of evolution. Learning from these biological topologies is a smarter and easier way to achieve brain-like intelligence with features of efficiency, robustness, and flexibility. Here we proposed a brain topology-improved spiking neural network (BT-SNN) for efficient reinforcement learning. First, hundreds of biological topologies are generated and selected as subsets of the Allen mouse brain topology with the help of the Tanimoto hierarchical clustering algorithm, which has been widely used in analyzing key features of the brain connectome. Second, a few biological constraints are used to filter out three key topology candidates, including but not limited to the proportion of node functions (e.g., sensation, memory, and motor types) and network sparsity. Third, the network topology is integrated with the hybrid numerical solver-improved leaky-integrated and fire neurons. Fourth, the algorithm is then tuned with an evolutionary algorithm named adaptive random search instead of backpropagation to guide synaptic modifications without affecting raw key features of the topology. Fifth, under the test of four animal-survival-like RL tasks (i.e., dynamic controlling in Mujoco), the BT-SNN can achieve higher scores than not only counterpart SNN using random topology but also some classical ANNs (i.e., long-short-term memory and multi-layer perception). This result indicates that the research effort of incorporating biological topology and evolutionary learning rules has much in store for the future.

An energy‐efficient tunable threshold spiking neuron with excitatory and inhibitory function

AbstractIn this work, a complementary metal‐oxide‐semiconductor (CMOS) based leaky‐integrate and fire neuron has been proposed and investigated for neuromorphic applications. The neuron has been designed in Cadence Virtuoso and validated experimentally. It has been observed that the neuron consumes a maximum energy of 68.87 fJ/spike. The response of the neuron to excitatory as well as inhibitory inputs has been studied. To verify the applicability, the proposed neuron has been explored for reconfigurable threshold logic to implement various linearly separable Boolean functions including OR, AND, NOT, NOR, and NAND. Moreover, the threshold tunability of the neuron has also been verified and this property has been exploited to design threshold‐controlled logic gates. Instead of adjusting the weights of the applied inputs, the functionality of such gates can be controlled by changing the threshold of the neuron, simplifying the synaptic architecture of a neural network. Finally, a multilayer network has been designed and the recognition ability of the proposed network for MNIST handwritten digits has been verified with an accuracy of 96.93%.

Physics-Based Self-Learning Spiking Neural Network enhanced time-integration scheme for computing viscoplastic structural finite element response

The present study introduces a new physics-based self-learning spiking neural framework to compute geometrically and physically nonlinear structural response. While the so-called traditional or second-generation deep neural networks are used in many applications in the class of physics-informed neural networks, we propose a hybrid model that consists of third-generation Leaky-Integrated and Fire (LIF) neurons, Recurrent Leaky-Integrated and Fire (RLIF) neurons and second-generation dense transformation. The third-generation neurons are inspired by the human brain’s energy-efficient functioning which introduces inherent temporal and sparse behavior leading to a more sustainable AI approach. However, the sparse nature of the spiking neurons poses a challenge to tackle nonlinear regression tasks. Thus, in the present study, we use an autoencoding strategy that converts the real-valued signals to its spiking representation and enables the spiking neurons to learn nonlinear material response. The proposed hybrid network is firstly pretrained with combined data-driven and physics-based loss functions and then deployed in the plastic corrector step of the implicit integration which is used in the Finite Element solver and is validated on a series of Boundary Value Problems (BVPs) consisting of plate elements. A self-learning strategy is introduced for the hybrid network to train itself during FE simulation using the proposed physics-based loss function. Two major advantages were observed: an overall computational gain in the excess of 30% and the self-learning/online training ability of the model that bolsters its convergence behavior. Finally, the proposed third-generation layers are deployed on the Xylo-Av2 neuromorphic chip and its energy performance is compared with the second-generation Recurrent Neural Networks (RNNs).

Flexible Unipolar IGZO Transistor-Based Integrate and Fire Neurons for Spiking Neuromorphic Applications.

In this article, three different implementations of an Axon-Hillock circuit are presented, one of the basic building blocks of spiking neural networks. In this work, we explored the design of such circuits using a unipolar thin-film transistor technology based on amorphous InGaZnO, often used for large-area electronics. All the designed circuits are fabricated by direct material deposition and patterning on top of a flexible polyimide substrate. Axon-Hillock circuits presented in this article consistently show great adaptability of the basic properties of a spiking neuron such as output spike frequency adaptation and output spike width adaptation. Additional degrees of adaptability are demonstrated with each of the Axon-Hillock circuit varieties: neuron circuit threshold voltage adaptation, differentiation between input signal importance, and refractory period modulation. The proposed neuron can change its firing frequency up to three orders of magnitude by varying a single voltage brought to a circuit terminal. This allows the neuron to function, and potentially learn, at vastly different timescales that coincide with the biologically meaningful timescales, going from milliseconds to seconds, relevant for circuits meant for interaction with the environment. Thanks to careful design choices, the average measured power consumption is kept in the nW range, realistically allowing upscaling towards the spiking neural networks in the future. The spiking neuron with refractory period modulation presented in this work has an area of 607.3 μm × 492.2 μm, it experimentally demonstrated firing rates as low as 11.926 mHz, and its energy consumption per spike is ≈ 700 pJ at 30 Hz.

Deep spiking neural networks with integrate and fire neuron using steep switching device

Deep learning has shown impressive capabilities in tasks like speech recognition and image classification. However, modern deep neural networks often demand a significant number of weights and extensive computational resources, creating efficiency challenges for applications on edge devices. To address these issues, researchers have introduced deep spiking neural networks (DSNNs) that leverage specialized hardware for synapses and neurons. DSNNs offer a potential solution by improving efficiency in edge-device applications. In this paper, the hardware based DSNN with integrate and fire neuron using steep switching device was investigated. We propose integrate and fire neuron using steep switching device to implement rate coding as input encoding method. Because the steep switching device has double-gate, the threshold voltage of the neuron circuits can be adaptively controlled, which changes the rates of input pulse. Hence, the adjustment of the threshold of neuron can be employed to mitigate the accuracy deterioration resulting from the transformation from deep neural networks (DNNs) to DSNNs. In addition, the off-current of proposed integrate and fire neuron circuit decreases significantly as the steep switching device has steep subthreshold swing. A system simulation of a hardware based DSNN shows that the adjustable threshold of the neuron circuit can achieve a high inference accuracy of 98.36 % which is comparable to that obtained with software based DNN.

A Single Schottky Barrier MOSFET-Based Leaky Integrate and Fire Neuron for Neuromorphic Computing

In this paper, a Schottky Barrier MOSFET (SB-MOSFET) based on Impact Ionization mechanism is used to design a leaky integrate and fire (LIF) neuron with considerable enhancement in area, energy and cost is proposed. Using 2D calibrated simulation, we confirmed that SB-MOSFET LIF is able to replicate the neuron behavior precisely without using external circuitry. The proposed LIF neuron shows significantly lower energy per spike of 4 pJ/spike, which is lowest among the single transistor based neurons present in the literature. The recognition precision of 89.2% has been accomplished for Modified National Institute of Standards and Technology (MNIST) image. Besides this, SB-MOSFET doesn’t require any doped regions, therefore it can be fabricated with low thermal budget.

FPGA Realization of a High-Speed Spiking Neural Network with Modified LIF Neurons for Pattern Recognition

This paper presents the multiplier-less Modified Leaky Integrate and Fire Neuron Unit (MLIFNU) and proposed a modified Spiking Neural Network (SNN) architecture for Pattern Recognition application. Synthesis results of implementation on Field-Programmable Gate Array are presented. Accordingly, the maximum frequency of the MLIFNU and SNN are 361.991 MHz and 281.442 MHz, respectively.

A tunable and versatile 28 nm FD-SOI crossbar output circuit for low power analog SNN inference with eNVM synapses

In this work we report a study and a co-design methodology of an analog SNN crossbar output circuit designed in a 28 nm FD-SOI technology node that comprises a tunable current attenuator and a leak-integrate and fire neurons that would enable the integration of emerging non-volatile memories (eNVMs) for synaptic arrays based on various technologies including phase change (PCRAM), oxide-based (OxRAM), spin transfer and spin orbit torque magnetic memories (STT, SOT-MRAM). Circuit SPICE simulation results and eNVM experimental data are used to showcase and estimate the neurons fan-in for each type of eNVM considering the technology constraints and design trade-offs that set its limits such as membrane capacitance, supply voltage, leakage, etc.

Spiking CMOS-NVM mixed-signal neuromorphic ConvNet with circuit- and training-optimized temporal subsampling.

We increasingly rely on deep learning algorithms to process colossal amount of unstructured visual data. Commonly, these deep learning algorithms are deployed as software models on digital hardware, predominantly in data centers. Intrinsic high energy consumption of Cloud-based deployment of deep neural networks (DNNs) inspired researchers to look for alternatives, resulting in a high interest in Spiking Neural Networks (SNNs) and dedicated mixed-signal neuromorphic hardware. As a result, there is an emerging challenge to transfer DNN architecture functionality to energy-efficient spiking non-volatile memory (NVM)-based hardware with minimal loss in the accuracy of visual data processing. Convolutional Neural Network (CNN) is the staple choice of DNN for visual data processing. However, the lack of analog-friendly spiking implementations and alternatives for some core CNN functions, such as MaxPool, hinders the conversion of CNNs into the spike domain, thus hampering neuromorphic hardware development. To address this gap, in this work, we propose MaxPool with temporal multiplexing for Spiking CNNs (SCNNs), which is amenable for implementation in mixed-signal circuits. In this work, we leverage the temporal dynamics of internal membrane potential of Integrate & Fire neurons to enable MaxPool decision-making in the spiking domain. The proposed MaxPool models are implemented and tested within the SCNN architecture using a modified version of the aihwkit framework, a PyTorch-based toolkit for modeling and simulating hardware-based neural networks. The proposed spiking MaxPool scheme can decide even before the complete spatiotemporal input is applied, thus selectively trading off latency with accuracy. It is observed that by allocating just 10% of the spatiotemporal input window for a pooling decision, the proposed spiking MaxPool achieves up to 61.74% accuracy with a 2-bit weight resolution in the CIFAR10 dataset classification task after training with back propagation, with only about 1% performance drop compared to 62.78% accuracy of the 100% spatiotemporal window case with the 2-bit weight resolution to reflect foundry-integrated ReRAM limitations. In addition, we propose the realization of one of the proposed spiking MaxPool techniques in an NVM crossbar array along with periphery circuits designed in a 130nm CMOS technology. The energy-efficiency estimation results show competitive performance compared to recent neuromorphic chip designs.

Solid State Neuroscience: Spiking Neural Networks as Time Matter

We aim at building a bridge between to a priori disconnected fields: Neuroscience and Material Science. We construct an analogy based on identifying spikes events in time with the positions of particles of matter. We show that one may think of the dynamical states of spiking neurons and spiking neural networks as time-matter. Namely, a structure of spike-events in time having analogue properties to that of ordinary matter. We can define for neural systems notions equivalent to the equations of state, phase diagrams and their phase transitions. For instance, the familiar Ideal Gas Law relation (Pυ = constant) emerges as analogue of the Ideal Integrate and Fire neuron model relation (Iin ISI = constant). We define the neural analogue of the spatial structure correlation function, that can characterize spiking states with temporal long-range order, such as regular tonic spiking. We also define the “neuro-compressibility” response function in analogy to the lattice compressibility. We show that similarly to the case of ordinary matter, the anomalous behavior of the neuro-compressibility is a precursor effect that signals the onset of changes in spiking states. We propose that the notion of neuro-compressibility may open the way to develop novel medical tools for the early diagnose of diseases. It may allow to predict impending anomalous neural states, such as Parkinson’s tremors, epileptic seizures, electric cardiopathies, and perhaps may even serve as a predictor of the likelihood of regaining consciousness.

BP-based supervised learning algorithm for multilayer photonic spiking neural network and hardware implementation.

We introduce a supervised learning algorithm for photonic spiking neural network (SNN) based on back propagation. For the supervised learning algorithm, the information is encoded into spike trains with different strength, and the SNN is trained according to different patterns composed of different spike numbers of the output neurons. Furthermore, the classification task is performed numerically and experimentally based on the supervised learning algorithm in the SNN. The SNN is composed of photonic spiking neuron based on vertical-cavity surface-emitting laser which is functionally similar to leaky-integrate and fire neuron. The results prove the demonstration of the algorithm implementation on hardware. To seek ultra-low power consumption and ultra-low delay, it is great significance to design and implement a hardware-friendly learning algorithm of photonic neural networks and realize hardware-algorithm collaborative computing.

Dynamic threshold integrate and fire neuron model for low latency spiking neural networks

Spiking Neural Networks (SNNs) operate with asynchronous discrete events which enable lower power and greater computational efficiency on event-driven hardware than Artificial Neural Networks (ANNs). Conventional ANN-to-SNN conversion methods usually employ Integrate and Fire (IF) neuron model with a fixed threshold to act as Rectified Linear Unit (ReLU). However, there is a large demand for the input spikes to reach the fixed threshold and fire, which leads to high inference latency. In this work, we propose a Dynamic Threshold Integrate and Fire (DTIF) neuron model by exploiting the biological neuron threshold variability, where the threshold is inversely related to the neuron input. The spike activity is increased by dynamically adjusting the threshold at each simulation time-step to reduce the latency. Compared to the state-of-the-art conversion methods, the ANN-to-SNN conversion using DTIF model has lower latency with competitive accuracy, which has been verified by deep architecture on image classification tasks including MNIST, CAIFAR-10, and CIFAR-100 datasets. Moreover, it achieves 7.14 × faster inference under 0.44 × energy consumption than the typical method of maximum normalization.

A Super-Efficient TinyML Processor for the Edge Metaverse

Although the Metaverse is becoming a popular technology in many aspects of our lives, there are some drawbacks to its implementation on clouds, including long latency, security concerns, and centralized infrastructures. Therefore, designing scalable Metaverse platforms on the edge layer can be a practical solution. Nevertheless, the realization of these edge-powered Metaverse ecosystems without high-performance intelligent edge devices is almost impossible. Neuromorphic engineering, which employs brain-inspired cognitive architectures to implement neuromorphic chips and Tiny Machine Learning (TinyML) technologies, can be an effective tool to enhance edge devices in such emerging ecosystems. Thus, a super-efficient TinyML processor to use in the edge-enabled Metaverse platforms has been designed and evaluated in this research. This processor includes a Winner-Take-All (WTA) circuit that was implemented via a simplified Leaky Integrate and Fire (LIF) neuron on an FPGA. The WTA architecture is a computational principle in a neuromorphic system inspired by the mini-column structure in the human brain. The resource consumption of the WTA architecture is reduced by employing our simplified LIF neuron, making it suitable for the proposed edge devices. The results have indicated that the proposed neuron improves the response speed to almost 39% and reduces resource consumption by 50% compared to recent works. Using our simplified neuron, up to 4200 neurons can be deployed on VIRTEX 6 devices. The maximum operating frequency of the proposed neuron and our spiking WTA is 576.319 MHz and 514.095 MHz, respectively.

RRAM-Based Energy Efficient Scalable Integrate and Fire Neuron With Built-In Reset Circuit

In this brief, we propose a Resistive Random Access Memory (RRAM) based self-resetting Integrate and Fire ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\text{I}\&amp;\text{F}$ </tex-math></inline-formula> ) neuron. The proposed neuron circuit does not require any external bias voltage and the integration of control unit required to reset RRAM into neuron circuit optimizes its overall power consumption. The neuron circuit proposed in this brief consists of two RRAMs for integrate and fire operations, whereas, pulse propogation and reset circuit consists of 22 CMOS transistors. It consumes <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$1.5~fJ$ </tex-math></inline-formula> per spike, which is 48% and 53% less than the recent neurons designed using, nanoscale FBFET and PDSOI-MOSFET, respectively. The operating frequency of proposed neuron ranges from <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$277~KHz$ </tex-math></inline-formula> to <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$03~MHz$ </tex-math></inline-formula> , which is at least 7.5% and 10% higher than the operating frequencies of above mentioned recent neurons, respectively. The inclusion of reset circuit into RRAM based neuron circuit enables the implementation of large scale spiking neural network (SNN), which makes it superior in terms of power and energy consumption.

An artificial bridge circuit approach between two biological neurons using nanoscale topologies towards paralytic disorders

The advent of Nanoscale IC technology towards pulse-based neural systems reactivates the dead nervous about restoring the functionality of paralytic disorders. This work reports in first time a design of a novel CMOS biological neuron system, which replaces a dead neuron between two neurons to restore communication in paralyzed individuals. The work binds into three stages: design of a spiking leaky Integrator and Fire (LIF) neuron with refractory period mechanisms, which achieves a low power consumption of 2.4 μW, in the first stage; an adaptive homeostatic synapse with short and long-term spike plasticity, that reconfigure the spiking neuron networks of multichannel sensor electrodes to record the electric signal from the active cell as second stage; the final stage presents a low-power common source current reuse regulated cascode (CS-CR-RGC) TIA for amplifying the weak synapse current signal, which achieves a high gain of 135.71 dBΩ with an optimized noise performance of 0.19 pA/Hz. The entire work is designed and implemented using a CMOS 65 nm commercial process that occupies a die area of 400 μm × 120 μm.

A Comparative Analysis of Numerical Methods for Solving the Leaky Integrate and Fire Neuron Model

The human nervous system is one of the most complex systems of the human body. Understanding its behavior is crucial in drug discovery and developing medical devices. One approach to understanding such a system is to model its most basic unit, neurons. The leaky integrate and fire (LIF) method models the neurons’ response to a stimulus. Given the fact that the model’s equation is a linear ordinary differential equation, the purpose of this research is to compare which numerical analysis method gives the best results for the simplified version of this model. Adams predictor and corrector (AB4-AM4) and Heun’s methods were then used to solve the equation. In addition, this study further researches the effects of different current input models on the LIF’s voltage output. In terms of the computational time, Heun’s method was 0.01191 s on average which is much less than that of the AB-AM4 method (0.057138) for a constant DC input. As for the root mean square error, the AB-AM4 method had a much lower value (0.0061) compared to that of Heun’s method (0.3272) for the same constant input. Therefore, our results show that Heun’s method is best suited for the simplified LIF model since it had the lowest computation time of 36 ms, was stable over a larger range, and had an accuracy of 72% for the varying sinusoidal current input model.

L-Shaped Double Gate Bipolar Impact Ionization MOSFET Based Energy Efficient Leaky Integrate and Fire Neuron for Spiking Neural Network

L-Shaped Double Gate Bipolar Impact Ionization MOSFET Based Energy Efficient Leaky Integrate and Fire Neuron for Spiking Neural Network

Design of Leaky Integrate and Fire Neuron for Spiking Neural Networks Using Trench Bipolar I-MOS

In this paper, using calibrated 2-D TCAD simulations we report a trench Bipolar I-MOS for the realization of a spiking neural network. We demonstrated that the proposed trench Bipolar I-MOS LIF neuron can emulate the biological neuronal nature and exhibits a low threshold voltage (-0.16 V), which is ∼ |400 mV| lower than the past reported LBIMOS LIF neuron. Moreover, the trench Bipolar I-MOS neuron consumes 0.35 pJ energy per spike, which is ∼ 100x lower in comparison to the PDSOI LIF neuron. Further, the proposed LIF neuron shows ∼ 10x reduction in energy per spike than the recently published Ge MOSFET and JLFET based LIF neurons. In addition, the proposed trench Bipolar I-MOS LIF neuron exhibits ∼ 6 orders high spiking frequency than the biological neuron. Also, the proposed device shows a ∼ 1.1x reduction in the breakdown voltage as compared to the conventional Bipolar I-MOS. This is due to the crowding of the electric field near the gate edges.

Mott Memristors and Neuronal Ion Channels: A Qualitative Analysis

Memristive devices have earned increased recognition over the past decade, owing to their application in memory and neuromorphic systems. Recently, memristive devices have found a propitious role in Spiking Neural Networks. This is partly attributed to an emerging family of memristive devices that are volatile. Volatile memristors, especially Mott memristors, have been used to implement Leaky Integrate and Fire neurons along with neuron circuits with richer dynamics, referred to as neuristors. The Mott memristor is often compared to voltage-gated ion channels in the Hodgkin-Huxley neuron model. However, voltage-gated ion channels in the Hodgkin-Huxley model do not exhibit voltage hysteresis in their quasi-static I-V characteristics, a critical property of Mott memristors. We discuss the association between Mott memristors based on joule heating and the Hodgkin-Huxley voltage-gated ion channels, and we prove that they are qualitatively dissimilar. Consequently, we propose a hypothetical potassium current-gated ion channel that exhibits voltage hysteresis. We simulate a Mott memristor Pearson-Anson oscillator and a current-gated ion channel oscillator. We use the nullcline plots and bifurcation diagrams of the circuits to demonstrate that both circuits are qualitatively similar. We further show that both circuits produce oscillations through a supercritical Andronov-Hopf bifurcation and have a similar oscillation-generation mechanism. We conclude that for the Mott memristor to share qualitative similarity to a neuronal ion channel, the channel must be current-gated rather than voltage-gated. Our aim is to supplement the understanding of how hardware neurons relate to their biological counterparts, which is a crucial first step towards implementing better brain-inspired systems.