Leaky Integrator Research Articles

A Temperature Sensory Leaky Integrate-and-Fire Artificial Neuron Based on Chitosan/PNIPAM Bilayer Volatile Complementary Resistive Switching Memristor.

The presence of neurons is crucial in neuromorphic computing systems as they play a vital role in modulating the strength of synapses through the release of either excitatory or inhibitory stimuli. Hence, the development of sensory neurons plays a pivotal role in broadening the scope of brain-inspired neural computing. The present study introduces an artificial sensory neuron, which is constructed using a temperature-sensitive volatile complementary resistance switch memristor based on the functional layer of the chitosan/PNIPAM bilayer. The resistive switching behavior arises from the formation and ionization of oxygen vacancy filaments, whereby the threshold voltage and low resistive resistance of the device exhibit a temperature-dependent increase within the range of 290-410 K. A functional replication of a neuron with leaky integration and firing has been successfully developed, effectively simulating essential biological functions such as firing triggered by threshold, refractory period implementation, and modulation of spiking frequency. The artificial sensory neuron exhibits characteristics similar to those of leaky integrated firing neurons that receive temperature inputs. It has the potential to control the output frequency and amplitude under varying temperature conditions, making it suitable for temperature-sensing applications. This study presents a potential hardware implementation for developing efficient artificial intelligence systems that can support temperature detections.

Carbon price forecasting using leaky integrator echo state networks with the framework of decomposition-reconstruction-integration

Carbon trading is one of the strategies to achieve the goal of dual carbon. However, carbon prices exhibit non-stationary and nonlinear characteristics, posing significant challenges for accurate forecasting. Inspired by the ideas of “division and conquest” and “granularity reconstruction”, a decomposition-reconstruction-integration framework is built for carbon price forecasting. The proposed model leverages a comprehensive ensemble empirical mode decomposition with adaptive noise (CEEMDAN) and sample entropy (SE) to extract features from two perspectives of time and complexity, and thus the one-dimensional time series of carbon price is reconstructed into different items. Subsequently, leaky integrator echo state networks (LiESNs) are utilized to forecast each of these items individually. These individual forecasts are then integrated to generate the final point forecast. To evaluate the uncertainty of the point forecast, a Gaussian process model is applied to interval forecast. The proposed model comprehensively considers the time series features of carbon price including the temporal modal features in the decomposition stage, the entropy probability features in the reconstruction stage, and the essential temporal dependencies via dynamic reservoir of interconnected neurons in the integration forecasting stage. Experimental results demonstrate the model's exceptional forecasting performance, surpassing the effectiveness of alternative approaches.

Tilted Perpendicular Anisotropy-Induced Spin-Orbit Ratchet Effects

Abstract Using micromagnetic simulations, in this study we demonstrate the tilted perpendicular anisotropy -induced spin-orbit ratchet effect. During spin-orbit torque (SOT)-induced magnetization switching, the critical currents required to switch between the two magnetization states (upward and downward magnetization) are asymmetric. In addition, in the nanowire structure, tilted anisotropy induces formation of tilted domain walls (DWs). The tilted DWs exhibit a ratchet behavior during motion. The ratchet effect during switching and DW motions can be tuned by changing the current direction with respect to the tilting direction of anisotropy. The ratchet motion of the DWs can be used to mimic the Leaky–Integrate–Fire function of a biological neuron, especially the asymmetric property of the “potential” and “reset” processes. Our results provide a full understanding of the influence of tilted perpendicular anisotropy on SOT-induced magnetization switching and DW motion, and are beneficial for design of further SOT-based devices.

Biophysics-inspired spike rate adaptation for computationally efficient phenomenological nerve modeling

This study introduces and evaluates the PHAST+ model, part of a computational framework designed to simulate the behavior of auditory nerve fibers in response to the electrical stimulation from a cochlear implant. PHAST+ incorporates a highly efficient method for calculating accommodation and adaptation, making it particularly suited for simulations over extended stimulus durations. The proposed method uses a leaky integrator inspired by classic biophysical nerve models. Through evaluation against single-fiber animal data, our findings demonstrate the model’s effectiveness across various stimuli, including short pulse trains with variable amplitudes and rates. Notably, the PHAST+ model performs better than its predecessor, PHAST (a phenomenological model by van Gendt et al.), particularly in simulations of prolonged neural responses. While PHAST+ is optimized primarily on spike rate decay, it shows good behavior on several other neural measures, such as vector strength and degree of adaptation. The future implications of this research are promising. PHAST+ drastically reduces the computational burden to allow the real-time simulation of neural behavior over extended periods, opening the door to future simulations of psychophysical experiments and multi-electrode stimuli for evaluating novel speech-coding strategies for cochlear implants.

Modeling realistic synaptic inputs of CA1 hippocampal pyramidal neurons and interneurons via Adaptive Generalized Leaky Integrate-and-Fire models

Computational models of brain regions are crucial for understanding neuronal network dynamics and the emergence of cognitive functions. However, current supercomputing limitations hinder the implementation of large networks with millions of morphological and biophysical accurate neurons. Consequently, research has focused on simplified spiking neuron models, ranging from the computationally fast Leaky Integrate and Fire (LIF) linear models to more sophisticated non-linear implementations like Adaptive Exponential (AdEX) and Izhikevic models, through Generalized Leaky Integrate and Fire (GLIF) approaches. However, in almost all cases, these models are tuned (and can be validated) only under constant current injections and they may not, in general, also reproduce experimental findings under variable currents. This study introduces an Adaptive GLIF (A-GLIF) approach that addresses this limitation by incorporating a new set of update rules. The extended A-GLIF model successfully reproduces both constant and variable current inputs, and it was validated against the results obtained using a biophysical accurate model neuron. This enhancement provides researchers with a tool to optimize spiking neuron models using classic experimental traces under constant current injections, reliably predicting responses to synaptic inputs, which can be confidently used for large-scale network implementations.

Impact of time-history terms on reservoir dynamics and prediction accuracy in echo state networks

The echo state network (ESN) is an excellent machine learning model for processing time-series data. This model, utilising the response of a recurrent neural network, called a reservoir, to input signals, achieves high training efficiency. Introducing time-history terms into the neuron model of the reservoir is known to improve the time-series prediction performance of ESN, yet the reasons for this improvement have not been quantitatively explained in terms of reservoir dynamics characteristics. Therefore, we hypothesised that the performance enhancement brought about by time-history terms could be explained by delay capacity, a recently proposed metric for assessing the memory performance of reservoirs. To test this hypothesis, we conducted comparative experiments using ESN models with time-history terms, namely leaky integrator ESNs (LI-ESN) and chaotic echo state networks (ChESN). The results suggest that compared with ESNs without time-history terms, the reservoir dynamics of LI-ESN and ChESN can maintain diversity and stability while possessing higher delay capacity, leading to their superior performance. Explaining ESN performance through dynamical metrics are crucial for evaluating the numerous ESN architectures recently proposed from a general perspective and for the development of more sophisticated architectures, and this study contributes to such efforts.

Realization of nociceptive receptors based on Mott memristors

Nociceptive receptors are primarily responsible for detecting and responding to potentially harmful stimuli, including painful sensations and tissue damage. In this letter, we designed Pt/Ag/NbO x /W memristors with threshold switching (TS) characteristics and low working voltage attributed to the diffusion of Ag ions within the device. Furthermore, this device successfully emulates the functions of a leaky integrate-and-fire neuron and nervous pain perception functions, respectively. The artificial neurons exhibit multiple neural functions, including leaky integration, threshold-driven firing, self-relaxation characteristics, and allodynia, hyperalgesia of the nociceptors. The proposed TS memristor may show great potential in the field of neuromorphic computing and creating intelligent systems that can replicate the complexity of the human brain.

All optical tunable RF filter using elemental antimony

Abstract In the past decade, the proliferation of modern telecommunication technologies, including 5G, and the widespread adoption of the Internet-of-things (IoT) have led to an unprecedented surge in data generation and transmission. This surge has created an escalating demand for advanced signal processing capabilities. Microwave photonic (MWP) processors offer a promising solution to satisfy this unprecedented demand for data processing by capitalising on the high bandwidth and low latency achievable by optical systems. In this work, we introduce an integrated MWP processing unit for all-optical RF filtering using elemental antimony. We exploit the crystallisation dynamics of antimony to demonstrate a photonic leaky integrator, which is configured to operate as a first-order low-pass filter with a bandwidth of 300 kHz and ultra-compact footprint of 16 × 16 μm2. We experimentally demonstrate the implementation of such a filter as an envelope detector to demodulate an amplitude-modulated signal. Finally, a discussion on achieving bandwidth tunability is presented.

Spiking neural network with local plasticity and sparse connectivity for audio classification

Purpose. Studying the possibility of implementing a data classification method based on a spiking neural network, which has a low number of connections and is trained based on local plasticity rules, such as Spike-Timing-Dependent Plasticity. Methods. As the basic architecture of a spiking neural network we use a network included an input layer and layers of excitatory and inhibitory spiking neurons (Leaky Integrate and Fire). Various options for organizing connections in the selected neural network are explored. We have proposed a method for organizing connectivity between layers of neurons, in which synaptic connections are formed with a certain probability, calculated on the basis of the spatial arrangement of neurons in the layers. In this case, a limited area of connectivity leads to a higher sparseness of connections in the overall network. We use frequency-based coding of data into spike trains, and logistic regression is used for decoding. Results. As a result, based on the proposed method of organizing connections, a set of spiking neural network architectures with different connectivity coefficients for different layers of the original network was implemented. A study of the resulting spiking network architectures was carried out using the Free Spoken Digits dataset, consisting of 3000 audio recordings corresponding to 10 classes of digits from 0 to 9. Conclusion. It is shown that the proposed method of organizing connections for the selected spiking neural network allows reducing the number of connections by up to 60% compared to a fully connected architecture. At the same time, the accuracy of solving the classification problem does not deteriorate and is 0.92...0.95 according to the F1 metric. This matches the accuracy of standard support vector machine, k-nearest neighbor, and random forest classifiers. The source code for this article is publicly available: https://github.com/sag111/Sparse-WTA-SNN.

BrainForest: Neuromorphic Multiplier-Less Bit-Serial Weight-Memory-Optimized 1024-Tree Brain-State Classification Processor.

Personalized brain implants have the potential to revolutionize the treatment of neurological disorders and augment cognition. Medical implants that deliver therapeutic stimulation in response to detected seizures have already been deployed for the treatment of epilepsy. These devices require low-power integrated circuits for life-long operation. This constraint impedes the integration of machine-learning driven classifiers that could improve treatment outcomes. This paper introduces BrainForest, a neuromorphic multiplier-less bit-serial weight-memory-optimized brain-state classification processor. The architecture achieves state-of-the-art energy efficiency using two layers of neuron models to implement the spectral and temporal functions needed for classification: 1) resonate-and-fire neurons are used to extract physiological signal band energy EEG biomarkers 2) leaky integrator neurons are used to build multi-timescale representations for classification. Sparse neural model firing activity is used to clock-gate device logic, thereby decreasing power consumption by 93%. An energy-optimized 1024-tree boosted decision forest performs the classification used to trigger stimulation in response to detected pathological brain states. The IC is implemented in 65nm CMOS with state-of-the-art power consumption (best case: 9.6μW, typical: 118μW), achieving a seizure sensitivity of 97.5% with a false detection rate of 2.08 per hour.

Employment of time-varying sensory evidence to test the mechanisms underlying flexible decision-making.

To make flexible decisions in dynamic environments, the brain must integrate behaviorally relevant information while simultaneously discarding irrelevant information. This study aimed to investigate the mechanisms responsible for discarding irrelevant information during context-dependent decision-making. We trained two macaque monkeys to switch between direction and depth discrimination tasks in successive trials. During decision-making, the strength of the motion or depth signal changes transiently at various times, introducing a brief pulse. We assessed the effects of pulse on behavioral choices. Consistent with previous findings, early relevant pulses, such as motion pulses during direction discrimination, had a significantly larger effect compared to late pulses. Critically, the effects of irrelevant pulses, such as motion pulses during depth discrimination, exhibited an initial minimal effect, followed by an increase and subsequent decrease as a function of pulse timing. Gating mechanisms alone, aimed at discarding irrelevant information, did not account for the observed time course of pulse effects. Instead, the observed increase in the effects of irrelevant pulses with time suggested the involvement of a leaky integration mechanism. The results suggested that the brain controls the amount of disposal in accumulating sensory evidence during flexible decision-making.

Generalised synchronisations, embeddings, and approximations for continuous time reservoir computers

We establish conditions under which a continuous time reservoir computer, such as a leaky integrator echo state network, admits a generalised synchronisation f between the source dynamics and reservoir dynamics. We show that multiple generalised synchronisations can exist simultaneously, and connect this to the multi-Echo-State-Property (multi-ESP). In the special case of a linear reservoir computer, we derive a closed form expression for the generalised synchronisation f. Furthermore, we establish conditions under which f is of class C1, and conditions under which f is a topological embedding on the fixed points of the source system. This embedding result is closely related to Takens’ embedding Theorem.We also prove that the embedding of fixed points occurs almost surely for randomly generated linear reservoir systems. With an embedding achieved, we discuss how the universal approximation theorem makes it possible to forecast the future dynamics of the source system and replicate its topological properties. We illustrate the theory by embedding a fixed point of the Lorenz-63 system into the reservoir space using numerical methods. Finally, we show that if the observations are perturbed by white noise, the GS is preserved up to a perturbation by an Ornstein–Uhlenbeck process.

Quantization-based simulation of spiking neurons: theoretical properties and performance analysis

ABSTRACT In this work we present an exhaustive analysis of the use of Quantized State Systems (QSS) algorithms for the discrete event simulation of Leaky Integrate and Fire models of spiking neurons. Making use of some properties of these algorithms, we first derive theoretical error bounds for the sub-threshold dynamics as well as estimates of the computational costs as a function of the accuracy settings. Then, we corroborate those results on different simulation experiments, where we also study how these algorithms scale with the size of the network and its connectivity. The results obtained show that the QSS algorithms, without any type of optimisation or specialisation, obtain accurate results with low computational costs even in large networks with a high level of connectivity.

A Single Schottky Barrier MOSFET-Based Leaky Integrate and Fire Neuron for Neuromorphic Computing

In this paper, a Schottky Barrier MOSFET (SB-MOSFET) based on Impact Ionization mechanism is used to design a leaky integrate and fire (LIF) neuron with considerable enhancement in area, energy and cost is proposed. Using 2D calibrated simulation, we confirmed that SB-MOSFET LIF is able to replicate the neuron behavior precisely without using external circuitry. The proposed LIF neuron shows significantly lower energy per spike of 4 pJ/spike, which is lowest among the single transistor based neurons present in the literature. The recognition precision of 89.2% has been accomplished for Modified National Institute of Standards and Technology (MNIST) image. Besides this, SB-MOSFET doesn’t require any doped regions, therefore it can be fabricated with low thermal budget.

FPGA Realization of a High-Speed Spiking Neural Network with Modified LIF Neurons for Pattern Recognition

This paper presents the multiplier-less Modified Leaky Integrate and Fire Neuron Unit (MLIFNU) and proposed a modified Spiking Neural Network (SNN) architecture for Pattern Recognition application. Synthesis results of implementation on Field-Programmable Gate Array are presented. Accordingly, the maximum frequency of the MLIFNU and SNN are 361.991 MHz and 281.442 MHz, respectively.

Multiple-Reservoir Hierarchical Echo State Network

Leaky Integrator Echo State Network (Leaky-ESN) is a useful training method for handling time series prediction problems. However, the singular coupling of all neurons in the reservoir makes Leaky-ESN less effective for sophisticated learning tasks. In this paper, we propose a new improvement to the Leaky-ESN model called the Multiple-Reservoir Hierarchical Echo State Network (MH-ESN). By introducing a new mechanism for constructing the reservoir, the efficiency of the network in handling training tasks is improved. The hierarchical structure is used in the process of constructing the reservoir mechanism of MH-ESN. The MH-ESN consists of multiple layers, each comprising a multi-reservoir echo state network model. The sub-reservoirs within each layer are linked via principal neurons, which mimics the functioning of a biological neural network. As a result, the coupling among neurons in the reservoir is decreased, and the internal dynamics of the reservoir are improved. Based on the analysis results, the MH-ESN exhibits significantly better prediction accuracy than Leaky-ESN for complex time series prediction.

Model Based Verification of Spiking Neural Networks in Cyber Physical Systems

Spiking Neural Networks (SNNs) have found increasing utility in designing safety-critical Cyber-Physical Systems (CPSs) such as implantable medical devices, autonomous vehicles, and space robotics due to their capability to operate on information represented in temporal coding and exhibit various behavioural modalities. Thus, there has been recent interest in formally verifying their timing behaviours and providing soundness guarantees of their diverse characteristics. However, beyond the simplistic Leaky Integrate and Fire (LIF) model, which only mimics 3 spiking behaviours, there is a lack of unifying methodology in literature to verify complex dynamics of biological neurons exhibiting 20 spiking behaviours as demonstrated by the pioneering work of Izhikevich. There is also a complete lack of formulation for the verification of SNN-based systems. This paper bridges these gaps by proposing a model-based approach for designing SNN-based controllers in CPS. We propose sound structural transformations translating any spiking neuron into networks of Timed Automata (TA), model the complex Izhikevich neural model and formally verify all 20 timing behaviours it exhibits for the first time. We then present two case studies that were modelled as SNNs using our approach: the PID controller, and the Car-Following controller, and subsequently attempt static model checking and statistical verification of their generated TA models for safety guarantees.

Sensitivity analysis of point neuron model simulations implemented on neuromorphic hardware.

With the ongoing growth in the field of neuro-inspired computing, newly arriving computational architectures demand extensive validation and testing against existing benchmarks to establish their competence and value. In our work, we break down the validation step into two parts-(1) establishing a methodological and numerical groundwork to establish a comparison between neuromorphic and conventional platforms and, (2) performing a sensitivity analysis on the obtained model regime to assess its robustness. We study the neuronal dynamics based on the Leaky Integrate and Fire (LIF) model, which is built upon data from the mouse visual cortex spanning a set of anatomical and physiological constraints. Intel Corp.'s first neuromorphic chip "Loihi" serves as our neuromorphic platform and results on it are validated against the classical simulations. After setting up a model that allows a seamless mapping between the Loihi and the classical simulations, we find that Loihi replicates classical simulations very efficiently with high precision. This model is then subjected to the second phase of validation, through sensitivity analysis, by assessing the impact on the cost function as values of the significant model parameters are varied. The work is done in two steps-(1) assessing the impact while changing one parameter at a time, (2) assessing the impact while changing two parameters at a time. We observe that the model is quite robust for majority of the parameters with slight change in the cost function. We also identify a subset of the model parameters changes which make the model more sensitive and thus, need to be defined more precisely.

Synaptic transistor with multiple biological functions based on metal-organic frameworks combined with the LIF model of a spiking neural network to recognize temporal information

Spiking neural networks (SNNs) have immense potential due to their utilization of synaptic plasticity and ability to take advantage of temporal correlation and low power consumption. The leaky integration and firing (LIF) model and spike-timing-dependent plasticity (STDP) are the fundamental components of SNNs. Here, a neural device is first demonstrated by zeolitic imidazolate frameworks (ZIFs) as an essential part of the synaptic transistor to simulate SNNs. Significantly, three kinds of typical functions between neurons, the memory function achieved through the hippocampus, synaptic weight regulation and membrane potential triggered by ion migration, are effectively described through short-term memory/long-term memory (STM/LTM), long-term depression/long-term potentiation (LTD/LTP) and LIF, respectively. Furthermore, the update rule of iteration weight in the backpropagation based on the time interval between presynaptic and postsynaptic pulses is extracted and fitted from the STDP. In addition, the postsynaptic currents of the channel directly connect to the very large scale integration (VLSI) implementation of the LIF mode that can convert high-frequency information into spare pulses based on the threshold of membrane potential. The leaky integrator block, firing/detector block and frequency adaptation block instantaneously release the accumulated voltage to form pulses. Finally, we recode the steady-state visual evoked potentials (SSVEPs) belonging to the electroencephalogram (EEG) with filter characteristics of LIF. SNNs deeply fused by synaptic transistors are designed to recognize the 40 different frequencies of EEG and improve accuracy to 95.1%. This work represents an advanced contribution to brain-like chips and promotes the systematization and diversification of artificial intelligence.

A neuromorphic physiological signal processing system based on VO2 memristor for next-generation human-machine interface

Physiological signal processing plays a key role in next-generation human-machine interfaces as physiological signals provide rich cognition- and health-related information. However, the explosion of physiological signal data presents challenges for traditional systems. Here, we propose a highly efficient neuromorphic physiological signal processing system based on VO2 memristors. The volatile and positive/negative symmetric threshold switching characteristics of VO2 memristors are leveraged to construct a sparse-spiking yet high-fidelity asynchronous spike encoder for physiological signals. Besides, the dynamical behavior of VO2 memristors is utilized in compact Leaky Integrate and Fire (LIF) and Adaptive-LIF (ALIF) neurons, which are incorporated into a decision-making Long short-term memory Spiking Neural Network. The system demonstrates superior computing capabilities, needing only small-sized LSNNs to attain high accuracies of 95.83% and 99.79% in arrhythmia classification and epileptic seizure detection, respectively. This work highlights the potential of memristors in constructing efficient neuromorphic physiological signal processing systems and promoting next-generation human-machine interfaces.