Spiking Neurons Research Articles

Benchmarking of hardware-efficient real-time neural decoding in brain–computer interfaces

Designing processors for implantable closed-loop neuromodulation systems presents a formidable challenge owing to the constrained operational environment, which requires low latency and high energy efficacy. Previous benchmarks have provided limited insights into power consumption and latency. However, this study introduces algorithmic metrics that capture the potential and limitations of neural decoders for closed-loop intra-cortical brain–computer interfaces in the context of energy and hardware constraints. This study benchmarks common decoding methods for predicting a primate’s finger kinematics from the motor cortex and explores their suitability for low latency and high energy efficient neural decoding. The study found that ANN-based decoders provide superior decoding accuracy, requiring high latency and many operations to effectively decode neural signals. Spiking neural networks (SNNs) have emerged as a solution, bridging this gap by achieving competitive decoding performance within sub-10 ms while utilizing a fraction of computational resources. These distinctive advantages of neuromorphic SNNs make them highly suitable for the challenging closed-loop neural modulation environment. Their capacity to balance decoding accuracy and operational efficiency offers immense potential in reshaping the landscape of neural decoders, fostering greater understanding, and opening new frontiers in closed-loop intra-cortical human-machine interaction.

A Biologically Inspired Movement Recognition System with Spiking Neural Networks for Ambient Assisted Living Applications

This study presents a novel solution for ambient assisted living (AAL) applications that utilizes spiking neural networks (SNNs) and reconfigurable neuromorphic processors. As demographic shifts result in an increased need for eldercare, due to a large elderly population that favors independence, there is a pressing need for efficient solutions. Traditional deep neural networks (DNNs) are typically energy-intensive and computationally demanding. In contrast, this study turns to SNNs, which are more energy-efficient and mimic biological neural processes, offering a viable alternative to DNNs. We propose asynchronous cellular automaton-based neurons (ACANs), which stand out for their hardware-efficient design and ability to reproduce complex neural behaviors. By utilizing the remote supervised method (ReSuMe), this study improves spike train learning efficiency in SNNs. We apply this to movement recognition in an elderly population, using motion capture data. Our results highlight a high classification accuracy of 83.4%, demonstrating the approach’s efficacy in precise movement activity classification. This method’s significant advantage lies in its potential for real-time, energy-efficient processing in AAL environments. Our findings not only demonstrate SNNs’ superiority over conventional DNNs in computational efficiency but also pave the way for practical neuromorphic computing applications in eldercare.

Spike-based computation using classical recurrent neural networks

Spiking neural networks (SNNs) are a type of artificial neural networks in which communication between neurons is only made of events, also called spikes. This property allows neural networks to make asynchronous and sparse computations and therefore drastically decrease energy consumption when run on specialized hardware. However, training such networks is known to be difficult, mainly due to the non-differentiability of the spike activation, which prevents the use of classical backpropagation. This is because state-of-the-art SNNs are usually derived from biologically-inspired neuron models, to which are applied machine learning methods for training. Nowadays, research about SNNs focuses on the design of training algorithms whose goal is to obtain networks that compete with their non-spiking version on specific tasks. In this paper, we attempt the symmetrical approach: we modify the dynamics of a well-known, easily trainable type of recurrent neural network (RNN) to make it event-based. This new RNN cell, called the spiking recurrent cell, therefore communicates using events, i.e. spikes, while being completely differen-tiable. Vanilla backpropagation can thus be used to train any network made of such RNN cell. We show that this new network can achieve performance comparable to other types of spiking networks in the MNIST benchmark and its variants, the Fashion-MNIST and the Neuromorphic-MNIST. Moreover, we show that this new cell makes the training of deep spiking networks achievable.

Fully neuromorphic vision and control for autonomous drone flight.

Biological sensing and processing is asynchronous and sparse, leading to low-latency and energy-efficient perception and action. In robotics, neuromorphic hardware for event-based vision and spiking neural networks promises to exhibit similar characteristics. However, robotic implementations have been limited to basic tasks with low-dimensional sensory inputs and motor actions because of the restricted network size in current embedded neuromorphic processors and the difficulties of training spiking neural networks. Here, we present a fully neuromorphic vision-to-control pipeline for controlling a flying drone. Specifically, we trained a spiking neural network that accepts raw event-based camera data and outputs low-level control actions for performing autonomous vision-based flight. The vision part of the network, consisting of five layers and 28,800 neurons, maps incoming raw events to ego-motion estimates and was trained with self-supervised learning on real event data. The control part consists of a single decoding layer and was learned with an evolutionary algorithm in a drone simulator. Robotic experiments show a successful sim-to-real transfer of the fully learned neuromorphic pipeline. The drone could accurately control its ego-motion, allowing for hovering, landing, and maneuvering sideways-even while yawing at the same time. The neuromorphic pipeline runs on board on Intel's Loihi neuromorphic processor with an execution frequency of 200 hertz, consuming 0.94 watt of idle power and a mere additional 7 to 12 milliwatts when running the network. These results illustrate the potential of neuromorphic sensing and processing for enabling insect-sized intelligent robots.

Hybrid synaptic structure for spiking neural network realization

Neural networks and neuromorphic computing represent fundamental paradigms as alternative approaches to Von-Neumann-based implementations, advancing in the applications of deep learning and machine vision. Nonetheless, conventional semiconductor circuits encounter challenges in achieving ultra-fast processing speed and low power consumption due to their dissipative properties. Conversely, single flux quantum circuits exhibit inherent spiking behavior, showcasing their characteristics as a promising candidate for spiking neural networks (SNNs). In this work, we present a compact hybrid synapse circuit to mimic the biological interconnect functionality, enabling the weighting operations for excitatory and inhibitory impulses. Additionally, the proposed structure facilitates input accumulation, which is performed before the activation function. In the experiments, our synaptic structure interfaces with a soma circuit fabricated using a commercial Nb process, underscoring its compatibility and supporting its potential for integration into efficient neural network architectures. The weight value on the synapse is configurable by utilizing cryo-CMOS circuits, providing adaptability to the inference networks. We’ve successfully designed, fabricated, and partially tested the JJ-Synapse within our cryocooler system, enabling high-speed inference implementation for SNNs.

Continuous adaptive nonlinear model predictive control using spiking neural networks and real-time learning

Model predictive control (MPC) is a prominent control paradigm providing accurate state prediction and subsequent control actions for intricate dynamical systems with applications ranging from autonomous driving to star tracking. However, there is an apparent discrepancy between the model’s mathematical description and its behavior in real-world conditions, affecting its performance in real-time. In this work, we propose a novel neuromorphic (brain-inspired) spiking neural network for continuous adaptive non-linear MPC. Utilizing real-time learning, our design significantly reduces dynamic error and augments model accuracy, while simultaneously addressing unforeseen situations. We evaluated our framework using real-world scenarios in autonomous driving, implemented in a physics-driven simulation. We tested our design with various vehicles (from a Tesla Model 3 to an Ambulance) experiencing malfunctioning and swift steering scenarios. We demonstrate significant improvements in dynamic error rate compared with traditional MPC implementation with up to 89.15% median prediction error reduction with 5 spiking neurons and up to 96.08% with 5,000 neurons. Our results may pave the way for novel applications in real-time control and stimulate further studies in the adaptive control realm with spiking neural networks.

Spintronic Artificial Synapses Using Voltage‐Controlled Multilevel Magnetic States

AbstractNeuromorphic computing offers energy‐efficient computations for large‐scale data processing compared to conventional von Neumann computing. The artificial synapse, a key element for learning and memory operations in neuromorphic computing, requires multi‐state characteristics and the capability to change and store its states. The implementation of hardware‐based artificial synapses using nonvolatile memory provides significant advantages in terms of energy consumption and circuit area compared to their currently employed CMOS‐based counterparts. In this regard, spintronic devices have emerged as a promising candidate due to their desirable properties for artificial synapses, including multilevel formability, non‐volatility, and outstanding writing performance. In this study, spintronic artificial synapses utilizing voltage‐controlled multilevel magnetic states and energy‐ and area‐efficient artificial neural network architectures associated with them are demonstrated. The multilevel states are created by gradually modulating the magnetic easy‐axis orientation from perpendicular to in‐plane and vice versa, which is achieved either by sequentially applying gate voltage pulses or by adjusting the pulse width of the gate voltage. Based on these spintronic artificial synapses, convolutional neural network (CNN) and spiking neural network (SNN) architectures are constructed, demonstrating high recognition accuracy for the MNIST dataset with improved energy efficiency and a reduced circuit area.

Diagnostic biomarker discovery from brain EEG data using LSTM, reservoir-SNN, and NeuCube methods in a pilot study comparing epilepsy and migraine

The study introduces a new online spike encoding algorithm for spiking neural networks (SNN) and suggests new methods for learning and identifying diagnostic biomarkers using three prominent deep learning neural network models: deep BiLSTM, reservoir SNN, and NeuCube. EEG data from datasets related to epilepsy, migraine, and healthy subjects are employed. Results reveal that BiLSTM hidden neurons capture biological significance, while reservoir SNN activities and NeuCube spiking dynamics identify EEG channels as diagnostic biomarkers. BiLSTM and reservoir SNN achieve 90 and 85% classification accuracy, while NeuCube achieves 97%, all methods pinpointing potential biomarkers like T6, F7, C4, and F8. The research bears implications for refining online EEG classification, analysis, and early brain state diagnosis, enhancing AI models with interpretability and discovery. The proposed techniques hold promise for streamlined brain-computer interfaces and clinical applications, representing a significant advancement in pattern discovery across the three most popular neural network methods for addressing a crucial problem. Further research is planned to study how early can these diagnostic biomarkers predict an onset of brain states.

Computational modeling to study the impact of changes in Nav1.8 sodium channel on neuropathic pain

ObjectiveNav1.8 expression is restricted to sensory neurons; it was hypothesized that aberrant expression and function of this channel at the site of injury contributed to pathological pain. However, the specific contributions of Nav1.8 to neuropathic pain are not as clear as its role in inflammatory pain. The aim of this study is to understand how Nav1.8 present in peripheral sensory neurons regulate neuronal excitability and induce various electrophysiological features on neuropathic pain.MethodsTo study the effect of changes in sodium channel Nav1.8 kinetics, Hodgkin–Huxley type conductance-based models of spiking neurons were constructed using the NEURON v8.2 simulation software. We constructed a single-compartment model of neuronal soma that contained Nav1.8 channels with the ionic mechanisms adapted from some existing small DRG neuron models. We then validated and compared the model with our experimental data from in vivo recordings on soma of small dorsal root ganglion (DRG) sensory neurons in animal models of neuropathic pain (NEP).ResultsWe show that Nav1.8 is an important parameter for the generation and maintenance of abnormal neuronal electrogenesis and hyperexcitability. The typical increased excitability seen is dominated by a left shift in the steady state of activation of this channel and is further modulated by this channel’s maximum conductance and steady state of inactivation. Therefore, modified action potential shape, decreased threshold, and increased repetitive firing of sensory neurons in our neuropathic animal models may be orchestrated by these modulations on Nav1.8.ConclusionComputational modeling is a novel strategy to understand the generation of chronic pain. In this study, we highlight that changes to the channel functions of Nav1.8 within the small DRG neuron may contribute to neuropathic pain.

Life-long learning and evolving associative memories in brain-inspired spiking neural networks

Life-long learning and evolving associative memories in brain-inspired spiking neural networks

Cell detection with convolutional spiking neural network for neuromorphic cytometry

Imaging flow cytometry (IFC) is an advanced cell-analytic technology offering rich spatial information and fluorescence intensity for multi-parametric characterization. Manual gating in cytometry data enables the classification of discrete populations from the sample based on extracted features. However, this expert-driven technique can be subjective and laborious, often presenting challenges in reproducibility and being inherently limited to bivariate analysis. Numerous AI-driven cell classifications have recently emerged to automate the process of including multivariate data with enhanced reproducibility and accuracy. Our previous work demonstrated the early development of neuromorphic imaging cytometry, evaluating its feasibility in resolving conventional frame-based imaging systems’ limitations in data redundancy, fluorescence sensitivity, and compromised throughput. Herein, we adopted a convolutional spiking neural network (SNN) combined with the YOLOv3 model (SNN-YOLO) to perform cell classification and detection on label-free samples under neuromorphic vision. Spiking techniques are inherently suitable post-processing techniques for neuromorphic vision sensing. The experiment was conducted with polystyrene-based microparticles, THP-1, and LL/2 cell lines. The network’s performance was compared with that of a traditional YOLOv3 model fed with event-generated frame data to serve as a baseline. In this work, our SNN-YOLO outperformed the YOLOv3 baseline by achieving the highest average class accuracy of 0.974, compared to 0.962 for YOLOv3. Both models reported comparable performances across other key metrics and should be further explored for future auto-gating strategies and cytometry applications.

Neuromorphic computing spiking neural network edge detection model for content based image retrieval

ABSTRACT In contemporary times, content-based image retrieval (CBIR) techniques have gained widespread acceptance as a means for end-users to discern and extract specific image content from vast repositories. However, it is noteworthy that a substantial majority of CBIR studies continue to rely on linear methodologies such as gradient-based and derivative-based edge detection techniques. This research explores the integration of bioinspired Spiking Neural Network (SNN) based edge detection within CBIR. We introduce an innovative, computationally efficient SNN-based approach designed explicitly for CBIR applications, outperforming existing SNN models by reducing computational overhead by 2.5 times. The proposed SNN-based edge detection approach is seamlessly incorporated into three distinct CBIR techniques, each employing conventional edge detection methodologies including Sobel, Canny, and image derivatives. Rigorous experimentation and evaluations are carried out utilizing the Corel-10k dataset and crop weed dataset, a widely recognized and frequently adopted benchmark dataset in the realm of image analysis. Importantly, our findings underscore the enhanced performance of CBIR methodologies integrating the proposed SNN-based edge detection approach, with an average increase in mean precision values exceeding 3%. This study conclusively demonstrated the utility of our proposed methodology in optimizing feature extraction, thereby establishing its pivotal role in advancing edge centric CBIR approaches.

A Hybrid Neural Coding Approach for Pattern Recognition With Spiking Neural Networks.

Recently, brain-inspired spiking neural networks (SNNs) have demonstrated promising capabilities in solving pattern recognition tasks. However, these SNNs are grounded on homogeneous neurons that utilize a uniform neural coding for information representation. Given that each neural coding scheme possesses its own merits and drawbacks, these SNNs encounter challenges in achieving optimal performance such as accuracy, response time, efficiency, and robustness, all of which are crucial for practical applications. In this study, we argue that SNN architectures should be holistically designed to incorporate heterogeneous coding schemes. As an initial exploration in this direction, we propose a hybrid neural coding and learning framework, which encompasses a neural coding zoo with diverse neural coding schemes discovered in neuroscience. Additionally, it incorporates a flexible neural coding assignment strategy to accommodate task-specific requirements, along with novel layer-wise learning methods to effectively implement hybrid coding SNNs. We demonstrate the superiority of the proposed framework on image classification and sound localization tasks. Specifically, the proposed hybrid coding SNNs achieve comparable accuracy to state-of-the-art SNNs, while exhibiting significantly reduced inference latency and energy consumption, as well as high noise robustness. This study yields valuable insights into hybrid neural coding designs, paving the way for developing high-performance neuromorphic systems.

Intelligent Event-Based Lip Reading Word Classification with SNN Using Spatio-Temporal Attention Features and Triplet Loss

Lip reading is a visual alternative to enhance the intelligence of traditional speech recognition, which can benefit from retina-like event cameras that focus on dynamic movements. Spiking Neural Networks (SNNs) are inherently well-suited to cooperate with event cameras. However, employing SNN for event-based lip reading presents significant challenges, particularly in effectively extracting spatio-temporal features from input events and distinguishing between phonetically similar elements. To address these challenges, this paper proposes a novel event-based lip-reading model that leverages the SNN framework, enriched by a designed Spatial-Temporal Attention Block (STAB) and a triplet loss. Specifically, STAB comprises both spatial and temporal attention branches, dynamically emphasizing those spatial and temporal characteristics most reflective of lip movements. STAB also includes a fusion mechanism to integrate these spatial and temporal insights, providing a comprehensive and focused representation of lip movements. In addition, we enhance the model by incorporating triplet loss into the SNN training process, further improving the model's ability to distinguish between visually similar words. Experimental results show the superior performance of our SNN and validate the effect of STAB and triplet loss. We also conduct the energy consumption analysis to affirm the model's energy efficiency.

SpikeExplorer: Hardware-Oriented Design Space Exploration for Spiking Neural Networks on FPGA

One of today’s main concerns is to bring artificial intelligence capabilities to embedded systems for edge applications. The hardware resources and power consumption required by state-of-the-art models are incompatible with the constrained environments observed in edge systems, such as IoT nodes and wearable devices. Spiking Neural Networks (SNNs) can represent a solution in this sense: inspired by neuroscience, they reach unparalleled power and resource efficiency when run on dedicated hardware accelerators. However, when designing such accelerators, the amount of choices that can be taken is huge. This paper presents SpikExplorer, a modular and flexible Python tool for hardware-oriented Automatic Design Space Exploration to automate the configuration of FPGA accelerators for SNNs. SpikExplorer enables hardware-centric multiobjective optimization, supporting target factors such as accuracy, area, latency, power, and various combinations during the exploration process. The tool searches the optimal network architecture, neuron model, and internal and training parameters leveraging Bayesian optimization, trying to reach the desired constraints imposed by the user. It allows for a straightforward network configuration, providing the full set of explored points for the user to pick the trade-off that best fits their needs. The potential of SpikExplorer is showcased using three benchmark datasets. It reaches 95.8% accuracy on the MNIST dataset, with a power consumption of 180 mW/image and a latency of 0.12 ms/image, making it a powerful tool for automatically optimizing SNNs.

An artificial visual neuron with multiplexed rate and time-to-first-spike coding.

Human visual neurons rely on event-driven, energy-efficient spikes for communication, while silicon image sensors do not. The energy-budget mismatch between biological systems and machine vision technology has inspired the development of artificial visual neurons for use in spiking neural network (SNN). However, the lack of multiplexed data coding schemes reduces the ability of artificial visual neurons in SNN to emulate the visual perception ability of biological systems. Here, we present an artificial visual spiking neuron that enables rate and temporal fusion (RTF) coding of external visual information. The artificial neuron can code visual information at different spiking frequencies (rate coding) and enables precise and energy-efficient time-to-first-spike (TTFS) coding. This multiplexed sensory coding scheme could improve the computing capability and efficacy of artificial visual neurons. A hardware-based SNN with the RTF coding scheme exhibits good consistency with real-world ground truth data and achieves highly accurate steering and speed predictions for self-driving vehicles in complex conditions. The multiplexed RTF coding scheme demonstrates the feasibility of developing highly efficient spike-based neuromorphic hardware.

SNNAuth: Sensor-Based Continuous Authentication on Smartphones Using Spiking Neural Networks

SNNAuth: Sensor-Based Continuous Authentication on Smartphones Using Spiking Neural Networks

Fusion synapse by memristor and capacitor for spiking neuromorphic systems

Research on neuromorphic computing with spiking neural networks and in-memory computing to achieve low-power consumption and high-speed operation has received great attention. In this study, we proposed a new synaptic device called a fusion synapse that consists of a memristor and a capacitor connected in a series and uses a change in the time constant as a synaptic weight. We also prepared both neuron and synchronizer circuits to utilize the time constants as synaptic weights and trained them with PyTorch according to two models. One model is based on previous research, and the other is an aware model of circuit information. We converted the synaptic weights generated by the training into conductance. When assembling two models for inference, 95% accuracy was achieved on MNIST, and the power consumption per inference was 640 nJ. The accuracy was comparable to related research, and considering Denard scaling, the power consumption was lower than the digital implementation.

Minicolumn-Based Episodic Memory Model With Spiking Neurons, Dendrites and Delays.

Episodic memory is fundamental to the brain's cognitive function, but how neuronal activity is temporally organized during its encoding and retrieval is still unknown. In this article, combining hippocampus structure with a spiking neural network (SNN), a new bionic spiking temporal memory (BSTM) model is proposed to explore the encoding, formation, and retrieval of episodic memory. For encoding episodic memory, the spike-timing-dependent-plasticity (STDP) learning algorithm and a proposed minicolumn selection algorithm are used to encode each input item into several active minicolumns. For the formation of episodic memory, a sequential memory algorithm is proposed to store the contexts between items. For retrieval of episodic memory, the local retrieval algorithm and the global retrieval algorithm are proposed to retrieve sequence information, achieving multisentence prediction and multitime step prediction. All functions of BSTM are based on bionic spiking neurons, which have biological characteristics including columnar and dendritic structures, firing and receiving spikes, and delaying transmission. To test the performance of the BSTM model, the Children's Book Test (CBT) data set was used to conduct a series of experiments under different settings, including changing the number of minicolumns, neurons and sequences, modifying sequence items, etc. Compared to other sequence memory algorithms, the experimental results show that the proposed BSTM achieves higher accuracy and better robustness.

Energy efficient artificial gustatory system for in-sensor computing

In this work, a novel bio-inspired artificial gustatory system is proposed for in-sensor neuromorphic computing. The system is demonstrated using a novel material-engineered compound-semiconductor double-gate ferroelectric tunnel FET. The behaviour of the device as a biosensor and as a spiking neuron is verified individually. Using extensive simulation in Atlas TCAD, the biosensor functionality is validated with ION sensitivity of 108. Likewise, the device shows excellent neuronal behaviour with energy consumption of 37 aJ/spike, without using any external circuitry. It also exhibits control over the spiking frequency through amplitude, frequency, and duty cycle of input synaptic current. By cascading the biosensor and neuron, a complete bio-mimicked gustatory system is designed to identify a separate pattern for different tastes. The proposed gustatory system integrates both the devices, and simultaneously performs sensing and spike encoding. The biosensor transduces the pH and dielectric constant of the target biomolecule, corresponding to gustatory neurons present in the taste buds of a biological gustatory system. The neuron performs spike encoding and acts as an input neuron in a classifying network, which corresponds to gustatory cortex of its biological counterpart. The system consumes an average power of 49.58 pW which is ∼1000 × lesser than the state-of-the-art artificial gustatory system, eliminating the need for complex hardware and exorbitant energy consumption. Therefore, the proposed gustatory system provides a highly efficient and compact solution for neuromorphic gustatory sensing and classification, with potential applications in portable, wearable, and implantable devices.