Spiking Neural Network Model Research Articles

Spiking Neural Networks for Real-Time Pedestrian Street-Crossing Detection Using Dynamic Vision Sensors in Simulated Adverse Weather Conditions

This study explores the integration of Spiking Neural Networks (SNNs) with Dynamic Vision Sensors (DVSs) to enhance pedestrian street-crossing detection in adverse weather conditions—a critical challenge for autonomous vehicle systems. Utilizing the high temporal resolution and low latency of DVSs, which excel in dynamic, low-light, and high-contrast environments, this research evaluates the effectiveness of SNNs compared to traditional Convolutional Neural Networks (CNNs). The experimental setup involved a custom dataset from the CARLA simulator, designed to mimic real-world variability, including rain, fog, and varying lighting conditions. Additionally, the JAAD dataset was adopted to allow for evaluations using real-world data. The SNN models were optimized using Temporally Effective Batch Normalization (TEBN) and benchmarked against well-established deep learning models, concerning their accuracy, computational efficiency, and energy efficiency in complex weather conditions. This study also conducted a comprehensive analysis of energy consumption, highlighting the significant reduction in energy usage achieved by SNNs when processing DVS data. The results indicate that SNNs, when integrated with DVSs, not only reduce computational overhead but also dramatically lower energy consumption, making them a highly efficient choice for real-time applications in autonomous vehicles (AVs).

Spike-based computational primitives to reproduce neural dynamics in the parietal cortex during motor preparation

Abstract Biologically plausible spiking neural network models of sensory cortices can be instrumental in understanding and validating their principles of computation.
 Models based on Cortical Computational Primitives (CCPs), such as Hebbian plasticity and Winner-Take-All (WTA) networks, have already been successful in this approach.
 However, the specific nature and roles of CCPs in sensorimotor cortices during cognitive tasks are yet to be fully deciphered.
 The evolution of motor intention in the Posterior Parietal Cortex (PPC) before arm-reaching movements is a well-suited cognitive process to assess the effectiveness of different CCPs.
 To this end, we propose a biologically plausible model composed of heterogeneous spiking neurons which implements and combines multiple CCPs, such as multi-timescale learning and soft WTA modules.
 By training the model to replicate the dynamics of in-vivo recordings from non-human primates, we show how it is effective in generating meaningful representations from unbalanced input data, and in faithfully reproducing the transition from motor planning to action selection.
 Our findings elucidate the importance of distributing spike-based plasticity across multi-timescales, and provide an explanation for the role of different CCPs in models of frontoparietal cortical networks for performing multisensory integration to efficiently inform action execution.

A Spatial-Channel-Temporal-Fused Attention for Spiking Neural Networks.

Spiking neural networks (SNNs) mimic brain computational strategies, and exhibit substantial capabilities in spatiotemporal information processing. As an essential factor for human perception, visual attention refers to the dynamic process for selecting salient regions in biological vision systems. Although visual attention mechanisms have achieved great success in computer vision applications, they are rarely introduced into SNNs. Inspired by experimental observations on predictive attentional remapping, we propose a new spatial-channel-temporal-fused attention (SCTFA) module that can guide SNNs to efficiently capture underlying target regions by utilizing accumulated historical spatial-channel information in the present study. Through a systematic evaluation on three event stream datasets (DVS Gesture, SL-Animals-DVS, and MNIST-DVS), we demonstrate that the SNN with the SCTFA module (SCTFA-SNN) not only significantly outperforms the baseline SNN (BL-SNN) and two other SNN models with degenerated attention modules, but also achieves competitive accuracy with the existing state-of-the-art (SOTA) methods. Additionally, our detailed analysis shows that the proposed SCTFA-SNN model has strong robustness to noise and outstanding stability when faced with incomplete data, while maintaining acceptable complexity and efficiency. Overall, these findings indicate that incorporating appropriate cognitive mechanisms of the brain may provide a promising approach to elevate the capabilities of SNNs.

Formation and retrieval of cell assemblies in a biologically realistic spiking neural network model of area CA3 in the mouse hippocampus

The hippocampal formation is critical for episodic memory, with area Cornu Ammonis 3 (CA3) a necessary substrate for auto-associative pattern completion. Recent theoretical and experimental evidence suggests that the formation and retrieval of cell assemblies enable these functions. Yet, how cell assemblies are formed and retrieved in a full-scale spiking neural network (SNN) of CA3 that incorporates the observed diversity of neurons and connections within this circuit is not well understood. Here, we demonstrate that a data-driven SNN model quantitatively reflecting the neuron type-specific population sizes, intrinsic electrophysiology, connectivity statistics, synaptic signaling, and long-term plasticity of the mouse CA3 is capable of robust auto-association and pattern completion via cell assemblies. Our results show that a broad range of assembly sizes could successfully and systematically retrieve patterns from heavily incomplete or corrupted cues after a limited number of presentations. Furthermore, performance was robust with respect to partial overlap of assemblies through shared cells, substantially enhancing memory capacity. These novel findings provide computational evidence that the specific biological properties of the CA3 circuit produce an effective neural substrate for associative learning in the mammalian brain.

Neuromorphic intermediate representation: A unified instruction set for interoperable brain-inspired computing

Spiking neural networks and neuromorphic hardware platforms that simulate neuronal dynamics are getting wide attention and are being applied to many relevant problems using Machine Learning. Despite a well-established mathematical foundation for neural dynamics, there exists numerous software and hardware solutions and stacks whose variability makes it difficult to reproduce findings. Here, we establish a common reference frame for computations in digital neuromorphic systems, titled Neuromorphic Intermediate Representation (NIR). NIR defines a set of computational and composable model primitives as hybrid systems combining continuous-time dynamics and discrete events. By abstracting away assumptions around discretization and hardware constraints, NIR faithfully captures the computational model, while bridging differences between the evaluated implementation and the underlying mathematical formalism. NIR supports an unprecedented number of neuromorphic systems, which we demonstrate by reproducing three spiking neural network models of different complexity across 7 neuromorphic simulators and 4 digital hardware platforms. NIR decouples the development of neuromorphic hardware and software, enabling interoperability between platforms and improving accessibility to multiple neuromorphic technologies. We believe that NIR is a key next step in brain-inspired hardware-software co-evolution, enabling research towards the implementation of energy efficient computational principles of nervous systems. NIR is available at neuroir.org

SpQuant-SNN: ultra-low precision membrane potential with sparse activations unlock the potential of on-device spiking neural networks applications.

Spiking neural networks (SNNs) have received increasing attention due to their high biological plausibility and energy efficiency. The binary spike-based information propagation enables efficient sparse computation in event-based and static computer vision applications. However, the weight precision and especially the membrane potential precision remain as high-precision values (e.g., 32 bits) in state-of-the-art SNN algorithms. Each neuron in an SNN stores the membrane potential over time and typically updates its value in every time step. Such frequent read/write operations of high-precision membrane potential incur storage and memory access overhead in SNNs, which undermines the SNNs' compatibility with resource-constrained hardware. To resolve this inefficiency, prior works have explored the time step reduction and low-precision representation of membrane potential at a limited scale and reported significant accuracy drops. Furthermore, while recent advances in on-device AI present pruning and quantization optimization with different architectures and datasets, simultaneous pruning with quantization is highly under-explored in SNNs. In this work, we present SpQuant-SNN, a fully-quantized spiking neural network with ultra-low precision weights, membrane potential, and high spatial-channel sparsity, enabling the end-to-end low precision with significantly reduced operations on SNN. First, we propose an integer-only quantization scheme for the membrane potential with a stacked surrogate gradient function, a simple-yet-effective method that enables the smooth learning process of quantized SNN training. Second, we implement spatial-channel pruning with membrane potential prior, toward reducing the layer-wise computational complexity, and floating-point operations (FLOPs) in SNNs. Finally, to further improve the accuracy of low-precision and sparse SNN, we propose a self-adaptive learnable potential threshold for SNN training. Equipped with high biological adaptiveness, minimal computations, and memory utilization, SpQuant-SNN achieves state-of-the-art performance across multiple SNN models for both event-based and static image datasets, including both image classification and object detection tasks. The proposed SpQuant-SNN achieved up to 13× memory reduction and >4.7× FLOPs reduction with < 1.8% accuracy degradation for both classification and object detection tasks, compared to the SOTA baseline.

Maintenance of delay-period activity in working memory task is modulated by local network structure.

Revealing the relationship between neural network structure and function is one central theme of neuroscience. In the context of working memory (WM), anatomical data suggested that the topological structure of microcircuits within WM gradient network may differ, and the impact of such structural heterogeneity on WM activity remains unknown. Here, we proposed a spiking neural network model that can replicate the fundamental characteristics of WM: delay-period neural activity involves association cortex but not sensory cortex. First, experimentally observed receptor expression gradient along the WM gradient network is reproduced by our network model. Second, by analyzing the correlation between different local structures and duration of WM activity, we demonstrated that small-worldness, excitation-inhibition balance, and cycle structures play crucial roles in sustaining WM-related activity. To elucidate the relationship between the structure and functionality of neural networks, structural circuit gradients in brain should also be subject to further measurement. Finally, combining anatomical data, we simulated the duration of WM activity across different brain regions, its maintenance relies on the interaction between local and distributed networks. Overall, network structural gradient and interaction between local and distributed networks are of great significance for WM.

Effective Active Learning Method for Spiking Neural Networks.

A large quantity of labeled data is required to train high-performance deep spiking neural networks (SNNs), but obtaining labeled data is expensive. Active learning is proposed to reduce the quantity of labeled data required by deep learning models. However, conventional active learning methods in SNNs are not as effective as that in conventional artificial neural networks (ANNs) because of the difference in feature representation and information transmission. To address this issue, we propose an effective active learning method for a deep SNN model in this article. Specifically, a loss prediction module ActiveLossNet is proposed to extract features and select valuable samples for deep SNNs. Then, we derive the corresponding active learning algorithm for deep SNN models. Comprehensive experiments are conducted on CIFAR-10, MNIST, Fashion-MNIST, and SVHN on different SNN frameworks, including seven-layer CIFARNet and 20-layer ResNet-18. The comparison results demonstrate that the proposed active learning algorithm outperforms random selection and conventional ANN active learning methods. In addition, our method converges faster than conventional active learning methods.

Spiking neural networks for biomedical signal analysis.

Artificial intelligence (AI) has had a significant impact on human life because of its pervasiveness across industries and its rapid development. Although AI has achieved superior performance in learning and reasoning, it encounters challenges such as substantial computational demands, privacy concerns, communication delays, and high energy consumption associated with cloud-based models. These limitations have facilitated a paradigm change in on-device AI processing, which offers enhanced privacy, reduced latency, and improved power efficiency through the direct execution of computations on devices. With advancements in neuromorphic systems, spiking neural networks (SNNs), often referred to as the next generation of AI, are currently in focus as on-device AI. These technologies aim to mimic the human brain efficiency and provide promising real-time processing with minimal energy. This study reviewed the application of SNNs in the analysis of biomedical signals (electroencephalograms, electrocardiograms, and electromyograms), and consequently, investigated the distinctive attributes and prospective future paths of SNNs models in the field of biomedical signal analysis.

Adaptive structure generation and neuronal differentiation for memory encoding in SNNs

Memory is the core of cognition. The exploration of the memory encoding mechanism or the representation mechanism of information in the Spiking Neural Network (SNN) is the basis for the in-depth study of memory. In this paper, we study the memory encoding mechanism of multilayer SNN models from a biomimetic perspective and explore a method using the high biological likelihood of SNN to enable the network to effectively simulate memory effects. We proposed a series of heuristic neuron-growing connection algorithms and supervised network weight learning algorithms, which were applied to the unsupervised and supervised training process of the presentation layer. These methods optimized the structure of the representation layer, achieved functional differentiation of neurons, and enabled the network to generate differentiated representations for different data modes. Under our algorithm, the proposed model achieves stable convergence with identical pattern inputs, demonstrating distinct representations and sensitivities to different visual modalities. To achieve stable information expression within the network, we conducted various comparative experiments to determine diverse parameters of the complex network. This paper contributes to the development of Brain-inspired Intelligence by bridging the gap between computer science and neuroscience by using simulations to validate biological hypotheses and guide machine learning.

Slow ramping emerges from spontaneous fluctuations in spiking neural networks

The capacity to initiate actions endogenously is critical for goal-directed behavior. Spontaneous voluntary actions are typically preceded by slow-ramping activity in medial frontal cortex that begins around two seconds before movement, which may reflect spontaneous fluctuations that influence action timing. However, the mechanisms by which these slow ramping signals emerge from single-neuron and network dynamics remain poorly understood. Here, we developed a spiking neural-network model that produces spontaneous slow ramping activity in single neurons and population activity with onsets ~2 s before threshold crossings. A key prediction of our model is that neurons that ramp together have correlated firing patterns before ramping onset. We confirmed this model-derived hypothesis in a dataset of human single neuron recordings from medial frontal cortex. Our results suggest that slow ramping signals reflect bounded spontaneous fluctuations that emerge from quasi-winner-take-all dynamics in clustered networks that are temporally stabilized by slow-acting synapses.

Estimating orientation in natural scenes: A spiking neural network model of the insect central complex.

The central complex of insects contains cells, organised as a ring attractor, that encode head direction. The 'bump' of activity in the ring can be updated by idiothetic cues and external sensory information. Plasticity at the synapses between these cells and the ring neurons, that are responsible for bringing sensory information into the central complex, has been proposed to form a mapping between visual cues and the heading estimate which allows for more accurate tracking of the current heading, than if only idiothetic information were used. In Drosophila, ring neurons have well characterised non-linear receptive fields. In this work we produce synthetic versions of these visual receptive fields using a combination of excitatory inputs and mutual inhibition between ring neurons. We use these receptive fields to bring visual information into a spiking neural network model of the insect central complex based on the recently published Drosophila connectome. Previous modelling work has focused on how this circuit functions as a ring attractor using the same type of simple visual cues commonly used experimentally. While we initially test the model on these simple stimuli, we then go on to apply the model to complex natural scenes containing multiple conflicting cues. We show that this simple visual filtering provided by the ring neurons is sufficient to form a mapping between heading and visual features and maintain the heading estimate in the absence of angular velocity input. The network is successful at tracking heading even when presented with videos of natural scenes containing conflicting information from environmental changes and translation of the camera.

High-performance deep spiking neural networks with 0.3 spikes per neuron

Communication by rare, binary spikes is a key factor for the energy efficiency of biological brains. However, it is harder to train biologically-inspired spiking neural networks than artificial neural networks. This is puzzling given that theoretical results provide exact mapping algorithms from artificial to spiking neural networks with time-to-first-spike coding. In this paper we analyze in theory and simulation the learning dynamics of time-to-first-spike-networks and identify a specific instance of the vanishing-or-exploding gradient problem. While two choices of spiking neural network mappings solve this problem at initialization, only the one with a constant slope of the neuron membrane potential at threshold guarantees the equivalence of the training trajectory between spiking and artificial neural networks with rectified linear units. For specific image classification architectures comprising feed-forward dense or convolutional layers, we demonstrate that deep spiking neural network models can be effectively trained from scratch on MNIST and Fashion-MNIST datasets, or fine-tuned on large-scale datasets, such as CIFAR10, CIFAR100 and PLACES365, to achieve the exact same performance as that of artificial neural networks, surpassing previous spiking neural networks. Our approach accomplishes high-performance classification with less than 0.3 spikes per neuron, lending itself for an energy-efficient implementation. We also show that fine-tuning spiking neural networks with our robust gradient descent algorithm enables their optimization for hardware implementations with low latency and resilience to noise and quantization.

Real-time execution of SNN models with synaptic plasticity for handwritten digit recognition on SIMD hardware.

Recent advancements in neuromorphic computing have led to the development of hardware architectures inspired by Spiking Neural Networks (SNNs) to emulate the efficiency and parallel processing capabilities of the human brain. This work focuses on testing the HEENS architecture, specifically designed for high parallel processing and biological realism in SNN emulation, implemented on a ZYNQ family FPGA. The study applies this architecture to the classification of digits using the well-known MNIST database. The image resolutions were adjusted to match HEENS' processing capacity. Results were compared with existing work, demonstrating HEENS' performance comparable to other solutions. This study highlights the importance of balancing accuracy and efficiency in the execution of applications. HEENS offers a flexible solution for SNN emulation, allowing for the implementation of programmable neural and synaptic models. It encourages the exploration of novel algorithms and network architectures, providing an alternative for real-time processing with efficient energy consumption.

Direct training high-performance deep spiking neural networks: a review of theories and methods.

Spiking neural networks (SNNs) offer a promising energy-efficient alternative to artificial neural networks (ANNs), in virtue of their high biological plausibility, rich spatial-temporal dynamics, and event-driven computation. The direct training algorithms based on the surrogate gradient method provide sufficient flexibility to design novel SNN architectures and explore the spatial-temporal dynamics of SNNs. According to previous studies, the performance of models is highly dependent on their sizes. Recently, direct training deep SNNs have achieved great progress on both neuromorphic datasets and large-scale static datasets. Notably, transformer-based SNNs show comparable performance with their ANN counterparts. In this paper, we provide a new perspective to summarize the theories and methods for training deep SNNs with high performance in a systematic and comprehensive way, including theory fundamentals, spiking neuron models, advanced SNN models and residual architectures, software frameworks and neuromorphic hardware, applications, and future trends.

Myelin dystrophy impairs signal transmission and working memory in a multiscale model of the aging prefrontal cortex.

Normal aging leads to myelin alterations in the rhesus monkey dorsolateral prefrontal cortex (dlPFC), which are positively correlated with degree of cognitive impairment. It is hypothesized that remyelination with shorter and thinner myelin sheaths partially compensates for myelin degradation, but computational modeling has not yet explored these two phenomena together systematically. Here, we used a two-pronged modeling approach to determine how age-related myelin changes affect a core cognitive function: spatial working memory. First, we built a multicompartment pyramidal neuron model fit to monkey dlPFC empirical data, with an axon including myelinated segments having paranodes, juxtaparanodes, internodes, and tight junctions. This model was used to quantify conduction velocity (CV) changes and action potential (AP) failures after demyelination and subsequent remyelination. Next, we incorporated the single neuron results into a spiking neural network model of working memory. While complete remyelination nearly recovered axonal transmission and network function to unperturbed levels, our models predict that biologically plausible levels of myelin dystrophy, if uncompensated by other factors, can account for substantial working memory impairment with aging. The present computational study unites empirical data from ultrastructure up to behavior during normal aging, and has broader implications for many demyelinating conditions, such as multiple sclerosis or schizophrenia.

Automatic detection of sleep apnea from a single-lead ECG signal based on spiking neural network model

BackgroundSleep apnea (SLA) is a commonly encountered sleep disorder characterized by repetitive cessation of respiration while sleeping. In the past few years, researchers have focused on developing less complex and more cost-effective diagnostic approaches for identifying SLA recipients, in contrast to the cumbersome, complicated, and expensive conventional methods. MethodThis study presents a biologically plausible learning approach of spiking neural networks (SNN) with temporal coding and a tempotron learning model for diagnosing SLA disorder using single-lead electrocardiogram (ECG) data information. The proposed framework utilizes temporal encoding and the leaky integrate and fire model to transform the ECG signal into spikes for capturing the signal's dynamic pattern nature and to simulate input response behaviors. The tempoton learning technique, a spike-based algorithm, trains the SNN model to identify SLA event patterns from encoded output spike trains. This study utilized ECG data to extract heart rate variability (HRV) and ECG-derived respiration (EDR) signals from 1-min segment data of ECG records for input to SNN model. Thirty-five recordings of both released and withheld data from the Apnea-ECG databases from Physionet have been applied to train the SNN model and validate the model's efficacy in identifying SLA occurrences. ResultsThe proposed method demonstrated substantial improvements compared to other SLA detection techniques, achieving a significant accuracy of 94.63 % for per-segment detection, along with specificity, sensitivity, F1-score and AUC values of 96.21 %, 92.04 %, 0.9285, and 0.9851 respectively. The accuracy for per-recording detection achieved 100 %, with a correlation coefficient value of 0.986. Additionally, the experiment used UCD data for validation methods, achieving an accuracy of 84.573 %. ConclusionsThese results suggest the effectiveness and accessibility of the presented approach for accurately identifying SLA cases. The suggested model enhances the performance of SLA detection when contrasted with various techniques based on feature engineering and feature learning.

A Novel Electronic Nose Using Biomimetic Spiking Neural Network for Mixed Gas Recognition

Odors existing in natural environment are typically mixtures of a large variety of chemical compounds in specific proportions. It is a challenging task for an electronic nose to recognize the gas mixtures. Most current research is based on the overall response of sensors and uses relatively simple datasets, which cannot be used for complex mixtures or rapid monitoring scenarios. In this study, a novel electronic nose (E-nose) using a spiking neural network (SNN) model was proposed for the detection and recognition of gas mixtures. The electronic nose integrates six commercial metal oxide sensors for automated gas acquisition. SNN with a simple three-layer structure was introduced to extract transient dynamic information and estimate concentration rapidly. Then, a dataset of mixed gases with different orders of magnitude was established by the E-nose to verify the model’s performance. Additionally, random forests and the decision tree regression model were used for comparison with the SNN-based model. Results show that the model utilizes the dynamic characteristics of the sensors, achieving smaller mean squared error (MSE &lt; 0.01) and mean absolute error (MAE) with less data compared to random forest and decision tree algorithms. In conclusion, the electronic nose system combined with the bionic model shows a high performance in identifying gas mixtures, which has a great potential to be used for indoor air quality monitoring in practical applications.

Bridging model and experiment in systems neuroscience with Cleo: the Closed-Loop, Electrophysiology, and Optophysiology simulation testbed.

Systems neuroscience has experienced an explosion of new tools for reading and writing neural activity, enabling exciting new experiments such as all-optical or closed-loop control that effect powerful causal interventions. At the same time, improved computational models are capable of reproducing behavior and neural activity with increasing fidelity. Unfortunately, these advances have drastically increased the complexity of integrating different lines of research, resulting in the missed opportunities and untapped potential of suboptimal experiments. Experiment simulation can help bridge this gap, allowing model and experiment to better inform each other by providing a low-cost testbed for experiment design, model validation, and methods engineering. Specifically, this can be achieved by incorporating the simulation of the experimental interface into our models, but no existing tool integrates optogenetics, two-photon calcium imaging, electrode recording, and flexible closed-loop processing with neural population simulations. To address this need, we have developed Cleo: the Closed-Loop, Electrophysiology, and Optophysiology experiment simulation testbed. Cleo is a Python package enabling injection of recording and stimulation devices as well as closed-loop control with realistic latency into a Brian spiking neural network model. It is the only publicly available tool currently supporting two-photon and multi-opsin/wavelength optogenetics. To facilitate adoption and extension by the community, Cleo is open-source, modular, tested, and documented, and can export results to various data formats. Here we describe the design and features of Cleo, validate output of individual components and integrated experiments, and demonstrate its utility for advancing optogenetic techniques in prospective experiments using previously published systems neuroscience models.

Exploring the potential of spiking neural networks in biomedical applications: advantages, limitations, and future perspectives.

In this paper, a comprehensive exploration is undertaken to elucidate the utilization of Spiking Neural Networks (SNNs) within the biomedical domain. The investigation delves into the experimentally validated advantages of SNNs in comparison to alternative models like LSTM, while also critically examining the inherent limitations of SNN classifiers or algorithms. SNNs exhibit distinctive advantages that render them particularly apt for targeted applications within the biomedical field. Over time, SNNs have undergone extensive scrutiny in realms such as neuromorphic processing, Brain-Computer Interfaces (BCIs), and Disease Diagnosis. Notably, SNNs demonstrate a remarkable affinity for the processing and analysis of biomedical signals, including but not limited to electroencephalogram (EEG), electromyography (EMG), and electrocardiogram (ECG) data. This paper initiates its exploration by introducing some of the biomedical applications of EMG, such as the classification of hand gestures and motion decoding. Subsequently, the focus extends to the applications of SNNs in the analysis of EEG and ECG signals. Moreover, the paper delves into the diverse applications of SNNs in specific anatomical regions, such as the eyes and noses. In the final sections, the paper culminates with a comprehensive analysis of the field, offering insights into the advantages, disadvantages, challenges, and opportunities introduced by various SNN models in the realm of healthcare and biomedical domains. This holistic examination provides a nuanced perspective on the potential transformative impact of SNN across a spectrum of applications within the biomedical landscape.