Spiking Neurons Research Articles

A Federated Learning Protocol for Spiking Neural Membrane Systems.

Although deep learning models have shown promising results in solving problems related to image recognition or natural language processing, they do not match how the biological brain works. Some of the differences include the amount of energy consumed, the way neurons communicate, or the way they learn. To close the gap between artificial neural networks and biological ones, researchers proposed the spiking neural network. Layered Spiking Neural P systems (LSN P systems) are networks of spiking neurons used to solve various classification problems. In this paper, we study the LSN P systems in the context of a federated learning client-server architecture over horizontally partitioned data. We analyze the privacy implications of pre-trained LSN P systems through membership inference attacks. We also perform experiments to assess the performance of an LSN P system trained in the federated learning setup. Our findings suggest that LSN P systems demonstrate higher accuracy and faster convergence compared to federated algorithms based on either perceptron or spiking neural networks.

Exploiting memristive autapse and temporal distillation for training spiking neural networks

Spiking neural networks (SNNs) have attracted widespread attention due to their brain-inspired information processing mechanism and low power, sparse accumulation computation on neuromorphic chips. The surrogate gradient method makes it possible to train deep SNNs using backpropagation and shows satisfactory performance on some tasks. However, as the network structure becomes deeper, the spike information may fail to transmit to deeper layers, thus causing the output layer to make wrong predictions in recognition tasks. Inspired by the autaptic structure in the cerebral cortex, which is formed by axons connecting to their own dendrites and capable of modulating neuronal activity, we use discrete memristors to build feedback-connected autapses to adaptively regulate the precision of the spikes. Further, to prevent outlier at a certain time step from affecting the overall output, we distill the averaged knowledge into sub-models at each time step to correct potential errors. By combining these two proposed methods, we propose a deep SNNs optimized by Leaky Integrate-and-Fire (LIF) model with memristive autapse and temporal distillation, referred to as MA-SNN. A series of experiments on static datasets (CIFAR10 and CIFAR100) as well as neuromorphic datasets (DVS-CIFAR10 and N-Caltech101) demonstrated the competitiveness of the proposed model and validated the effectiveness of its components. Code for MA-SNN is available at: https://github.com/CHNtao/MA-SNN.

Hesitant fuzzy clustering with convolutional spiking neural network for movie recommendations

The movie recommender system is one of the most influential and practical tools for aiding individuals in quickly selecting films to watch. Despite numerous academic efforts to employ recommender systems for various purposes, such as movie-watching and book-buying, many studies have overlooked user-specific movie recommendations. This paper introduces a novel approach for movie recommendations that combines the hesitant fuzzy clustering with a convolutional spiking neural network movie recommender system. The initial step involves acquiring input data from benchmark datasets like MovieLens 100K and MovieLens 1M. Further, content-based features are extracted from the dataset using ternary pattern and discrete wavelet transforms. After that hesitant fuzzy linguistic Bi-objective clustering (HFLBC) is applied for cluster selection based on the extracted features. Subsequently, a movie recommender scheme utilizing a convolutional spiking neural network is introduced to predict user film preferences. The efficiency of the proposed model is compared to existing methods such as multi-modal trust-dependent recommender scheme and graph-dependent hybrid recommendation scheme. The results show a significant improvement, with the proposed model achieving 13.79% and 16.47% higher accuracy than the existing methods. The findings highlight the potential of proposed system in enhancing the accuracy and personalization of movie recommendations.

SpikeLog: Log-Based Anomaly Detection via Potential-Assisted Spiking Neuron Network

SpikeLog: Log-Based Anomaly Detection via Potential-Assisted Spiking Neuron Network

NeuDen: a framework for the integration of neuromorphic evolving spiking neural networks with dynamic evolving neuro-fuzzy systems for predictive and explainable modelling of streaming data

NeuDen: a framework for the integration of neuromorphic evolving spiking neural networks with dynamic evolving neuro-fuzzy systems for predictive and explainable modelling of streaming data

Integration of CeO2-Based Memristor with Vertically Aligned Nanocomposite Thin Film: Enabling Selective Conductive Filament Formation for High-Performance Electronic Synapses.

The CeO2-based memristor has attracted significant attention due to its intrinsic resistive switching (RS) properties, large on/off ratio, and great plasticity, making it a promising candidate for artificial synapses. However, significant challenges such as high power consumption and poor device reliability hinder its broad application in neuromorphic microchips. To tackle these issues, in this work, we design a novel bilayer (BL) memristor by integrating a CeO2-based memristor with a Co-CeO2 vertically aligned nanocomposite (VAN) layer and compare it with the single layer (SL) memristor. Preliminary electrical testing reveals that the BL memristor offers a reduced set/reset voltage (∼67% lower), a higher on/off ratio (∼5 × 102), enhanced device reliability, and improved device-to-device variation compared to the SL memristor. Insight from COMSOL simulation, coupled with microstructural analysis, provides a comprehensive elucidation on how the VAN layer facilitates the selective conductive filament (CF) formation. Subsequently, the plasticity of the BL memristor is evaluated through long-term potentiation/depression (LTP/LTD), paired-pulse facilitation (PPF), and spike-time-dependent plasticity (STDP). The spiking neural network (SNN) built upon the BL memristor achieves remarkable accuracy (∼94%) after only 12 iterations, underscoring its potential for high-performance neural networks.

Performing hardness classification using diffusive memristor based artificial neurons

Abstract Artificial neurons and synapses are the building blocks for constructing a neuromorphic system such as Spiking Neural Network (SNN) or Artificial Neural Network (ANN). Recently, there has been tremendous interest in using memristors to develop neuromorphic technologies that can be used in advanced SNNs and ANNs. Memristors, because of their simple device structure, easy and high-density fabrication, and integration with other semiconductor electronics are suitable candidates for the construction of neuromorphic concepts. However, not much has been discussed about using memristors for the development of sensors that can be utilized for object- classification especially their rigidity, shape and structure. In this article, we propose the application of memristors, specifically silver nanoparticle based diffusive memristor, in conjunction with a piezoelectric sensor within a robotics gripper, serving as one receptor (a tactile sensor) that triggers neuron circuitry with memristors to generate spikes. Furthermore, to perform hardness classification, we utilized various objects to collect data and generated multiple spikes corresponding to each object. This data was then utilized with a machine learning algorithm. The outcomes were compared with the accuracy of commercial FSR tactile sensors. Our approach demonstrated the capability of diffusive memristors in generating neuron spikes from tactile stimuli for hardness classification, achieving accuracy ranging from 82% to 100% during the validation of 20% test data across various algorithms, while the FSR sensors achieved an accuracy range of 95% to 98%.

Can human brain connectivity explain verbal working memory?

ABSTRACT The ability of humans to store spoken words in verbal working memory and build extensive vocabularies is believed to stem from evolutionary changes in cortical connectivity across primate species. However, the underlying neurobiological mechanisms remain unclear. Why can humans acquire vast vocabularies, while non-human primates cannot? This study addresses this question using brain-constrained neural networks that realize between-species differences in cortical connectivity. It investigates how these structural differences support the formation of neural representations for spoken words and the emergence of verbal working memory, crucial for human vocabulary building. We develop comparative models of frontotemporal and occipital cortices, reflecting human and non-human primate neuroanatomy. Using meanfield and spiking neural networks, we simulate auditory word recognition and examine verbal working memory function. The “human models”, characterized by denser inter-area connectivity in core language areas, produced larger cell assemblies than the “monkey models”, with specific topographies reflecting semantic properties of the represented words. Crucially, longer-lasting reverberant neural activity was observed in human versus monkey architectures, compatible with robust verbal working memory, a necessary condition for vocabulary building. Our findings offer insights into the structural basis of human-specific symbol learning and verbal working memory, shedding light on humans’ unique capacity for large vocabulary acquisition.

DEVELOPMENT AND SIMULATION OF SPIKE NEURAL NETWORK ARCHITECTURE

A review of different models of spike neural networks has been conducted. A spike type neural network is developed in the article. The mathematical model of exchange of neurotransmitters between neurons is proposed. An algorithm for network operation and neuron interaction is proposed, as well as an approach to training the network based on the formation of new connections from dendrites and increasing the signal transmission coefficient. The model of neural network was created in Python using the developed algorithm and mathematical model. To speed up the calculations, they were paralleled using the Numba library. The library Matplotlib was used for visual modelling and plotting the number of neurotransmitters on neurons. Experimental studies of the developed model of biological type network were carried out. It was shown that the developed network after training responds faster to the signals that were applied to it in the process of training.

Efficient coding in biophysically realistic excitatory-inhibitory spiking networks.

The principle of efficient coding posits that sensory cortical networks are designed to encode maximal sensory information with minimal metabolic cost. Despite the major influence of efficient coding in neuroscience, it has remained unclear whether fundamental empirical properties of neural network activity can be explained solely based on this normative principle. Here, we derive the structural, coding, and biophysical properties of excitatory-inhibitory recurrent networks of spiking neurons that emerge directly from imposing that the network minimizes an instantaneous loss function and a time-averaged performance measure enacting efficient coding. We assumed that the network encodes a number of independent stimulus features varying with a time scale equal to the membrane time constant of excitatory and inhibitory neurons. The optimal network has biologically-plausible biophysical features, including realistic integrate-and-fire spiking dynamics, spike-triggered adaptation, and a non-specific excitatory external input. The excitatory-inhibitory recurrent connectivity between neurons with similar stimulus tuning implements feature-specific competition, similar to that recently found in visual cortex. Networks with unstructured connectivity cannot reach comparable levels of coding efficiency. The optimal ratio of excitatory vs inhibitory neurons and the ratio of mean inhibitory-to-inhibitory vs excitatory-to-inhibitory connectivity are comparable to those of cortical sensory networks. The efficient network solution exhibits an instantaneous balance between excitation and inhibition. The network can perform efficient coding even when external stimuli vary over multiple time scales. Together, these results suggest that key properties of biological neural networks may be accounted for by efficient coding.

An electrodiffusive network model with multicompartmental neurons and synaptic connections.

Most computational models of neurons assume constant ion concentrations, disregarding the effects of changing ion concentrations on neuronal activity. Among the models that do incorporate ion concentration dynamics, simplifications are often made that sacrifice biophysical consistency, such as neglecting the effects of ionic diffusion on electrical potentials or the effects of electric drift on ion concentrations. A subset of models with ion concentration dynamics, often referred to as electrodiffusive models, account for ion concentration dynamics in a way that ensures a biophysical consistent relationship between ion concentrations, electric charge, and electrical potentials. These models include compartmental single-cell models, geometrically explicit models, and domain-type models, but none that model neuronal network dynamics. To address this gap, we present an electrodiffusive network model with multicompartmental neurons and synaptic connections, which we believe is the first compartmentalized network model to account for intra- and extracellular ion concentration dynamics in a biophysically consistent way. The model comprises an arbitrary number of "units," each divided into three domains representing a neuron, glia, and extracellular space. Each domain is further subdivided into a somatic and dendritic layer. Unlike conventional models which focus primarily on neuronal spiking patterns, our model predicts intra- and extracellular ion concentrations (Na+, K+, Cl-, and Ca2+), electrical potentials, and volume fractions. A unique feature of the model is that it captures ephaptic effects, both electric and ionic. In this paper, we show how this leads to interesting behavior in the network. First, we demonstrate how changing ion concentrations can affect the synaptic strengths. Then, we show how ionic ephaptic coupling can lead to spontaneous firing in neurons that do not receive any synaptic or external input. Lastly, we explore the effects of having glia in the network and demonstrate how a strongly coupled glial syncytium can prevent neuronal depolarization blocks.

A VOx-based optoelectronic memristor for application in visual perception

Abstract While biological vision systems excel at in-memory processing with low power consumption, traditional silicon-based vision chips struggle with high energy demands. This gap motivates the exploration of alternative materials for artificial intelligence applications. This paper presents a VOx-based optoelectronic synaptic memristive device. The proposed artificial synaptic device ITO/VO x /Pt mimics biological functions such as potentiation (P), depression (D), long-term memory, short-term memory (STM), and paired-pulse facilitation (PPF). The PPF index, standing at 105%, suggests a favorable pattern in STM function. The device served as synapses within a spiking neural network showing an achievable pattern classification accuracy of 88.68%, highlighting the potential of the VO x synaptic device for pattern classification tasks. The suggested VO x -based synaptic devices could represent an efficient pattern recognition and visual perception application.

Energy‐Efficient Hardware Implementation of Spiking‐Restricted Boltzmann Machines Using Pseudo‐Synaptic Sampling

Stochastic sampling is performed to reduce hardware energy consumption and prevent overfitting by reducing parameters, because not all data are required for learning. In this study, a new approach, pseudo‐synaptic sampling (PS2) method, which approximates the conventional synaptic sampling machine (S2M) method through a hardware‐friendly implementation while demonstrating superior efficiency, is introduced. By sampling in front of neurons rather than at each synapse, the PS2 method improves hardware energy efficiency and ensures scalability. Furthermore, it improves energy and area efficiency by eliminating the additional circuit required by other techniques, such as the random walk (RW) method previously used which requires an additional circuit to frequently charge/discharge the membrane potential. Herein, the average firing rate equation for the S2M method is modified to suit the experimental conditions of this study. Through this numerical simulations, it is confirmed that the activation function of the PS2 method aligns with that of the S2M method and verified that the PS2 method can implement stochasticity for restricted Boltzmann machine (RBM) neurons. Experimental validation of the PS2 method, compared to the RW method, for Modified National Institute of Standards and Technology database (MNIST) training and inference on field‐programmable‐gate‐array‐implemented spiking RBM chips reveals promising results. In an MNIST 100‐handwritten digit experiment, the PS2 method exhibits on‐chip training accuracy (92%) comparable to that of the RW method (93%). Furthermore, in the energy consumption analysis, it is shown that the PS2 method reduces power consumption by 94.94% compared to the RW method, highlighting its enhanced power efficiency due to a reduced number of circuit elements. In the investigation into the impact of increasing the frequency at which random bits are generated, it is shown that the RW method experiences accuracy degradation even with slight increases, whereas the PS2 method maintains accuracy over significantly longer periods. This enables further power reduction by allowing for a longer period during random bit generation. In this study, a foundation is laid for maximizing the energy efficiency of spiking neural network processors by optimizing internal noise generation mechanisms.

Entropy of Neuronal Spike Patterns

Neuronal spike patterns are the fundamental units of neural communication in the brain, which is still not fully understood. Entropy measures offer a quantitative framework to assess the variability and information content of these spike patterns. By quantifying the uncertainty and informational content of neuronal patterns, entropy measures provide insights into neural coding strategies, synaptic plasticity, network dynamics, and cognitive processes. Here, we review basic entropy metrics and then we provide examples of recent advancements in using entropy as a tool to improve our understanding of neuronal processing. It focuses especially on studies on critical dynamics in neural networks and the relation of entropy to predictive coding and cortical communication. We highlight the necessity of expanding entropy measures from single neurons to encompass multi-neuronal activity patterns, as cortical circuits communicate through coordinated spatiotemporal activity patterns, called neuronal packets. We discuss how the sequential and partially stereotypical nature of neuronal packets influences the entropy of cortical communication. Stereotypy reduces entropy by enhancing reliability and predictability in neural signaling, while variability within packets increases entropy, allowing for greater information capacity. This balance between stereotypy and variability supports both robustness and flexibility in cortical information processing. We also review challenges in applying entropy to analyze such spatiotemporal neuronal spike patterns, notably, the “curse of dimensionality” in estimating entropy for high-dimensional neuronal data. Finally, we discuss strategies to overcome these challenges, including dimensionality reduction techniques, advanced entropy estimators, sparse coding schemes, and the integration of machine learning approaches. Thus, this work summarizes the most recent developments on how entropy measures contribute to our understanding of principles underlying neural coding.

Stress-induced artificial neuron spiking in diffusive memristors

Diffusive memristors owing to their ability to produce current spiking when a constant or slowly changing voltage is applied are competitive candidates for development of artificial electronic neurons. These artificial neurons can be integrated into various prospective autonomous and robotic systems as sensors, e.g. ones implementing object grasping and classification. We report here Ag nanoparticle-based diffusive memristor prepared on a flexible polyethylene terephthalate substrate in which the electric spiking behaviour was induced by the electric voltage under an additional stimulus of external mechanical impact. By changing the magnitude and frequency of the mechanical impact, we are able to manipulate the spiking response of our artificial neuron. This functionality to control the spiking characteristics paves a pathway for the development of touch-perception sensors that can convert local pressure into electrical spikes for further processing in neural networks. We have proposed a mathematical model which captures the operation principle of the fabricated memristive sensors and qualitatively describes the measured spiking behaviour. Employing such flexible diffusive memristors that can directly translate tactile information into spikes, similar to force and pressure sensors, could offer substantial benefits for various applications in robotics.

Enhanced Tuberculosis Detection in Chest X-Rays Using Optimal Feature Selection with Hybrid Binary Particle Swarm Optimization (HBPSO) and a Dual Classifier Framework Combining LSTM and Spiking Neural Networks

Tuberculosis (TB) remains a significant global health challenge, particularly in developing countries, where rapid and accurate diagnosis is crucial for effective treatment. Recent advancements in machine learning and image processing have shown promise in enhancing the detection of TB from chest X-ray images. This research introduces a novel approach that integrates a hybrid feature selection method with a dual classifier framework to improve the accuracy of tuberculosis detection. Feature extraction is conducted using Multi-Scale Local Binary Patterns (MSLBP), Speeded-Up Robust Features (SURF) with Bag of Words (BoW), and Cartoon Texture Features, combining them into a comprehensive feature vector. The Hybrid Binary Particle Swarm Optimization (HBPSO) is employed for optimal feature selection, reducing redundancy and improving classification efficiency. Classification is performed using a dual framework involving a Long Short-Term Memory (LSTM) network and a Spiking Neural Network (SNN), which work in tandem to provide robust predictions. Majority voting is utilized to finalize predictions, ensuring high accuracy and adaptability across varying TB presentations. Extensive evaluation on publicly available datasets shows that this hybrid approach significantly outperforms existing methods, achieving a highest accuracy of 99.73%, along with superior precision, recall, and F1 scores.

The backpropagation algorithm implemented on spiking neuromorphic hardware

The capabilities of natural neural systems have inspired both new generations of machine learning algorithms as well as neuromorphic, very large-scale integrated circuits capable of fast, low-power information processing. However, it has been argued that most modern machine learning algorithms are not neurophysiologically plausible. In particular, the workhorse of modern deep learning, the backpropagation algorithm, has proven difficult to translate to neuromorphic hardware. This study presents a neuromorphic, spiking backpropagation algorithm based on synfire-gated dynamical information coordination and processing implemented on Intel’s Loihi neuromorphic research processor. We demonstrate a proof-of-principle three-layer circuit that learns to classify digits and clothing items from the MNIST and Fashion MNIST datasets. To our knowledge, this is the first work to show a Spiking Neural Network implementation of the exact backpropagation algorithm that is fully on-chip without a computer in the loop. It is competitive in accuracy with off-chip trained SNNs and achieves an energy-delay product suitable for edge computing. This implementation shows a path for using in-memory, massively parallel neuromorphic processors for low-power, low-latency implementation of modern deep learning applications.

A cloud-edge framework for energy-efficient event-driven control: an integration of online supervised learning, spiking neural networks and local plasticity rules

Abstract This paper presents a novel cloud-edge framework for addressing energy constraints in complex control systems. Our approach centers around a learning-based controller using Spiking Neural Networks (SNN) on physical plants. By integrating a biologically plausible learning method with local plasticity rules, we harness the energy efficiency, scalability of the newtwork, and low latency of SNNs. This design replicates control signals from a cloud-based controller directly on the plant, reducing the need for constant plant-cloud communication. The plant updates weights only when errors surpass predefined thresholds, ensuring efficiency and robustness in various conditions. Applied to linear workbench systems and satellite rendezvous scenarios, including obstacle avoidance, our architecture dramatically lowers normalized tracking error by 96% with increased network size. The event-driven nature of SNNs minimizes energy consumption, utilizing only about 11.1 × 104 pJ (0.3% of conventional computing requirements). The results demonstrate the system’s adjustment to changing work environments and its efficient use of energy resources, with a moderate increase in energy consumption of 37% for dynamic obstacles, compared to non-obstacle scenarios.

Intermittent rate coding and cue-specific ensembles support working memory.

Persistent, memorandum-specific neuronal spiking activity has long been hypothesized to underlie working memory1,2. However, emerging evidence suggests a potential role for 'activity-silent' synaptic mechanisms3-5. This issue remains controversial because evidence for either view has largely relied eitheron datasets that fail to capture single-trial population dynamics or onindirect measures of neuronal spiking. We addressed this controversyby examining the dynamics of mnemonic information on single trials obtained from large, local populations of lateral prefrontal neurons recorded simultaneously in monkeys performing a working memory task. Here we show that mnemonic information does not persist in the spiking activity of neuronal populations during memory delays, but instead alternates between coordinated 'On' and 'Off' states. At the level of single neurons, Off states are drivenby both a loss of selectivity for memoranda and a return of firing rates to spontaneous levels. Further exploiting the large-scale recordings used here, we show that mnemonic information is available in the patterns of functional connections among neuronal ensembles during Off states. Our results suggest that intermittent periods of memorandum-specific spiking coexist with synaptic mechanisms to support working memory.

MXene/WO3 Sensor Array with Improved SNN Algorithm for Accurate Identification of Toxic Gases.

Gas sensing is pivotal in critical areas such as industrial production and food safety. This study explores the gas classification capabilities of MXene-based gas sensors. Pure V2CTx MXene and an MXene/WO3 nanocomposite were synthesized, and MXene-based gas sensors were integrated into a 2 × 2 rudimentary electronic nose array. The tests on gas sensitivity revealed that the inclusion of WO3 nanoparticles (NPs) boosted the sensor's response to 10 ppm of NO2 from 2.82 to 3.45 at room temperature. Moreover, the sensor showcased a rapid response/recovery duration of 74.5/149.0 s, excellent environmental stability, and long-term reliable sensing performance. Furthermore, we have improved the method of accurately identifying four toxic gases detected by an MXene-based sensor array using a spiking neural network (SNN) based on the memristive system. Also, the performance of this identification method revealed that the method achieved 95.83% accuracy in the identification of the four gases. Notably, the improved SNN demonstrated approximately 5% higher accuracy than the other gas recognition algorithm. These results highlight the potential of SNN as a powerful tool to accurately and reliably identify toxic gases based on the gas sensor array.