Spiking Convolutional Neural Network Research Articles

Cell detection with convolutional spiking neural network for neuromorphic cytometry

Imaging flow cytometry (IFC) is an advanced cell-analytic technology offering rich spatial information and fluorescence intensity for multi-parametric characterization. Manual gating in cytometry data enables the classification of discrete populations from the sample based on extracted features. However, this expert-driven technique can be subjective and laborious, often presenting challenges in reproducibility and being inherently limited to bivariate analysis. Numerous AI-driven cell classifications have recently emerged to automate the process of including multivariate data with enhanced reproducibility and accuracy. Our previous work demonstrated the early development of neuromorphic imaging cytometry, evaluating its feasibility in resolving conventional frame-based imaging systems’ limitations in data redundancy, fluorescence sensitivity, and compromised throughput. Herein, we adopted a convolutional spiking neural network (SNN) combined with the YOLOv3 model (SNN-YOLO) to perform cell classification and detection on label-free samples under neuromorphic vision. Spiking techniques are inherently suitable post-processing techniques for neuromorphic vision sensing. The experiment was conducted with polystyrene-based microparticles, THP-1, and LL/2 cell lines. The network’s performance was compared with that of a traditional YOLOv3 model fed with event-generated frame data to serve as a baseline. In this work, our SNN-YOLO outperformed the YOLOv3 baseline by achieving the highest average class accuracy of 0.974, compared to 0.962 for YOLOv3. Both models reported comparable performances across other key metrics and should be further explored for future auto-gating strategies and cytometry applications.

Hierarchical Mapping of Large-Scale Spiking Convolutional Neural Networks Onto Resource-Constrained Neuromorphic Processor

Hierarchical Mapping of Large-Scale Spiking Convolutional Neural Networks Onto Resource-Constrained Neuromorphic Processor

Convolutional spiking neural networks for intent detection based on anticipatory brain potentials using electroencephalogram

Spiking neural networks (SNNs) are receiving increased attention because they mimic synaptic connections in biological systems and produce spike trains, which can be approximated by binary values for computational efficiency. Recently, the addition of convolutional layers to combine the feature extraction power of convolutional networks with the computational efficiency of SNNs has been introduced. This paper studies the feasibility of using a convolutional spiking neural network (CSNN) to detect anticipatory slow cortical potentials (SCPs) related to braking intention in human participants using an electroencephalogram (EEG). Data was collected during an experiment wherein participants operated a remote-controlled vehicle on a testbed designed to simulate an urban environment. Participants were alerted to an incoming braking event via an audio countdown to elicit anticipatory potentials that were measured using an EEG. The CSNN’s performance was compared to a standard CNN, EEGNet and three graph neural networks via 10-fold cross-validation. The CSNN outperformed all the other neural networks, and had a predictive accuracy of 99.06% with a true positive rate of 98.50%, a true negative rate of 99.20% and an F1-score of 0.98. Performance of the CSNN was comparable to the CNN in an ablation study using a subset of EEG channels that localized SCPs. Classification performance of the CSNN degraded only slightly when the floating-point EEG data were converted into spike trains via delta modulation to mimic synaptic connections.

An efficient intrusion detection model based on convolutional spiking neural network

Many intrusion detection techniques have been developed to ensure that the target system can function properly under the established rules. With the booming Internet of Things (IoT) applications, the resource-constrained nature of its devices makes it urgent to explore lightweight and high-performance intrusion detection models. Recent years have seen a particularly active application of deep learning (DL) techniques. The spiking neural network (SNN), a type of artificial intelligence that is associated with sparse computations and inherent temporal dynamics, has been viewed as a potential candidate for the next generation of DL. It should be noted, however, that current research into SNNs has largely focused on scenarios where limited computational resources and insufficient power sources are not considered. Consequently, even state-of-the-art SNN solutions tend to be inefficient. In this paper, a lightweight and effective detection model is proposed. With the help of rational algorithm design, the model integrates the advantages of SNNs as well as convolutional neural networks (CNNs). In addition to reducing resource usage, it maintains a high level of classification accuracy. The proposed model was evaluated against some current state-of-the-art models using a comprehensive set of metrics. Based on the experimental results, the model demonstrated improved adaptability to environments with limited computational resources and energy sources.

Darwin3: a large-scale neuromorphic chip with a novel ISA and on-chip learning.

Spiking neural networks (SNNs) are gaining increasing attention for their biological plausibility and potential for improved computational efficiency. To match the high spatial-temporal dynamics in SNNs, neuromorphic chips are highly desired to execute SNNs in hardware-based neuron and synapse circuits directly. This paper presents a large-scale neuromorphic chip named Darwin3 with a novel instruction set architecture, which comprises 10 primary instructions and a few extended instructions. It supports flexible neuron model programming and local learning rule designs. The Darwin3 chip architecture is designed in a mesh of computing nodes with an innovative routing algorithm. We used a compression mechanism to represent synaptic connections, significantly reducing memory usage. The Darwin3 chip supports up to 2.35 million neurons, making it the largest of its kind on the neuron scale. The experimental results showed that the code density was improved by up to 28.3× in Darwin3, and that the neuron core fan-in and fan-out were improved by up to 4096× and 3072× by connection compression compared to the physical memory depth. Our Darwin3 chip also provided memory saving between 6.8× and 200.8× when mapping convolutional spiking neural networks onto the chip, demonstrating state-of-the-art performance in accuracy and latency compared to other neuromorphic chips.

A novel bionic olfactory network combined with an electronic nose for identification of industrial exhaust

Traditional electronic nose (E-nose) signal processing methods and classification techniques are often cumbersome, heavily dependent on the experience of the implementers and poor in biological interpretability. In view of the above problems, a novel bionic olfactory network is proposed for processing E-nose data for industrial exhaust identification. First, the bionic olfactory network (BON) consists of a bionic olfactory bulb system (BOBS) and a bionic olfactory cortex (BOC). The BOBS highly simulates the odor information conduction function of the mammalian olfactory bulb and is composed of a novel bionic olfactory bulb model and recurrence quantification analysis (RQA), which can transform the original sensor responses into neuron spike sequences and extract features from the sequences, simplifying traditional data processing steps. The BOC is a convolutional spiking neural network that primarily imitates the odor information recognition function of the olfactory cortex in the mammalian olfactory system. The whole BON exhibits excellent biological interpretability due to its unique mode of information transmission. Finally, the experimental data of ten types of industrial exhaust were collected by using a self-built E-nose system. The experimental results indicate that the classification accuracy of the BON is 94.4%, which is much better than that of the combination of traditional signal processing methods and classification techniques. An even better recognition accuracy of 79.2% can be obtained in small sample training, which is still the best performance.

A Deep Convolutional Spiking Neural Network for embedded applications

Deep neural networks (DNNs) have received a great deal of interest in solving everyday tasks in recent years. However, their computational and energy costs limit their use on mobile and edge devices. The neuromorphic computing approach called spiking neural networks (SNNs) represents a potential solution for bridging the gap between performance and computational expense. Despite the potential benefits of energy efficiency, the current SNNs are being used with datasets such as MNIST, Fashion-MNIST, and CIFAR10, limiting their applications compared to DNNs. Therefore, the applicability of SNNs to real-world applications, such as scene classification and forecasting epileptic seizures, must be demonstrated yet. This paper develops a deep convolutional spiking neural network (DCSNN) for embedded applications. We explore a convolutional architecture, Visual Geometry Group (VGG16), to implement deeper SNNs. To train a spiking model, we convert the pre-trained VGG16 into corresponding spiking equivalents with nearly comparable performance to the original one. The trained weights of VGG16 were then transferred to the equivalent SNN architecture while performing a proper weight–threshold balancing. The model is evaluated in two case studies: land use and land cover classification, and epileptic seizure detection. Experimental results show a classification accuracy of 94.88%, and seizure detection specificity of 99.45% and a sensitivity of 95.06%. It is confirmed that conversion-based training SNNs are promising, and the benefits of DNNs, such as solving complex and real-world problems, become available to SNNs.

Health Monitoring in IoT with Context-Aware Deep Convolutional Spiking Neural Network and Woodpecker Mating Algorithm

ABSTRACT In recent years, many context-sensitive approaches have been established to offer physiological information about the wellness of each person. While the patient’s health is being monitored, there are delays in the cloud data transfer. To overcome these delays, a Deep Convolutional Spike Neural Network (DCSNN) with a Woodpecker Mating Algorithm (WMA) is proposed in this manuscript for monitoring the healthcare data in the Internet of Things (IoT)-based context-aware architecture (DCSNN-WMA-HCM-IoT-CAA). Here, the input data are amassed from real-time datasets. Then the data are supplied to the pre-processing. The pre-processing data involve the assortment phase, data storage phase, and data redundancy phase. In the redundancy phase, the Kernel co-relation method is employed to remove the repeated data. The pre-processing output is given to the feature extraction. The feature is extracted under the fast discrete Curvelet transform method. After that, the extracted features are given to DCSNN optimized with WMA to classify regular, irregular, and critical conditions of the patient. The proposed DCSNN-WMA-HCM-IoT-CAA method is activated in OMNeT++. The proposed DCSNN-WMA-HCM-IoT-CAA approach attains 4.08%, 8.17%, 9.32%, and 5.17% high accuracy, 14.3%, 15.4%, 19.51%, and 27.81% lower computation Time and 12.29%, 15.36%, 11.55%, and 13.91% higher AUC compared with existing methods.

DCSNN optimized with hybrid Border Collie optimization and Archimedes optimization algorithms for solid waste prediction in Chennai

The rapid growth of smart cities and industry causes an increase in waste production. The amount of municipal solid waste (MSW) increases by several factors, including population growth, economic status, and consumption trends. The inadequacy of basic trash data is a major issue for managing MSW. Numer-ous existing models based on solid waste prediction have been presented so far, but none of them predict solid waste accurately and it consumes more time. To address these concerns, a deep convolutional spiking neural network for solid waste prediction (DCSNN-SWP) is proposed in this paper. Here, the real-time solid waste prediction data are gathered from the quantity of municipal corporation of Chennai (MCC), landfill, garden garbage, and coconut shell reports in Tamil Nadu (Chennai), such as Zone 9 (Nungambak-kam), Zone 10 (Kodambakkam) and Zone 13 (Adyar). Then the collected solid waste data are pre-processed using the kernel correlation model. Then the pre-processing data is given to DCSNN-hybrid BCMO and Archimedes optimization algorithm which accurately predicts the solid waste as wet waste, dry waste, horti-culture waste, and dumping yard for 2022–2032 years. The proposed DCSNN-SWP method has been implemented in Python.

Recognition and detection of unusual activities in ATM using dual-channel capsule generative adversarial network

Real-time detection of human activity has become essential for monitoring and security of public spaces such as bank ATMs and workplaces, due to daily rise in criminal activities. Currently, monocular CCTV cameras that only record RGB video are used for monitoring such restricted areas. In addition to RGB data, the RGB + D sensor also offers depth information about the scene. To overcome the issue of online recognition of anomalous activities in Bank ATMs, a supervised deep learning method utilizing Dual-Channel Capsule Generative Adversarial Network (DCCGAN) and RGB + D sensor is proposed. The input RGB + D Dataset is given to super pixel motion detection method to find the region of interest (ROI). Motion detection is a significant stage examination of wide scene for background subtraction and foreground detection. After arranging the motion detection, its region is positioned frame by frame. Then, the detected ROI is given to fast discrete curvelet transform with the wrapping (FDCT-WRP) method. FDCT-WRP feature extraction method. These extracting features are supplied to Deep Convolutional Spiking neural network (DCSNN), which realize the object. RGB and depth video segments are used to create motion templates from the RGB + D data online video stream. These templates are trained on DCCGANs to identify distrustful events in current activity and categorized as normal and abnormal. Additionally, a unique RGB + D dataset is employed because there was no existing dataset available for analyzing human activity in ATMs. The proposed DUA-DCCGAN-ATM approach is assessed on qualitative with quantitative statistical evaluation parameters and identify suspect occurrence with 0.932 precision and 98.2 % accuracy. The detailed statistical analysis exemplifies that the proposed technique can identify distrustful events in a real-time online manner prior completing the anomalous activity.

Correction to: Deep convolutional spiking neural network fostered automatic detection and classification of breast cancer from mammography images

Correction to: Deep convolutional spiking neural network fostered automatic detection and classification of breast cancer from mammography images

An AER-based spiking convolution neural network system for image classification with low latency and high energy efficiency

Spiking neural network is used to solve the problems of power consumption and computation requirement of the traditional artificial neural network. This paper proposes an event-driven multilayer spiking convolutional neural network (SCNN) hardware system for the image classification task. The proposed hardware system consists of a microcontroller unit (MCU) and an SCNN processor. Input images are temporally encoded to spike trains in the form of address event representation (AER) by MCU, and sent to the SCNN processor to execute AER-based spike-centric convolution in order to reduce redundant operations and get the classification result efficiently. By rearranging the storage structure of weights of synapses and membrane potential (MP) of spiking neurons, multi-channel parallel MP updating is realized to reduce latency. In addition, FIFOs are inserted between the spiking layers to construct a pipeline structure. The hardware architecture of the proposed system is implemented on the Xilinx ZYNQ-7000 platform. At the operating frequency of 100 MHz, the proposed SCNN processor has a speed of 400classifications/s and 80uJ/inference.

Neuromorphic Speech Recognition With Photonic Convolutional Spiking Neural Networks

Spiking neural network (SNN) have attracted lots of attention due to its event-driven nature and powerful computation capability. However, it is still limited to simple task due to the training difficulty. In this work, we propose a hybrid architecture of photonic convolutional spiking neural network (PCSNN) to realize the speech recognition task. In the PCSNN, the feature extraction is realized by a convolution SNN with unsupervised learning algorithm, the classification is realized by a photonic SNN with modified time-based supervised training algorithm. The TIDIGITS dataset is used to test the speech recognition performance of the proposed PCSNN, and the highest testing accuracy is 93.75%. The proposed PCSNN provides a solution for architecture and algorithm co-design for the speech recognition task, which is helpful for extending the applications of photonic SNN.

Deep convolutional spiking neural network fostered automatic detection and classification of breast cancer from mammography images

Deep convolutional spiking neural network fostered automatic detection and classification of breast cancer from mammography images

Spiking neural networks for frame-based and event-based single object localization

Spiking neural networks (SNNs) have shown much promise as an energy-efficient alternative to artificial neural networks (ANNs). Such methods trained by surrogate gradient (SG) descent are now capable of dealing with frame- or event-based vision tasks beyond classification (e.g. object detection, semantic segmentation). However, important questions about their behavior w.r.t fundamental design choices remain under-explored (e.g. how a specific neural coding scheme impacts the performance/robustness? Does a higher temporal latency necessarily imply better results?). In this paper, we focus on single object localization as a context to analyze deep convolutional SNNs on (1) the importance of temporal latency (i.e. the number of time-steps) on performance ; (2) their robustness against sensor corruptions ; and (3) the impact of neural coding schemes on performance with static images. We design a simple SNN baseline for frame- and event-based single object localization and compare it against a similar ANN architecture. Our experiments show that our approach can achieve competitive or better performance in accuracy and robustness against common sensor corruptions with significantly lower energy consumption. More importantly, our experimental analysis draws conclusions significantly different from well-known studies focused on SNNs trained with bio-plausible learning rules, which helps in the design of SG-trained architectures, and offers insight to design priorities in future neuromorphic technologies.

A Novel Event-Driven Spiking Convolutional Neural Network for Electromyography Pattern Recognition.

Electromyography (EMG) pattern recognition is an important technology for prosthesis control and human-computer interaction etc. However, the practical application of EMG pattern recognition is hampered by poor accuracy and robustness due to electrode shift caused by repeated wearing of the signal acquisition device. Moreover, the user's acceptability is low due to the heavy training burden, which is caused by the need for a large amount of training data by traditional methods. In order to explore the advantage of spiking neural network (SNN) in solving the poor robustness and heavy training burden problems in EMG pattern recognition, a spiking convolutional neural network (SCNN) composed of cyclic convolutional neural network (CNN) and fully connected modules is proposed and implemented in this study. High density surface electromyography (HD-sEMG) signals collected from 6 gestures of 10 subjects at 6 electrode positions are taken as the research object. Compared to CNN with the same structure, CNN-Long Short Term Memory (CNN-LSTM), linear kernel linear discriminant analysis classifier (LDA) and spiking multilayer perceptron (Spiking MLP), the accuracy of SCNN is 50.69%, 33.92%, 32.94% and 9.41% higher in the small sample training experiment, 6.50%, 4.23%, 28.73%, and 2.57% higher in the electrode shifts experiment respectively. In addition, the power consumption of SCNN is about 1/93 of CNN. The advantages of the proposed framework in alleviating user training burden, mitigating the adverse effect of electrode shifts and reducing power consumption make it very meaningful for promoting the development of user-friendly real-time myoelectric control system.

Neural encoding with unsupervised spiking convolutional neural network

Accurately predicting the brain responses to various stimuli poses a significant challenge in neuroscience. Despite recent breakthroughs in neural encoding using convolutional neural networks (CNNs) in fMRI studies, there remain critical gaps between the computational rules of traditional artificial neurons and real biological neurons. To address this issue, a spiking CNN (SCNN)-based framework is presented in this study to achieve neural encoding in a more biologically plausible manner. The framework utilizes unsupervised SCNN to extract visual features of image stimuli and employs a receptive field-based regression algorithm to predict fMRI responses from the SCNN features. Experimental results on handwritten characters, handwritten digits and natural images demonstrate that the proposed approach can achieve remarkably good encoding performance and can be utilized for “brain reading” tasks such as image reconstruction and identification. This work suggests that SNN can serve as a promising tool for neural encoding.

Hybrid photonic deep convolutional residual spiking neural networks for text classification.

Spiking neural networks (SNNs) offer powerful computation capability due to its event-driven nature and temporal processing. However, it is still limited to shallow structure and simple tasks due to the training difficulty. In this work, we propose a deep convolutional residual spiking neural network (DCRSNN) for text classification tasks. In the DCRSNN, the feature extraction is achieved via a convolution SNN with residual connection, using the surrogate gradient direct training technique. Classification is performed by a fully-connected network. We also suggest a hybrid photonic DCRSNN, in which photonic SNNs are used for classification with a converted training method. The accuracy of hard and soft reset methods, as well as three different surrogate functions, were evaluated and compared across four different datasets. Results indicated a maximum accuracy of 76.36% for MR, 91.03% for AG News, 88.06% for IMDB and 93.99% for Yelp review polarity. Soft reset methods used in the deep convolutional SNN yielded slightly better accuracy than their hard reset counterparts. We also considered the effects of different pooling methods and observation time windows and found that the convergence accuracy achieved by convolutional SNNs was comparable to that of convolutional neural networks under the same conditions. Moreover, the hybrid photonic DCRSNN also shows comparable testing accuracy. This work provides new insights into extending the SNN applications in the field of text classification and natural language processing, which is interesting for the resources-restrained scenarios.

SDenseNet-An Improved DenseNet Model for Spiking Neural Networks

In order to port DenseNet to a spiking neural network, its activation function must be modified to spiking neurons. In addition, the direct conversion model suffers from the inability to transmit non-peak sequences across layers and a sluggish training pace. A deep convolutional spiking neural network (DCSNN) architecture called SDenseNet has been designed in order to address these problems. The first step is to adjust the connection sequence of each module layer in the network to ensure that the output of each layer is the spiking sequence; Next, convolutional kernels of different sizes are used in parallel in the Transition layer to extract different features and spliced along the length of the channel in an attempt to build up the performance of the feature extraction. The framework suggested in this paper has demonstrated improvement in benchmark indicators such as training speed, accuracy rate, and mean loss when compared to the framework directly converted to the spiking neural network by training the two publicly image data sets named CIFAR-10 and CIFAR-100.

Spiking CMOS-NVM mixed-signal neuromorphic ConvNet with circuit- and training-optimized temporal subsampling.

We increasingly rely on deep learning algorithms to process colossal amount of unstructured visual data. Commonly, these deep learning algorithms are deployed as software models on digital hardware, predominantly in data centers. Intrinsic high energy consumption of Cloud-based deployment of deep neural networks (DNNs) inspired researchers to look for alternatives, resulting in a high interest in Spiking Neural Networks (SNNs) and dedicated mixed-signal neuromorphic hardware. As a result, there is an emerging challenge to transfer DNN architecture functionality to energy-efficient spiking non-volatile memory (NVM)-based hardware with minimal loss in the accuracy of visual data processing. Convolutional Neural Network (CNN) is the staple choice of DNN for visual data processing. However, the lack of analog-friendly spiking implementations and alternatives for some core CNN functions, such as MaxPool, hinders the conversion of CNNs into the spike domain, thus hampering neuromorphic hardware development. To address this gap, in this work, we propose MaxPool with temporal multiplexing for Spiking CNNs (SCNNs), which is amenable for implementation in mixed-signal circuits. In this work, we leverage the temporal dynamics of internal membrane potential of Integrate & Fire neurons to enable MaxPool decision-making in the spiking domain. The proposed MaxPool models are implemented and tested within the SCNN architecture using a modified version of the aihwkit framework, a PyTorch-based toolkit for modeling and simulating hardware-based neural networks. The proposed spiking MaxPool scheme can decide even before the complete spatiotemporal input is applied, thus selectively trading off latency with accuracy. It is observed that by allocating just 10% of the spatiotemporal input window for a pooling decision, the proposed spiking MaxPool achieves up to 61.74% accuracy with a 2-bit weight resolution in the CIFAR10 dataset classification task after training with back propagation, with only about 1% performance drop compared to 62.78% accuracy of the 100% spatiotemporal window case with the 2-bit weight resolution to reflect foundry-integrated ReRAM limitations. In addition, we propose the realization of one of the proposed spiking MaxPool techniques in an NVM crossbar array along with periphery circuits designed in a 130nm CMOS technology. The energy-efficiency estimation results show competitive performance compared to recent neuromorphic chip designs.