Hierarchical Spiking Neural Network Research Articles

Speech intelligibility prediction based on a physiological model of the human ear and a hierarchical spiking neural network.

A speech intelligibility (SI) prediction model is proposed that includes an auditory preprocessing component based on the physiological anatomy and activity of the human ear, a hierarchical spiking neural network, and a decision back-end processing based on correlation analysis. The auditory preprocessing component effectively captures advanced physiological details of the auditory system, such as retrograde traveling waves, longitudinal coupling, and cochlear nonlinearity. The ability of the model to predict data from normal-hearing listeners under various additive noise conditions was considered. The predictions closely matched the experimental test data under all conditions. Furthermore, we developed a lumped mass model of a McGee stainless-steel piston with the middle-ear to study the recovery of individuals with otosclerosis. We show that the proposed SI model accurately simulates the effect of middle-ear intervention on SI. Consequently, the model establishes a model-based relationship between objective measures of human ear damage, like distortion product otoacoustic emissions, and speech perception. Moreover, the SI model can serve as a robust tool for optimizing parameters and for preoperative assessment of artificial stimuli, providing a valuable reference for clinical treatments of conductive hearing loss.

Hierarchical spiking neural network auditory feature based dry-type transformer fault diagnosis using convolutional neural network

Dry-type transformer fault diagnosis (DTTFD) presents a significant challenge because of its complex internal structure and sensitivity to noise. To address this challenge, we propose a DTTFD method that combines hierarchical spike neural network auditory features (HSNNAF) with convolutional neural networks (CNN). By leveraging the hierarchical structure of the central auditory system and sequential nonlinear feature extraction to compute the HSNNAF, we enhanced the relevant clues of transformer faults while removing non-fault source noise. Subsequently, the obtained HSNNAF were fed into a CNN for fault classification. The proposed method demonstrated high accuracy in DTTFD, with a diagnostic accuracy of 99.52%. Even at a signal-to-noise ratio of 0 dB, the diagnostic accuracy remains as high as 95.88%. These results indicate that the method can accurately diagnose faults in dry-type transformers while exhibiting excellent noise resistance capabilities.

A bio-inspired hierarchical spiking neural network with biological synaptic plasticity for event camera object recognition

With inherent sparse spike-based coding and asynchronous event-driven computation, spiking neural network (SNN) is naturally suitable for processing event stream data of event cameras. In order to improve the feature extraction and classification performance of bio-inspired hierarchical SNNs, in this paper an event camera object recognition system based on biological synaptic plasticity is proposed. In our system input event streams were firstly segmented adaptively using spiking neuron potential to improve computational efficiency of the system. Multi-layer feature learning and classification are implemented by our bio-inspired hierarchical SNN with synaptic plasticity. After Gabor filter-based event-driven convolution layer which extracted primary visual features of event streams, we used a feature learning layer with unsupervised spiking timing dependent plasticity (STDP) rule to help the network extract frequent salient features, and a feature learning layer with reward-modulated STDP rule to help the network learn diagnostic features. The classification accuracies of the network proposed in this paper on the four benchmark event stream datasets were better than the existing bio-inspired hierarchical SNNs. Moreover, our method showed good classification ability for short event stream input data, and was robust to input event stream noise. The results show that our method can improve the feature extraction and classification performance of this kind of SNNs for event camera object recognition.

Deep Learning with Optimal Hierarchical Spiking Neural Network for Medical Image Classification

Medical image classification becomes a vital part of the design of computer aided diagnosis (CAD) models. The conventional CAD models are majorly dependent upon the shapes, colors, and/or textures that are problem oriented and exhibited complementary in medical images. The recently developed deep learning (DL) approaches pave an efficient method of constructing dedicated models for classification problems. But the maximum resolution of medical images and small datasets, DL models are facing the issues of increased computation cost. In this aspect, this paper presents a deep convolutional neural network with hierarchical spiking neural network (DCNN-HSNN) for medical image classification. The proposed DCNN-HSNN technique aims to detect and classify the existence of diseases using medical images. In addition, region growing segmentation technique is involved to determine the infected regions in the medical image. Moreover, NADAM optimizer with DCNN based Capsule Network (CapsNet) approach is used for feature extraction and derived a collection of feature vectors. Furthermore, the shark smell optimization algorithm (SSA) based HSNN approach is utilized for classification process. In order to validate the better performance of the DCNN-HSNN technique, a wide range of simulations take place against HIS2828 and ISIC2017 datasets. The experimental results highlighted the effectiveness of the DCNN-HSNN technique over the recent techniques interms of different measures. Please type your abstract here.

A Bio-Inspired Hierarchical Spiking Neural Network With Reward-Modulated STDP Learning Rule for AER Object Recognition

The inherent spike-based and event-driven computation makes spiking neural networks (SNNs) naturally suitable to provide efficient and low-latency solution in neuromorphic vision processing. In this work we propose an address-event representation (AER) object recognition system which consists of a noise filter, an event stream segmentation module and a hierarchical convolutional SNN. In order to improve the feature learning capacity of bio-inspired shallow hierarchical SNNs for AER object recognition, we apply a convolutional SNN with R-STDP learning rule and first-spike coding to further extract features after primary feature learning using Gabor filter-based event-driven convolution. The R-STDP learning rule enables the network to adjust weights by combining local STDP learning rule with a global reward signal and to make classification without using an external classifier. Experimental results show that our method significantly outperforms the existing hierarchical SNN-based recognition methods on four popular AER datasets. Moreover, our method has much better recognition ability when using very short input event stream, and can effectively learn from small size training sets. In addition, the robustness of the network to input noise is improved. Our method is very beneficial for developing neuromorphic-vision object recognition algorithm in resource-constrained applications when event streams are short and training samples are limited.

A temporal hierarchical feedforward model explains both the time and the accuracy of object recognition

Brain can recognize different objects as ones it has previously experienced. The recognition accuracy and its processing time depend on different stimulus properties such as the viewing conditions, the noise levels, etc. Recognition accuracy can be explained well by different models. However, most models paid no attention to the processing time, and the ones which do, are not biologically plausible. By modifying a hierarchical spiking neural network (spiking HMAX), the input stimulus is represented temporally within the spike trains. Then, by coupling the modified spiking HMAX model, with an accumulation-to-bound decision-making model, the generated spikes are accumulated over time. The input category is determined as soon as the firing rates of accumulators reaches a threshold (decision bound). The proposed object recognition model accounts for both recognition time and accuracy. Results show that not only does the model follow human accuracy in a psychophysical task better than the well-known non-temporal models, but also it predicts human response time in each choice. Results provide enough evidence that the temporal representation of features is informative, since it can improve the accuracy of a biologically plausible decision maker over time. In addition, the decision bound is able to adjust the speed-accuracy trade-off in different object recognition tasks.

Pattern Recognition of Spiking Neural Networks Based on Visual Mechanism and Supervised Synaptic Learning.

Electrophysiological studies have shown that mammalian primary visual cortex are selective for the orientations of visual stimuli. Inspired by this mechanism, we propose a hierarchical spiking neural network (SNN) for image classification. Grayscale input images are fed through a feed-forward network consisting of orientation-selective neurons, which then projected to a layer of downstream classifier neurons through the spiking-based supervised tempotron learning rule. Based on the orientation-selective mechanism of the visual cortex and tempotron learning rule, the network can effectively classify images of the extensively studied MNIST database of handwritten digits, which achieves 96% classification accuracy based on only 2000 training samples (traditional training set is 60000). Compared with other classification methods, our model not only guarantees the biological plausibility and the accuracy of image classification but also significantly reduces the needed training samples. Considering the fact that the most commonly used deep learning neural networks need big data samples and high power consumption in image recognition, this brain-inspired computational neural network model based on the layer-by-layer hierarchical image processing mechanism of the visual cortex may provide a basis for the wide application of spiking neural networks in the field of intelligent computing.

An ensemble unsupervised spiking neural network for objective recognition

It is now known that the spiking neuron is a basic unit of spiking neural networks (SNNs). Spiking neurons modulate the nervous cells via receiving external incentives, generation of action potential and firing spikes. The SNNs usually used for pattern recognition tasks or complex computation depending on the brain-like characteristic. Although the SNNs have no advantages comparing with the deep neural networks in terms of classification accuracy, the SNNs have more characteristics of biological neurons. In this paper, a hierarchical SNN, comprising convolutional and pooling layers, is designed. The proposed SNN consists of excitatory and inhibitory neurons based on the mechanism of the primate brain. A temporal coding (rank order) manner is used to encode the input patterns. It depends on the rank of the spike arrival on post synapses to establish the priority of input spikes for a particular pattern. The spike-timing-dependent plasticity (STDP) learning rule is used in convolutional layers to extract visual features in an unsupervised learning manner. During the classification stage, a lateral inhibition mechanism is used to prevent the non-firing neurons and produce distinguishable results. In order to improve the performance of our SNN, an ensemble SNN architecture using the voting method is proposed, and transfer learning is used to avoid re-training the SNN when solving the different tasks. The hand-written digits classification task on MNIST, CIFAR-10, and BreaKHis databases are used to verify the performance of the proposed SNN. Experimental results show that by using the ensemble architecture and transfer learning, the classification accuracy of 99.27% for the MNIST database, overall accuracy is 93% for the CIFAR-10 database, and overall accuracy is 96.97% for BreaKHis database. In the meantime, this work achieves a better performance than the benchmarking approaches. Taken together, the results of our work suggest that the ensemble SNN architecture with transfer learning is key to improving the performance of the SNN.

Spiking network optimized for word recognition in noise predicts auditory system hierarchy.

The auditory neural code is resilient to acoustic variability and capable of recognizing sounds amongst competing sound sources, yet, the transformations enabling noise robust abilities are largely unknown. We report that a hierarchical spiking neural network (HSNN) optimized to maximize word recognition accuracy in noise and multiple talkers predicts organizational hierarchy of the ascending auditory pathway. Comparisons with data from auditory nerve, midbrain, thalamus and cortex reveals that the optimal HSNN predicts several transformations of the ascending auditory pathway including a sequential loss of temporal resolution and synchronization ability, increasing sparseness, and selectivity. The optimal organizational scheme enhances performance by selectively filtering out noise and fast temporal cues such as voicing periodicity, that are not directly relevant to the word recognition task. An identical network arranged to enable high information transfer fails to predict auditory pathway organization and has substantially poorer performance. Furthermore, conventional single-layer linear and nonlinear receptive field networks that capture the overall feature extraction of the HSNN fail to achieve similar performance. The findings suggest that the auditory pathway hierarchy and its sequential nonlinear feature extraction computations enhance relevant cues while removing non-informative sources of noise, thus enhancing the representation of sounds in noise impoverished conditions.

Unsupervised Learning of a Hierarchical Spiking Neural Network for Optical Flow Estimation: From Events to Global Motion Perception.

The combination of spiking neural networks and event-based vision sensors holds the potential of highly efficient and high-bandwidth optical flow estimation. This paper presents the first hierarchical spiking architecture in which motion (direction and speed) selectivity emerges in an unsupervised fashion from the raw stimuli generated with an event-based camera. A novel adaptive neuron model and stable spike-timing-dependent plasticity formulation are at the core of this neural network governing its spike-based processing and learning, respectively. After convergence, the neural architecture exhibits the main properties of biological visual motion systems, namely feature extraction and local and global motion perception. Convolutional layers with input synapses characterized by single and multiple transmission delays are employed for feature and local motion perception, respectively; while global motion selectivity emerges in a final fully-connected layer. The proposed solution is validated using synthetic and real event sequences. Along with this paper, we provide the cuSNN library, a framework that enables GPU-accelerated simulations of large-scale spiking neural networks. Source code and samples are available at https://github.com/tudelft/cuSNN.

A spiking neural network for extraction of features in colour opponent visual pathways and FPGA implementation

The human vision system is one of the most complex and the most superior systems that have been known to date. Inspired by the visual ventral pathways, a hierarchical spiking neural network is proposed to extract features from colour opponent visual pathways. The network is composed of multiple visual processing channels to simulate processing mechanism of the visual system. The firing rate map of each channel is recorded and further processed. Useful features are extracted from firing rate map with the characteristics of colour contrast sensitivity, orientation selective and illuminant constancy. These characteristics greatly enrich the expression of colour features and enhance the ability of object recognition. Simulation algorithm of the spiking neural network is based on integrated-and-fire neuron model and a set of receptive fields. Simulation results show that the network has the merit of capability of dealing with complex pattern recognition tasks. Furthermore, a SNN blockset is developed to implement the spiking neural network on FPGAs. It greatly speeds up the hardware modeling of the spiking neural network so that SNNs can be efficiently used in the hardware systems such as robots.

Categorization and decision-making in a neurobiologically plausible spiking network using a STDP-like learning rule

Understanding how the human brain is able to efficiently perceive and understand a visual scene is still a field of ongoing research. Although many studies have focused on the design and optimization of neural networks to solve visual recognition tasks, most of them either lack neurobiologically plausible learning rules or decision-making processes. Here we present a large-scale model of a hierarchical spiking neural network (SNN) that integrates a low-level memory encoding mechanism with a higher-level decision process to perform a visual classification task in real-time. The model consists of Izhikevich neurons and conductance-based synapses for realistic approximation of neuronal dynamics, a spike-timing-dependent plasticity (STDP) synaptic learning rule with additional synaptic dynamics for memory encoding, and an accumulator model for memory retrieval and categorization. The full network, which comprised 71,026 neurons and approximately 133 million synapses, ran in real-time on a single off-the-shelf graphics processing unit (GPU). The network was constructed on a publicly available SNN simulator that supports general-purpose neuromorphic computer chips. The network achieved 92% correct classifications on MNIST in 100 rounds of random sub-sampling, which is comparable to other SNN approaches and provides a conservative and reliable performance metric. Additionally, the model correctly predicted reaction times from psychophysical experiments. Because of the scalability of the approach and its neurobiological fidelity, the current model can be extended to an efficient neuromorphic implementation that supports more generalized object recognition and decision-making architectures found in the brain.

A visual attention model based on hierarchical spiking neural networks

Based on the information processing functionalities of spiking neurons, hierarchical spiking neural networks are proposed to simulate visual attention. Using spiking neural networks inspired by the visual system, an image can be decomposed into multiple visual image components. Based on specific visual image components and image features, a visual attention system is proposed to extract attention areas according to top–down volition-controlled signals. The hierarchical spiking neural networks are constructed with a conductance-based integrate-and-fire neuron model and a set of specific receptive fields in different levels. The simulation algorithm and properties of the networks are detailed in this paper. Simulation results show that the attention system is able to perform visual attention of objects based on specific image components or features, and a demonstration shows how the attention system can detect a house in a visual image. Using the proposed saliency index, attention areas of interest can be extracted from spike rate maps of multiple visual pathways, such as ON/OFF colour pathways. According to this visual attention principle, the visual image processing system can quickly focus on specific areas while ignoring other areas.

Processing visual stimuli using hierarchical spiking neural networks

Based on spiking neuron models and different receptive field models, hierarchical networks are proposed to process visual stimuli, in which multiple overlapped objects are represented by different orientation bars. The main purpose of this paper is to show that hierarchical spiking neural networks are able to segment the objects and bind their pixels to form shapes of objects using local excitatory lateral connections. The presented architecture is based on biologically inspired hierarchical structures. Segmentation is achieved through temporal correlation of neuron activities. The properties of these networks are demonstrated using a series of visual scenes representing different stimuli settings.