Evolving Spiking Neural Networks Research Articles

Design of spiking neural networks for blood pressure prediction during general anesthesia: considerations for optimizing results

The ability to predict blood pressure changes during general anesthesia would assist anesthetists minimize the risk of complications due to hypotensive events. However, such prediction is not trivial. Evolving spiking neural networks are a relatively new computational method that may have application to this problem. NeuCubeST consists of a 3-dimensional network of locally connected neurons called a Spiking Neural Network reservoir (SNNr) and can be used to classify time series data for prediction. There are a number of design considerations when using NeuCubeST as a classifier of time-series data: what pre-processing of the raw data is required (pre-processing), how to convert the time-series data into a spike train (input-encoding), which neurons the data are connected to (input-mapping), and how many nearest neighbours to use in classification (classification). However, it is still unclear how sensitive NeuCubeST-based systems are to perturbations of any of the above. In this paper we evaluate the contribution of these design factors to blood pressure prediction using NeuCubeST. 6000 possible combinations of those NeuCubeST options were tested for each of 100 patients and for each a Signal to Noise Ratio was obtained. All four investigated design factors showed significant contribution to SNR. Intra-operative MAP prediction using NeuCubeST can be effective but performance is sensitive to the design choices.

A spiking neural network model for obstacle avoidance in simulated prosthetic vision

Limited by visual percepts elicited by existing visual prosthesis, it’s necessary to enhance its functionality to fulfill some challenging tasks for the blind such as obstacle avoidance. This paper argues that spiking neural networks (SNN) are effective techniques for object recognition and introduces for the first time a SNN model for obstacle recognition to assist blind people wearing prosthetic vision devices by modelling and classifying spatio-temporal (ST) video data. The proposed methodology is based on a novel spiking neural network architecture, called NeuCube as a general framework for video data modelling in simulated prosthetic vision. As an integrated environment including spiking trains encoding, input variable mapping, unsupervised reservoir training and supervised classifier training, the NeuCube consists of a spiking neural network reservoir (SNNr) and a dynamic evolving spiking neural network classifier (deSNN). First, input data is captured by visual prosthesis, then ST feature extraction is utilized in the low-resolution prosthetic vision generated by prostheses. Finally such ST features are fed to the NeuCube to output classification result of obstacle analysis for an early warning system to be activated. Experiments on collected video data and comparison with other computational intelligence methods indicate promising results. This makes it possible to directly utilize available neuromorphic hardware chips, embedded in visual prostheses, to enhance significantly their functionality. The proposed NeuCube-based obstacle avoidance methodology provides useful guidance to the blind, thus offering a significant improvement of current prostheses and potentially benefiting future prosthesis wearers.

Initial Experiments Evolving Spiking Neural Networks with Supervised Learning Capability

There is currently much research activity aimed at synaptic plasticity methods for spiking neural networks. While many methods have been proposed, there are few that provide for supervised learning. A fundamental premise of the work reported here is that the network topology is key to defining the network’s capabilities: the topology IS the algorithm. Hence, learning at the level of the whole network is an emergent phenomenon of the learning mechanism operating on individual synapses and the topology. Therefore, the topology and the learning mechanism(s) must be designed together, and evolutionary computation (EC) is a suitable technology for this.We report on initial experiments on a relatively simple test problem, the tonic burster, using several types of learning including supervised. We see that EC can locate seemingly good solutions that actually do not solve the desired task; they “cheat” by simply exploiting the supervisory signals. A simple modification of the train-test protocol can solve this. We introduce an approach we call “artificial neurology” for systematically examining the behaviour of a SNN in order to understand how it achieves its performance. Experiments indicate that a combination of Hebbian and supervised learning works best for this task.

A hybrid differential evolution algorithm for parameter tuning of evolving spiking neural network

In this paper, differential evolution DE has been utilised to solve the problem of tuning the parameters of evolving spiking neural network ESNN manually. As ESNN is sensitive to its parameters as other models, optimal integration of parameters leads to better classification accuracy. A hybrid differential evolution for parameter tuning of evolving spiking neural network DEPT-ESNN is presented for parameter optimisation for determining the optimal number of evolving spiking neural network ESNN parameters: modulation factor Mod, similarity factor Sim and threshold factor C. The best values of parameters are adaptively selected by differential evolution DE to avoid selecting suitable values for a particular problem by trial-and-error approach. Several standard datasets from UCI machine learning are used for evaluating the performance of this hybrid model. It has been found that the classification accuracy and other performance measures can be increased by using hybrid method with differential evolution DEPT_ESNN.

Mapping, Learning, Visualization, Classification, and Understanding of fMRI Data in the NeuCube Evolving Spatiotemporal Data Machine of Spiking Neural Networks.

This paper introduces a new methodology for dynamic learning, visualization, and classification of functional magnetic resonance imaging (fMRI) as spatiotemporal brain data. The method is based on an evolving spatiotemporal data machine of evolving spiking neural networks (SNNs) exemplified by the NeuCube architecture [1]. The method consists of several steps: mapping spatial coordinates of fMRI data into a 3-D SNN cube (SNNc) that represents a brain template; input data transformation into trains of spikes; deep, unsupervised learning in the 3-D SNNc of spatiotemporal patterns from data; supervised learning in an evolving SNN classifier; parameter optimization; and 3-D visualization and model interpretation. Two benchmark case study problems and data are used to illustrate the proposed methodology-fMRI data collected from subjects when reading affirmative or negative sentences and another one-on reading a sentence or seeing a picture. The learned connections in the SNNc represent dynamic spatiotemporal relationships derived from the fMRI data. They can reveal new information about the brain functions under different conditions. The proposed methodology allows for the first time to analyze dynamic functional and structural connectivity of a learned SNN model from fMRI data. This can be used for a better understanding of brain activities and also for online generation of appropriate neurofeedback to subjects for improved brain functions. For example, in this paper, tracing the 3-D SNN model connectivity enabled us for the first time to capture prominent brain functional pathways evoked in language comprehension. We found stronger spatiotemporal interaction between left dorsolateral prefrontal cortex and left temporal while reading a negated sentence. This observation is obviously distinguishable from the patterns generated by either reading affirmative sentences or seeing pictures. The proposed NeuCube-based methodology offers also a superior classification accuracy when compared with traditional AI and statistical methods. The created NeuCube-based models of fMRI data are directly and efficiently implementable on high performance and low energy consumption neuromorphic platforms for real-time applications.

Evolving Spiking Neural Networks for Recognition of Aged Voices

The aging of the voice, known as presbyphonia, is a natural process that can cause great change in vocal quality of the individual. This is a relevant problem to those people who use their voices professionally, and its early identification can help determine a suitable treatment to avoid its progress or even to eliminate the problem. This work focuses on the development of a new model for the identification of aging voices (independently of their chronological age), using as input attributes parameters extracted from the voice and glottal signals. The proposed model, named Quantum binary-real evolving Spiking Neural Network (QbrSNN), is based on spiking neural networks (SNNs), with an unsupervised training algorithm, and a Quantum-Inspired Evolutionary Algorithm that automatically determines the most relevant attributes and the optimal parameters that configure the SNN. The QbrSNN model was evaluated in a database composed of 120 records, containing samples from three groups of speakers. The results obtained indicate that the proposed model provides better accuracy than other approaches, with fewer input attributes.

Adaptive cow movement detection using evolving spiking neural network models

For the need of automatic and intelligent dairy cow farming, it is important to combine more information technologies with the surveillance system. Different gestures may provide different healthy status information of the cow, in order to timely detect the abnormal activity and reduce the workload of breeder, a system which imitates the cognition of human brain is proposed in this paper. First, AdaBoost method is used to detect the position of cow in the surveillance video sequence, and then the 3D Evolving Spiking Neural Network Model is training to recognize different gestures and classify the activity in the real-time video surveillance. Experiment shows the average accuracy of detection and classification of the proposed system is about 80 %, and the performance is robust in complex natural environment. The proposed method can be used as the base of whole alert system which can help breeders to discover abnormal activity and to prevent the diseases in advance, so as to improve the welfare of dairy cow and the quality of milk.

An integrated harmony search algorithm-based multi-objective differential evolution of evolving spiking neural network

In this paper, an integrated harmony search algorithm based on multi-objective differential evolution of evolving spiking neural network HSMODE-ESNN is presented to determine the optimal pre-synaptic neurons network structure and accuracy performance for classification problems simultaneously. This proposed method uses the harmony search HS algorithm in selecting the offspring by using all individuals rather than two in differential evolution DE. This feature enhances the flexibility of the HS algorithm in producing better solutions which is utilised to overcome the disadvantage of DE. Several standard datasets from UCI machine learning are used for evaluating the performance of this hybrid model. The experimental results have proven that the hybrid HSMODE-ESNN gives better results in terms of accuracy and complexity.

Development of a Self-Regulating Evolving Spiking Neural Network for classification problem

This paper presents a new spiking neural network for pattern classification problems, referred to as the Self-Regulating Evolving Spiking Neural (SRESN) classifier, that regulates the learning process of the network. It uses a two layered spiking neural network and the input layer consists of receptive field neurons, which convert a real valued input to spikes using the population coding scheme without any delays. The output layer consists of leaky integrate-and-fire neurons. Since SRESN does not use any delays, the number of network parameters for SRESN is significantly lower than that used by other spiking neural networks, used in this study. During training, the learning algorithm for SRESN, automatically evolves neurons in the output layer based on the training data stream and the current knowledge stored in the network. Depending on the knowledge in the sample and the class specific knowledge stored in the network, it can choose to either add a neuron or update the network parameters or skip learning the sample resulting in self-regulation of the learning process. In case of neuron addition, the weights for the newly added neuron are initialized using a modified rank order scheme which facilitates SRESN for use in online/sequential as well as batch learning modes. The parameter update strategy in SRESN ensures that connections with non-zero postsynaptic potential at the time of the spike are alone updated which helps prevent over training. While evaluating the performance of SRESN, first a study is conducted to assess the impact of various parameters on its performance and establish guidelines to choose suitable values for these parameters. Next, the performance of SRESN, operating in batch mode, is compared with other spiking neural classifiers, including SpikeProp and MuSpiNN, for the UCI benchmark problems of Iris flower classification and Wisconsin breast cancer. Subsequently, the performance of SRESN in online and batch learning mode is compared with an evolving spiking neural classifier for five benchmark data sets from the UCI machine learning repository. Finally, SRESN is applied to solve the practical problem of Epilepsy detection. The performance comparison clearly indicates that SRESN provides a higher generalization accuracy using fewer network parameters.

Memetic Harmony Search Algorithm Based on Multi-objective Differ-ential Evolution of Evolving Spiking Neural Networks

Spiking neural network (SNN) plays an essential role in classification problems. Although there are many models of SNN, Evolving Spiking Neural Network (ESNN) is widely used in many recent research works. Evolutionary algorithms, mainly differential evolution (DE) have been used for enhancing ESNN algorithm. However, many realworld optimization problems include several contradictory objectives. Rather than single optimization, Multi-Objective Optimization (MOO) can be utilized as a set of optimal solutions to solve these problems. In this paper, Harmony Search (HS) and memetic approach was used to improve the performance of MOO with ESNN. Consequently, Memetic Harmony Search Multi-Objective Differential Evolution with Evolving Spiking Neural Network (MEHSMODEESNN) was applied to improve ESNN structure and accuracy rates. Standard data sets from the UCI machine learning are used for evaluating the performance of this enhanced multi objective hybrid model. The experimental results have proved that the Memetic Harmony Search Multi-Objective Differential Evolution with Evolving Spiking Neural Network (MEHSMODE-ESNN) gives better results in terms of accuracy and network structure.

Quantum-Inspired Features and Parameter Optimization of Spiking Neural Networks for a Case Study from Atmospheric

Abstract Identified cluster of atmospheric discharges, sufficiently near from transmissions line, could be an important alarm to support real time decisions. Lightning are important events that affect the electrical power system operation, which are often responsible for transmission lines outages, and can trigger a sequence of events that lead to system collapse. The Brazilian lightning network detection monitors nearly 18 million events monthly and all this data must be processed and analyzed. This paper uses a hybrid model named the Quantum binary-real evolving Spiking Neural Network (QbrSNN) for clustering problem, where the features and parameters of a spiking neural network (SNN) are optimized using the Quantum-Inspired Evolutionary Algorithm with representation Binary-Real (QIEA-BR). The proposed model is applied to atmospheric discharges data, with a significantly higher clustering accuracy than traditional techniques.

Evolving spiking neural networks for personalised modelling, classification and prediction of spatio-temporal patterns with a case study on stroke

The paper presents a novel method and system for personalised (individualised) modelling of spatio/spectro-temporal data (SSTD) and prediction of events. A novel evolving spiking neural network reservoir system (eSNNr) is proposed for the purpose. The system consists of spike-time encoding module of continuous value input information into spike trains; a recurrent 3D SNNr; eSNN as an evolving output classifier. Such system is generated for every new individual, using existing data of similar individuals. Subject to proper training and parameter optimisation, the system is capable of accurate spatio-temporal pattern recognition (STPR) and of early prediction of individual events. The method and the system are generic, applicable to various SSTD and classification and prediction problems. As a case study, the method is applied for early prediction of occurrence of stroke on an individual basis. Preliminary experiments demonstrated a significant improvement in accuracy and time of event prediction when using the proposed method when compared with standard machine learning methods, such as MLR, SVM, and MLP. Future development and applications are discussed.

From evolving artificial gene regulatory networks to evolving spiking neural networks for signal processing

From evolving artificial gene regulatory networks to evolving spiking neural networks for signal processing

From evolving artificial gene regulatory networks to evolving spiking neural networks for pattern recognition

The GReaNs platform (which stands for Gene Regulatory evolutionary artificial Networks) is an artificial life software platform that has been previously shown to allow the evolution of networks for signal processing and animat control [1]. The structure of the resulting network in this model is encoded in a linear genome, without imposing any restrictions on the size of the genome or the size of the network. We extended this model to encode spiking neural networks [4]. In previous work using GReaNs [1], each node in the artificial network has been considered to be an analog of a biological transcriptional unit. However, the nodes can also be seen as artificial neurons. Our extension has introduced to GReaNs two models of spiking neurons (LIF: leaky integrate and fire neurons with a fixed threshold [2] and AdEx: adaptive-exponential integrate and fire [3]). Here we use GReaNs for the evolution of spiking networks for pattern recognition. The genetic algorithm used in GReaNS is based on evolving the number of the neurons and the weights of the connections, but the synaptic delays are constant. We evolved a network that generates spikes at a maximum firing rate when the network is presented with one spike from five input neurons in a specific order, and that stays silent when the order is reversed. We also used three inputs, each with one spike, requiring the network to fire at am aximum rate for a specific temporal order of spikes and to remain silent for all the other permutations. In further work, we plan to investigate the generalization abilities of the evolved networks and the robustness of their behaviour to noise. GReaNs can export a PyNN script, which allows the simulation of an evolved network on different backends, including neuromorphic real-time hardware for real time simulation.

Evolving spiking neural network—a survey

This paper provides a comprehensive literature survey on the evolving Spiking Neural Network (eSNN) architecture since its introduction in 2006 as a further extension of the ECoS paradigm introduced by Kasabov in 1998. We summarize the functioning of the method, discuss several of its extensions and present a number of applications in which the eSNN method was employed. We focus especially on some proposed extensions that allow the processing of spatio-temporal data and for feature and parameter optimisation of eSNN models to achieve better accuracy on classification/prediction problems and to facilitate new knowledge discovery. Finally, some open problems are discussed and future directions highlighted.

Dynamic evolving spiking neural networks for on-line spatio- and spectro-temporal pattern recognition

On-line learning and recognition of spatio- and spectro-temporal data (SSTD) is a very challenging task and an important one for the future development of autonomous machine learning systems with broad applications. Models based on spiking neural networks (SNN) have already proved their potential in capturing spatial and temporal data. One class of them, the evolving SNN (eSNN), uses a one-pass rank-order learning mechanism and a strategy to evolve a new spiking neuron and new connections to learn new patterns from incoming data. So far these networks have been mainly used for fast image and speech frame-based recognition. Alternative spike-time learning methods, such as Spike-Timing Dependent Plasticity (STDP) and its variant Spike Driven Synaptic Plasticity (SDSP), can also be used to learn spatio-temporal representations, but they usually require many iterations in an unsupervised or semi-supervised mode of learning. This paper introduces a new class of eSNN, dynamic eSNN, that utilise both rank-order learning and dynamic synapses to learn SSTD in a fast, on-line mode. The paper also introduces a new model called deSNN, that utilises rank-order learning and SDSP spike-time learning in unsupervised, supervised, or semi-supervised modes. The SDSP learning is used to evolve dynamically the network changing connection weights that capture spatio-temporal spike data clusters both during training and during recall. The new deSNN model is first illustrated on simple examples and then applied on two case study applications: (1) moving object recognition using address-event representation (AER) with data collected using a silicon retina device; (2) EEG SSTD recognition for brain–computer interfaces. The deSNN models resulted in a superior performance in terms of accuracy and speed when compared with other SNN models that use either rank-order or STDP learning. The reason is that the deSNN makes use of both the information contained in the order of the first input spikes (which information is explicitly present in input data streams and would be crucial to consider in some tasks) and of the information contained in the timing of the following spikes that is learned by the dynamic synapses as a whole spatio-temporal pattern.

Dynamic Quantum-Inspired Particle Swarm Optimizationas Feature and Parameter Optimizer for Evolving Spiking Neural Networks

This paper proposes a new structure for Quantum-inspired Particle Swarm Optimization (QiPSO) to enhance feature and parameter optimization of Evolving Spiking Neural Networks (ESNN). The new Dynamic Quantum-inspired Particle Swarm Optimization (DQiPSO) will be integrated within ESNN where features and parameters are simultaneously and more efficiently optimized. The features are modeled as a quantum bit vector, where probability computation is added to perform the feature selection task. For the parameters, values are presented as real numbers. A hybrid particle structure is required for these two different data types. In addition, an improved search strategy has been introduced to find the most relevant features and eliminate irrelevant features on a synthetic dataset. The results show that the proposed optimizer structure yields promising outcomes in identifying the most relevant features, and obtaining the best combination of ESNN parameters with faster and more accurate classification.

Evolving spiking neural networks for robot control

We describe a sequence of experiments in which a robot “brain” was evolved to mimic the behaviours captured under control of a heuristic rule program (imitation learning). The task was light-seeking while avoiding obstacles using binocular light sensors and a trio of IR proximity sensors. The “brain” was a spiking neural network simulator whose parameters were tuned by a genetic algorithm, where fitness was assessed by the closeness to target output spike trains. Spike trains were frequency encoded. The network topology was manually designed, and then modified in response to observed difficulties during evolution. We noted that good performance seems best approached by judicious mixing of excitation and inhibition. Besides robotic applications, the domain of “smart” prosthetics also appears promising.

KNOWLEDGE EXTRACTION FROM EVOLVING SPIKING NEURAL NETWORKS WITH RANK ORDER POPULATION CODING

This paper demonstrates how knowledge can be extracted from evolving spiking neural networks with rank order population coding. Knowledge discovery is a very important feature of intelligent systems. Yet, a disproportionally small amount of research is centered on the issue of knowledge extraction from spiking neural networks which are considered to be the third generation of artificial neural networks. The lack of knowledge representation compatibility is becoming a major detriment to end users of these networks. We show that a high-level knowledge can be obtained from evolving spiking neural networks. More specifically, we propose a method for fuzzy rule extraction from an evolving spiking network with rank order population coding. The proposed method was used for knowledge discovery on two benchmark taste recognition problems where the knowledge learnt by an evolving spiking neural network was extracted in the form of zero-order Takagi-Sugeno fuzzy IF-THEN rules.

Integrated feature and parameter optimization for an evolving spiking neural network: Exploring heterogeneous probabilistic models

This study introduces a quantum-inspired spiking neural network (QiSNN) as an integrated connectionist system, in which the features and parameters of an evolving spiking neural network are optimized together with the use of a quantum-inspired evolutionary algorithm. We propose here a novel optimization method that uses different representations to explore the two search spaces: A binary representation for optimizing feature subsets and a continuous representation for evolving appropriate real-valued configurations of the spiking network. The properties and characteristics of the improved framework are studied on two different synthetic benchmark datasets. Results are compared to traditional methods, namely a multi-layer-perceptron and a naïve Bayesian classifier (NBC). A previously used real world ecological dataset on invasive species establishment prediction is revisited and new results are obtained and analyzed by an ecological expert. The proposed method results in a much faster convergence to an optimal solution (or a close to it), in a better accuracy, and in a more informative set of features selected.