Evolving Spiking Neural Networks Research Articles

Effective air pollution prediction by combining time series decomposition with stacking and bagging ensembles of evolving spiking neural networks

In this article, we introduce a new approach to air pollution prediction using the CEEMDAN time series decomposition method combined with the two-layered ensemble of predictors created based on the stacking and bagging techniques. The proposed ensemble approach is outperforming other selected state-of-the-art models when the bagging ensemble consisting of evolving Spiking Neural Networks (eSNNs) is used in the second layer of the stacking ensemble. In our experiments, we used the PM10 air pollution and weather dataset for Warsaw. As the results of the experiments show, the proposed ensemble can achieve the following error and agreement values over the tested dataset: error RMSE 6.91, MAE 5.14 and MAPE 21%; agreement IA 0.94. In addition, this article provides the computational and space complexity analysis of eSNNs predictors and offers a new encoding method for spiking neural networks that can be effectively applied for values of skewed distributions.

The hybrid feature extraction method for classification of adolescence idiopathic scoliosis using Evolving Spiking Neural Network

Scoliosis is the most commonly occurring spinal condition in adolescents. Photogrammetry is an alternative to radiography for examining adolescents with idiopathic scoliosis (AIS) to measure the curvature of the human back and avoid radiography exposure. Currently, a manual approach is applied to detect the spine curve. Machine Learning (ML) models are introduced to overcome this issue and reduce human error. Thus, an appropriate Features Extraction (FE) method is crucial to producing a good ML classification model. Local Binary Pattern (LBP) is capable of providing the desired features from the surface of a human body. However, it produces a high number of parameters for each image, affecting the classification performance. Therefore, a new FE method has been proposed to reduced the number of parameters. A fusion of LBP and 1DCNN (1-Dimensional Convolutional Neural Network) FE methods that can reduce the number of parameters by 50% is proposed. These features are classified using Evolving Spiking Neural Network (LBP-1DCNN-ESNN) and achieve an accuracy of 90.00%. It leads to faster and more accurate results. ESNN can produce efficient and fast decisions through a one-pass learning process generated from the neuron repository. The proposed LBP-1DCNN-ESNN model benefits healthcare when quick solutions are demanded.

Hyperparameter Optimization of Evolving Spiking Neural Network for Time-Series Classification

Hyperparameter Optimization of Evolving Spiking Neural Network for Time-Series Classification

Unsupervised anomaly detection in multivariate time series with online evolving spiking neural networks

With the increasing demand for digital products, processes and services the research area of automatic detection of signal outliers in streaming data has gained a lot of attention. The range of possible applications for this kind of algorithms is versatile and ranges from the monitoring of digital machinery and predictive maintenance up to applications in analyzing big data healthcare sensor data. In this paper we present a method for detecting anomalies in streaming multivariate times series by using an adapted evolving Spiking Neural Network. As the main components of this work we contribute (1) an alternative rank-order-based learning algorithm which uses the precise times of the incoming spikes for adjusting the synaptic weights, (2) an adapted, realtime-capable and efficient encoding technique for multivariate data based on multi-dimensional Gaussian Receptive Fields and (3) a continuous outlier scoring function for an improved interpretability of the classifications. Spiking neural networks are extremely efficient when it comes to process time dependent information. We demonstrate the effectiveness of our model on a synthetic dataset based on the Numenta Anomaly Benchmark with various anomaly types. We compare our algorithm to other streaming anomaly detecting algorithms and can prove that our algorithm performs better in detecting anomalies while demanding less computational resources for processing high dimensional data.

Integrated Evolving Spiking Neural Network and Feature Extraction Methods for Scoliosis Classification

Adolescent Idiopathic Scoliosis (AIS) is a deformity of the spine that affects teenagers. The current method for detecting AIS is based on radiographic images which may increase the risk of cancer growth due to radiation. Photogrammetry is another alternative used to identify AIS by distinguishing the curves of the spine from the surface of a human’s back. Currently, detecting the curve of the spine is manually performed, making it a time-consuming task. To overcome this issue, it is crucial to develop a better model that automatically detects the curve of the spine and classify the types of AIS. This research proposes a new integration of ESNN and Feature Extraction (FE) methods and explores the architecture of ESNN for the AIS classification model. This research identifies the optimal Feature Extraction (FE) methods to reduce computational complexity. The ability of ESNN to provide a fast result with a simplicity and performance capability makes this model suitable to be implemented in a clinical setting where a quick result is crucial. A comparison between the conventional classifier (Support Vector Machine (SVM), Multi-layer Perceptron (MLP) and Random Forest (RF)) with the proposed AIS model also be performed on a dataset collected by an orthopedic expert from Hospital Universiti Kebangsaan Malaysia (HUKM). This dataset consists of various photogrammetry images of the human back with different types of Malaysian AIS patients to solve the scoliosis problem. The process begins by pre-processing the images which includes resizing and converting the captured pictures to gray-scale images. This is then followed by feature extraction, normalization, and classification. The experimental results indicate that the integration of LBP and ESNN achieves higher accuracy compared to the performance of multiple baseline state-of-the-art Machine Learning for AIS classification. This demonstrates the capability of ESNN in classifying the types of AIS based on photogrammetry images.

Unsupervised Anomaly Detection in Stream Data with Online Evolving Spiking Neural Networks

Unsupervised anomaly discovery in stream data is a research topic with many practical applications. However, in many cases, it is not easy to collect enough training data with labeled anomalies for supervised learning of an anomaly detector in order to deploy it later for identification of real anomalies in streaming data. It is thus important to design anomalies detectors that can correctly detect anomalies without access to labeled training data. Our idea is to adapt the Online evolving Spiking Neural Network (OeSNN) classifier to the anomaly detection task. As a result, we offer an Online evolving Spiking Neural Network for Unsupervised Anomaly Detection algorithm (OeSNN-UAD), which, unlike OeSNN, works in an unsupervised way and does not separate output neurons into disjoint decision classes. OeSNN-UAD uses our proposed new two-step anomaly detection method. Also, we derive new theoretical properties of neuronal model and input layer encoding of OeSNN, which enable more effective and efficient detection of anomalies in our OeSNN-UAD approach. The proposed OeSNN-UAD detector was experimentally compared with state-of-the-art unsupervised and semi-supervised detectors of anomalies in stream data from the Numenta Anomaly Benchmark and Yahoo Anomaly Datasets repositories. Our approach outperforms the other solutions provided in the literature in the case of data streams from the Numenta Anomaly Benchmark repository. Also, in the case of real data files of the Yahoo Anomaly Benchmark repository, OeSNN-UAD outperforms other selected algorithms, whereas in the case of Yahoo Anomaly Benchmark synthetic data files, it provides competitive results to the results recently reported in the literature.

Evolving Spiking Neural Network Model for PM2.5 Hourly Concentration Prediction Based on Seasonal Differences: A Case Study on Data from Beijing and Shanghai

ABSTRACT In recent years, the dangers that air pollutants pose to human health and the environment have received widespread attention. Although accurately predicting the air quality is essential to managing pollution and developing control policies, traditional forecasting models have not been able to simulate the seasonal and diurnal variation in air pollutant concentrations. Furthermore, inadequate processing of the available spatio-temporal data has precluded the capture of predictive historical patterns. Therefore, we have developed a staging evolving spiking neural network (eSNN) model named Staging-eSNN that first employs a time series clustering algorithm to distinguish the seasonal from the diurnal variation in the PM2.5 concentration. We then predict the concentrations in Beijing and Shanghai 1, 3, 6, 12 and 24 hours in advance. Various evaluation indicators show that the Staging-eSNN model achieves higher performance than the support vector regression (SVR), random forest (RF) and other eSNN models.

Botnet detection used fast-flux technique, based on adaptive dynamic evolving spiking neural network algorithm

A botnet refers to a group of machines. These machines are controlled distantly by a specific attacker. It represents a threat facing the web and data security. Fast-flux service network (FFSN) has been engaged by bot herders for cover malicious botnet activities. It has been engaged by bot herders for increasing the lifetime of malicious servers through changing the IP addresses of the domain name quickly. In the present research, we aimed to propose a new system. This system is named fast flux botnet catcher system (FFBCS). This system can detect FF-domains in an online mode using an adaptive dynamic evolving spiking neural network algorithm. Comparing with two other related approaches the proposed system shows a high level of detection accuracy, low false positive and negative rates, respectively. It shows a high performance. The algorithm's proposed adaptation increased the accuracy of the detection. For instance, this accuracy reached (98.76%) approximately.

Botnet detection used fast-flux technique, based on adaptive dynamic evolving spiking neural network algorithm

A botnet refers to a group of machines. These machines are controlled distantly by a specific attacker. It represents a threat facing the web and data security. Fast-flux service network (FFSN) has been engaged by bot herders for cover malicious botnet activities. It has been engaged by bot herders for increasing the lifetime of malicious servers through changing the IP addresses of the domain name quickly. In the present research, we aimed to propose a new system. This system is named fast flux botnet catcher system (FFBCS). This system can detect FF-domains in an online mode using an adaptive dynamic evolving spiking neural network algorithm. Comparing with two other related approaches the proposed system shows a high level of detection accuracy, low false positive and negative rates, respectively. It shows a high performance. The algorithm's proposed adaptation increased the accuracy of the detection. For instance, this accuracy reached (98.76%) approximately.

P300 ERP Component on Eating Habits Profiling Using Dynamic Evolving Spiking Neural Network (deSNN)

— Unhealthy eating habits have become a big issue that often causes many chronic diseases in various countries in recent years. The current assessment to identify the status of eating habits is to use self-assessment. However, self-assessment is known to have an error or uncertainty value due to cognitive factors from respondents that affect the results of the assessment. A person's profile is potentially measured by reviewing Event-related potential (ERP) which is an ideal technique for understanding perception and attention. This study uses images of healthy and unhealthy foods as a stimulus when recording EEG data. The method used for classification is dynamic evolving spiking neural network (deSSN) based on the Neucube architecture. The results showed that the mean amplitude of the P300 component discovered in the Parietal and Occipital lobes was higher for healthy food in the healthy eating habits group. Whereas the unhealthy eating habits group was higher for unhealthy foods. The deSNN classification is proven to operate in learning ERP data but the accuracy rate is not too high due to inadequate sample training

Evolved Neuromorphic Control for High Speed Divergence-Based Landings of MAVs

Flying insects are capable of vision-based navigation in cluttered environments, reliably avoiding obstacles through fast and agile maneuvers, while being very efficient in the processing of visual stimuli. Meanwhile, autonomous micro air vehicles still lag far behind their biological counterparts, displaying inferior performance at a much higher energy consumption. In light of this, we want to mimic flying insects in terms of their processing capabilities, and consequently show the efficiency of this approach in the real world. This letter does so through evolving spiking neural networks for controlling landings of micro air vehicles using optical flow divergence from a downward-looking camera. We demonstrate that the resulting neuromorphic controllers transfer robustly from a highly abstracted simulation to the real world, performing fast and safe landings while keeping network spike rate minimal. Furthermore, we provide insight into the resources required for successfully solving the problem of divergence-based landing, showing that high-resolution control can be learned with only a single spiking neuron. To the best of our knowledge, this work is the first to integrate spiking neural networks in the control loop of a real-world flying robot. Videos of the experiments can be found at https://bit.ly/neuro-controller .

Hydrofluoroether Impurities—Chemical Detection Using a Deep Learning Laser Speckle Contrast Evolving Spiking Neural Network

Hydrofluoroether (HFE) impurities detection is an issue related to detecting chemical contamination within a high volume manufacturing (HVM) chiller caused by a rapid emulated environmental attack. The aftereffects of the cycle emulated attack may eventually create a micro-crack in the heat-exchanger. This event eventually causes the contamination of HFE due to multiple chemical interactions. This study proposes a new classification methodology to detect HFE chemical impurities using induction by a 532nm laser. The purpose of the laser induction is to leverage its laser speckle contrast attributes and amplify its detection. One of the reasons for choosing the 532nm laser spectrum is its highest quantum efficiency. Once amplification of detection is achieved, the detector tends to be in a highly sensitive mode due to low marginality to differentiate between two different conditions. This mode is prone to be stochastic. Thus, a new form of architecture known as Deep Learning Laser Speckle Contrast Evolving Spiking Neural Network (DL-LSC-ESNN) is proposed. The architecture utilizes speckle contrast domain conversion, dimensional additivity of the receptive neuron of evolving spiking neural network (ESNN), followed by the strength of Convolution Neural Network (CNN) feature extraction (FE) capability. Ultimately, the evolving and adaptive ability of ESNN is assimilated and integrated seamlessly. CNN acted to extract only an important spike train, and the result is its essences of important spikes or spike feature maps. The spike feature maps are then fed into the ESNN neuron repository, which either assimilates or creates a new neuron repository. The proposed methods show significant improvement in the accuracy of detection against multiple baseline state of the art CNN architecture and ultimately demonstrated its capability to detect real-time contamination of HFE thus improved the detection rate significantly for the HVM environment.

Exploiting the stimuli encoding scheme of evolving Spiking Neural Networks for stream learning

Stream data processing has lately gained momentum with the arrival of new Big Data scenarios and applications dealing with continuously produced information flows. Unfortunately, traditional machine learning algorithms are not prepared to tackle the specific challenges imposed by data stream processing, such as the need for learning incrementally, limited memory and processing time requirements, and adaptation to non-stationary data, among others. To face these paradigms, Spiking Neural Networks have emerged as one of the most promising stream learning techniques, with variants such as Evolving Spiking Neural Networks capable of efficiently addressing many of these challenges. Interestingly, these networks resort to a particular population encoding scheme – Gaussian Receptive Fields – to transform the incoming stimuli into temporal spikes. The study presented in this manuscript sheds light on the predictive potential of this encoding scheme, focusing on how it can be applied as a computationally lightweight, model-agnostic preprocessing step for data stream learning. We provide informed intuition to unveil under which circumstances the aforementioned population encoding method yields effective prediction gains in data stream classification with respect to the case where no preprocessing is performed. Results obtained for a variety of stream learning models and both synthetic and real stream datasets are discussed to empirically buttress the capability of Gaussian Receptive Fields to boost the predictive performance of stream learning methods, spanning further research towards extrapolating our findings to other machine learning problems.

Evolving spiking neural networks methods for classification problem: a case study in flood events risk assessment

<p>Analysing environmental events such as predicting the risk of flood is considered as a challenging task due to the dynamic behaviour of the data. One way to correctly predict the risk of such events is by gathering as much of related historical data and analyse the correlation between the features which contribute to the event occurrences. Inspired by the brain working mechanism, the spiking neural networks have proven the capability of revealing a significant association between different variables spike behaviour during an event. Personalised modelling, on the other hand, allows a personal model to be created for a specific data model and experiment. Therefore, a personalised modelling method incorporating spiking neural network is used to create a personalised model for assessing a real-world flood case study in Kuala Krai, Kelantan based on historical data of 2012-2016 provided by Malaysian Meteorological Department. The result shows that the method produces the highest accuracy among the selected compared algorithms.</p>

Gryphon: a semi-supervised anomaly detection system based on one-class evolving spiking neural network

The backbone of the economy, security and sustainability of a state is inseparably linked to the security of its critical infrastructure. Critical infrastructures define goods, systems or subsystems that are essential to maintain the vital functions of society, health, physical protection, security plus economic and social well-being of citizens. The digital security of critical infrastructures is a very important priority for the well-being of every country, especially nowadays, because of the direct threats dictated by the current international conjuncture and due to the emerging interactions or interconnections developed between the National Critical Infrastructures, internationally. The aim of this research is the development and testing of an Anomaly Detection intelligent algorithm that has the advantage to run very fast with a small portion of the available data and to perform equally well with the existing approaches. Such a system must be characterized by high efficiency and very fast execution. Thus, we present the Gryphon advanced intelligence system. Gryphon is a Semi-Supervised Unary Anomaly Detection System for big industrial data which is employing an evolving Spiking Neural Network (eSNN) One-Class Classifier (eSNN-OCC). This machine learning algorithm corresponds to a model capable of detecting very fast and efficiently, divergent behaviors and abnormalities associated with cyberattacks, which are known as Advanced Persistent Threat (APT). The training process is performed on data related to the normal function of a critical infrastructure.

Air pollution prediction with clustering-based ensemble of evolving spiking neural networks and a case study for London area

In this article, we propose a novel Clustering-based Ensemble model (CEeSNN) for air pollution prediction based on evolving Spiking Neural Networks (eSNN), where each eSNN network is trained on a separate set of time series. In our approach, we generate training sets by clustering an initial set of time series with respect to pollution values. Each obtained cluster of time series is used to build a single eSNN network. In the experiments, we forecasted ozone and PM10 pollution for Greater London Area for 1, 3, and 6 h ahead based on data from three monitoring sites located there. The prediction quality of the proposed CEeSNN model, as well as the singleton NeuCube model, an MLP network and the ARIMA model was assessed by means of several quality measures. The experimental results show that the proposed ensemble model is able to give significantly better forecasting results than the other three models.

Modelling gene interaction networks from time-series gene expression data using evolving spiking neural networks

The genetic mechanisms responsible for the differentiation, metabolism, morphology and function of a cell in both normal and abnormal conditions can be uncovered by the analysis of transcriptomes. Mining big data such as the information encoded in nucleic acids, proteins, and metabolites has challenged researchers for several years now. Even though bioinformatics and system biology techniques have improved greatly and many improvements have been done in these fields of research, most of the processes that influence gene interaction over time are still unknown. In this study, we apply state-of-the art spiking neural network techniques to model, analyse and extract information about the regulatory processes of gene expression over time. A case study of microarray profiling in human skin during elicitation of eczema is used to examine the temporal association of genes involved in the inflammatory response, by means of a gene interaction network. Spiking neural network techniques are able to learn the interaction between genes using information encoded from the time-series gene expression data as spikes. The temporal interaction is learned, and the patterns of activity extracted and analysed with a gene interaction network. Results demonstrated that useful knowledge can be extracted from the data by using spiking neural network, unlocking some of the possible mechanisms involved in the regulatory process of gene expression.

Adaptive long-term traffic state estimation with evolving spiking neural networks

Due to the nature of traffic itself, most traffic forecasting models reported in literature aim at producing short-term predictions, yet their performance degrades when the prediction horizon is increased. The scarce long-term estimation strategies currently found in the literature are commonly based on the detection and assignment to patterns, but their performance decays when unexpected events provoke non predictable changes, or if the allocation to a traffic pattern is inaccurate. This work introduces a method to obtain long-term pattern forecasts and adapt them to real-time circumstances. To this end, a long-term estimation scheme based on the automated discovery of patterns is proposed and integrated with an on-line change detection and adaptation mechanism. The framework takes advantage of the architecture of evolving Spiking Neural Networks (eSNN) to perform adaptations without retraining the model, allowing the whole system to work autonomously in an on-line fashion. Its performance is assessed over a real scenario with 5 min data of a 6-month span of traffic in the center of Madrid, Spain. Significant accuracy gains are obtained when applying the proposed on-line adaptation mechanism on days with special, non-predictable events that degrade the quality of their long-term traffic forecasts.

SpikeCD: a parameter-insensitive spiking neural network with clustering degeneracy strategy

A clustering degeneracy algorithm, called SpikeCD, with spiking RBF neurons for classification is proposed in this paper. Unlike traditional spiking RBF networks where their performance severely relies on the time-costing process of parameter optimization, SpikeCD uses a clustering degeneracy strategy to adjust the number and centers of spiking RBF neurons, which is insensitive to parameters. A supervised learning is followed to improve network’s classification ability. Its performance is demonstrated on several benchmark datasets from the UCI Machine Learning Repository and image datasets. The results show SpikeCD can achieve good classification accuracy with simple structure. Moreover, the variation of parameters has a little effect on it. We hope this algorithm can be a new inspiration for improving the robustness of evolving spiking neural networks and other machine learning methods.

Design of spiking neural networks for blood pressure prediction during general anesthesia: considerations for optimizing results

The ability to predict blood pressure changes during general anesthesia would assist anesthetists minimize the risk of complications due to hypotensive events. However, such prediction is not trivial. Evolving spiking neural networks are a relatively new computational method that may have application to this problem. NeuCubeST consists of a 3-dimensional network of locally connected neurons called a Spiking Neural Network reservoir (SNNr) and can be used to classify time series data for prediction. There are a number of design considerations when using NeuCubeST as a classifier of time-series data: what pre-processing of the raw data is required (pre-processing), how to convert the time-series data into a spike train (input-encoding), which neurons the data are connected to (input-mapping), and how many nearest neighbours to use in classification (classification). However, it is still unclear how sensitive NeuCubeST-based systems are to perturbations of any of the above. In this paper we evaluate the contribution of these design factors to blood pressure prediction using NeuCubeST. 6000 possible combinations of those NeuCubeST options were tested for each of 100 patients and for each a Signal to Noise Ratio was obtained. All four investigated design factors showed significant contribution to SNR. Intra-operative MAP prediction using NeuCubeST can be effective but performance is sensitive to the design choices.