Evolutionary Deep Learning Research Articles

An evolutionary deep learning approach using flexible variable-length dynamic stochastic search for anomaly detection of robot joints

Anomaly detection is crucial for condition monitoring of robot joints. An increasing number of anomaly detection methods based on deep learning have been investigated. However, since the deep learning architectures for anomaly detection are manually designed by trial and error, the design process is time-consuming, and the designed deep learning architectures may not be optimal for specific anomaly detection tasks. In this paper, a Flexible Variable-Length Dynamic Stochastic Search (FVLDSS) is proposed by designing and embedding the encoding and alignment strategies into the original Dynamic Stochastic Search (DSS). Subsequently, an Evolutionary Deep Learning Approach Using FVLDSS (EDLDSS) is proposed to automatically search for an optimal deep learning architecture for anomaly detection. In EDLDSS, Convolutional Neural Network (CNN) is used as the feature extractor, k-nearest neighbors trained only with normal samples is used as the anomaly detector, and FVLDSS is used to simultaneously optimize the network structure and hyperparameters of CNN. Furthermore, an anomaly detection method based on EDLDSS is developed to detect the anomalies in robot joints, in which S-transform spectrograms of vibration signals normalized by Z-score normalization are used as the input of CNN. To validate the performance of EDLDSS, a condition monitoring system using fiber Bragg grating sensors is first established for the industrial robot to acquire vibration signals from robot joints and conduct three experiments. The statistical results demonstrate the feasibility and satisfactory performance of EDLDSS in handling the anomaly detection problem of robot joints.

An evolutionary deep learning model based on XGBoost feature selection and Gaussian data augmentation for AQI prediction

Accurate prediction of air quality is crucial for ensuring the scientific validity and effectiveness of air pollution control measures. This study proposes a combined deep learning (DL) model (XGBoost-GDA-TCN-IMRFO-GRU) for predicting hourly air quality index (AQI) data in four cities. The model integrates Extreme gradient boosting (XGBoost) for feature selection, Gaussian data augmentation (GDA), improved manta ray foraging optimization (IMRFO) algorithm, temporal convolutional network (TCN), and gated recurrent unit (GRU). XGBoost calculates the scores of pollutant gases affecting AQI, selecting the top four important pollutants (PM2.5, PM10, NO2, O3) based on their importance rankings. GDA enhances the robustness of the DL models and addresses the limitations of insufficient and overfitting training datasets. Additionally, the IMRFO algorithm, with two improved strategies, is applied to enhance the GRU model. TCN extracts spatiotemporal features of AQI, while GRU constructs a temporal model for efficient computations. Compared to eleven benchmark models, the proposed model demonstrates superior performance in terms of MAE, RMSE, MAPE, and NSE, achieving high accuracy and optimal prediction performance. Specifically, the XGBoost-GDA-TCN-IMRFO-GRU model reduces RMSE, MAE, and MAPE by 33–60 %, 39–68 %, and 39–66 %, respectively, compared to the TCN model. Therefore, the XGBoost-GDA-TCN-IMRFO-GRU model can provide reliable early warnings for air quality, which is of great significance for air pollution prevention and the sustainable development of society.

Optimization of Resources Allocation using Evolutionary Deep Learning

Optimization of Resources Allocation using Evolutionary Deep Learning

Matching workloads ro systems with deep reinforcement learning

Along with the evolution of computer microarchitecture over the years, the number of dies, cores, and embedded multi-die interconnect bridges has grown. Optimizing the workload running on a central processing unit (CPU) to improve the computer performance has become a challenge. Matching workloads to systems with optimal system configurations to achieve the desired system performance is an open challenge in both academic and industrial research. In this paper, we propose two reinforcement learning (RL) approaches, a deep reinforcement learning (DRL) approach and an evolutionary deep reinforcement learning (EDRL) approach, to find an optimal system configuration for a given computer workload with a system performance objective. The experimental results demonstrate that both approaches can determine the optimal system configuration with the desired performance objective. The comparison studies illustrate that the DRL approach outperforms the standard RL approaches. In the future, these DRL approaches can be leveraged in system performance auto-tuning studies.

Evolutionary Deep Learning for Sequential Data Processing in Music Education

Evolutionary Deep Learning for Sequential Data Processing in Music Education

Genetic Programming-Based Evolutionary Deep Learning for Data-Efficient Image Classification

Data-efficient image classification is a challenging task that aims to solve image classification using small training data. Neural network-based deep learning methods are effective for image classification, but they typically require large-scale training data and have major limitations such as requiring expertise to design network architectures and having poor interpretability. Evolutionary deep learning is a recent hot topic that combines evolutionary computation with deep learning. However, most evolutionary deep learning methods focus on evolving architectures of neural networks, which still suffer from limitations such as poor interpretability. To address this, this paper proposes a new genetic programming-based evolutionary deep learning approach to data-efficient image classification. The new approach can automatically evolve variable-length models using many important operators from both image and classification domains. It can learn different types of image features from colour or gray-scale images, and construct effective and diverse ensembles for image classification. A flexible multi-layer representation enables the new approach to automatically construct shallow or deep models/trees for different tasks and perform effective transformations on the input data via multiple internal nodes. The new approach is applied to solve five image classification tasks with different training set sizes. The results show that it achieves better performance in most cases than deep learning methods for data-efficient image classification. A deep analysis shows that the new approach has good convergence and evolves models with high interpretability, different lengths/sizes/shapes, and good transferability.

An Evolutionary Deep Learning Method Based on Improved Heap-Based Optimization for Medical Image Classification and Diagnosis

An Evolutionary Deep Learning Method Based on Improved Heap-Based Optimization for Medical Image Classification and Diagnosis

Adaptive Sampling Path Planning for a 3D Marine Observation Platform Based on Evolutionary Deep Reinforcement Learning

Adaptive sampling of the marine environment may improve the accuracy of marine numerical prediction models. This study considered adaptive sampling path optimization for a three-dimensional (3D) marine observation platform, leading to a path-planning strategy based on evolutionary deep reinforcement learning. The low sampling efficiency of the reinforcement learning algorithm is improved by evolutionary learning. The combination of these two components as a new algorithm has become a current research trend. We first combined the evolutionary algorithm with different reinforcement learning algorithms to verify the effectiveness of the combination of algorithms with different strategies. Experimental results indicate that the fusion of the two algorithms based on a maximum-entropy strategy is more effective for adaptive sampling using a 3D marine observation platform. Data assimilation experiments indicate that adaptive sampling data from a 3D mobile observation platform based on evolutionary deep reinforcement learning improves the accuracy of marine environment numerical prediction systems.

An evolutionary deep learning model based on EWKM, random forest algorithm, SSA and BiLSTM for building energy consumption prediction

Accurate prediction of building energy consumption is crucial to the rational scheme of building energy. Combining the entropy-weighted K-means (EWKM) with the random forest (RF) method, a feature selection (EWKM-RF) method is proposed in this paper. Based on the proposed EWKM-RF method, the classification and feature selection of the energy consumption influencing factors can be achieved exclusively. Meanwhile, based on the EWKM-RF method and the bi-directional long-short-term memory neural network (BiLSTM) optimized by the sparrow search algorithm (SSA), an RF-SSA-BiLSTM prediction model for building energy consumption is established in this paper. As the weight, learning rate, and hidden layer node parameters of the BiLSTM neural network are optimized with the SSA, the constraints for manually adjusting parameters are avoided in the proposed prediction model. To examine the accuracy of the proposed model, energy consumption data of a civil public building in Dalian city are collected and tested. Results show the prediction error of RF-SSA-BiLSTM after feature selection is reduced by 24.55 % in high and low energy consumption months. Compared with RF-BiLSTM, RF-PSO-BiLSTM, and RF-CNN-BiLSTM, the RF-SSA-BiLSTM has strong robustness. The average MAE, RMSE and MAPE values of energy consumption prediction in the four seasons are 1.30, 1.63 and 0.02.

Metaheuristic evolutionary deep learning model based on temporal convolutional network, improved aquila optimizer and random forest for rainfall-runoff simulation and multi-step runoff prediction

Accurate and reliable runoff prediction is of great significance to water resources management, disaster monitoring and rational development and utilization of water resources. In this paper, a metaheuristic evolutionary deep learning model based on Temporal Convolutional Network (TCN), Improved Aquila Optimizer (IAO) and Random Forest (RF) is proposed for rainfall-runoff simulation and multi-step runoff prediction. In this study, the influence of various input variables on the prediction accuracy is discussed. First of all, in order to avoid the dimensional disaster problems and reduce the calculation time, RF is used to calculate the correlation between the input variables and the prediction object, and the data with high correlation is selected as the final inputs. Then, the filtered data are sent to the TCN model, and the parameters of the TCN model are optimized using the IAO algorithm, and the final prediction results are obtained. In this study, the rainfall and runoff data of five stations in the middle reaches of Jinsha River, China were selected, and the runoff of Panzhihua station was simulated and predicted by establishing multiple models. By analyzing and comparing the predictive results of several models, it shows that the models and improvements proposed in this study are effective.

Retraction Notice: An Evolutionary Deep Learning Anomaly Detection Framework for In-Vehicle Networks—CAN Bus

Retraction Notice: An Evolutionary Deep Learning Anomaly Detection Framework for In-Vehicle Networks—CAN Bus

A novel multi-gradient evolutionary deep learning approach for few-shot wind power prediction using time-series GAN

Due to the lack of historical data, accurate prediction is a great challenge for newly constructed wind farms (NWFs). How to guarantee satisfactory prediction accuracy with limited data has become a bottleneck issue in wind power prediction (WPP). To solve the few-shot learning problem in NWFs, a novel multi-gradient evolutionary deep learning neural network (EATDLNN) prediction model is proposed, which incorporates the time-series GAN (TimeGAN) and multivariate variational mode decomposition (MVMD) method. TimeGAN adds an auto-encoder network and a supervised loss function to GAN, which enables it to capture the dynamic time series correlation characteristics of wind data and generate high-quality samples. MVMD is then employed to simultaneously decompose nonstationary sequences. In the EATDLNN prediction model, an evolution-based multi-gradient training approach is proposed to train the DLNN by integrating three gradient descent methods (i.e., Adam, MBGD-Momentum, RMSprop). Compared with the traditional single-gradient training approach, the multi-gradient training approach provides different gradient descent paths and the gradient can adaptively choose to update towards the path with the smallest error based on the evolutionary framework, which can help the model jump out of the local optimum and enhance the generalization ability. Massive experiments prove the excellent performance of the proposed method, especially in one-step prediction, the root mean square errors reduce by 79.40%∼91.05% compared with other state-of-the-art methods.

Survey on Evolutionary Deep Learning: Principles, Algorithms, Applications, and Open Issues

Over recent years, there has been a rapid development of deep learning (DL) in both industry and academia fields. However, finding the optimal hyperparameters of a DL model often needs high computational cost and human expertise. To mitigate the above issue, evolutionary computation (EC) as a powerful heuristic search approach has shown significant merits in the automated design of DL models, so-called evolutionary deep learning (EDL). This article aims to analyze EDL from the perspective of automated machine learning (AutoML). Specifically, we first illuminate EDL from DL and EC and regard EDL as an optimization problem. According to the DL pipeline, we systematically introduce EDL methods ranging from data preparation, model generation, to model deployment with a new taxonomy (i.e., what and how to evolve/optimize), and focus on the discussions of solution representation and search paradigm in handling the optimization problem by EC. Finally, key applications, open issues, and potentially promising lines of future research are suggested. This survey has reviewed recent developments of EDL and offers insightful guidelines for the development of EDL.

Neuroevolution with box mutation: An adaptive and modular framework for evolving deep neural networks

The pursuit of self-evolving neural networks has driven the emerging field of Evolutionary Deep Learning, which combines the strengths of Deep Learning and Evolutionary Computation. This work presents a novel method for evolving deep neural networks by adapting the principles of Geometric Semantic Genetic Programming, a subfield of Genetic Programming, and Semantic Learning Machine. Our approach integrates evolution seamlessly through natural selection with the optimization power of backpropagation in deep learning, enabling the incremental growth of neural networks’ neurons across generations. By evolving neural networks that achieve nearly 89% accuracy on the CIFAR-10 dataset with relatively few parameters, our method demonstrates remarkable efficiency, evolving in GPU minutes compared to the field standard of GPU days.

Influence Maximization in Complex Networks by Using Evolutionary Deep Reinforcement Learning

Influence maximization (IM) in complex networks tries to activate a small subset of seed nodes that could maximize the propagation of influence. The studies on IM have attracted much attention due to their wide applications such as item recommendation, viral marketing, information propagation and disease immunization. Existing works mainly model the IM problem as a discrete optimization problem, and use either approximate or meta-heuristic algorithms to address this problem. However, these works are hard to find a good tradeoff between effectiveness and efficiency due to the NP-hard and large-scale network properties of the IM problem. In this article, we propose an evolutionary deep reinforcement learning algorithm (called EDRL-IM) for IM in complex networks. First, EDRL-IM models the IM problem as a continuous weight parameter optimization of deep Q network (DQN). Then, it combines an evolutionary algorithm (EA) and a deep reinforcement learning algorithm (DRL) to evolve the DQN. The EA simultaneously evolves a population of individuals, and each of which represents a possible DQN and returns a solution to the IM problem through a dynamic markov node selection strategy, while the DRL integrates all information and network-specific knowledge of DQNs to accelerate their evolution. Systematic experiments on both benchmark and real-world networks show the superiority of EDRL-IM over the state-of-the-art IM methods in finding seed nodes.

Inverse design of ultra-wideband transparent frequency selective surface absorbers based on evolutionary deep learning

Conventional frequency selective surface (FSS) absorbers design is time-consuming, involving multiple electromagnetic (EM) simulations for parameter scanning. A novel reverse design method is proposed utilizing evolutionary deep learning (EDL) based on an improved bacterial foraging optimization (IBFO) algorithm and a deep belief network. It establishes the relationship between the geometric structure and EM response. The combination of IBFO and EDL facilitates an efficient optimization for structural parameters, mitigating the ‘one-to-many’ problem and accelerating the design process. An optically transparent FSS absorber with an ultra-bandwidth of 8–18 GHz is designed to verify the proposed method’s capability. The simulation and experimental results demonstrate that the absorber displays exceptional characteristics such as polarization insensitivity and robustness under a 45° oblique incidence angle, making it a suitable candidate for radar stealth and photovoltaic solar energy applications. The proposed method can be applied to the design and optimization of various absorbers and complex EM devices.

Developing an evolutionary deep learning framework with random forest feature selection and improved flow direction algorithm for NOx concentration prediction

The continuous change of load makes it difficult for the power plant to control the emission of pollutants. A reliable NOx concentration prediction model is important to energy conservation and emission reduction. In this study, an evolutionary deep learning framework is proposed to predict the NOx emission of coal fired boiler. In the process of NOx emission, several variables will affect the emission concentration. Therefore, firstly, the variables with high importance to NOx emission are selected through random forest (RF), and the initial input data is filtered to obtain a new input data. Secondly, the convolutional neural network (CNN) is used to further extract the deep characteristics of the new data set. Thirdly, Using Chaos strategy and iterative local search (ILS) method to improve flow direction algorithm (FDA) and enhance the optimization ability of the algorithm. Finally, the improved FDA (IFDA) is used to optimize the hyperparameters of Bi-directional gated recurrent unit (BiGRU), and the optimized BiGRU model is used to forecast the NOx emission concentration. Three different data sets from coal-fired power plants were obtained to verify the proposed hybrid RF-CNN-FDA-BiGRU model. Experimental results show that the proposed model can accurately predict NOx concentration and has better prediction performance.

A Novel Hybrid Artificial Bee Colony-Based Deep Convolutional Neural Network to Improve the Detection Performance of Backscatter Communication Systems

Backscatter communication (BC) is a promising technology for low-power and low-data-rate applications, though the signal detection performance is limited since the backscattered signal is usually much weaker than the original signal. When the detection performance is poor, the backscatter device (BD) may not be able to accurately detect and interpret the incoming signal, leading to errors and degraded communication quality. This can result in data loss, slow data transfer rates, and reduced reliability of the communication link. This paper proposes a novel approach to improve the detection performance of backscatter communication systems using evolutionary deep learning. In particular, we focus on training deep convolutional neural networks (DCNNs) to improve the detection performance of BC. We first develop a novel hybrid algorithm based on artificial bee colony (ABC), biogeography-based optimization (BBO), and particle swarm optimization (PSO) to optimize the architecture of the DCNN, followed by training using a large set of benchmark datasets. To develop the hybrid ABC, the migration operator of the BBO is used to improve the exploitation. Moving towards the global best of PSO is also proposed to improve the exploration of the ABC. Then, we take advantage of the proposed deep architecture to improve the bit-error rate (BER) performance of the studied BC system. The simulation results demonstrate that the proposed algorithm has the best performance in training the benchmark datasets. The results also show that the proposed approach significantly improves the detection performance of backscattered signals compared to existing works.

Coupling Progressive Deep Learning with the AdaBoost Framework for Landslide Displacement Rate Prediction in the Baihetan Dam Reservoir, China

Disasters caused by landslides pose a considerable threat to people’s lives and property, resulting in substantial losses each year. Landslide displacement rate prediction (LDRP) provides a useful fundamental tool for mitigating landslide disasters. However, more accurately predicting LDRP remains a challenge in the study of landslides. Lately, ensemble deep learning algorithms have shown promise in delivering a more precise and effective spatial modeling solution. The core aims of this research are to explore and evaluate the prediction capability of three progressive evolutionary deep learning (DL) techniques, i.e., a recurrent neural network (RNN), long short-term memory (LSTM), and a gated recurrent unit (GRU) ensemble AdaBoost algorithm for modeling rainfall-induced and reservoir-induced landslides in the Baihetan reservoir area in China. The outcomes show that the ensemble DL model could predict the Wangjiashan landslide in the Baihetan reservoir area with improved accuracy. The highest accuracy was achieved in the testing set when the window length equaled 30. However, assembling two predictors outperformed the accuracy of assembling three predictors, with the mean absolute error and root mean square error reaching 1.019 and 1.300, respectively. These findings suggest that the combination of strong learners and DL can yield satisfactory prediction results.

Smart city urban planning using an evolutionary deep learning model

Smart city urban planning using an evolutionary deep learning model