Ensemble Deep Learning Research Articles

Dynamic-Deep-Ensemble-Learning Scheme for Probabilistic Voltage Stability Margin Estimation to Enhance Resilient Power Grid Monitoring

Modern power grids are characterized by significant penetration of renewable energy sources (RES), variable power demand, and aging transmission infrastructure, all of which contribute to a high degree of operational uncertainty. Such uncertainty complicates the assessment of the static voltage stability in power grids. In response to this challenge, this paper proposes a novel deep ensemble learning-based approach to assess the probabilistic voltage stability margin (PVSM) for strengthening the resilience of power grid monitoring. First, the estimation of the PVSM is formulated as a quantile regression problem. Then, an improved deep quantile regression (iDQR) is utilized to generate a set of quantiles under specific nominal proportions. Next, a dynamic deep ensemble learning (D2EL) scheme based on diverse iDQR models and an improved Choquet fuzzy integral (iCFI) algorithm is proposed to enhance the overall performance of predictive quantiles for the PVSM. The proposed D2EL-based PVSM estimation approach is capable of accommodating system changes in a timely manner, thus providing higher estimation accuracy and stronger adaptability than conventional approaches. A comprehensive numerical study of several test systems is carried out, taking into account uncertain RES and loads, as well as topology changes. The results reveal the impressive performance of the proposed approach in the PVSM assessment.

Novel Human Activity Recognition by graph engineered ensemble deep learning model

This research delves into the domain of Human Activity Recognition (HAR) through sensor data analysis, offering a comprehensive exploration of three diverse datasets: UniMiB-SHAR, Motion Sense, and WISDM Actitracker. The UniMiB-SHAR dataset encompasses a diverse array of linear as well as non-linear and complex activities which involve the movement of more than one joint or muscle (for example Hitting Obstacles, jogging and falling with face down). This motion generates highly correlated sensor readings over a certain period of time. In this case, Convolution Neural Networks (CNNs) are effective in feature extraction as well as classification of HAR activities, but they may not fully grasp the combined features of spatial as well as temporal aspects in the HAR datasets and heavily rely on labelled data. Whereas, Graph convolution networks (GCN), with their capacity to model complex interactions through graph structure, complement CNN’s capabilities in classifying non-linear activities in the HAR dataset. By leveraging the Knowledge graph structure and acquiring the feature embeddings from the GCN model, in this study, a Noval ensemble CNN model is proposed for the classification of activities. The novel HAR pipeline is termed as Graph Engineered EnsemCNN HAR (GE-EnsemCNN-HAR) and its performance is evaluated on HAR datasets. Proposed model demonstrated a noteworthy accuracy of 93.5% on UniMiB-SHAR dataset, surpassing the Shallow CNN model with GNN with an improvement of 20.14%. The proposed model achieved a notable accuracy rate of 96.18% and 98% when evaluated on the Motion Sense and WISDM Actitracker dataset.

Improving Facial Expression Classification through Ensemble Deep Learning Models

Improving Facial Expression Classification through Ensemble Deep Learning Models

An ensemble deep learning model for automatic classification of cotton leaves diseases

<p>Cotton plant (Gossypium herbaceum), is one of the significant fiber crop grown worldwide. However, the crop is quite prone to leaves diseases, for which deep learning (DL) techniques can be utilized for early disease prediction and prevent stakeholders from losing the harvest. The objective of this paper is to develop a novel ensemble based deep convolutional neural network (DCNN) model developed on two base pretrained models named: VGG16 and InceptionV3 for early detection of cotton leaves diseases. The proposed ensemble model trained on cotton leaves dataset reports higher training and testing prediction accuracies as compared to the base pretrained models. Given that, deep learning architectures have hyper-parameters, this paper presents exhaustive experimental evaluations on ensemble model to tune hyper-parameters named learning rate, optimizer and no of epochs. The suggested hyper-parameter settings can be directly utilized while employing the ensemble model for cotton plant leaves disease detection and prediction. With suggested hyper-parameters settings of learning rate 0.0001, 20 epochs and stochastic gradient descent (SGD) optimizer, ensemble model reported training and testing accuracies of 98% and 95% respectively, which was higher than the training and testing accuracies of VGG16 and InceptionV3 pretrained DCNN models.</p>

A Deep Learning Ensemble Framework for Fake News Stance Detection and Classification

A Deep Learning Ensemble Framework for Fake News Stance Detection and Classification

ACPPfel: Explainable deep ensemble learning for anticancer peptides prediction based on feature optimization.

Background: Cancer is a significant global health problem that continues to cause a high number of deaths worldwide. Traditional cancer treatments often come with risks that can compromise the functionality of vital organs. As a potential alternative to these conventional therapies, Anticancer peptides (ACPs) have garnered attention for their small size, high specificity, and reduced toxicity, making them as a promising option for cancer treatments. Methods: However, the process of identifying effective ACPs through wet-lab screening experiments is time-consuming and requires a lot of labor. To overcome this challenge, a deep ensemble learning method is constructed to predict anticancer peptides (ACPs) in this study. To evaluate the reliability of the framework, four different datasets are used in this study for training and testing. During the training process of the model, integration of feature selection methods, feature dimensionality reduction measures, and optimization of the deep ensemble model are carried out. Finally, we explored the interpretability of features that affected the final prediction results and built a web server platform to facilitate anticancer peptides prediction, which can be used by all researchers for further studies. This web server can be accessed at http://lmylab.online:5001/. Results: The result of this study achieves an accuracy rate of 98.53% and an AUC (Area under Curve) value of 0.9972 on the ACPfel dataset, it has improvements on other datasets as well.

Software bug localization based on optimized and ensembled deep learning models

AbstractAn automated task for finding the essential buggy files among software projects with the help of a given bug report is termed bug localization. The conventional approaches suffer from the challenges of performing lexical matching. Particularly, the terms utilized for describing the bugs in the bug reports are observed to be irrelevant to the terms used in the source code files. To resolve these problems, we propose an optimized and ensemble deep learning model for software bug localization. These features are reduced by the principle component analysis (PCA). Then, they are selected by the weighted convolutional neural network (CNN) model with the support of the Modified Scatter Probability‐based Coyote Optimization Algorithm (MSP‐COA). Finally, the optimal features are subjected to the ensemble deep neural network and long short‐term memory (DNN‐LSTM), with parameter tuning by the MSP‐COA. Experimental results show that the proposed approach can achieve higher bug localization accuracy than individual models.

Enhanced Tropical Cyclone Precipitation Prediction in the Northwest Pacific Using Deep Learning Models and Ensemble Techniques

This study focuses on optimizing precipitation forecast induced by tropical cyclones (TCs) in the Northwest Pacific region, with lead times ranging from 6 to 72 h. The research employs deep learning models, such as U-Net, UNet3+, SE-Net, and SE-UNet3+, which utilize precipitation forecast data from the Global Forecast System (GFS) and real-time GFS environmental background data using a U-Net structure. To comprehensively make use of the precipitation forecasts from these models, we additionally use probabilistic matching (PM) and simple averaging (AVR) in rainfall prediction. The precipitation data from the Global Precipitation Measurement (GPM) Mission serves as the rainfall observation. The results demonstrate that the root mean squared errors (RMSEs) of U-Net, UNet3+, SE-UNet, SE-UNet3+, AVR, and PM are lowered by 8.7%, 10.1%, 9.7%, 10.0%, 11.4%, and 11.5%, respectively, when compared with the RMSE of the GFS TC precipitation forecasts, while the mean absolute errors are reduced by 9.6%, 11.3%, 9.0%, 12.0%, 12.8%, and 13.0%, respectively. Furthermore, the neural network model improves the precipitation threat scores (TSs). On average, the TSs of U-Net, UNet3+, SE-UNet, SE-UNet3+, AVR, and PM are raised by 12.8%, 21.3%, 19.3%, 20.7%, 22.5%, and 22.9%, respectively, compared with the GFS model. Notably, AVR and PM outperform all other individual models, with PM’s performance slightly better than AVR’s. The most important feature variables in optimizing TC precipitation forecast in the Northwest Pacific region based on the UNet-based neural network include GFS precipitation forecast data, land and sea masks, latitudinal winds at 500 hPa, and vertical winds at 500 hPa.

Deep ensemble learning approach for lower limb movement recognition from multichannel sEMG signals

Deep ensemble learning approach for lower limb movement recognition from multichannel sEMG signals

A Novel Approach for Air Quality Index Prognostication using Hybrid Optimization Techniques

This research presents an innovative deep learning approach for forecasting the Air Quality Index (AQI), a crucial public health concern in both developed and developing countries. The proposed methodology encompasses four stages: (a) Pre-processing, involving data cleaning and transformation; (b) Feature Extraction, capturing central tendency, dispersion, higher order statistics, and Spearman's rank correlation; (c) Feature Selection, using a novel hybrid optimization model, Particle Updated Grey Wolf Optimizer (PUGWO); and (d) an ensembled deep learning model for AQI prediction, integrating a Convolutional Neural Network (CNN), an optimized Bi-directional Long Short-Term Memory (Bi-LSTM), and an Auto-encoder. The CNN and Auto-encoder are trained on the extracted features, and their outputs are fed into the optimized Bi-LSTM for final AQI prediction. Implemented on the PYTHON platform, this model is evaluated through R^2, MAE, and RMSE error metrics. The proposed HRFKNN model demonstrates superior performance with an R-Square of 0.961, RMSE of 11.92, and MAE of 10.29, outperforming traditional models like Logistic Regression, HRFLM, and HRFDT. This underscores its effectiveness in delivering precise and reliable AQI predictions.

Deep Ensemble Learning Model for Diagnosis of Lung Diseases from Chest X -Ray Images

Objectives: This study aims to develop a robust medical recognition system using deep learning for the identification of various lung diseases, including COVID-19, pneumonia, lung opacity, and normal states, from chest X-ray images. The focus is on implementing ensemble fixed features learning methods to enhance diagnostic capabilities, contributing to the development of a cost-effective and reliable diagnostic tool for combating the global epidemic of lung disorders. Methods: The study utilizes a Kaggle dataset containing COVID-19 chest radiography images. Raw X-ray images undergo preprocessing for contrast enhancement and noise removal while addressing dataset imbalance through near-miss resampling. Ensemble learning techniques, including two and three-level methods, are employed to harness the strengths of individual base learners—VGG16, InceptionV3, and MobileNetV2. The model's performance is evaluated using metrics such as accuracy, recall, precision, and F1-score. For remote access, a user interface and a shared web link are developed using Python Gradio. Findings: In two-level ensembles, features from base learners are concatenated and classified using a support vector machine. Three-level ensembles use concatenated features classified by three machine learning classifiers, employing a majority voting system for the final prediction. The two-level method achieved 93% accuracy, precision, recall, and F1 score. The three-level ensemble model demonstrates superior performance, achieving 94% accuracy in detecting four lung diseases, namely COVID-19, pneumonia, lung opacity, and normal states. Novelty: This research contributes to the field by showcasing the efficacy of deep learning technology, particularly ensemble learning, in enhancing the detection of lung diseases from raw chest X-ray images. The model employs three modified and efficient pretrained networks for automatic feature extraction, eliminating the need for manual feature engineering. The developed model stands as a promising decision-support tool for healthcare professionals, particularly in low-resource environments. Keywords: Convolutional Neural Network (CNN), Deep Learning (DL), Transfer Learning (TL), Ensemble learning (EL), Fixed feature extraction, Chest X­rays (CXR), Lung diseases

An advanced accident avoidance system based on imbalance-control ensemble and deep learning fusion design

The concept of endorsing AI in embedded systems is growing in all sectors including the development of Accident Avoidance Systems. Although real-time road crash prediction is vital for enhancing road user safety, there has been limited focus on the analysis of real-time crash events within ensemble and deep learning fused systems. The main aim of this paper is to design an advanced Accident Avoidance System established on a deep learning and ensemble fusion strategy in order to acquire more performant crash predictions. As such, four highly optimized models for crash prediction have been designed based on the popular ensemble techniques: CatBoost, AdaBoost and Bagging and the deep learning CNN. Additionally, four categories of features, including driver inputs, vehicle kinematics, driver states and weather conditions, were measured during the execution of various driving tasks performed on a driving simulator. Moreover, given the infrequent nature of crash events, an imbalance-control procedure was adopted using the SMOTE and ADASYN techniques. The highest performances results have been acquired using CatBoost along with ADASYN on almost all the adopted metrics during the different weather conditions, and more than 50% of all crashes have occurred in rainy weather conditions, whereas 31% have been exhibited in fog patterns. The sensitivity analysis results indicate that the fusing all the acquired features has the highest impact on the prediction performance. To our knowledge, there has been a limited interest, if not at all, at adopting a fused ensemble deep learning system examining the real-time impact of the adopted features’ combinations on the prediction of road crashes while taking into account class imbalance. The findings provide new insights into crash prediction and emphasize the relevance of the explanatory features which can be endorsed in designing efficient Accident Avoidance Systems.

Classification of intrusion cyber‐attacks in smart power grids using deep ensemble learning with metaheuristic‐based optimization

AbstractThe most advanced power grid design, known as a ‘smart power grid’, integrates information and communication technology (ICT) with a conventional grid system to enable remote management of electricity distribution. The intelligent cyber‐physical architecture enables bidirectional, real‐time data sharing between electricity suppliers and consumers through smart meters and advanced metering infrastructure (AMI). Data protection issues, such as data tampering, firmware exploitation, and the leakage of sensitive information arise due to the smart power grid's substantial reliance on ICT. To maintain reliable and efficient power distribution, these issues must be identified and resolved quickly. Intrusion detection is essential for providing secure services and alerting system administrators in the case of adversary attacks. This paper proposes an intrusion classification scheme that identifies several types of cyber attacks on modern smart power grids. Grey‐Wolf metaheuristic optimization‐based feature selection is used to learn non‐linear, overlapping, and complex electrical grid properties. An extended deep‐stacked ensemble technique is advanced by putting predictions from weak learners (CNNs) into a meta‐learner (MLP). The outcomes of this approach are explained and confirmed using explainable AI (XAI). The publicly available dataset from Mississippi State University and Oak Ridge National Laboratory (MSU‐ORNL) is used to conduct experiments. The experimental results show that the proposed method achieved a peak accuracy of 96.6% while scrutinizing the original MSU‐ORNL data feature set and a maximum accuracy of 99% when analysing the selected feature set. Therefore, the proposed intrusion classification scheme may protect smart power grid systems against cyber security attacks.

An Ensemble of Transfer Learning based InceptionV3 and VGG16 Models for Paddy Leaf Disease Classification

Paddy is a crucial food crop providing essential nutrients and energy and serving more than half the global population. Diagnosing and preventing plant diseases at an early stage is crucial for the health and productivity of crops. Automated disease diagnosis eliminates the need for experts and delivers accurate outcomes. This research will diagnose paddy leaf diseases with Deep Learning technology. The diseases such as bacterial blight, blast, tungro, brown spot, and healthy leaf classes are diagnosed and classified in this study. The dataset contains 160 images from each class with 800 images. Our proposed model is an ensemble of transfer-learned InceptionV3 and VGG16 architectures, which utilizes the strength of individual models to improve overall performance. The use of transfer-learned ensemble deep learning architectures achieved impressive accuracy rates of 97.03%, 94.97%, and 98.87% for training, validation and testing respectively. The results indicating that model is not overfit and generalizes well to unseen data. The model's performance is evaluated with confusion matrix with the parameters like precision, recall, F1-score, and support. We also tested the model's performance against other proposed deep learning techniques with and without transfer learning techniques. Moreover, this research advances reliable automated disease detection systems, fostering sustainable agriculture and enhancing global food security.

SleepEGAN: A GAN-enhanced ensemble deep learning model for imbalanced classification of sleep stages

Deep neural networks have played an important role in the automatic classification of sleep stages due to their strong representation and in-model feature transformation abilities. However, class imbalance and individual heterogeneity which typically exist in raw EEG signals of sleep data can significantly affect the classification performance of any machine learning algorithms. To solve these two problems, this paper develops a generative adversarial network (GAN)-powered ensemble deep learning model, named SleepEGAN, for the imbalanced classification of sleep stages. To alleviate class imbalance, we propose a new GAN (called EGAN) architecture adapted to the features of EEG signals for data augmentation. The generated samples for minority classes are used in the training process. In addition, we design a cost-free ensemble learning strategy to reduce the model estimation variance caused by the heterogeneity between the validation and test sets, to enhance the accuracy and robustness of prediction performance. We show that the proposed method improves classification accuracy compared to several existing state-of-the-art methods. The overall classification accuracy and macro F1-score obtained by our SleepEGAN method on three public sleep datasets are: Sleep-EDF-39: 86.8% and 81.9%; Sleep-EDF-153: 83.8% and 78.7%; SHHS: 88.0% and 82.1%.

Comparing conceptual and super ensemble deep learning models for streamflow simulation in data-scarce catchments

Study RegionSore and Masha river catchments in Baro Akobo river basin: Ethiopia. Study FocusThis research addresses the challenges associated with conventional data-driven streamflow modelling, which often exhibits inconsistent performance across different variability states. To bridge this gap, we explore the efficiency of model ensembles, a popular hydrological approach that harnesses the strengths of multiple models while preserving the fundamental characteristics of the data. Specifically, we compare three modified super ensemble meta-learners with eight single and hybrid machine learning base learners. The study also incorporates a semi-distributed HBV-light conceptual hydrological model and utilizes a decade of remote sensing and ground hydro-meteorological daily time series data. Different remote sensing-based vegetation index data products are employed to simulate single-step daily streamflow. The Recursive Feature Elimination (RFE) algorithm is applied to extract influential input parameters. New Hydrological Insights for the RegionOur findings consistently demonstrate that the three super ensemble learners outperform both the eight base models and the HBV-light model. The top-ranked Extra Tree Regression Super Ensemble (ETRSE) model exhibits superior performance, surpassing the HBV-light model by 24% according to the R2 performance measure. This study highlights the positive impact of selecting influential input parameters on the overall performance of machine learning models, providing valuable insights for hydrological modelling in the region.

A secondary modal decomposition ensemble deep learning model for groundwater level prediction using multi-data

Groundwater level (GWL) prediction is important for ecological protection and resource utilization; it helps in formulating policies for artificial groundwater recharge, modifying the number of extraction wells, etc., and can support sustainable human development as well as inform water resource management decisions. However, climate change, anthropogenic impacts, and the complex coupling between surface water and groundwater increase the difficulty of predicting groundwater levels. The model proposed in this paper combines external data as well as multiple models. The method leverages long and short-term memory (LSTM) and convolutional neural network (CNN) models, combined with secondary modal decomposition and slime mould algorithm (SMA), together with an adaptive weight module (AWM). The study applies this method to predict GWL for three different hydrological conditions in China, specifically for the Jinan Baotu Spring, Heihu Spring, and Zhongtianshe watershed of Taihu Lake. A comparison of metrics such as mean absolute error and Nash efficiency coefficient for single and hybrid models shows that the model in this paper is more advantageous than the single model and other hybrid models. The interpretability of the model is enhanced by SHAP values that demonstrate the degree of contribution of the input variables. This paper uses SHAP analyses to identify the key drivers affecting groundwater levels. These factors must be detected in order to develop groundwater resource protection measures.

Towards Detection and Attribution of Cyber Attacks in IoT Enabled Cyber-Physical Systems

Internet of Things (IoT) enabled cyber physical systems such as Industrial equipment’s and operational IT to send and receive data over internet. This equipment’s will have sensors to sense equipment condition and report to centralized server using internet connection. Sometime some malicious users may attack or hack such sensors and then alter their data and this false data will be report to centralized server and false action will be taken. Due to false data many countries equipment and production system got failed and many algorithms was developed to detect attack, but all these algorithms suffer from data imbalance (one class my contains huge records (for example NORMAL records and other class like attack may contains few records which lead to imbalance problem and detection algorithms may failed to predict accurately). To deal with data imbalance, existing algorithms were using OVER and UNDER sampling which will generate new records for FEWER class only. Securing Internet of Things (IoT)-enabled cyberphysical systems (CPS) can be challenging, as security solutions developed for general information / operational technology (IT / OT) systems may not be as effective in a CPS setting. Thus, this paper presents a two-level ensemble attack detection and attribution framework designed for CPS, and more specifically in an industrial control system (ICS). At the first level, a decision tree combined with a novel ensemble deep representation learning model is developed for detecting attacks imbalanced ICS environments. At the second level, an ensemble deep neural network is designed for attack attribution.The connection between ICS or IIoT-based systems with public networks, however, increases their attack surfaces and risks of being targeted by cyber criminals.To overcome from this issue, we are introducing novel technique without using any under or oversampling algorithms. The proposed technique consists of 2 parts. Auto encoder: It will get trained on imbalanced dataset and then extract features from it and these extracted features will get trained with DECISION TREE algorithm to predict label for known or unknown attacks. Decision tree get trained on reduced number of features obtained from PCA (principal component analysis) algorithm. Deep Neural Network (DNN): In this level, DNN algorithm get trained on known and unknown attacks. Key Word: Cyber Physical Systems (CPS), Industrial Control System (ICS).

An ensemble deep learning model with empirical wavelet transform feature for oral cancer histopathological image classification

An ensemble deep learning model with empirical wavelet transform feature for oral cancer histopathological image classification

Melody generation based on deep ensemble learning using varying temporal context length

Melody generation based on deep ensemble learning using varying temporal context length