Deep Ensemble Research Articles

Deep ensemble of classifier for intrusion detection in WSN and improved attack mitigation process

SummaryAttacks have a significant negative impact on the wireless sensor network's network operations. They target to weaken the network layer, if they are not completely eradicated, the networks' ability to execute their desired function becomes collapsed. While intended to monitor such unexpected attacks, anomaly‐based intrusion detection systems (AIDS) have a high risk of false positives. This article intends to propose a Deep Ensemble Intrusion detection with a Self‐adaptive Sewing Training‐Based Optimization (DIED‐SASTO) model that includes the following steps: (a) preprocessing, (b) feature extraction, (c) optimal feature selection, and (d) detection. Initially, an improved class imbalance processing will take place to solve the imbalance problem during the preprocessing step. Subsequently, the features are extracted including higher‐order statistical features, improved entropy, and correlation‐based features during the feature extraction step. From the extracted feature set, as the curse of dimensionality becomes a serious challenge, it is significant to choose the optimal features. In the optimal feature selection process, a new Self‐adaptive Sewing Training‐Based Optimization (SASTO) is used. In addition, the intrusion detection will be employed based on the selected optimal features and trained using deep ensemble models like Deep maxout, Deep Belief Network (DBN), and Bidirectional Long short‐term memory (Bi‐LSTM). Once the presence of an attack is detected, it is mandatory to mitigate the attacker node form the network. For Attack Mitigation, improved entropy will be progressed. The effectiveness of the DIED‐SASTO is evaluated over conventional methods in terms of various metrics.

Research on a Deep Ensemble Learning Model for the Ultra-Short-Term Probabilistic Prediction of Wind Power

An accurate method for predicting wind power is crucial in effectively mitigating wind energy fluctuations and ensuring a stable power supply. Nevertheless, the inadequacy of the stability of wind energy severely hampers the consistent functioning of the power grid and the reliable provision of electricity. To enhance the accuracy of wind power forecasting, this paper proposes an ensemble model named the complete ensemble empirical mode decomposition with adaptive noise (CEEMDAN) and convolutional bidirectional long short-term memory (CNN-BiLSTM), which incorporates a data preprocessing technique, feature selection method, deep ensemble model, and adaptive control. Initially, CEEMDAN is utilized to decompose wind speed and power sequences and hence obtain decomposed subsequences for further analysis. Subsequently, the CNN is used to extract features from each subsequence, whereas each subsequence is processed by BiLSTM to obtain an ultra-short-term deterministic prediction model. Additionally, the adaptive kernel density estimation (AKDE) method is employed to estimate the probabilistic distribution of prediction error, enabling ultra-short-term probabilistic wind power prediction. Finally, based on real datasets, the reliability of the model in probabilistic prediction is verified through the evaluation metrics of multi-step prediction intervals (PIs).

Automated Classification of Agricultural Species through Parallel Artificial Multiple Intelligence System–Ensemble Deep Learning

The classification of certain agricultural species poses a formidable challenge due to their inherent resemblance and the absence of dependable visual discriminators. The accurate identification of these plants holds substantial importance in industries such as cosmetics, pharmaceuticals, and herbal medicine, where the optimization of essential compound yields and product quality is paramount. In response to this challenge, we have devised an automated classification system based on deep learning principles, designed to achieve precision and efficiency in species classification. Our approach leverages a diverse dataset encompassing various cultivars and employs the Parallel Artificial Multiple Intelligence System–Ensemble Deep Learning model (P-AMIS-E). This model integrates ensemble image segmentation techniques, including U-Net and Mask-R-CNN, alongside image augmentation and convolutional neural network (CNN) architectures such as SqueezeNet, ShuffleNetv2 1.0x, MobileNetV3, and InceptionV1. The culmination of these elements results in the P-AMIS-E model, enhanced by an Artificial Multiple Intelligence System (AMIS) for decision fusion, ultimately achieving an impressive accuracy rate of 98.41%. This accuracy notably surpasses the performance of existing methods, such as ResNet-101 and Xception, which attain 93.74% accuracy on the testing dataset. Moreover, when applied to an unseen dataset, the P-AMIS-E model demonstrates a substantial advantage, yielding accuracy rates ranging from 4.45% to 31.16% higher than those of the compared methods. It is worth highlighting that our heterogeneous ensemble approach consistently outperforms both single large models and homogeneous ensemble methods, achieving an average improvement of 13.45%. This paper provides a case study focused on the Centella Asiatica Urban (CAU) cultivar to exemplify the practical application of our approach. By integrating image segmentation, augmentation, and decision fusion, we have significantly enhanced accuracy and efficiency. This research holds theoretical implications for the advancement of deep learning techniques in image classification tasks while also offering practical benefits for industries reliant on precise species identification.

Innovative Use of Self-Attention-Based Ensemble Deep Learning for Suicide Risk Detection in Social Media Posts

Suicidal ideation constitutes a critical concern in mental health, adversely affecting individuals and society at large. The early detection of such ideation is vital for providing timely support to individuals and mitigating its societal impact. With social media serving as a platform for self-expression, it offers a rich source of data that can reveal early symptoms of mental health issues. This paper introduces an innovative ensemble learning method named LSTM-Attention-BiTCN, which fuses LSTM and BiTCN models with a self-attention mechanism to detect signs of suicidality in social media posts. Our LSTM-Attention-BiTCN model demonstrated superior performance in comparison to baseline models in the realm of classification and suicidal ideation detection, boasting an accuracy of 0.9405, a precision of 0.9385, a recall of 0.9424, and an F1-score of 0.9405. Our proposed model can aid healthcare professionals in recognizing suicidal tendencies among social media users accurately, thereby contributing to efforts to reduce suicide rates.

Ensemble Deep Learning Model to Predict Lymphovascular Invasion in Gastric Cancer.

Lymphovascular invasion (LVI) is one of the most important prognostic factors in gastric cancer as it indicates a higher likelihood of lymph node metastasis and poorer overall outcome for the patient. Despite its importance, the detection of LVI(+) in histopathology specimens of gastric cancer can be a challenging task for pathologists as invasion can be subtle and difficult to discern. Herein, we propose a deep learning-based LVI(+) detection method using H&E-stained whole-slide images. The ConViT model showed the best performance in terms of both AUROC and AURPC among the classification models (AUROC: 0.9796; AUPRC: 0.9648). The AUROC and AUPRC of YOLOX computed based on the augmented patch-level confidence score were slightly lower (AUROC: -0.0094; AUPRC: -0.0225) than those of the ConViT classification model. With weighted averaging of the patch-level confidence scores, the ensemble model exhibited the best AUROC, AUPRC, and F1 scores of 0.9880, 0.9769, and 0.9280, respectively. The proposed model is expected to contribute to precision medicine by potentially saving examination-related time and labor and reducing disagreements among pathologists.

Hierarchical Pruning of Deep Ensembles with Focal Diversity

Deep neural network ensembles combine the wisdom of multiple deep neural networks to improve the generalizability and robustness over individual networks. It has gained increasing popularity to study and apply deep ensemble techniques in the deep learning community. Some mission-critical applications utilize a large number of deep neural networks to form deep ensembles to achieve desired accuracy and resilience, which introduces high time and space costs for ensemble execution. However, it still remains a critical challenge whether a small subset of the entire deep ensemble can achieve the same or better generalizability and how to effectively identify these small deep ensembles for improving the space and time efficiency of ensemble execution. This article presents a novel deep ensemble pruning approach, which can efficiently identify smaller deep ensembles and provide higher ensemble accuracy than the entire deep ensemble of a large number of member networks. Our hierarchical ensemble pruning approach (HQ) leverages three novel ensemble pruning techniques. First, we show that the focal ensemble diversity metrics can accurately capture the complementary capacity of the member networks of an ensemble team, which can guide ensemble pruning. Second, we design a focal ensemble diversity based hierarchical pruning approach, which will iteratively find high quality deep ensembles with low cost and high accuracy. Third, we develop a focal diversity consensus method to integrate multiple focal diversity metrics to refine ensemble pruning results, where smaller deep ensembles can be effectively identified to offer high accuracy, high robustness and high ensemble execution efficiency. Evaluated using popular benchmark datasets, we demonstrate that the proposed hierarchical ensemble pruning approach can effectively identify high quality deep ensembles with better classification generalizability while being more time and space efficient in ensemble decision making. We have released the source codes on GitHub at https://github.com/git-disl/HQ-Ensemble .

Deep Ensemble Network for Sentiment Analysis in Bi-lingual Low-resource Languages

Sentiment analysis (SA) is the systematic identification, extraction, quantification, and study of affective states and subjective information using natural language processing. It is widely used for analyzing users’ feedback, such as reviews or social posts. Recently, SA has been one of the favorite research domains in NLP due to their wide range of applications, including E-commerce, healthcare, hotel business, and others. Many machine learning and deep learning-based models exist to predict the sentiment of the user’s post. However, the sentiment analysis in low-resource languages such as Kannada, Malayalam, Telugu, and Tamil received less attention due to language complexity and the low availability of required resources. This research fills the gap by proposing an ensemble model for predicting the sentiment of code-mixed Kannada and Malayalam languages. The ensemble of transformer-based models achieved a promising weighted F 1 -score of 0.66 for Kannada code-mixed language. In contrast, the ensemble model of the deep learning framework performed best by achieving a weighted F 1 -score of 0.72 for the Malayalam dataset, outperforming existing research.

Contactless face video based vital signs detection framework for continuous health monitoring using feature optimization and hybrid neural network.

Continuous monitoring of vital signs such as respiration and heart rate is essential to detect and predict conditions that may affect the patient's well-being. To detect these vital signs most medical systems use contact sensors. They are not feasible for long term monitoring and are not repeatable. Vital signs using facial video-noncontact monitoring are becoming increasingly important. Researchers in the last few years although considerable progress has been made, challenging datasets absence timing of assessment process and the technology still has some limitations such as time consuming nature and lack of computer portability. To solve those problems, we propose a contactless video based vital signs detection framework for continuous health monitoring using feature optimization and hybrid neural network. In the proposed technique, modified war strategy optimization algorithm is proposed to segment the face portion from the input video frames. Then, we utilize the known data acquisition models to extract vital signs from the segmented face portions are heart rate, blood pressure, respiratory rate and oxygen saturation. An improved neural network structure (Lifting Net) is further used to achieve the adaptive extraction of deep hidden features for specific signs, for realizing the high precision of human health monitoring. The Hughes effect or dimensionality issue affects detection accuracy in sign classification when there are fewer training instances relative to the number of spectral features. The problem can be overcome through feature optimization here Northern goshawk optimization algorithm is used to select optimal best features which reduces the data dimensionality issue. Furthermore, hybrid deep ensemble reinforcement learning classifier is proposed for the human vital sign detection and classification which ensures the early detection of patient abnormality. Finally, we validate our framework using benchmark video datasets such as TokyoTechrPPG, PURE and COHFACE. To proves the effectiveness of proposed technique using simulation results and comparative analysis.

MRI brain tumor detection using deep learning and machine learning approaches

The development of aberrant brain cells, some of which may become cancerous, is known as a brain tumour. The quality of life and life expectancy of patients are enhanced by early and timely illness identification and treatment plans. Magnetic Resonance Imaging (MRI) scans are the most common approach for finding brain tumors. However, the ability of radiologists and other clinical experts to identify, segment, and remove contaminated tumour regions from MRI images is a critical factor in a process that is iterative and labor-intensive and relies on those individuals' abilities in these areas. Concepts for image processing may envision the diverse human organ anatomical structures. It is difficult to find abnormal brain regions using simple imaging methods. Over the last several years, interest in the emerging machine learning field of “Deep Learning (DL)" has grown significantly. It was extensively used in numerous applications and shown to be an effective Machine Learning (ML) technique for many of the challenging issues. This research suggests a novel MRI brain tumour detection method based on DL and ML. Initially the MRI images are collected and preprocessed using Adaptive Contrast Enhancement Algorithm (ACEA) and median filter. Fuzzy c-means based segmentation is done to segment the preprocessed images. The features like energy, mean, entropy and contrast are extracted using Gray-level co-occurrence matrix (GLCM). The abnormal tissues are classified using the proposed Ensemble Deep Neural Support Vector Machine (EDN-SVM) classifier. The numerical findings reveal a better accuracy (97.93 %), sensitivity (92 %), and specificity (98 %) in recognizing aberrant and normal tissue from brain MRIimages, which supports the effectiveness of the approach that was recommended.

Regularized Multi-Decoder Ensemble for an Error-Aware Scene Representation Network.

Feature grid Scene Representation Networks (SRNs) have been applied to scientific data as compact functional surrogates for analysis and visualization. As SRNs are black-box lossy data representations, assessing the prediction quality is critical for scientific visualization applications to ensure that scientists can trust the information being visualized. Currently, existing architectures do not support inference time reconstruction quality assessment, as coordinate-level errors cannot be evaluated in the absence of ground truth data. By employing the uncertain neural network architecture in feature grid SRNs, we obtain prediction variances during inference time to facilitate confidence-aware data reconstruction. Specifically, we propose a parameter-efficient multi-decoder SRN (MDSRN) architecture consisting of a shared feature grid with multiple lightweight multi-layer perceptron decoders. MDSRN can generate a set of plausible predictions for a given input coordinate to compute the mean as the prediction of the multi-decoder ensemble and the variance as a confidence score. The coordinate-level variance can be rendered along with the data to inform the reconstruction quality, or be integrated into uncertainty-aware volume visualization algorithms. To prevent the misalignment between the quantified variance and the prediction quality, we propose a novel variance regularization loss for ensemble learning that promotes the Regularized multi-decoder SRN (RMDSRN) to obtain a more reliable variance that correlates closely to the true model error. We comprehensively evaluate the quality of variance quantification and data reconstruction of Monte Carlo Dropout (MCD), Mean Field Variational Inference (MFVI), Deep Ensemble (DE), and Predicting Variance (PV) in comparison with our proposed MDSRN and RMDSRN applied to state-of-the-art feature grid SRNs across diverse scalar field datasets. We demonstrate that RMDSRN attains the most accurate data reconstruction and competitive variance-error correlation among uncertain SRNs under the same neural network parameter budgets. Furthermore, we present an adaptation of uncertainty-aware volume rendering and shed light on the potential of incorporating uncertain predictions in improving the quality of volume rendering for uncertain SRNs. Through ablation studies on the regularization strength and decoder count, we show that MDSRN and RMDSRN are expected to perform sufficiently well with a default configuration without requiring customized hyperparameter settings for different datasets.

Uncertainty-Aware Deep Neural Representations for Visual Analysis of Vector Field Data.

The widespread use of Deep Neural Networks (DNNs) has recently resulted in their application to challenging scientific visualization tasks. While advanced DNNs demonstrate impressive generalization abilities, understanding factors like prediction quality, confidence, robustness, and uncertainty is crucial. These insights aid application scientists in making informed decisions. However, DNNs lack inherent mechanisms to measure prediction uncertainty, prompting the creation of distinct frameworks for constructing robust uncertainty-aware models tailored to various visualization tasks. In this work, we develop uncertainty-aware implicit neural representations to model steady-state vector fields effectively. We comprehensively evaluate the efficacy of two principled deep uncertainty estimation techniques: (1) Deep Ensemble and (2) Monte Carlo Dropout, aimed at enabling uncertainty-informed visual analysis of features within steady vector field data. Our detailed exploration using several vector data sets indicate that uncertaintyaware models generate informative visualization results of vector field features. Furthermore, incorporating prediction uncertainty improves the resilience and interpretability of our DNN model, rendering it applicable for the analysis of non-trivial vector field data sets.

Predicting and screening high-performance polyimide membranes using negative correlation based deep ensemble methods.

Polyimide polymer membranes have become critical materials in gas separation and storage applications due to their high selectivity and excellent permeability. However, with over 107 known types of polyimides, relying solely on experimental research means potential high-performance candidates are likely to be overlooked. This study employs a deep learning method optimized by negative correlation ensemble techniques to predict the gas permeability and selectivity of polyimide structures, enabling rapid and efficient material screening. We propose a deep neural network model based on negative correlation deep ensemble methods (DNN-NCL), using Morgan molecular fingerprints as input. The DNN-NCL model achieves an R2 value of approximately 0.95 on the test set, which is a 4% improvement over recent model performance, and effectively mitigates overfitting with a maximum discrepancy of less than 0.03 between the training and test sets. High-throughput screening of over 8 million hypothetical polymers identified hundreds of promising candidates for gas separation membranes, with 14 structures exceeding the Robeson upper bound for CO2/N2 separation. Visualization of high-throughput predictions shows that although the Robeson upper bound was never explicitly used as a model constraint, the majority of predictions are compressed below this limit, demonstrating the deep learning model's ability to reflect real-world physical conditions. Reverse analysis of model predictions using SHAP analysis achieved interpretability of the deep learning model's predictions and identified three key functional groups deemed important by the deep neural network for gas permeability: carbonyl, thiophene, and ester groups. This established a bridge between the structure and properties of polyimide materials. Additionally, we confirmed that two polyimide structures predicted by the model to have excellent CO2/N2 selectivity, namely 6-methylpyrimidin-5-amine and 1,4,5,6-tetrahydropyrimidin-2-amine, have been experimentally validated in previous studies. This research demonstrates the feasibility of using deep learning methods to explore the vast chemical space of polyimides, providing a powerful tool for discovering high-performance gas separation membranes.

Deep Ensemble Transformers for Dimensionality Reduction.

We propose deep ensemble transformers (DETs), a fast, scalable approach for dimensionality reduction problems. This method leverages the power of deep neural networks and employs cascade ensemble techniques as its fundamental feature extraction tool. To handle high-dimensional data, our approach employs a flexible number of intermediate layers sequentially. These layers progressively transform the input data into decision tree predictions. To further enhance prediction performance, the output from the final intermediate layer is fed through a feed-forward neural network architecture for final prediction. We derive an upper bound of the disparity between the generalization error and the empirical error and demonstrate that it converges to zero. This highlights the generalizability of our method to parameter estimation and feature selection problems. In our experimental evaluations, DETs outperform existing models in terms of prediction accuracy, representation learning ability, and computational time. Specifically, the method achieves over 95% accuracy in gene expression data and can be trained on average 50% faster than traditional artificial neural networks (ANNs).

Colorectal Cancer Classification based on Deep Ensemble Model with Self Adaptive Training Model

Colorectal Cancer Classification based on Deep Ensemble Model with Self Adaptive Training Model

Steel Surface Defect Recognition in Smart Manufacturing Using Deep Ensemble Transfer Learning-Based Techniques

Steel Surface Defect Recognition in Smart Manufacturing Using Deep Ensemble Transfer Learning-Based Techniques

A Framework for Early Detection of Acute Lymphoblastic Leukemia and Its Subtypes From Peripheral Blood Smear Images Using Deep Ensemble Learning Technique

A Framework for Early Detection of Acute Lymphoblastic Leukemia and Its Subtypes From Peripheral Blood Smear Images Using Deep Ensemble Learning Technique

A Novel Deep Ensemble Method for Selective Classification of Electrocardiograms.

Telehealth paradigms are essential for remotely managing patients with chronic conditions. To assist clinicians in handling the large volumes of data collected through these systems, clinical decision support systems (CDSSs) have been developed. However, the effectiveness of CDSSs depends on the quality of remotely recorded physiological data and the reliability of the algorithms used for processing this data. This study aims to reliably detect atrial fibrillation (AF) from short-term single-lead (STSL) electrocardiogram (ECG) recordings obtained in unsupervised telehealth environments. A novel deep ensemble-based method was developed for detecting AF from STSL ECG recordings. Following this, a postprocessing algorithm was created to assess uncertainty in classified STSL ECGs and to refrain from interpretation when confidence is low. The proposed method was validated through a 5-fold cross-validation on the Cardiology Challenge 2017 (CinC2017) dataset. The deep ensemble method achieved 83.5 ± 1.5% sensitivity, 98.4 ± 0.2% specificity, and an F 1-score of 0.847 ± 0.016in AF detection. Implementing the selective classification algorithm resulted in significant improvements, with sensitivity increasing to 92.8 ± 2.2%, specificity to 99.7 ± 0.0%, and an F 1-score of 0.919 ± 0.016. The proposed method demonstrates the feasibility of accurately detecting AF from STSL ECG recordings. The selective classification approach offers a substantial enhancement to automated ECG interpretation algorithms in telehealth solutions. These findings highlight the potential for improving the utility of telehealth systems by integrating advanced CDSSs capable of managing uncertainty and ensuring higher accuracy, thereby improving patient outcomes in remote healthcare settings.

Streamlined and Resource-Efficient Estimation of Epistemic Uncertainty in Deep Ensemble Classification Decision via Regression

Streamlined and Resource-Efficient Estimation of Epistemic Uncertainty in Deep Ensemble Classification Decision via Regression

IncepV3Dense: Deep Ensemble based Average Learning Strategy for identification of Micro-nutrient deficiency in banana crop

IncepV3Dense: Deep Ensemble based Average Learning Strategy for identification of Micro-nutrient deficiency in banana crop

Augmenting Internet of Medical Things Security: Deep Ensemble Integration and Methodological Fusion

Augmenting Internet of Medical Things Security: Deep Ensemble Integration and Methodological Fusion