Ensemble Framework Research Articles

Information fusion-based Bayesian optimized heterogeneous deep ensemble model based on longitudinal neuroimaging data

The fusion of multimodal longitudinal data is difficult but crucial for enhancing the accuracy of deep learning models for disease identification and helps provide tailored and patient-centric decisions. This study explores the fusion of multimodal data to detect the progression of Alzheimer’s disease (AD) using ensemble learning. We propose a heterogeneous ensemble framework of Bayesian-optimized time-series deep learning models to identify progressive deterioration of brain damage. Experimental results show that the heterogeneous ensemble of three models with patient’s temporal data outperforms all other variants of ensemble models by achieving an average performance of 95% for accuracy. We also propose a novel explainability approach, which enables domain experts and practitioners to better comprehend the model's final decision. The visual explainability of infected brain regions and the model's robustness is evaluated by our two medical domain experts showing its promising use in real medical environment. To evaluate the model’s generalizability and robustness, our optimized model is tested on a dataset with different distribution. The experiments demonstrate that the proposed model, which was trained on ADNI data, exhibits reliable generalization to NACC data with an average precision of 90%, recall of 91%, F1-score of 89%, AUC of 88%, and accuracy of 88%.

Robust fracture intensity estimation from petrophysical logs and mud loss data: a multi-level ensemble modeling approach

This study presents a pioneering machine learning approach to continuously model fracture intensity in hydrocarbon reservoirs using solely conventional well logs and mud loss data. While machine learning has previously been applied to predict discrete fracture properties, this is among the first attempts to leverage well logs for continuous fracture intensity modeling leveraging advanced ensemble techniques. A multi-level stacked ensemble methodology systematically combines the strengths of diverse algorithms like gradient boosting, random forest and XGBoost through a tiered approach, enhancing predictive performance beyond individual models. Nine base machine learning algorithms generate initial fracture intensity predictions which are combined through linear regression meta-models and further stacked using ridge regression into an integrated super-learner model. This approach achieves significant improvements over individual base models, with the super-learner attaining a mean absolute error of 0.083 and R^2 of 0.980 on test data. By quantifying the crucial fracture intensity parameter continuously as a function of depth, this data-driven methodology enables more accurate reservoir characterization compared to traditional methods. The ability to forecast fracture intensity solely from conventional well logs opens new opportunities for rapid, low-cost quantification of this parameter along new wells without requiring advanced logging tools. When incorporated into reservoir simulators, these machine learning fracture intensity models can help optimize production strategies and recovery management. This systematic stacked ensemble framework advances continuous fracture intensity modeling exclusively from well logs, overcoming limitations of prior techniques. Novel insights gained via rigorous model evaluation deepen the understanding of naturally fractured reservoirs.

A deep learning‐based multivariate decomposition and ensemble framework for container throughput forecasting

AbstractTraditional linear models struggle to capture the intricate relationship between dynamic container throughput and its complex interplay with economic fluctuations. This study introduces a novel, deep learning‐based multivariate framework for precision in demanding landscapes. The framework consistently outperforms eight established benchmark models by employing vital economic indicators like GDP and port tonnage, identified through rigorous predictor importance analysis of an initial set of four variables, including imports and exports. Statistical significance is demonstrably achieved through the Diebold–Mariano and Wilcoxon rank‐sum tests. Utilizing the Port of Singapore as a case study, the framework offers agile adaptability for the ever‐evolving global supply chain. Comprehensive analyses ensure robustness, decoding intricate throughput dynamics. Incorporating noise‐assisted multivariate empirical mode decomposition (NA‐MEMD) for nonlinear decomposition and bidirectional long short‐term memory (BiLSTM) for time series dependencies, this innovative approach holds promise for revolutionizing container throughput forecasting and enhancing competitiveness in the global market through optimized resource allocation and streamlined operations.

An Ensemble Learning Framework for Anomaly Detection of Important Geographical Entities

Abstract. Due to the complex landforms and the limited resolution of remote sensing imagery, it is difficult to avoid the problem of incorrectly capturing geographical entities, such as buildings. Therefore, anomaly detection of important geographical entities is of great significance to ensure the authenticity and accuracy of geographical entity data. In this paper, we propose an ensemble learning framework for anomaly detection of geographical entity by aggregating the predicted labels generated by multiple deep learning models. In detail, we explore multiple change detection and semantic segmentation model and fully utilize the advantages of various deep learning neural network architectures. The proposed anomaly detection strategy of buildings has been performed on two benchmark datasets, including WHU Building change detection dataset and LEVIR building change detection dataset, the experimental results prove that the proposed method can achieve a more robust and better performance than using single change detection model in terms of quantitative performance and visual performance.

A comprehensive ensemble pruning framework based on dual-objective maximization trade-off

A comprehensive ensemble pruning framework based on dual-objective maximization trade-off

Traffic Flow Prediction by an Ensemble Framework with Data Denoising and Functional Principal Components Analysis

Accurate traffic flow data are important for congestion identification and real-time traffic control. Raw traffic flow data may be disturbed by different noises during the acquisition process, which leads to the degradation of model prediction performance. To address this problems, this paper proposes a data denoising method based on the improved complete ensemble empirical mode decomposition with adaptive noise (ICEEMDAN) combined with wavelet threshold denoising (WTD) to suppress potential outliers in the data, and then introduces functional principal component analysis to accomplish the traffic forecasting task. We use the proposed framework for real traffic flow prediction and validate the prediction performance of the proposed framework using root mean square error and mean absolute error. In the prediction performance comparison, five denoising methods are considered: empirical mode decomposition (EMD), ensemble empirical mode decomposition, complete ensemble empirical mode decomposition with adaptive noise, ICEEMDAN, and WTD. The prediction results show that the model combined with the denoising method outperforms the model without the denoising method. ICEEMDAN combined with WTD’s denoising method improves prediction performance more than other methods. Furthermore, the WTD method with dmey type achieves higher accuracy than other types.

Ensemble framework for concept drift detection and class imbalance in data streams

Ensemble framework for concept drift detection and class imbalance in data streams

A novel ensemble learning framework based on a genetic algorithm for the classification of pneumonia

Pneumonia is a disease that can be detected by the opacity changes in chest X-rays and can lead to fatal consequences. Medical image analysis has several challenges, such as limited labeled datasets, imbalanced class distribution, image noise, and overfitting, so individual Convolutional Neural Networks (CNNs) are insufficient to detect pneumonia accurately. Although ensemble CNN models have been used in previous studies, the literature lacks guidance on identifying the optimal CNN models and weight ratio to combine them. In this study, we propose a novel ensemble CNN framework to accurately detect pneumonia, with optimum weights set by a Genetic Algorithm (GA). Firstly, a noise outside the lung was removed, and the model performance was enhanced by performing lung segmentation on Chest X-ray. The performances of several CNN models were analyzed by hyperparameter optimization. The framework combines the three models that give the best accuracy and the two models that provide the lowest false-negative value with the ensemble method in the ratio of the most appropriate weights. The proposed framework provided the best performance on the public test dataset with an accuracy of 97.23% and an F1-score of 97.45% compared to state-of-the-art methods. The study's main contributions are determining suitable models and their optimal weights of the ensemble method based on the GA. The proposed framework enables a rapid and effective diagnostic process, less costly healthcare services, and more efficient use of resources. The demo-link is https://www.youtube.com/watch?v=KZ50K3HL70U.

Multi-perspective option price forecasting combining parametric and non-parametric pricing models with a new dynamic ensemble framework

This article introduces a dynamic ensemble framework that integrates parametric and non-parametric pricing models. Within this framework, we propose a time-varying parametric pricing model optimized using artificial intelligence algorithms. Additionally, we construct a non-parametric pricing model using a 2-dimensional convolutional neural network (2D-CNN) to capture the interactions among options, enhancing the existing non-parametric pricing model. Validation using China's SSE 50 ETF options trading data reveals several key findings: Firstly, the dynamic integration method proposed in this study not only improves prediction accuracy but also enhances stability. Secondly, previous parametric pricing models do not effectively utilize their pricing performance, while our proposed time-varying parametric pricing model significantly enhances accuracy. Lastly, the 2D-CNN model, which considers interactions among options trades, proves to be reasonable and effective, outperforming common non-parametric pricing models. The dynamic ensemble framework proposed in this study effectively combines the strengths of both parametric and non-parametric pricing models. This research serves as an important reference for risk managers, institutional investors, and other stakeholders. Furthermore, it provides valuable research ideas for future scholars in the field.

Small-Sample-Learning-Based Lithium-Ion Batteries Health Assessment: An Optimized Ensemble Framework

Small-Sample-Learning-Based Lithium-Ion Batteries Health Assessment: An Optimized Ensemble Framework

Multi-source deep domain adaptation ensemble framework for cross-dataset motor imagery EEG transfer learning

Objective. Electroencephalography (EEG) is an important kind of bioelectric signal for measuring physiological activities of the brain, and motor imagery (MI) EEG has significant clinical application prospects. Convolutional neural network has become a mainstream algorithm for MI EEG classification, however lack of subject-specific data considerably restricts its decoding accuracy and generalization performance. To address this challenge, a novel transfer learning (TL) framework using auxiliary dataset to improve the MI EEG classification performance of target subject is proposed in this paper. Approach. We developed a multi-source deep domain adaptation ensemble framework (MSDDAEF) for cross-dataset MI EEG decoding. The proposed MSDDAEF comprises three main components: model pre-training, deep domain adaptation, and multi-source ensemble. Moreover, for each component, different designs were examined to verify the robustness of MSDDAEF. Main results. Bidirectional validation experiments were performed on two large public MI EEG datasets (openBMI and GIST). The highest average classification accuracy of MSDDAEF reaches 74.28% when openBMI serves as target dataset and GIST serves as source dataset. While the highest average classification accuracy of MSDDAEF is 69.85% when GIST serves as target dataset and openBMI serves as source dataset. In addition, the classification performance of MSDDAEF surpasses several well-established studies and state-of-the-art algorithms. Significance. The results of this study show that cross-dataset TL is feasible for left/right-hand MI EEG decoding, and further indicate that MSDDAEF is a promising solution for addressing MI EEG cross-dataset variability.

Deep Leaning for Malnutrition Detection: Implications for Global Health

Malnutrition continues to be a major global health concern, impacting millions of people from various demographic backgrounds. To lessen the grave health effects of malnutrition, early detection and focused intervention are essential. Conventional evaluation techniques, like physical examinations, body weight calculations, and blood testing, are frequently expensive, time-consuming, and unpredictable. This research describes a revolutionary machine learning strategy for successfully diagnosing malnutrition. Specifically, we leverage transfer learning approaches from pre-trained models and employ NumPy for linear algebra, pandas for data processing, and matplotlib for interactive graph plotting. Function approximation, prediction robustness, and classification accuracy are all improved by the ensemble framework. After processing the data set, our model divides it into four groups: normal, stunting, obesity, and wasting. The main goal is to accurately classify malnutrition so that customized therapies may be implemented for each group. By doing this, we hope to lower the risk of death, health issues, and physical disabilities brought on by malnutrition. In summary, our suggested approach marks a substantial development in the efficient identification of malnutrition and the provision of tailored care. Accurate classification is ensured with the incorporation of ensemble learning algorithms, opening the door for focused treatments to enhance patient outcomes. Keywords—Analysis, Machine learning, Ensemble learning, NumPy, Matplotlib, Seaborn, Malnutrition.

Ensemble learning prediction framework for EGFR amplification status of glioma based on terahertz spectral features

Epidermal growth factor receptor (EGFR) plays a pivotal role in the initiation and progression of gliomas. In particular, in glioblastoma, EGFR amplification emerges as a catalyst for invasion, proliferation, and resistance to radiotherapy and chemotherapy. Current approaches are not capable of providing rapid diagnostic results of molecular pathology. In this study, we propose a terahertz spectroscopic approach for predicting the EGFR amplification status of gliomas for the first time. A machine learning model was constructed using the terahertz response of the measured glioma tissues, including the absorption coefficient, refractive index, and dielectric loss tangent. The novelty of our model is the integration of three classical base classifiers, i.e., support vector machine, random forest, and extreme gradient boosting. The ensemble learning method combines the advantages of various base classifiers, this model has more generalization ability. The effectiveness of the proposed method was validated by applying an individual test set. The optimal performance of the integrated algorithm was verified with an area under the curve (AUC) maximum of 85.8 %. This signifies a significant stride toward more effective and rapid diagnostic tools for guiding postoperative therapy in gliomas.

An Ensemble Framework for Projecting the Impact of Lymphatic Filariasis Interventions Across Sub-Saharan Africa at a Fine Spatial Scale.

Lymphatic filariasis (LF) is a neglected tropical disease targeted for elimination as a public health problem by 2030. Although mass treatments have led to huge reductions in LF prevalence, some countries or regions may find it difficult to achieve elimination by 2030 owing to various factors, including local differences in transmission. Subnational projections of intervention impact are a useful tool in understanding these dynamics, but correctly characterizing their uncertainty is challenging. We developed a computationally feasible framework for providing subnational projections for LF across 44 sub-Saharan African countries using ensemble models, guided by historical control data, to allow assessment of the role of subnational heterogeneities in global goal achievement. Projected scenarios include ongoing annual treatment from 2018 to 2030, enhanced coverage, and biannual treatment. Our projections suggest that progress is likely to continue well. However, highly endemic locations currently deploying strategies with the lower World Health Organization recommended coverage (65%) and frequency (annual) are expected to have slow decreases in prevalence. Increasing intervention frequency or coverage can accelerate progress by up to 5 or 6 years, respectively. While projections based on baseline data have limitations, our methodological advancements provide assessments of potential bottlenecks for the global goals for LF arising from subnational heterogeneities. In particular, areas with high baseline prevalence may face challenges in achieving the 2030 goals, extending the "tail" of interventions. Enhancing intervention frequency and/or coverage will accelerate progress. Our approach facilitates preimplementation assessments of the impact of local interventions and is applicable to other regions and neglected tropical diseases.

An automated learning model for twitter sentiment analysis using Ranger AdaBelief optimizer based Bidirectional Long Short Term Memory

AbstractSentiment analysis is an automated approach which is utilized in process of analysing textual data to describe public opinion. The sentiment analysis has major role in creating impact in the day‐to‐day life of individuals. However, a precise interpretation of text still relies as a major concern in classifying sentiment. So, this research introduced Bidirectional Long Short Term Memory with Ranger AdaBelief Optimizer (Bi‐LSTM RAO) to classify sentiment of tweets. Initially, data is obtained from Twitter API, Sentiment 140 and Stanford Sentiment Treebank‐2 (SST‐2). The raw data is pre‐processed and it is subjected to feature extraction which is performed using Bag of Words (BoW) and Term Frequency‐Inverse Document Frequency (TF‐IDF). The feature selection is performed using Gazelle Optimization Algorithm (GOA) which removes the irrelevant or redundant features that maximized model performance and classification is performed using Bi LSTM–RAO. The RAO optimizes the loss function of Bi‐LSTM model that maximized accuracy. The classification accuracy of proposed method for Twitter API, Sentiment 140 and SST 2 dataset is obtained as 909.44%, 99.71% and 99.86%, respectively. These obtained results are comparably higher than ensemble framework, Robustly Optimized BERT and Gated Recurrent Unit (RoBERTa‐GRU), Logistic Regression‐Long Short Term Memory (LR‐LSTM), Convolutional Bi‐LSTM, Sentiment and Context Aware Attention‐based Hybrid Deep Neural Network (SCA‐HDNN) and Stochastic Gradient Descent optimization based Stochastic Gate Neural Network (SGD‐SGNN).

Design an optimization based ensemble machine learning framework for detecting rice leaf diseases

Design an optimization based ensemble machine learning framework for detecting rice leaf diseases

RobEns: Robust Ensemble Adversarial Machine Learning Framework for Securing IoT Traffic.

Recently, Machine Learning (ML)-based solutions have been widely adopted to tackle the wide range of security challenges that have affected the progress of the Internet of Things (IoT) in various domains. Despite the reported promising results, the ML-based Intrusion Detection System (IDS) proved to be vulnerable to adversarial examples, which pose an increasing threat. In fact, attackers employ Adversarial Machine Learning (AML) to cause severe performance degradation and thereby evade detection systems. This promoted the need for reliable defense strategies to handle performance and ensure secure networks. This work introduces RobEns, a robust ensemble framework that aims at: (i) exploiting state-of-the-art ML-based models alongside ensemble models for IDSs in the IoT network; (ii) investigating the impact of evasion AML attacks against the provided models within a black-box scenario; and (iii) evaluating the robustness of the considered models after deploying relevant defense methods. In particular, four typical AML attacks are considered to investigate six ML-based IDSs using three benchmarking datasets. Moreover, multi-class classification scenarios are designed to assess the performance of each attack type. The experiments indicated a drastic drop in detection accuracy for some attempts. To harden the IDS even further, two defense mechanisms were derived from both data-based and model-based methods. Specifically, these methods relied on feature squeezing as well as adversarial training defense strategies. They yielded promising results, enhanced robustness, and maintained standard accuracy in the presence or absence of adversaries. The obtained results proved the efficiency of the proposed framework in robustifying IDS performance within the IoT context. In particular, the accuracy reached 100% for black-box attack scenarios while preserving the accuracy in the absence of attacks as well.

An effective ensemble classification framework for intrusion detection system using LSTM autoencoder

In intrusion detection, the curse of dimensionality and the trade-off between maintaining a low false alarm rate and achieving a high detection rate are significant challenges. This research suggests a unique strategy based on dimensionality reduction methods to improve the performance of network intrusion detection systems (NIDS). Compressing high-dimensional network traffic data using a Long Short-Term Memory Autoencoder (LSTMAE) allows the reduced characteristics to be submitted to a classifier to identify anomalies that may indicate an attack. Using standard datasets, including Network Security Laboratory - Knowledge Discovery in Datasets (NSL-KDD), UNSW-NB15, and Canadian Institute for Cyber Security - Intrusion Detection Systems (CICIDS2017), the proposed model is tested with classifiers like Random Forest (RF) and LightGBM (Light Gradient Boosting Machines). It is hoped that by adopting this method, NIDS response times may be improved while costs associated with storing and processing data are minimized. Precision, recall, F-score, accuracy, detection rate (DR), and false alarm rate (FAR) are only a few of the performance measures used to assess the quality of the suggested models. The experimental findings show that the proposed LSTMAE model reduces prediction errors more effectively than classic machine learning techniques such as Random Forest (RF), Gradient Boosting (GB), Support Vector Machines (SVM), Deep Belief Networks (DBN), Deep Neural Networks (DNN), Autoencoder (AE), and Long Short-Term Memory (LSTM). The results also show that the proposed solution outperforms the state-of-the-art methods of detection accuracy and computing complexity using accuracy, precision, recall, F1_Score, detection rate, and FAR.

Advanced forecasting of COVID-19 epidemic: Leveraging ensemble models, advanced optimization, and decomposition techniques

In the global effort to address the outbreak of the new coronavirus pneumonia (COVID-19) pandemic, accurate forecasting of epidemic patterns has become crucial for implementing successful interventions aimed at preventing and controlling the spread of the disease. The correct prediction of the course of COVID-19 outbreaks is a complex and challenging task, mainly because of the significant volatility in the data series related to COVID-19. Previous studies have been limited by the exclusive use of individual forecasting techniques in epidemic modeling, disregarding the integration of diverse prediction procedures. The lack of attention to detail in this situation can yield worse-than-ideal results. Consequently, this study introduces a novel ensemble framework that integrates three machine learning methods (kernel ridge regression (KRidge), Deep random vector functional link (dRVFL), and ridge regression) within a linear relationship (L-KRidge-dRVFL-Ridge). The optimization of this framework is accomplished through a distinctive approach, specifically adaptive differential evolution and particle swarm optimization (A-DEPSO). Moreover, an effective decomposition method, known as time-varying filter empirical mode decomposition (TVF-EMD), is employed to decompose the input variables. A feature selection technique, specifically using the light gradient boosting machine (LGBM), is also implemented to extract the most influential input variables. The daily datasets of COVID-19 collected from two countries, namely Italy and Poland, were used as the experimental examples. Additionally, all models are implemented to forecast COVID-19 at two-time horizons: 10- and 14-day ahead (t+10 and t+14). According to the results, the proposed model can yield higher correlation coefficient (R) for both case studies: Italy (t+10 = 0.965, t+14 = 0.961) and Poland (t+10 = 0.952, t+14 = 0.940) than the other models. The experimental results demonstrate that the model suggested in this paper has outstanding results in various kinds of complex epidemic prediction situations. The proposed ensemble model demonstrates exceptional accuracy and resilience, outperforming all similar models in terms of efficacy.

Online prediction of ship maneuvering motions based on adaptive weighted ensemble learning under dynamic changes

Dynamic changes in ship maneuverability challenge the accuracy and effectiveness of ship maneuvering models. This paper proposes an online prediction method based on the adaptive weighted ensemble learning framework, which can adaptively update the model according to changes in maneuverability, especially for reoccurring changes. The method contains two main mechanisms: the change monitoring mechanism and the adaptive weighting mechanism. The former identifies the change in ship dynamics and decides when to incorporate a new base model; the latter adjusts the weights of the base models to align with current scenarios, thus ensuring the predictive accuracy. To assess the method’s effectiveness under varying ship dynamics, the online prediction of ship maneuvering motions under speed-induced dynamic changes is investigated. Compared with the offline model, the result demonstrates the superiority of the adaptive weighted ensemble model. The proposed method can consistently provide accurate predictions in the scenarios with reoccurring changes, and can also enhance the model capability by adjusting weights to cope with some unencountered changes.