Ensemble Machine Learning Research Articles

An Optimized Approach for Detection and Classification of Spam Email’s Using Ensemble Methods

AbstractSince the advent of email services, spam emails have been a major concern because users’ security depends on the classification of emails as ham or spam. It’s a malware attack that has been used for spear phishing, whaling, clone phishing, website forgery, and other harmful activities. However, various ensemble Machine Learning (ML) algorithms used for the detection and filtering of spam emails have been less explored. In this research, we offer a ML-based optimized algorithm for detecting spam emails that have been enhanced using Hyper-parameter tuning approaches. The proposed approach uses two feature extraction modules, namely Count-Vectorizer and TFIDF-Vectorizer that provide the most effective classification results when we apply them to three different publicly available email data sets: Ling Spam, UCI SMS Spam, and the Proposed dataset. Moreover, to extend the performance of classifiers we used various ML methods such as Naive Bayes (NB), Logistic Regression (LR), Extra Tree, Stochastic Gradient Descent (SGD), XG-Boost, Support Vector Machine (SVM), Random Forest (RF), Multi-layer Perception (MLP), and parameter optimization approaches such as Manual search, Random search, Grid search, and Genetic algorithm. For all three data sets, the SGD outperformed other algorithms. All of the other ensembles (Extra Tree, RF), linear models (LR, Linear-SVC), and MLP performed admirably, with relatively high precision, recall, accuracies, and F1-score.

Stacked ensemble machine learning approach for electroencephalography based major depressive disorder classification using temporal statistics

Major depressive disorder (MDD) is a serious and widespread mental health condition that remains challenging to diagnose accurately. Traditional psychological assessments, which can be subjective and sometimes unreliable, emphasize the need for more objective diagnostic tools. In this study, we present a machine learning (ML) model designed to diagnose depression by analysing statistical time-domain features extracted from Electroencephalography (EEG) data. The model is built using a stacked ensemble ML approach, incorporating nine-base estimators with various meta-classifiers. Through multiple trials, the model achieved an accuracy of 98.01%, with precision and recall rates of 97.78% and 96.61%, respectively with Adaptive Boosting (AdaBoost) as the meta-classifer. We also investigated the effects of data sampling and the number of base classifiers on the model’s performance. The findings demonstrate that the stacked ensemble approach significantly enhances the accuracy of diagnosing MDD and that the proposed model outperforms the methods used in previous studies. This model offers a promising tool for psychologists and medical professionals to diagnose depression more reliably, potentially leading to better treatment outcomes for those affected by the disorder.

Threshold-based inventory for flood susceptibility assessment of the world’s largest river island using multi-temporal SAR data and ensemble machine learning algorithms

Threshold-based inventory for flood susceptibility assessment of the world’s largest river island using multi-temporal SAR data and ensemble machine learning algorithms

Forecasting Blue and Green Water Footprint of Wheat Based on Single, Hybrid, and Stacking Ensemble Machine Learning Algorithms Under Diverse Agro-Climatic Conditions in Nile Delta, Egypt

The aim of this research is to develop and compare single, hybrid, and stacking ensemble machine learning models under spatial and temporal climate variations in the Nile Delta regarding the estimation of the blue and green water footprint (BWFP and GWFP) for wheat. Thus, four single machine learning models (XGB, RF, LASSO, and CatBoost) and eight hybrid machine learning models (XGB-RF, XGB-LASSO, XGB-CatBoost, RF-LASSO, CatBoost-LASSO, CatBoost-RF, XGB-RF-LASSO, and XGB-CatBoost-LASSO) were used, along with stacking ensembles, with five scenarios including climate and crop parameters and remote sensing-based indices. The highest R2 value for predicting wheat BWFP was achieved with XGB-LASSO under scenario 4 at 100%, while the minimum was 0.16 with LASSO under scenario 3 (remote sensing indices). To predict wheat GWFP, the highest R2 value of 100% was achieved with RF-LASSO across scenario 1 (all parameters), scenario 2 (climate parameters), scenario 4 (Peeff, Tmax, Tmin, and SA), and scenario 5 (Peeff and Tmax). The lowest value was recorded with LASSO and scenario 3. The use of individual and hybrid machine learning models showed high efficiency in predicting the blue and green water footprint of wheat, with high ratings according to statistical performance standards. However, the hybrid programs, whether binary or triple, outperformed both the single models and stacking ensemble.

Inverse Design of Singlet‐Fission Materials with Uncertainty‐Controlled Genetic Optimization

AbstractSinglet fission has shown potential for boosting the efficiency of solar cells, but the scarcity of suitable molecular materials hinders its implementation. We introduce an uncertainty‐controlled genetic algorithm (ucGA) based on ensemble machine learning predictions from different molecular representations that concurrently optimizes excited state energies, synthesizability, and exciton size for the discovery of singlet fission materials. The ucGA allows us to efficiently explore the chemical space spanned by the reFORMED fragment database, which consists of 45,000 cores and 5,000 substituents derived from crystallographic structures assembled in the FORMED repository. Running the ucGA in an exploitative setup performs local optimization on variations of known singlet fission scaffolds, such as acenes. In an explorative mode, hitherto unknown candidates displaying excellent excited state properties for singlet fission are generated. We suggest a class of heteroatom‐rich mesoionic compounds as acceptors for charge‐transfer mediated singlet fission. When included in larger donor‐acceptor systems, these units exhibit localization of the triplet state, favorable diradicaloid character and suitable triplet energies for exciton injection into semiconductor solar cells.

NIMG-01. RADIOMIC DATA ANALYSIS WITH ENSEMBLE MACHINE LEARNING TECHNIQUES PREDICT GRADE, HISTOPATHOLOGIC SUBTYPE, AND 1P/19Q CO-DELETION STATUS IN PATIENTS WITH LOW-GRADE GLIOMAS

Abstract Low-grade gliomas (LGGs) are a diverse group of primary brain tumors characterized by their variable prognosis and treatment response. Traditionally, the management of LGGs has relied on histopathological analysis, which sometimes offers limited insight into tumor behavior and treatment outcomes. Recent advances in radiomics have enabled the extraction of detailed quantitative features from routine imaging, providing a non-invasive method to assess tumor characteristics. These radiomic features can capture underlying tumor heterogeneity more comprehensively than conventional methods. Integrating radiomic data with ensemble machine learning techniques offers a promising approach to enhance the prediction of tumor grade, histopathologic subtype, and genetic alterations such as 1p/19q co-deletion status in LGGs, potentially leading to more tailored therapeutic strategies. We combined radiomic-genomic data from 159 magnetic resonance imaging (MRI) scans from the updated LGG-1p19qDeletion dataset (Erickson 2020) and 114 MRI scans from the ReMIND dataset (Juvekar 2024). Expert radiologists performed segmentations. A total of 2446 extracted radiomic features were initially extracted. Feature selection was performed using wrapper- and filter-based techniques with cross-validation. Prediction models were built using machine learning techniques, including random forest and extreme gradient boosting. Using Random Forest, we achieved high accuracy in predicting histopathologic subtype (88.14%, AUC 96.68%) with precision of 87.86% and Dice score of 87.88%. For grade prediction (accuracy 76.19%, AUC 91.84%), precision was 73.91%, and the Dice score was 77.27%. For 1p/19q co-deletion status (accuracy 75.61%, AUC 82.74%), precision was 77.78%, and the Dice score was 73.68%. Our study establishes a robust radiomic workflow to predict tumor grade, histopathologic subtype, and 1p/19q co-deletion status in low-grade gliomas, enhancing diagnostic accuracy and treatment personalization. This methodology can be applied clinically to improve the stratification of treatment plans for low-grade gliomas, potentially leading to effective, personalized therapy options. Using multicentric data potentially improved external validity, something which past research lacked. Future research should focus on validating and deploying this model using observational multicentric study designs.

Height prediction of individuals with osteogenesis imperfecta by machine learning

BackgroundOsteogenesis imperfecta (OI) is a genetic disorder characterized by low bone mass, bone fragility and short stature. There is a significant gap in knowledge regarding the growth patterns across different types of OI, and the prediction of height in individuals with OI was not adequately addressed. In this study, we described the growth patterns and predicted the height of individuals with OI employing multiple machine learning (ML) models. Accurate height prediction enables effective monitoring and facilitates the development of personalized intervention plans for managing OI.MethodThis study included cross-sectional data for 323 participants with OI, and the median height Z-score for OI types I, III and IV were − 0.62 (-5.93 ~ 3.24), -3.97 (-10.44 ~ -0.02) and − 1.64 (-6.67 ~ 2.44), respectively. Based on the cross-sectional data of participants, the height curves across different gender and OI types were plotted and compared. Subsequently, feature selection techniques, specifically the filter and wrapper methods, were employed to identify predictive factors for the height of participants. Finally, multiple machine learning (ML) models were constructed for height prediction, and the performance of each model was systematically evaluated.ResultsThe analysis of height curves revealed that male with OI are significantly taller than female with OI from the age of 14 (p = 0.045), individuals with OI type III are statistically shorter than those with OI types I and IV starting from 3 years old (p = 0.006), and those with OI type IV are statistically shorter than those with OI type I from the age of 10 (p = 0.028). The application of filter and wrapper methods identified gender (p = 0.001), age (p < 0.001), Sillence types (p = 0.007), weight Z-score (p < 0.001) and aBMD Z-score (p = 0.021) as significant predictive factors for height. The optimal performance of predictive models was registered by gradient boosting classifier (GB) (bias = 5.783, accuracy = 92.59%, R2 = 0.828), random forest (RF) (bias = 6.155, accuracy = 90.12%, R2 = 0.788), ensemble machine learning (EML) (bias = 6.250, accuracy = 91.36%, R2 = 0.825) and deep neuron networks (DNNs) (bias = 6.223, accuracy = 90.12%, R2 = 0.821).ConclusionThis study analyzed a large cohort of individuals with OI and provided detailed height patterns across different gender and OI types that are crucial for assessing overall growth. Gender, age, Sillence types, weight Z-score and aBMD Z-score were identified as predictive factors for height. The predictive models of GB, RF, EML and DNNs had higher accuracy to evaluate the height of individuals with OI. This study allows guardians and physicians to timely monitor the height parameters, and facilitate the creation of personalized intervention schedules tailored to the needs of individuals with OI.

Investigating the Performance of Data-Driven Ensemble Machine Learning Models in Preliminary Designing Multi-Stage Friction Pendulum Bearings

Over the last few decades, the field of base isolation systems has demonstrated enormous advancements through the innovation of novel systems, and there has been an enhancement in the performance of structures equipped with isolation under conditions of moderate and severe seismic activity. One of the most important systems is the multi-stage friction pendulum, which offers a spectrum of effective pendulums across different regimes, thereby achieving enhanced capability for energy dissipation. The difficulty in designing these bearings at the preliminary stage comes from the fairly long process of trials and errors in selecting the properties of each sliding surface so that the required effective period, effective damping, and displacement capacities are achieved. This study investigates the performance of various data-driven ensemble machine learning models in directly designing multi-stage friction pendulum bearings. Within the context of the analysis, the most recent generation of friction pendulum that has been introduced to the literature, the quintuple friction pendulum, is used as the investigation employs a case to assess the dependability of the design strategy and the efficacy of a multi-output machine learning techniques. In general, the findings of this study have shown that machine learning models provide an accurate way to directly design quintuple friction pendulum isolators and attain the necessary properties of the isolators. The practicality of this research lies in highlighting how machine learning can be applied to considerably expedite the design and optimization of multi-stage friction pendulum bearings and in defining the most suitable machine learning technique to adopt in such tasks.

Prediction of Strength and Failure Modes in Reinforced Concrete Beam–Column Joints using Ensemble Machine Learning Techniques

The beam-column joint is one of the important structural elements that control the structural behaviour during seismic excitation. Moreover, failure of any of these components may lead to the partial or overall collapse of the entire structure. In view of the safety concern, there is a need to evaluate the response properties, such as failure modes and ultimate shear capacity of the beam-column joints. Conventional methods to evaluate the shear capacity of beam-column joints are usually time-consuming. Therefore, in this study, an attempt has been made to evaluate the joint shear capacity and mode of failure of an exterior beam-column joint using ensembled machine learning (ML) approaches like Decision Tree, Random Forest, AdaBoost, CatBoost, and LightGBM. The present study highlights the potential use of the CatBoost model as a useful tool that can assist in predicting the shear strength and failure mode. The results of the study reveal that this model has performed well in predicting the load-carrying capacity and shear strength with high correlation coefficients of 0.9836 and 0.9978, respectively. Moreover, it has been observed that the prediction model provided by the CatBoost algorithm exhibits the highest accuracy for classification in the failure mode of beam-column joints.

Robust Health Condition Prediction of Helicopter Turboshaft Engines Using Ensemble Machine Learning Models

This paper presents a novel ensemble approach that combines multiple machine-learning algorithms to deliver robust predictions of helicopter turboshaft engine health status (nominal or faulty) using operational data. Engine health is evaluated through the torque margin, defined as the percentage difference between the measured and target torque values. A Gaussian process model is used to estimate the torque margin as a probability distribution function (PDF), and these predictions are incorporated as features into various machine-learning models. These models are then employed to perform binary classification, determining the engine's health state. To enhance performance, a reference set is defined for each unseen data point, allowing a comparison of the relative performances of the models, with the best performer selected for the final prediction. Our ensemble method achieves high accuracy in health classification while providing precise torque margin estimates. The results demonstrate that ensemble models offer superior generalization and reliability compared to individual machine-learning algorithms, especially when applied to complex, multivariate datasets like those from helicopter turboshaft engines.

Evidence of repeated zoonotic pathogen spillover events at ecological boundaries

Anthropogenic modifications to the landscape have altered several ecological processes worldwide, creating new ecological boundaries at the human/wildlife interface. Outbreaks of zoonotic pathogens often occur at these ecological boundaries, but the mechanisms behind new emergences remain drastically understudied. Here, we test for the influence of two types of ecosystem boundaries on spillover risk: (1) biotic transition zones such as species range edges and transitions between ecoregions and (2) land use transition zones where wild landscapes occur in close proximity to heavily impacted areas of high human population density. Using ebolavirus as a model system and an ensemble machine learning modeling framework, we investigated the role of likely reservoir (bats) and accidental host (primates) range edges and patterns of land use (defined using SEDAC categories) on past spillover events. Our results show that overlapping species range edges and heightened habitat diversity increase ebolavirus outbreaks risk. Moreover, we show that gradual transition zones, represent by high proportion of rangelands, acts as a buffer to reduces outbreak risks. With increasing landscape changes worldwide, we provide novel ecological and evolutionary insights into our understanding of zoonotic pathogen emergence and highlight the risk of aggressively developing ecological boundaries.

A Spatial Landslide Risk Assessment Based on Hazard, Vulnerability, Exposure, and Adaptive Capacity

Landslides threaten human life, property, and vital infrastructure in most mountainous regions. As climate change intensifies extreme weather patterns, the landslide risk is likely to increase, resulting in challenges for disaster management, sustainability development, and community resilience. This study presents a comprehensive framework for assessing landslide risk, integrating advanced machine learning models with the Iyengar–Sudarshan method. Our case study is Son La province, the Northwest region of Vietnam, with data collected from 1771 historical landslide occurrences and fifteen influencing factors for developing landslide susceptibility maps using advanced ensemble machine learning models. The Iyengar–Sudarshan method was applied to determine the weights for landslide exposure, vulnerability, and adaptive capacity indicators. The resulting landslide risk map shows that the highest-risk districts in Son La province are located in the central and northeastern regions, including Mai Son, Phu Yen, Thuan Chau, Yen Chau, Song Ma, and Bac Yen. These districts experience high landslide hazards, exposure, and vulnerability, often affecting densely populated urban and village areas with vulnerable populations, such as young children, the elderly, and working-age women. In contrast, due to minimal exposure, Quynh Nhai and Muong La districts have lower landslide risks. Despite having high exposure and vulnerability, Son La City is situated in a low-susceptibility zone with high adaptive capacity, resulting in a low landslide risk for this region. The proposed framework provides a reference tool for mitigating risk and enhancing strategic decision making in areas susceptible to landslides while advancing our understanding of landslide dynamics and fostering community resilience and long-term disaster prevention.

Hybrid Classifier for Detecting Zero-Day Attacks on IoT Networks

Recently, Internet of Things (IoT) networks have been exposed to many electronic attacks, giving rise to concerns about the security of these networks, where their weaknesses and gaps can be exploited to access or steal data. These networks are threatened by several cyberattacks, one of which is the zero-day distributed denial-of-service (DDoS) attack, which is considered one of the dangerous attacks targeting network security. As such, it is necessary to find smart solutions to address such attacks swiftly. To address these attacks, this research proposed a hybrid IDS to detect cyber-attacks on IoT networks via machine learning (ML) algorithms, namely, XGBoost, K-nearest neighbors, and stochastic gradient descent (SGD), while classifiers are combined via an ML ensemble. Grid search CV was used to find the best hyperparameters for each classifier at each classification stage. Random projection was used to select the relevant features for training the model. In the evaluation and performance testing phase of the model, two cybersecurity datasets (CIC-IDS2017 and CIC-DDoS2019) were used to test the efficiency of the model in detecting zero-day threats. The best results were obtained for the CIC-DDoS2019 dataset, where 20 features out of the total selection were used. The model was able to achieve an accuracy of 99.91% and an intrusion detection time of 0.22 seconds. The confusion matrix results also revealed a reduction in false alarms. The results and their comparison with those of recent relevant studies demonstrated the effectiveness of the hybrid model in securing IoT networks from zero-day attacks as well as its superiority in terms of accuracy and intrusion detection time. This study is an important step in enhancing security in the IoT environment by presenting a new hybrid model that is capable of dealing with zero-day attacks that are difficult to detect with traditional models.

Securing transactions: a hybrid dependable ensemble machine learning model using IHT-LR and grid search

Financial institutions and businesses face an ongoing challenge from fraudulent transactions, prompting the need for effective detection methods. Detecting credit card fraud is crucial for identifying and preventing unauthorized transactions. While credit card fraud incidents are relatively rare, they can result in substantial financial losses, particularly due to the high monetary value associated with fraudulent transactions. Timely detection of fraud enables investigators to take swift actions to mitigate further losses. However, the investigation process is often time-consuming, limiting the number of alerts that can be thoroughly examined each day. Therefore, the primary objective of a fraud detection model is to provide accurate alerts while minimizing false alarms and missed fraud cases. In this paper, we introduce a state-of-the-art hybrid ensemble (ENS) dependable machine learning (ML) model that intelligently combines multiple algorithms with proper weighted optimization using grid search, including decision tree (DT), random forest (RF), K-nearest neighbor (KNN), and multilayer perceptron (MLP), to enhance fraud identification. To address the data imbalance issue, we employ the instant hardness threshold (IHT) technique in conjunction with logistic regression (LR), surpassing conventional approaches. Our experiments are conducted on a publicly available credit card dataset comprising 284,807 transactions. The proposed model achieves impressive accuracy rates of 99.66%, 99.73%, 98.56%, and 99.79%, and a perfect 100% for the DT, RF, KNN, MLP and ENS models, respectively. The hybrid ensemble model outperforms existing works, establishing a new benchmark for detecting fraudulent transactions in high-frequency scenarios. The results highlight the effectiveness and reliability of our approach, demonstrating superior performance metrics and showcasing its exceptional potential for real-world fraud detection applications.

Enhancing Bipolar Disorder Detection using Heterogeneous Ensemble Machine Learning Techniques

This paper introduces a novel Heterogeneous Ensemble Machine Learning (HEML) approach designed to detect bipolar disorder, a significant healthcare challenge that demands precise and prompt diagnosis for effective treatment. The HEML method integrates multiple machines learning models, incorporating various physiological, behavioral, and contextual data from patients. By using a comprehensive feature selection technique, relevant features are extracted from each data source and utilized to train individual classifiers for detecting mental disorders. The classifiers include Adaboost, Decision Tree, K-nearest neighbors, Multilayer Perceptron, Random Forest, Relevance Vector Machine, and XGB, with Logistic Regression serving as the meta-model. This ensemble of classifiers enhances overall performance by capturing a wider range of characteristics related to mental disorders. The research evaluates the HEML method across three bipolar disorder datasets: Dataset1 (a multimodal dataset), Dataset2 (a sensor-based dataset), and Dataset3 (a real-time dataset). The HEML approach surpasses traditional methods, achieving superior accuracy rates of 95.21% with Dataset 1, 99.28% with Dataset 2, and 99% with Dataset 3. It outperforms individual models in detecting bipolar disorder, delivering the best Precision, Recall, F1 score, and Kappa Score. This comparative analysis advances the field of mental health diagnosis by leveraging the strengths of ensemble machine learning to improve accuracy and reliability in detection methods.

Accurate Prediction of Essential Proteins Using Ensemble Machine Learning

Abstract Essential proteins are crucial for biological processes and can be identified through both experimental and computational methods. While experimental approaches are highly accurate, they often demand extensive time and resources. To address these challenges, we present a computational ensemble learning framework designed to identify essential proteins more efficiently. Our method begins by using node2vec to transform proteins in the Protein-Protein Interaction (PPI) network into continuous, low-dimensional vectors. We also extract a range of features from protein sequences, including graph-theory-based, information-based, compositional, and physiochemical attributes. Additionally, we leverage deep learning techniques to analyze high-dimensional Position-Specific Scoring Matrices (PSSMs) and capture evolutionary information. We then combine these features for classification using various machine learning algorithms. To enhance performance, we integrate the outputs of these algorithms through ensemble methods such as voting, weighted averaging, and stacking. This approach effectively addresses data imbalances and improves both robustness and accuracy. Our ensemble learning framework achieved an AUC of 0.948 and an accuracy of 0.9252, outperforming other computational methods. These results demonstrate the effectiveness of our approach in accurately identifying essential proteins and highlight its superior feature extraction capabilities.

Analysis of ensemble machine learning classification comparison on the skin cancer MNIST dataset

This study aims to analyze the performance of various ensemble machine learning methods, such as Adaboost, Bagging, and Stacking, in the context of skin cancer classification using the skin cancer MNIST dataset. We also evaluate the impact of handling dataset imbalance on the classification model’s performance by applying imbalanced data methods such as random under sampling (RUS), random over sampling (ROS), synthetic minority over-sampling technique (SMOTE), and synthetic minority over-sampling technique with edited nearest neighbor (SMOTEENN). The research findings indicate that Adaboost is effective in addressing data imbalance, while imbalanced data methods can significantly improve accuracy. However, the selection of imbalanced data methods should be carefully tailored to the dataset characteristics and clinical objectives. In conclusion, addressing data imbalance can enhance skin cancer classification accuracy, with Adaboost being an exception that shows a decrease in accuracy after applying imbalanced data methods.

A spatially explicit multi-hazard framework for assessing flood, landslide, wildfire, and drought susceptibilities

Sustainable development goals require evaluating vulnerabilities and examining natural and climatic hazards for effective planning that reduces their impact on economic, social, and developmental efforts. Key hazards like floods, landslides, wildfires, and droughts have significantly affected terrestrial ecosystems and human societies, emphasizing the importance of comprehending these hazards. This study aimed to predict and spatially map multi-hazard, identifying historical and potential risks to inform sustainable development and construction programs that mitigate risks and promote resilience. A 34-year drought magnitude map was generated using long-term data, and ensemble and individual machine learning techniques were used to produce maps of flood, landslide, and wildfire hazards in a northwest region of Iran. Results demonstrated that ensemble learning models outperformed individual models, with the top-performing models being the weighted average (WA) of the two best models, random forest, extreme gradient boosting, WA models with over 80% accuracy, and WA incorporating all models, respectively. The CART model performed best among individual models. Variable importance analysis revealed that slope and precipitation were crucial factors for identifying high-hazard landslide areas, distance from waterways, vegetation cover, and topographic humidity index emerged as the most crucial factor for identifying flood hazard areas, while vegetation, rainfall, and proximity to roads significantly impacted wildfire hazard. The multi-hazard map produced by our study indicated that about 30% of the study area was highly and very highly susceptible to floods, landslides, wildfires, and droughts and the hazards mitigation efforts should be primarily directed to these specific portions of the study area. Our study underscored the importance of integrating long-term data and machine learning techniques in multi-hazard prediction and mapping, ultimately guiding mitigation efforts and promoting resilience in the face of natural and climatic hazards.

Correction: Uncertainty Assessment of Surface Water Salinity Using Standalone, Ensemble, and Deep Machine Learning Methods: A Case Study of Lake Urmia

Correction: Uncertainty Assessment of Surface Water Salinity Using Standalone, Ensemble, and Deep Machine Learning Methods: A Case Study of Lake Urmia

Retraction notice: Diagnosis of COVID-19 through blood sample using ensemble genetic algorithms and machine learning classifier

Retraction notice: Diagnosis of COVID-19 through blood sample using ensemble genetic algorithms and machine learning classifier