Ensemble Machine Learning Techniques Research Articles

A Novel Approach for Enhanced Feature Selection Over Retails Sales Data Using Ensemble Machine Learning Technique

A Novel Approach for Enhanced Feature Selection Over Retails Sales Data Using Ensemble Machine Learning Technique

Application of ensemble machine learning techniques to the diagnosis of the combustion in a gas turbine

Application of ensemble machine learning techniques to the diagnosis of the combustion in a gas turbine

Reconstructing Balloon‐Observed Gravity Wave Momentum Fluxes Using Machine Learning and Input From ERA5

AbstractGlobal atmospheric models rely on parameterizations to capture the effects of gravity waves (GWs) on middle atmosphere circulation. As they propagate upwards from the troposphere, the momentum fluxes associated with these waves represent a crucial yet insufficiently constrained component. The present study employs three tree‐based ensemble machine learning (ML) techniques to probe the relationship between large‐scale flow and small‐scale GWs within the tropical lower stratosphere. The measurements collected by eight superpressure balloons from the Strateole 2 campaign, comprising a cumulative observation period of 680 days, provide valuable estimates of the gravity wave momentum fluxes (GWMFs). Multiple explanatory variables, including total precipitation, wind, and temperature, were interpolated from the ERA5 reanalysis at each balloon's location. The ML methods are trained on data from seven balloons and subsequently utilized to estimate reference GWMFs of the remaining balloon. We observed that parts of the GW signal are successfully reconstructed, with correlations typically around 0.54 and exceeding 0.70 for certain balloons. The models show significantly different performances from one balloon to another, whereas they show rather comparable performances for any given balloon. In other words, limitations from training data are a stronger constraint than the choice of the ML method. The most informative inputs generally include precipitation and winds near the balloons' level. However, different models highlight different informative variables, making physical interpretation uncertain. This study also discusses potential limitations, including the intermittent nature of GWMFs and data scarcity, providing insights into the challenges and opportunities for advancing our understanding of these atmospheric phenomena.

Software development effort estimation using boosting algorithms and automatic tuning of hyperparameters with Optuna

AbstractConsidering the increasing need for software projects, estimating software development efforts is essential and can lead to improved project delivery quality. Machine learning methods are widely used to improve the accuracy of estimation. The boosting method is an ensemble machine learning technique less used in this field. In this research, five boosting algorithms including Adaboost, Gradient boosting, XGBoost, LightGBM, and CatBoost were implemented with the hyperparameter tuning framework Optuna on the ISBSG database. The Optuna is a next‐generation optimization method for automatically tuning hyperparameters of algorithms. Six evaluation criteria MMRE, MdMRE, MAE, MSE, Pred(0.25), and SA were used to evaluate the findings. The results show that the hyperparameter automatic tuning by Optuna increases the accuracy of prediction provided by all five models. When the Catboost algorithm uses Optuna to tune its hyperparameters has made the best prediction among the five algorithms studied in this research. Using Optuna, compared to the case where the algorithm uses its default settings, the highest percentage of prediction improvement was observed in the XGBoost algorithm (except for the SA criterion). Based on the criteria of MMRE, Pred(0.25), and SA, this study has a better prediction than some relatively similar articles.

Enhancing Truthfulness Assessment of News Articles Using Ensemble Machine Learning Techniques

Enhancing Truthfulness Assessment of News Articles Using Ensemble Machine Learning Techniques

AI-powered in the digital age: Ensemble innovation personalizes the food recommendations

This study proposes and evaluates a novel approach utilizing ensemble machine learning techniques for personalized meal services to address a critical gap in understanding AI-powered decision-making within the food delivery and restaurant industry. We draw inspiration from diverse fields, including non-traditional simulation methodologies and open innovation dynamics, to create a framework that leverages the combined strengths of individual algorithms. Three machine learning algorithms – decision trees, logistic regression, and naïve Bayes – are rigorously evaluated for their efficacy in classifying and assigning algorithms within an ensemble model for a new service. A simulated dataset, informed by expert tagging, is the training ground, ensuring practical relevance. We employ the voting probability metric on a held-out test set to provide a robust measure of accuracy in this critical task. Our findings reveal the significant potential of AI-powered personalized meal services. Ensemble models demonstrate high accuracy, showcasing the collaboration of combining individual algorithms. This originality lies in applying ensemble techniques to a business case with far-reaching implications for management and societal well-being. Beyond technical success, we explore this technology’s broader impact. AI-powered food recommendations can enhance accessibility for individuals with dietary needs, promote healthier lifestyles through nutritious meal suggestions, and generate new job opportunities. Acknowledging limitations and future research avenues, we invite further exploration of diverse machine learning algorithms and broader applications across various domains.

Automated Classification of COVID-19 Chest X-ray Images Using Ensemble Machine Learning Methods

This study delves into the efficacy of ensemble machine learning techniques for classifying chest X-ray images into three distinct categories: Normal, COVID-19, and Lung Opacity. Employing the Random Forest Classifier and a rigorous k-5 cross-validation framework, we aimed to enhance diagnostic accuracy for one of the most urgent medical challenges today—rapid and reliable COVID-19 detection. The analysis revealed an average accuracy of 51%, with varying precision and recall across different folds. The F1-score remained consistently around 35%, indicating a need for improved balance between precision and recall. Visualizations such as performance metric trends and a confusion matrix provided further insight into the classifier's performance, highlighting a notable degree of misclassification. Despite moderate success in the automated classification of the images, our research illustrates the complexity of applying machine learning to medical imaging, especially in differentiating between diseases with overlapping radiographic features. The study’s findings emphasize the potential of machine learning models to support diagnostic processes and suggest the necessity of advanced pre-processing techniques and extended datasets for enhanced model training. The research contributes to the growing body of knowledge in computational diagnostics and underscores the importance of developing robust, accurate machine learning tools to aid in the global healthcare crisis precipitated by the pandemic.

Gait alterations associated with worsening knee pain and physical function: a machine-learning approach with wearable sensors in the Multicenter Osteoarthritis Study.

The objective of this study is to identify gait alterations related to worsening knee pain, and to worsening physical function, using machine learning approaches applied to wearable-sensor derived data from a large observational cohort. Participants in the Multicenter Osteoarthritis Study (MOST) completed a 20-meter walk test wearing inertial sensors on their lower back and ankles. Parameters describing spatiotemporal features of gait were extracted from this data. We used an ensemble machine learning technique ("super learning") to optimally discriminate between those with and without worsening physical function and, separately, those with and without worsening pain over 2 years. We then used log-binomial regression to evaluate associations of the top ten influential variables selected with super-learning with each outcome. We also assessed whether the relation of altered gait with worsening function was mediated by changes in pain. Of 2324 participants, 29% and 24% had worsening knee pain and function over 2-years, respectively. From the super-learner, several gait parameters were found to be influential for worsening pain and for worsening function. After adjusting for confounders, greater gait asymmetry, longer average step length, and lower dominant frequency were associated with worsening pain, and lower cadence was associated with worsening function. Worsening pain partially mediated the association of cadence with function. We identified gait alterations associated with worsening knee pain and those associated with worsening physical function. These alterations could be assessed with wearable sensors in clinical settings. Further research should determine whether they might be therapeutic targets to prevent worsening pain and worsening function.

Predicting Road Traffic Collisions Using a Two-Layer Ensemble Machine Learning Algorithm

Road traffic collisions are among the world’s critical issues, causing many casualties, deaths, and economic losses, with a disproportionate burden falling on developing countries. Existing research has been conducted to analyze this situation using different approaches and techniques at different stretches and intersections. In this paper, we propose a two-layer ensemble machine learning (ML) technique to assess and predict road traffic collisions using data from a driving simulator. The first (base) layer integrates supervised learning techniques, namely k- Nearest Neighbors (k-NN), AdaBoost, Naive Bayes (NB), and Decision Trees (DT). The second layer predicts road collisions by combining the base layer outputs by employing the stacking ensemble method, using logistic regression as a meta-classifier. In addition, the synthetic minority oversampling technique (SMOTE) was performed to handle the data imbalance before training the model. To simplify the model, the particle swarm optimization (PSO) algorithm was used to select the most important features in our dataset. The proposed two-layer ensemble model had the best outcomes with an accuracy of 88%, an F1 score of 83%, and an AUC of 86% as compared with k-NN, DT, NB, and AdaBoost. The proposed two-layer ensemble model can be used in the future for theoretical as well as practical applications, such as road safety management for improving existing conditions of the road network and formulating traffic safety policies based on evidence.

Stacked ensemble modeling for improved tuberculosis treatment outcome prediction in pediatric cases

SummaryThe promising results of ML (machine learning) methods in various disciplines have led to the frequent use of these methods in health fields such as disease diagnosis, personalized medicine, medical image‐based diagnosis, and predicting the number of deaths and cases in a pandemic. However, a neglected area in the field of healthcare is the lack of study with ML to predict treatment outcomes for tuberculosis (TB) patients, particularly children experiencing failed treatment. This need has become more apparent as the coronavirus pandemic has reversed the gains of health institutions with TB disease, especially in children. Therefore, this article conducted a study using the stacked ensemble ML method to early predict the risk for children experiencing a failed treatment outcome of TB. To fulfill this need and determine the most appropriate technique, a two‐stage methodology was followed in this work. First, predictions were obtained by combining the information gain feature selection (IGFS) approach with a variety of single‐based ML algorithms, including logistic regression (LR), deep belief neural networks (DBN), random forest (RF), and decision tree (DT). Second, the proposed method, which includes a stacked ensemble ML technique, was used. The latter model uses LR as a meta‐learner and the aforementioned single‐based ML algorithms (DBN, LR, RF, and DT). The performance results of ML models used in the two stages were compared, and the proposed model which is the combination of the stack‐based ensemble learning model and the IGFS technique provided better ROC curves, accuracy, precision, and recall results.

Application of machine learning boosting and bagging methods to predict compressive and flexural strength of marble cement mortar

Compressive (CS) and flexural strength (FS) of sustainable mortar made from waste materials were estimated using machine learning (ML) tools. Ensemble ML techniques, including extreme gradient boosting (XGB) and bagging regressor (BR), were utilized to accomplish the study goals. The outcomes of the estimation models were compared with target values, and statistical checks were performed to assess and compare the models. Additionally, in order to determine the influence of mortar constituents on CS and FS estimation of mortar, the Shapley Additive exPlanations (SHAP) method was implemented. XGB exhibited superior predictive capabilities for the CS and FS of mortar compared to the BR approach, as evidenced by R2 values of 0.98 and 0.94 for the CS-XGB and FS-XGB models, respectively. Statistical validation checks provided additional evidence that the ML models were effective in estimating the strength of sustainable mortar. The results of the SHAP study showed that the most promising components that contributed favorably to strength were testing age, fly ash, and rice husk ash; however, waste-derived cement had a negative impact. Establishing models that can determine the CS and FS of mortar for different values of the input parameters could save time and money compared to doing the tests in a lab.

Removal of Ni(II) ions from wastewater by ion exchange resin: Process optimization using response surface methodology and ensemble machine learning techniques

This study investigates the removal of nickel ions from aqueous solutions using Amberlite IR120 Na resin. Response surface methodology (RSM) and Ensemble machine learning models were employed to predict and evaluate the adsorption capacity of the resin. The effects of the initial concentration of nickel ions (10–90 mg/L), resin dose (0.1–0.7 g/L), initial pH (3−9), and temperature (10–40 ºC) were analyzed as independent variables. In order to assess the efficiency of Ensemble models, five methods, including random forest, extra tree, AdaBoost, gradient boosting, and XGboost models, were employed. The results demonstrated that the XGboost model predicts the adsorption capacity of the resin for Ni(II) with very high accuracy in the training phase (R2 = 1) and the test phase (R2 =0.96). The maximum adsorption capacity of the resin in the optimization step was achieved for CCD (initial Ni(II) concentration of 45.69 mg/L, resin dose of 0.1 g/L, pH of 6.64, and temperature of 39.68 ºC) and Bayesian optimization (initial Ni(II) concentration of 54.78 mg/L, resin dose of 0.11 g/L, pH of 4.72, and temperature of 39.02 ºC) techniques by 142.67 mg/g and 139.60 mg/g, respectively. Moreover, the Bayesian optimization method resulted in a lower error (0.63 mg/g) than the CCD model (1.53 mg/g). Based on statistical measures, it can be concluded that XGBoost outperforms CCD.

Performance Comparison of different Disease Detection using Stacked Ensemble Learning Model

Malignancy risks and genetic disorders have long been challenging due to procedures that lack precision and predictability, thereby complicating the precise identification of diseases and their root causes. Machine learning classifiers have emerged as more suitable and effective tools. Various machine learning classifiers have been utilized to examine different genetic disorders, and the results from these classifiers have been further compared to determine their superiority. In this study, a variety of classifiers, including the SVM, KNN, decision tree, random forest, and logistic regression algorithms, are examined. These classifiers utilize specific training variables to analyze how input values correspond to the respective class. After successfully implementing each classifier, we proceeded to employ Stacking, an ensemble machine learning technique that aggregates predictions from individual classifiers on the same dataset. Four datasets, including the breast cancer, diabetes, Parkinson’s, and genomic datasets, were successfully implemented using the aforementioned methods, and the results obtained showed how the input values correspond to the class using a few training variables. SVM classifier was shown to be the most effective of the five described classifiers, having the highest accuracy in most of the cases. It provided accuracies of 97.43%, 97.46%, 97.45%, and 97.44% for each of the genome cancer, diabetes, Parkinson’s, and breast cancer datasets. The KNN and Random Forest models also came out to be very effective, with accuracy around 95% and 91%, respectively, for various disease datasets. The Logistic Regression and Decision Tree models also worked well. However, the ensemble method of Stacking proved to be highly efficient above all other base models and generated accuracies above 97.5% for all the aforementioned diseases.

Enhancing software defect prediction: a framework with improved feature selection and ensemble machine learning

Effective software defect prediction is a crucial aspect of software quality assurance, enabling the identification of defective modules before the testing phase. This study aims to propose a comprehensive five-stage framework for software defect prediction, addressing the current challenges in the field. The first stage involves selecting a cleaned version of NASA’s defect datasets, including CM1, JM1, MC2, MW1, PC1, PC3, and PC4, ensuring the data’s integrity. In the second stage, a feature selection technique based on the genetic algorithm is applied to identify the optimal subset of features. In the third stage, three heterogeneous binary classifiers, namely random forest, support vector machine, and naïve Bayes, are implemented as base classifiers. Through iterative tuning, the classifiers are optimized to achieve the highest level of accuracy individually. In the fourth stage, an ensemble machine-learning technique known as voting is applied as a master classifier, leveraging the collective decision-making power of the base classifiers. The final stage evaluates the performance of the proposed framework using five widely recognized performance evaluation measures: precision, recall, accuracy, F-measure, and area under the curve. Experimental results demonstrate that the proposed framework outperforms state-of-the-art ensemble and base classifiers employed in software defect prediction and achieves a maximum accuracy of 95.1%, showing its effectiveness in accurately identifying software defects. The framework also evaluates its efficiency by calculating execution times. Notably, it exhibits enhanced efficiency, significantly reducing the execution times during the training and testing phases by an average of 51.52% and 52.31%, respectively. This reduction contributes to a more computationally economical solution for accurate software defect prediction.

Differential diagnosis of erythemato-squamous diseases using a hybrid ensemble machine learning technique

Erythemato-squamous Diseases (ESD) encompass a group of common skin conditions, including psoriasis, seborrheic dermatitis, lichen planus, pityriasis rosea, chronic dermatitis, and pityriasis rubra pilaris. These dermatological conditions affect a significant portion of the population and present a current challenge for accurate diagnosis and classification. Traditional classification methods struggle due to shared characteristics among these diseases. Machine Learning offers a valuable tool for aiding clinical decision-making in ESD classification. In this study, we leverage the UC Irvine (UCI) dermatology dataset by applying necessary preprocessing steps to handle missing data. We conduct a comparative analysis of two feature selection methods: One-way ANOVA and Chi-square test. To enhance the model’s performance, we employ hyper-parameter tuning through GridSearchCV. The training process encompasses various algorithms, including Support Vector Machine (SVM), Logistic Regression, k-Nearest Neighbors (kNN), and Decision Trees. The culmination of our work is a hybrid ensemble machine learning model that combines the strengths of the trained classifiers. This ensemble classifier achieves an impressive accuracy of 98.9% when validated using a 10-fold cross-validation approach.

Accelerating Elastic Property Prediction in Fe-C Alloys through Coupling of Molecular Dynamics and Machine Learning.

The scarcity of high-quality data presents a major challenge to the prediction of material properties using machine learning (ML) models. Obtaining material property data from experiments is economically cost-prohibitive, if not impossible. In this work, we address this challenge by generating an extensive material property dataset comprising thousands of data points pertaining to the elastic properties of Fe-C alloys. The data were generated using molecular dynamic (MD) calculations utilizing reference-free Modified embedded atom method (RF-MEAM) interatomic potential. This potential was developed by fitting atomic structure-dependent energies, forces, and stress tensors evaluated at ground state and finite temperatures using ab-initio. Various ML algorithms were subsequently trained and deployed to predict elastic properties. In addition to individual algorithms, super learner (SL), an ensemble ML technique, was incorporated to refine predictions further. The input parameters comprised the alloy's composition, crystal structure, interstitial sites, lattice parameters, and temperature. The target properties were the bulk modulus and shear modulus. Two distinct prediction approaches were undertaken: employing individual models for each property prediction and simultaneously predicting both properties using a single integrated model, enabling a comparative analysis. The efficiency of these models was assessed through rigorous evaluation using a range of accuracy metrics. This work showcases the synergistic power of MD simulations and ML techniques for accelerating the prediction of elastic properties in alloys.

A Vast Review of Recognizing the Presence of Android Malware Based on Ensemble Machine Learning Technique

A Vast Review of Recognizing the Presence of Android Malware Based on Ensemble Machine Learning Technique

Optimizing Loop Tiling in Computing Systems through Ensemble Machine Learning Techniques

This work investigates the use of ensemble machine-learning algorithms to optimize loop-tiling in computing systems, with the goal of improving performance by predicting optimal tile sizes. It compares two approaches: independent training and averaging (soft voting) and an ensemble technique (hard voting) that employs models such as linear regression, ridge regression, and random forests. Experiments on an Intel Core i7-8565U CPU with several benchmark programs revealed that the hard voting Ensemble Approach beat the soft voting technique, providing more dependable and accurate predictions across a range of computing environments. The hard voting technique reduced execution time by around 87.5% for dynamic features and 89.89% for static features, whereas the soft voting approach showed an average drop of 75.45% for dynamic features and 78.13% for static characteristics. This work demonstrates the effectiveness of hard voting ensemble machine learning approaches in improving cache efficiency and total execution time, opening the way for future advances in high-performance computing settings.

Comparative study of ensemble machine learning techniques for airline delay prediction

Comparative study of ensemble machine learning techniques for airline delay prediction

Employing ensemble machine learning techniques for predicting the thermohydraulic performance of double pipe heat exchanger with and without turbulators

In this study, advanced machine learning techniques were utilized to forecast the thermohydraulic performance of a double pipe heat exchanger (DPHE). Key variables, including friction factor (f), Nusselt number (Nu), effectiveness (ɛ), and number transfer units (NTU), were modeled as functions of Reynolds number (Re), the number of turbulators (N), and the length of turbulators (L). To predict the DPHE's thermohydraulic performance, datasets were gathered from real-time experiments (case-1 and case-2) and literature (case-3). For clarity and analytical purposes, case-1 and case-2 were dimensioned to match the specifications outlined in literature (case-3). Case-1 represented a conventional DPHE without turbulators, while case-2 involved a DPHE with a gear disc turbulator. Additionally, datasets for DPHE with dolphin ring turbulators (case-3) were obtained for validation from literature. Subsequently, two ensemble boosting algorithms (extreme gradient and categorical) and one bagging algorithm (random forest) were applied to the dataset. The results highlighted that the categorical boosting model exhibited the highest accuracy, achieving a coefficient of determination (R2) of 0.9987 and an average absolute percent relative error (AAPRE) of 0.1837%. Furthermore, a sensitivity analysis was conducted for the best model (Random Forest), revealing relationships between input and output parameters. The Leverage statistical technique was also employed to identify valid, outlier, and out-of-range data points.