Ensemble Learning Research Articles

Classification of Dragon Fruit Varieties Based on Morphological Properties: Multi-Class Classification Approach

The classification of agricultural products is of great importance for quality control, optimized marketing, efficient logistics, research progress, consumer satisfaction, and sustainability. Dragon fruit has many varieties that need to be identified quickly and accurately for packaging and marketing. Considering the increasing demand for dragon fruit, it is obvious that an automated classification system has significant commercial and scientific value by increasing sorting efficiency and reducing manual labor costs. This study aimed to classify four commonly produced dragon fruit varieties according to their color, mechanical, and physical properties using machine learning models. Data were collected from 224 dragon fruits (53 American beauty, 57 Dark star, 65 Vietnamese white, and 49 Pepino dulce variety). Classification was performed using measurable physical and mechanical properties obtained through digital image processing, colorimetry, electronic weighing, and stress–strain testing. These methods provided objective and reproducible data collection for the models. Three models—Random Forest, Gradient Boosting, and Support Vector Classification—were implemented and their performances were evaluated using accuracy, precision, recall, Matthews’s correlation coefficient, Cohen’s Kappa, and F1-Score. The Random Forest model showed the highest performance in all metrics, achieving 98.66% accuracy, while the Support Vector Classification model had the lowest success. The superior performance of the Random Forest model can be attributed to its ability to handle complex, nonlinear relationships among multiple variables while preventing overfitting through ensemble learning. However, potential challenges in dragon fruit classification include variations due to environmental factors, genetic variation, and hybridization. Future research can focus on incorporating biochemical or genetic markers and improving real-time classification for industrial applications.

Blended Ensemble Learning for Robust Normal Behavior Modeling of Wind Turbines

ABSTRACTThe increasing scale of wind farms demands more efficient approaches to turbine monitoring and maintenance. Here, we present an innovative framework that combines enhanced kernel principal component analysis (KPCA) with ensemble learning to revolutionize normal behavior modeling (NBM) of wind turbines. By integrating random kitchen sinks (RKS) algorithm with KPCA, we achieved a 25.21% reduction in computational time while maintaining model accuracy. Our mixed ensemble approach, synthesizing LightGBM, random forest, and decision tree algorithms, demonstrated exceptional performance across diverse operational conditions, achieving R² values of 0.9995 in primary testing. The framework reduced mean absolute error by 25.1% and mean absolute percentage error by 33.4% compared to conventional methods. Notably, when tested across three distinct operational environments, the model maintained robust performance (R² > 0.97), demonstrating strong generalization capability. The system automatically detects anomalies using a 0.1% threshold, enabling real‐time monitoring of 78 variables across 136,000+ operational records. This scalable approach integrates seamlessly with existing SCADA infrastructure, offering a practical solution for large‐scale wind farm management. Our findings establish a new paradigm for wind turbine monitoring, combining computational efficiency with unprecedented accuracy in normal behavior prediction.

SELFprot: Effective and Efficient Multitask Finetuning Methods for Protein Parameter Prediction.

Accurately predicting protein-ligand interactions and enzymatic kinetics remains a challenge for computational biology. Here, we present SELFprot, a suite of modular transformer-based machine learning architectures that leverage the ESM2-35M model architecture for protein sequence and small molecule embeddings to improve predictions of complex biochemical interactions. SELFprot employs multitask learning and parameter-efficient finetuning through low-rank adaptation, allowing for adaptive, data-driven model refinement. Furthermore, ensemble learning techniques are used to enhance the robustness and reduce the prediction variance. Evaluated on the BindingDB and CatPred-DB data sets, SELFprot achieves competitive performance with notable improvements in parameter-efficient prediction of kcat, Km, Ki, Kd, IC50, and EC50 values as well as the classification of functional site residues. With comparable accuracy to existing models and an order of magnitude fewer parameters, SELFprot demonstrates versatility and efficiency, making it a valuable tool for protein-ligand interaction studies in bioengineering.

A clinical data-driven machine learning approach for predicting the effectiveness of piperacillin-tazobactam in treating lower respiratory tract infections

BackgroundIn hospitalized patients, inadequate antibiotic dosage leading to bacterial resistance and increased antimicrobial use intensity due to overexposure to antibiotics are common problems. In the present study, we constructed a machine learning model based on patients’ clinical information to predict the clinical effectiveness of Piperacillin-tazobactam (TZP) (4:1) in treating bacterial lower respiratory tract infections (LRTIs), to assist clinicians in making better clinical decisions.MethodsWe collected data from patients diagnosed with LRTIs or equivalent diagnoses admitted to the Department of Pulmonary and Critical Care Medicine at Shanghai Pudong Hospital, Shanghai, between January 1, 2021, and July 31, 2023. A total of 26 relevant clinical features were extracted from this cohort. Following data preprocessing, we trained four models: Logistic Regression, Random Forest, Support Vector Machine, and Gaussian Naive Bayes. The dataset was split into training and test sets using a 7:3 ratio. The top-performing models, as determined by Receiver Operating Characteristic (ROC)-Area Under the Curve (AUC) on the independent test set, were subsequently ensembled. Ensemble model (EL) performance was evaluated using bootstrap resampling on the training set and ROC-AUC, recall, accuracy, precision, F1-score, and log loss on an independent test set. The optimal model was then deployed as a web application for clinical outcome prediction.ResultsA total of 1,314 patients primarily treated with TZP as initial empiric antibiotic therapy were enrolled in the analysis. The success group comprised 995 patients (75.7%), while the failure group consisted of 319 patients (24.3%). We constructed an ensemble learning model based on the Logistic Regression, Support Vector Machine and Random Forest models, which showed better overall performance. The EL model demonstrated robust performance on an independent test set, exhibiting a ROC-AUC of 0.69, a recall of 0.69, an accuracy of 0.64, a precision of 0.40, a F1-score of 0.50, and a log loss of 0.66. A corresponding web application was then developed and made available at http://106.12.146.54:1020/.ConclusionsIn this study, we successfully developed and validated an EL model that effectively predicts the clinical effectiveness of TZP (4:1) in treating bacterial LRTIs. The model achieved a balanced performance across key evaluation metrics, demonstrating the model’s potential utility in clinical decision-making. The web-based application makes this model readily accessible to clinicians, potentially helping optimize antibiotic dosing decisions and reduce both inadequate treatment and overexposure. While promising, future studies with larger datasets and prospective validation are needed to further improve the model’s performance and validate its clinical utility. This work represents a step forward in using machine learning to support antimicrobial stewardship and personalized antibiotic therapy.

RRMSE-enhanced weighted voting regressor for improved ensemble regression.

Ensemble regression methods are widely used to improve prediction accuracy by combining multiple regression models, especially when dealing with continuous numerical targets. However, most ensemble voting regressors use equal weights for each base model's predictions, which can limit their effectiveness, particularly when there is no specific domain knowledge to guide the weighting. This uniform weighting approach doesn't consider that some models may perform better than others on different datasets, leaving room for improvement in optimizing ensemble performance. To overcome this limitation, we propose the RRMSE (Relative Root Mean Square Error) Voting Regressor, a new ensemble regression technique that assigns weights to each base model based on their relative error rates. By using an RRMSE-based weighting function, our method gives more importance to models that demonstrate higher accuracy, thereby enhancing the overall prediction quality. We tested the RRMSE Voting Regressor on six popular regression datasets and compared its performance with several state-of-the-art ensemble regression algorithms. The results show that the RRMSE Voting Regressor consistently achieves lower prediction errors than existing methods across all tested datasets. This improvement highlights the effectiveness of using relative error metrics for weighting in ensemble models. Our approach not only fills a gap in current ensemble regression techniques but also provides a reliable and adaptable method for boosting prediction performance in various machine learning tasks. By leveraging the strengths of individual models through smart weighting, the RRMSE Voting Regressor offers a significant advancement in the field of ensemble learning.

Exploring the Climate-Suitable Forestation Area Under Species Distribution and Growth Modeling for Larix kaempferi and Chamaecyparis obtusa in the Republic of Korea

Climate change has been transforming forest ecosystems globally, affecting the sustainability of conventional forest management practices. This study investigates the suitable forestation area (SFA) for Larix kaempferi and Chamaecyparis obtusa and their growth potential in South Korea under various climate change scenarios. Using species distribution models (SDMs) based on machine learning ensembles, we analyzed potential spatial shifts in the climatic suitability for these species. Growth models based on field data were also developed to evaluate growth variation between Köppen–Geiger climatic zones. The results indicate a substantial reduction in the SFA for L. kaempferi, with its habitat range confined to high-altitude regions due to rising temperatures. In contrast, the forestation potential for C. obtusa is predicted to expand nationwide, particularly in inland areas, under climate change scenarios. However, extreme increases in temperature and atmospheric CO2 concentrations exceeding 600 ppm may inhibit growth, highlighting the need for the development of adaptive management strategies. This study provides useful information for climate-resilient forestation planning by combining growth-weighted suitability indices with projected habitat shifts. These findings emphasize the importance of prioritizing high-altitude conservation zones for L. kaempferi and employing C. obtusa for inland afforestation as a means to ensure sustainable forest management and carbon neutrality objectives.

In silico screening to search for selective sodium channel blockers: When size matters.

In silico screening to search for selective sodium channel blockers: When size matters.

Multi-model integration for dynamic forecasting (MIDF): a framework for wind speed and direction prediction

Accurate forecasting of wind speed and direction is critical for the efficient integration of wind power into energy systems, ensuring reliable renewable energy production and grid stability. Traditional methods often struggle with capturing nonlinear interdependencies, quantifying uncertainties, and providing reliable long-term predictions, particularly in complex atmospheric conditions. To address these challenges, this study introduces multi-model Integration for dynamic forecasting (MIDF), an ensemble machine learning framework that combines the strengths of DeepAR and temporal fusion transformer (TFT) models through a two-step meta-learning process. MIDF leverages DeepAR’s probabilistic forecasting capabilities and TFT’s attention mechanisms to enhance accuracy, robustness, and interpretability. Using a custom meteorological dataset spanning January 2010 to May 2023, the model was evaluated against standalone alternatives across multiple metrics, including MSE, RMSE, and R2. MIDF achieved superior performance, with MSE, RMSE, and R2 values of 0.0035, 0.01913, and 0.89 for wind speed, and 0.00052, 0.02507, and 0.86 for wind direction, significantly reducing errors compared to existing methods. These results underscore the potential of ensemble learning in advancing wind forecasting accuracy, enabling more reliable renewable energy management, operational planning, and risk mitigation in meteorological applications.

Ensemble learning based on bi-directional gated recurrent unit and convolutional neural network with word embedding module for bioactive peptide prediction.

Ensemble learning based on bi-directional gated recurrent unit and convolutional neural network with word embedding module for bioactive peptide prediction.

Ensemble of Sequential Learning Models With Distributed Data Centers and Its Applications.

Handling massive datasets poses a significant challenge in modern data analysis, particularly within epidemiology and medicine. In this study, we introduce a novel approach using sequential ensemble learning to effectively analyze extensive datasets. Our method prioritizes efficiency from both statistical and computational perspectives, addressing challenges such as data communication and privacy, as discussed in federated learning literature. To demonstrate the efficacy of our approach, we present compelling real-world examples using COVID-19 data alongside simulation studies.

Techniques of Ensemble Learning for Human Emotional Classification and Detection

The identification of emotions from textual data has attracted attention due to developments in natural language processing (NLP), which provides insightful information about human behaviour in a variety of contexts. This study evaluates the performance of several Using an emotion dataset that is available on Hugging Face Hub, machine learning methods, such as Support Vector Machine (SVM), Logistic Regression, Random Forest, and Multinomial Naive Bayes, are used to classify emotions. The models we tested had different accuracy percentages: multinomial naive Bayes produced 65.90%, logistic regression performed well at 82.40%, random forest produced 85.12%, and support vector machine (SVM) produced 81.56%. Moreover, utilizing ensemble learning methodologies to capitalize on the advantages of many models has improved overall efficiency. Our ensemble learning approach demonstrates the effectiveness of mixing multiple models for improved emotion recognition, achieving an amazing 98% accuracy. Ensemble learning is a promising approach for emotion identification in textual data because it leverages the combined knowledge of separate models, reducing flaws and increasing robustness. Keywords--- Machine learning, Support Vector Machine, Random Forest, logarithmic regression, Multinomial Naive Bayes, Ensemble Voting Classifier, and Emotion Identification.

Short‐term prediction of railway track degradation using ensemble deep learning

AbstractShort‐term prediction of track degradation facilitates flexible and efficient maintenance, thereby meeting the railway system's escalating demands for track safety and smoothness. However, the track condition evolution presents challenges to accurate prediction, with diverse influential factors resulting in heterogeneous degradation patterns across space and time. In a short‐term context, time series derived from historical records are length‐limited, with sparse sampling points complicating feature identification. Actual activities, particularly minor repairs, lack strict periodicity, leading to irregular spans in continuous degradation curves, yielding nonuniform samples. This study leverages dynamic inspection and influential factors to propose an ensemble learning using the Transformer model. The outer framework employs unsupervised learning to group the sections based on specific time periods and track lengths. It assigns fuzzy logic categories to these groups to capture differentiated patterns and guides the division of samples into fuzzy subsets and assigns them to the learners corresponding to each cluster. The loosely coupled structure aids task decomposition and enhances local performance. The inner model refines the Transformer design for a new scenario, introducing a prediction objective transformation based on the interdependencies among multidimensional indicators to strengthen feature extraction. The prediction performance is evaluated using over 2 years of records from 560 km railway lines, offering insights for improving onsite track management.

Soil temperature prediction based on ensemble tree bagger machine learning algorithm for agricultural decision making

Soil temperature prediction based on ensemble tree bagger machine learning algorithm for agricultural decision making

Ensemble machine learning models for predicting concrete compressive strength incorporating various sand types

Ensemble machine learning models for predicting concrete compressive strength incorporating various sand types

Deep learning models in classifying primary bone tumors and bone infections based on radiographs

Primary bone tumors (PBTs) present significant diagnostic challenges due to their heterogeneous nature and similarities with bone infections. This study aimed to develop an ensemble deep learning framework that integrates multicenter radiographs and extensive clinical features to accurately differentiate between PBTs and bone infections. We compared the performance of the ensemble model with four imaging models based solely on radiographs utilizing EfficientNet B3, EfficientNet B4, Vision Transformer, and Swin Transformers. The patients were split into external dataset (N = 423) and internal dataset [including training (N = 1044), test (N = 354), and validation set (N = 171)]. The ensemble model outperformed imaging models, achieving areas under the curve (AUCs) of 0.948 and 0.963 on internal and external sets, respectively, with accuracies of 0.881 and 0.895. Its performance surpassed junior and mid-level radiologists and was comparable to senior radiologists (accuracy: 83.6%). These findings underscore the potential of deep learning in enhancing diagnostic precision for PBTs and bone infections (Research Registration Unique Identifying Number (UIN): researchregistry10483 and with details are available at https://www.researchregistry.com/register-now#home/registrationdetails/6693845995ba110026aeb754/).

Safeguarding Pipeline Integrity Through Stacked Ensemble Learning and Data Fusion

Safeguarding Pipeline Integrity Through Stacked Ensemble Learning and Data Fusion

Multi-domain Urdu fake news detection using pre-trained ensemble model

Fake News (FN) dissemination on websites and online platforms influences human behaviours, socio-political domains, and the sovereignty of a country. The outpour of biased news and propaganda on online portals can be addressed by restricting online propaganda using an automated mechanism. Proving the authenticity of news and information on online platforms in regional languages, such as Urdu, with limited resources and datasets, is challenging. Furthermore, limited research in resource-constrained languages has created language bias in Artificial Intelligence (AI) research, which is concentrated in this study. Natural Language Processing (NLP) techniques have been used for Fake News Detection (FND) for English news and various language-related tasks. Previous studies used Machine Learning (ML), Deep Learning (DL), and individual Pre-trained Language Models (PLMs) for Urdu FND. ML-based ensemble model showed better performance than pre-trained models for Urdu FND. We propose a methodology for Urdu FND by applying stacked ensemble learning of PLMs, ELECTRA, mBERT and XLM-RoBERTa after apposite fine-tuning and hyperparameter optimization. To overcome the limitations of each pre-trained transformer model, these are fine-tuned individually using a publicly available Urdu dataset. The prediction performance results of the proposed stacking approach surpass the performance of each pre-trained model. An Accuracy of 0.914, a Matthews Correlation Co-efficient (MCC) value of 0.898, and an F1-score of 0.904 validate the efficacy of the proposed ensemble model.

Optimizing oil production forecasts in Iranian oil fields: a comprehensive analysis using ensemble learning techniques

This study introduces the application of Stacking Ensemble Learning in petroleum engineering, marking a significant advancement in oil production rate forecasting. Unlike traditional forecasting methods, which often rely on single-model approaches with limited adaptability to complex, the methodology integrates multiple machine learning algorithms each optimized using distinct, hyperparameter tuning techniques. The inclusion of advanced optimization strategies, such as Genetic Algorithm (GA), Teaching-Learning-Based Optimization (TLBO), and Particle Swarm Optimization (PSO), ensures that each base model reaches its highest potential performance. By employing a diverse and extensive dataset of 7245 data points from 10 oil wells, the study achieves a high level of model generalizability across varying operational conditions, including choke sizes, wellhead pressures, and gas-oil ratio values. The ensemble approach demonstrated superior performance compared to individual models, with significant improvements in prediction accuracy and robustness. Models such as Radial Basis Function Neural Network (RBFNN) (74.4%), DNN-TLBO (93.3%), and DNN-GA (92.8%) exhibited enhanced capabilities but were outperformed by the integrated ensemble method (93.9%), highlighting its effectiveness in oil production forecasting. This work presents a novel application of Stacking Ensemble Learning in petroleum engineering, offering an innovative solution for improving oil production rate forecasting. The study’s integration of diverse optimization techniques and ensemble modeling provides new insights and tools to advance predictive analytics in the oil and gas industry.Graphical abstract

Machine learning based estimation of density of binary blends of cyclohexanes in normal alkanes

Given the application of cycloalkanes in surrogate blends for aviation fuels, their determination of critical characteristics pertinent to fuel transportation and combustion becomes imperative. In this study, we aim to construct intelligent models based on machine learning methods of random forest (RF), adaptive boosting, decision tree (DT), ensemble learning, K-nearest neighbors (KNN), support vector machine (SVM), multi-layer perceptron (MLP) artificial neural network and convolutional neural network (CNN) to predict the density of binary blends of ethylcyclohexane or methylcyclohexane with n-hexadecane/n-dodecane/n-tetradecane in terms of operational conditions (pressure and temperature) and cycloalkane mole fractions in n-alkanes, utilizing laboratory data extracted from existing scholarly publications. The reliability of the data used is affirmed using an outlier detection algorithm, and the relevancy factor concept is utilized to find the relative effects of the input parameters on the output parameter. The preciseness of the developed models is checked and compared comprehensively via statistical and graphical methods. The obtained results indicate that temperature has the most effect on density, with a relevancy value of − 0.9619, and pressure has the least effective value, with a relevancy value of 0.041, which explains the straight relationship between pressure and density. The modeling results illustrate that DT and RF algorithms have the best performance in calculating density with R2 values of 0.9985 and 0.09982, respectively. The MLP and Adaboosting models exhibit the weakest performance in this field, with R2 values of 0.9455 and 0.9477, respectively. The current paper indicates robust tools for the accurate prediction of the density of binary blends of ethyl cyclohexane or methylcyclohexane with n-hexadecane/n-dodecane/n-tetradecane, which are required for fuel transportation and combustion studies.

Physics-informed data-driven ensemble and transfer learning approaches for prediction of temperature field and cutting force during machining IN625 superalloy

Physics-informed data-driven ensemble and transfer learning approaches for prediction of temperature field and cutting force during machining IN625 superalloy