Ensemble Learning Research Articles

Prediction of non-stationary daily streamflow series based on ensemble learning: a case study of the Wei River Basin, China

Prediction of non-stationary daily streamflow series based on ensemble learning: a case study of the Wei River Basin, China

Ensemble deep learning and EfficientNet for accurate diagnosis of diabetic retinopathy

Diabetic Retinopathy (DR) stands as a significant global cause of vision impairment, underscoring the critical importance of early detection in mitigating its impact. Addressing this challenge head-on, this study introduces an innovative deep learning framework tailored for DR diagnosis. The proposed framework utilizes the EfficientNetB0 architecture to classify diabetic retinopathy severity levels from retinal images. By harnessing advanced techniques in computer vision and machine learning, the proposed model aims to deliver precise and dependable DR diagnoses. Continuous testing and experimentation shows to the efficiency of the architecture, showcasing promising outcomes that could help in the transformation of both diagnosing and treatment of DR. This framework takes help from the EfficientNet Machine Learning algorithms and employing advanced CNN layering techniques. The dataset utilized in this study is titled ’Diagnosis of Diabetic Retinopathy’ and is sourced from Kaggle. It consists of 35,108 retinal images, classified into five categories: No Diabetic Retinopathy (DR), Mild DR, Moderate DR, Severe DR, and Proliferative DR. Through rigorous testing, the framework yields impressive results, boasting an average accuracy of 86.53% and a loss rate of 0.5663. A comparison with alternative approaches underscores the effectiveness of EfficientNet in handling classification tasks for diabetic retinopathy, particularly highlighting its high accuracy and generalizability across DR severity levels. These findings highlight the framework’s potential to significantly advance the field of DR diagnosis, given more advanced datasets and more training resources which leads it to be offering clinicians a powerful tool for early intervention and improved patient outcomes.

Machine Learning‐Assisted Hybrid Package of White Light‐Emitting Diodes Employing Quantum Dots and Phosphor

AbstractWhite light‐emitting diodes (WLEDs), known for their high brightness, high efficiency, and long lifetime, are widely utilized in the backlight of liquid crystal displays. However, it is still difficult to improve the color gamut of WLEDs while maintaining the L50 lifetime. The luminous characteristics of WLEDs employing different combinations of quantum dots and phosphor are investigated in this work. Additionally, investigations on the L50 lifetime for WLEDs are carried out by employing a two‐step accelerated stress method. Finally, an ensemble machine learning model is proposed to predict the L50 lifetime of WLEDs, achieving high predictive accuracy with R2 of 0.986.

A Rice Leaf Area Index Monitoring Method Based on the Fusion of Data from RGB Camera and Multi-Spectral Camera on an Inspection Robot

Automated monitoring of the rice leaf area index (LAI) using near-ground sensing platforms, such as inspection robots, is essential for modern rice precision management. These robots are equipped with various complementary sensors, where specific sensor capabilities partially overlap to provide redundancy and enhanced reliability. Thus, leveraging multi-sensor fusion technology to improve the accuracy of LAI monitoring has become a crucial research focus. This study presents a rice LAI monitoring model based on the fused data from RGB and multi-spectral cameras with an ensemble learning algorithm. The results indicate that the estimation accuracy of the rice LAI monitoring model is effectively improved by fusing the vegetation index and textures from RGB and multi-spectral sensors. The model based on the LightGBM regression algorithm has the most improvement in accuracy, with a coefficient of determination (R2) of 0.892, a root mean square error (RMSE) of 0.270, and a mean absolute error (MAE) of 0.160. Furthermore, the accuracy of LAI estimation in the jointing stage is higher than in the heading stage. At the jointing stage, both LightGBM based on optimal RGB image features and Random Forest based on fused features achieved an R2 of 0.95. This study provides a technical reference for automatically monitoring rice growth parameters in the field using inspection robots.

Optimization of Cocoa Pods Maturity Classification Using Stacking and Voting with Ensemble Learning Methods in RGB and LAB Spaces

Determining the maturity of cocoa pods early is not just about guaranteeing harvest quality and optimizing yield. It is also about efficient resource management. Rapid identification of the stage of maturity helps avoid losses linked to a premature or late harvest, improving productivity. Early determination of cocoa pod maturity ensures both the quality and quantity of the harvest, as immature or overripe pods cannot produce premium cocoa beans. Our innovative research harnesses artificial intelligence and computer vision technologies to revolutionize the cocoa industry, offering precise and advanced tools for accurately assessing cocoa pod maturity. Providing an objective and rapid assessment enables farmers to make informed decisions about the optimal time to harvest, helping to maximize the yield of their plantations. Furthermore, by automating this process, these technologies reduce the margins for human error and improve the management of agricultural resources. With this in mind, our study proposes to exploit a computer vision method based on the GLCM (gray level co-occurrence matrix) algorithm to extract the characteristics of images in the RGB (red, green, blue) and LAB (luminance, axis between red and green, axis between yellow and blue) color spaces. This approach allows for in-depth image analysis, which is essential for capturing the nuances of cocoa pod maturity. Next, we apply classification algorithms to identify the best performers. These algorithms are then combined via stacking and voting techniques, allowing our model to be optimized by taking advantage of the strengths of each method, thus guaranteeing more robust and precise results. The results demonstrated that the combination of algorithms produced superior performance, especially in the LAB color space, where voting scored 98.49% and stacking 98.71%. In comparison, in the RGB color space, voting scored 96.59% and stacking 97.06%. These results surpass those generally reported in the literature, showing the increased effectiveness of combined approaches in improving the accuracy of classification models. This highlights the importance of exploring ensemble techniques to maximize performance in complex contexts such as cocoa pod maturity classification.

Res-UNet Ensemble Learning for Semantic Segmentation of Mineral Optical Microscopy Images

In geology and mineralogy, optical microscopic images have become a primary research focus for intelligent mineral recognition due to their low equipment cost, ease of use, and distinct mineral characteristics in imaging. However, due to their close reflectivity or transparency, some minerals are not easily distinguished from other minerals or background. Secondly, the number of background pixels often vastly exceeds the number of pixels for individual mineral particles, and the number of pixels of different mineral particles in the image also varies significantly. These have led to the issue of data imbalance. This imbalance results in lower recognition accuracy for categories with fewer samples. To address these issues, a flexible ensemble learning for semantic segmentation based on multiple optimized Res-UNet models is proposed, introducing dice loss and focal loss functions and incorporating a pre-positioned spatial transformer networks block. Twelve optimized Res-UNet models were used to construct multiple Res-UNet ensemble learnings using heterogeneous ensemble strategies. The results demonstrate that the system integrated with five learners using the weighted voting fusion method (RUEL-5-WV) achieved the best performance with a mean Intersection over Union (mIOU) of 91.65 across all nine categories and an IOU of 84.33 for the transparent mineral (gangue). The results indicate that this ensemble learning scheme outperforms individual optimized Res-UNet models. Compared to the classical Deeplabv3 and PSPNet, this scheme also exhibits significant advantages.

EEG channel and feature investigation in binary and multiple motor imagery task predictions

IntroductionMotor Imagery (MI) Electroencephalography (EEG) signals are non-stationary and dynamic physiological signals which have low signal-to-noise ratio. Hence, it is difficult to achieve high classification accuracy. Although various machine learning methods have already proven useful to that effect, the use of many features and ineffective EEG channels often leads to a complex structure of classifier algorithms. State-of-the-art studies were interested in improving classification performance with complex feature extraction and classification methods by neglecting detailed EEG channel and feature investigation in predicting MI tasks from EEGs. Here, we investigate the effects of the statistically significant feature selection method on four different feature domains (time-domain, frequency-domain, time-frequency domain, and non-linear domain) and their two different combinations to reduce the number of features and classify MI-EEG features by comparing low-dimensional matrices with well-known machine learning algorithms.MethodsOur main goal is not to find the best classifier performance but to perform feature and channel investigation in MI task classification. Therefore, the detailed investigation of the effect of EEG channels and features is implemented using a statistically significant feature distribution on 22 EEG channels for each feature set separately. We used the BCI Competition IV Dataset IIa and 288 samples per person. A total of 1,364 MI-EEG features were analyzed in this study. We tested nine distinct classifiers: Decision tree, Discriminant analysis, Logistic regression, Naive Bayes, Support vector machine, k-Nearest neighbor, Ensemble learning, Neural networks, and Kernel approximation.ResultsAmong all feature sets considered, classifications performed with non-linear and combined feature sets resulted in a maximum accuracy of 63.04% and 47.36% for binary and multiple MI task predictions, respectively. The ensemble learning classifier achieved the maximum accuracy in almost all feature sets for binary and multiple MI task classifications.DiscussionOur research thus shows that the statistically significant feature-based feature selection method significantly improves the classification performance with fewer features in almost all feature sets, enabling detailed and effective EEG channel and feature investigation.

Hybrid bagging and boosting with SHAP based feature selection for enhanced predictive modeling in intrusion detection systems

The novelty and growing sophistication of cyber threats mean that high accuracy and interpretable machine learning models are needed more than ever before for Intrusion Detection and Prevention Systems. This study aims to solve this challenge by applying Explainable AI techniques, including Shapley Additive explanations feature selection, to improve model performance, robustness, and transparency. The method systematically employs different classifiers and proposes a new hybrid method called Hybrid Bagging-Boosting and Boosting on Residuals. Then, performance is taken in four steps: the multistep evaluation of hybrid ensemble learning methods for binary classification and fine-tuning of performance; feature selection using Shapley Additive explanations values retraining the hybrid model for better performance and reducing overfitting; the generalization of the proposed model for multiclass classification; and the evaluation using standard information metrics such as accuracy, precision, recall, and F1-score. Key results indicate that the proposed methods outperform state-of-the-art algorithms, achieving a peak accuracy of 98.47% and an F1 score of 96.19%. These improvements stem from advanced feature selection and resampling techniques, enhancing model accuracy and balancing precision and recall. Integrating Shapley Additive explanations-based feature selection with hybrid ensemble methods significantly boosts the predictive and explanatory power of Intrusion Detection and Prevention Systems, addressing common pitfalls in traditional cybersecurity models. This study paves the way for further research on statistical innovations to enhance Intrusion Detection and Prevention Systems performance.

A stacked ensemble deep learning model for predicting the intensive care unit patient mortality

Accurate mortality prediction in intensive care units (ICUs) is essential for optimizing patient treatment, nursing care, and resource allocation. Traditional models, such as Acute Physiology and Chronic Health Evaluation and Simplified Acute Physiology Score, have been very important in clinical practice, but they frequently have issues with prediction accuracy and adaptability, especially when dealing with complex and evolving patient data. These issues can be resolved, and the accuracy of mortality prediction increased due to recent developments in machine learning, especially deep learning. The present study introduces a new deep learning ensemble model that achieves a significant improvement over existing methods. Using stacked ensemble learning, our approach combines the advantages of one Random Forests model and two CatBoost models. We achieved a notable performance in mortality prediction by carefully training and optimizing this ensemble using the electronic ICU Collaborative Research Database. Our model boasts an accuracy of 94.19%, precision of 94.097%, recall of 94.29%, and F1-score of 94.191%, demonstrating a substantial improvement over conventional approaches. The prediction of ICU mortality has been significantly improved using ensemble learning, which helps medical and nursing staff to better treat patients individually, allocate resources efficiently, and enhance patient outcomes. This approach gives healthcare experts the ability to make data-driven decisions, leading to more effective and efficient care within the ICU.

Optimizing quality of service forecasting in mobile networks through modified walrus optimization and multivariate approaches

This paper presents Ensemble-based Service Quality Prediction (EAQP), an automated method for predicting service quality under changing mobile network conditions. EAQP incorporates data preparation methods such as transformation, purification, & imputation, and then performs feature extraction utilizing statistical, geographical, as well as temporal approaches. An improved feature selection method, using a unique weighting approach and optimized by a modified Walrus Optimization Algorithm, improves the accuracy of predictions. EAQP utilizes a variety of prediction models such as support vector regression, recurrent neural network models, bi-directional short-term long-term memory networks, extreme learning machines, along with multi-layer perceptron neural networks to enhance predictive accuracy. EAQP uses complex optimization algorithms and ensemble learning approaches to provide precise and dependable predictions about service quality in real-time. This helps in proactive network management as well as improvement. This comprehensive approach shows potential for boosting network efficiency, optimizing the distribution of resources, and enhancing the end-user experience when using mobile communications systems.

A Machine Learning-Based Parameterized Tropical Cyclone Precipitation Model

AbstractCurrent simulation models considerably underestimate local-scale, short-duration extreme precipitation induced by tropical cyclones (TCs). This problem needs to be addressed to establish active response policies for TC-induced disasters. Taking Shanghai, a coastal megacity, as a study area and based on the observations from 192 meteorological stations in the city during 2005–2018, this study optimized the parameterized Tropical Cyclone Precipitation Model (TCPM) initially designed for TCs at the national scale (China) to the local or regional scales by using machine learning (ML) methods, including the random forest (RF), extreme gradient boosting (XGBoost), and ensemble learning (EL) algorithms. The TCPM-ML was applied for multiple temporal scale hazard assessment. The results show that: (1) The TCPM-ML not only improved TCPM performance for simulating hourly extreme precipitations, but also preserved the physical meaning of the results, contrary to ML methods; (2) Machine learning algorithms enhanced the TCPM ability to reproduce observations, although the hourly extreme precipitations remained slightly underestimated; (3) Best performance was obtained with the XGBoost or EL algorithms. Combining the strengths of both XGBoost and RF, the EL algorithm yielded the best overall performance. This study provides essential model support for TC disaster risk assessment and response at the local and regional scales in China.

Enhancing brain tumor classification by integrating radiomics and deep learning features: A comprehensive study utilizing ensemble methods on MRI scans

Background and objective This study aims to assess the effectiveness of combining radiomics features (RFs) with deep learning features (DFs) for classifying brain tumors—specifically Glioma, Meningioma, and Pituitary Tumor—using MRI scans and advanced ensemble learning techniques. Methods A total of 3064 T1-weighted contrast-enhanced brain MRI scans were analyzed. RFs were extracted using Pyradiomics, while DFs were obtained from a 3D convolutional neural network (CNN). These features were used both individually and together to train a range of machine learning models, including Support Vector Machines (SVM), Decision Trees (DT), Random Forests (RF), AdaBoost, Bagging, k-Nearest Neighbors (KNN), and Multi-Layer Perceptrons (MLP). To enhance the accuracy of these models, ensemble approaches such as Stacking, Voting, and Boosting were employed. LASSO feature selection and five-fold cross-validation were utilized to ensure the models’ robustness. Results The results demonstrated that combining RFs and DFs significantly improved the model's performance compared to using either feature set alone. The best performance was achieved using the combined RF + DF approach with ensemble methods, particularly Boosting, which resulted in an accuracy of 95.0%, an AUC of 0.92, a sensitivity of 88%, and a specificity of 90%. Conversely, models utilizing only RFs or DFs showed lower performance, with RFs reaching an AUC of 0.82 and DFs achieving an AUC of 0.85. Conclusion The integration of RFs and DFs, along with advanced ensemble methods, significantly improves the accuracy and reliability of brain tumor classification using MRI. This approach shows strong clinical potential, with opportunities for further enhancing generalizability and precision through additional MRI sequences and advanced machine learning techniques.

Recognition of Conus species using a combined approach of supervised learning and deep learning-based feature extraction.

Cone snails are venomous marine gastropods comprising more than 950 species widely distributed across different habitats. Their conical shells are remarkably similar to those of other invertebrates in terms of color, pattern, and size. For these reasons, assigning taxonomic signatures to cone snail shells is a challenging task. In this report, we propose an ensemble learning strategy based on the combination of Random Forest (RF) and XGBoost (XGB) methods. We used 47,600 cone shell images of uniform size (224 x 224 pixels), which were split into an 80:20 train-test ratio. Prior to performing subsequent operations, these images were subjected to pre-processing and transformation. After applying a deep learning approach (Visual Geometry Group with a 16-layer deep model architecture) for feature extraction, model specificity was further assessed by including multiple related and unrelated seashell images. Both classifiers demonstrated comparable recognition ability on random test samples. The evaluation results suggested that RF outperformed XGB due to its high accuracy in recognizing Conus species, with an average precision of 95.78%. The area under the receiver operating characteristic curve was 0.99, indicating the model's optimal performance. The learning and validation curves also demonstrated a robust fit, with the training score reaching 1 and the validation score gradually increasing to 95 as more data was provided. These values indicate a well-trained model that generalizes effectively to validation data without significant overfitting. The gradual improvement in the validation score curve is crucial for ensuring model reliability and minimizing the risk of overfitting. Our findings revealed an interactive visualization. The performance of our proposed model suggests its potential for use with datasets of other mollusks, and optimal results may be achieved for their categorization and taxonomical characterization.

Applications and Progress of Machine Learning Techniques in the Ladle Furnace Refining Process: A Review

Machine learning techniques have now become key technologies in optimizing the ladle furnace (LF) refining process, significantly improving prediction accuracy and operational efficiency in various aspects such as temperature control, slag optimization, and alloying processes. This article first reviews the principles of neural networks, utilizing their significant advantages in data processing capability and model generalization to explore the current applications and development trends of machine learning techniques in the LF refining process. It analyzes the advantages of hybrid dynamic models and ensemble learning methods in improving prediction accuracy and handling complex data. Additionally, this article explains the optimization of the LF refining process by combining machine learning techniques with traditional methods to reduce costs and improve production efficiency. It demonstrates the application effects and potential value of machine learning techniques in actual production, discusses the strategies and challenges in real‐time control and prediction accuracy, and envisions the future development direction of intelligent models for the LF refining process, providing a scientific basis and technical reference for the subsequent development of LF refining technology.

ML-Enabled Solar PV Electricity Generation Projection for a Large Academic Campus to Reduce Onsite CO2 Emissions

Mitigating CO2 emissions is essential to reduce climate change and its adverse effects on ecosystems. Photovoltaic electricity is 30 times less carbon-intensive than coal-based electricity, making solar PV an attractive option in reducing electricity demand from fossil-fuel-based sources. This study looks into utilizing solar PV electricity production on a large university campus in an effort to reduce CO2 emissions. The study involved investigating 153 buildings on the campus, spanning nine years of data, from 2015 to 2023. The study comprised four key phases. In the first phase, PVWatts gathered data to predict PV-generated energy. This was the foundation for Phase II, where a novel tree-based ensemble learning model was developed to predict monthly PV-generated electricity. The SHAP (SHapley Additive exPlanations) technique was incorporated into the proposed framework to enhance model explainability. Phase III involved calculating historical CO2 emissions based on past energy consumption data, providing a baseline for comparison. A meta-learning algorithm was implemented in Phase IV to project future CO2 emissions post-solar PV installation. This comparison estimated a potential emissions reduction and assessed the university’s progress toward its net-zero emissions goals. The study’s findings suggest that solar PV implementation could reduce the campus’s CO2 footprint by approximately 18% for the studied cluster of buildings, supporting sustainability and cleaner energy use on the campus.

A deep learning approach to predict differentiation outcomes in hypothalamic-pituitary organoids

We use three-dimensional culture systems of human pluripotent stem cells for differentiation into pituitary organoids. Three-dimensional culture is inherently characterized by its ability to induce heterogeneous cell populations, making it difficult to maintain constant differentiation efficiency. That is why the culture process involves empirical aspects. In this study, we use deep-learning technology to create a model that can predict from images of organoids whether differentiation is progressing appropriately. Our models using EfficientNetV2-S or Vision Transformer, employing VENUS-coupled RAX expression, predictively class bright-field images of organoids into three categories with 70% accuracy, superior to expert-observer predictions. Furthermore, the model obtained by ensemble learning with the two algorithms can predict RAX expression in cells without RAX::VENUS, suggesting that our model can be deployed in clinical applications such as transplantation.

Ensemble Learning for Oat Yield Prediction Using Multi-Growth Stage UAV Images

Accurate crop yield prediction is crucial for optimizing cultivation practices and informing breeding decisions. Integrating UAV-acquired multispectral datasets with advanced machine learning methodologies has markedly refined the accuracy of crop yield forecasting. This study aimed to construct a robust and versatile yield prediction model for multi-genotyped oat varieties by investigating 14 modeling scenarios that combine multispectral data from four key growth stages. An ensemble learning framework, StackReg, was constructed by stacking four base algorithms—ridge regression (RR), support vector machines (SVM), Cubist, and extreme gradient boosting (XGBoost)—to predict oat yield. The results show that, for single growth stages, base models achieved R2 values within the interval of 0.02 to 0.60 and RMSEs ranging from 391.50 to 620.49 kg/ha. By comparison, the StackReg improved performance, with R2 values extending from 0.25 to 0.61 and RMSEs narrowing to 385.33 and 542.02 kg/ha. In dual-stage and multi-stage settings, the StackReg consistently surpassed the base models, reaching R2 values of up to 0.65 and RMSE values as low as 371.77 kg/ha. These findings underscored the potential of combining UAV-derived multispectral imagery with ensemble learning for high-throughput phenotyping and yield forecasting, advancing precision agriculture in oat cultivation.

Ensemble Bagging Model for Predicting Flexural Strength of Geopolymer Concrete

Waste materials, such as fly ash and lime mortar, are used in the concrete industry to create an environmentally friendly environment. However, since the experimental studies will take time, it is necessary to predict the flexural strength (FS) and properties of Geopolymer concrete (GPC) using ensemble Learning (EL) algorithms in order to shorten the experimental work process and save money and time. In this study, a new ensemble the Bagging prediction model using gradient boosting regressor estimator is proposed to predict the FS of GPC using lime mortar. The performance of the proposed model was evaluated using the performance metrics [Formula: see text], RMSE, MSE, MAE, and MAPE. The proposed model was compared using the individual learning algorithms and validated using [Formula: see text]-fold cross-validation technique. From the SHAP plot obtained using the best proposed EL model BGR, ICE, and PDP analysis, it is seen that the blast furnace slag content has the most significant effect on the FS of GPC.

Performance Assessment of Individual and Ensemble Learning Models for Gully Erosion Susceptibility Mapping in a Mountainous and Semi-Arid Region

High-accuracy gully erosion susceptibility maps play a crucial role in erosion vulnerability assessment and risk management. The principal purpose of the present research is to evaluate the predictive power of individual machine learning models such as random forest (RF), decision tree (DT), and support vector machine (SVM), and ensemble machine learning approaches such as stacking, voting, bagging, and boosting with k-fold cross validation resampling techniques for modeling gully erosion susceptibility in the Oued El Abid watershed in the Moroccan High Atlas. A dataset comprising 200 gully points, identified through field observations and high-resolution Google Earth imagery, was used, alongside 21 gully erosion conditioning factors selected based on their importance, information gain, and multi-collinearity analysis. The exploratory results indicate that all derived gully erosion susceptibility maps had a good accuracy for both individual and ensemble models. Based on the receiver operating characteristic (ROC), the RF and the SVM models had better predictive performances, with AUC = 0.82, than the DT model. However, ensemble models significantly outperformed individual models. Among the ensembles, the RF-DT-SVM stacking model achieved the highest predictive accuracy, with an AUC value of 0.86, highlighting its robustness and superior predictive capability. The prioritization results also confirmed the RF-DT-SVM ensemble model as the best. These findings highlight the superiority of ensemble learning models over individual ones and underscore their potential for application in similar geo-environmental contexts.

Skin Disease Detection Using Ensemble Learning

Skin diseases affect millions of people worldwide, making early detection and accurate diagnosis crucial for effective treatment. However, the process of diagnosing skin conditions often requires specialized dermatological expertise, which can be time-consuming and expensive. To address this challenge, we have developed a web-based Skin Disease Detection System using cutting-edge machine learning and ensemble techniques. Our system leverages the HAM10000 dataset, consisting of 10,000 dermoscopic images representing seven common skin lesion types. We employ an ensemble learning approach, integrating ResNet50, EfficientNetB0, and DenseNet121 as base models and combining their predictions using a logistic regression meta-model. This architecture enhances the accuracy and robustness of the predictions. The backend of our application is built with Flask, responsible for model processing, authentication, and handling API requests. The React frontend provides an intuitive user interface where users can upload skin lesion images and receive real-time predictions along with confidence scores. The system also incorporates authentication and token verification for secure user access, ensuring data privacy and integrity. Our project aims to provide an accessible, reliable, and cost-effective tool to assist healthcare professionals and individuals in the early detection of skin diseases, potentially improving patient outcomes and reducing the burden on dermatology services