Ensemble Learning Method Research Articles

A Machine Learning‐Based Approach for the Prediction of Anticoagulant Activity of Hypericum perforatum L. and Evaluation of Compound Activity

ABSTRACTIntroductionHypericum perforatum L. (HPL) is extensively researched domestically and internationally as a medicinal plant. However, no reports of studies related to the anticoagulant activity of HPL have been retrieved. The specific bioactive components are unknown.ObjectiveThe aim of this study was to develop a machine learning (ML) method for rapid prediction of anticoagulant activity in HPL and evaluation of compound activity. Materials and methods.First, an in vitro anticoagulant activity assay was developed for the determination of the bioactivity of various medicinal parts of HPL. Then, the peak areas of compounds in HPL were integrated using UPLC‐Q‐TOF‐MS analysis. Subsequently, nine independent ML methods and two ensemble learning methods have been established to predict the anticoagulant activity of HPL and to evaluate the contribution of compounds. Feature importance scores were used for models visualization.ResultsA total of 24 compounds were shown to exhibited superior anticoagulant activity. Molecular docking experiments likewise confirmed this result. The results show that the branches of HPL have excellent anticoagulant activity, which has been previously overlooked. The established ML model demonstrated good performance in the prediction of the activity of HPL.ConclusionThe results were accurate and reliable, which significantly improved the efficiency of active compounds screening, and further exploration in this area is warranted.

Customer Churn Prediction Model Based on Adaptive Clustering Mixed-Sampling

Predicting the probability of customer churn is an important reference for formulating and implementing customer retention strategies. Compared with single classification method, ensemble learning method can obtain better generalization ability and ensemble learning-based customer churn prediction method has gradually become a research focus. However, the prediction data of customer churn is usually imbalanced in class, and there are problems of overlapping and class-imbalance, so the prediction effect of ensemble learning model is ineffective. Therefore, an ensemble learning method based on adaptive clustering mixed-sampling (CUS-Ensemble) is proposed. This method is based on Bagging and regarded the gradient boosting decision tree as classifier, Firstly, cleanse the date of the whole training set, Secondly, the clustering under-sampling method is used to sample majority class instances, so that the sampled majority class instances are slightly more than the minority class instances, then, the sampled majority class instances and all minority class instances form a new training subset and provide a data cleansing again, next, using Borderline-SMOTE to balance the cleaned subset, and input it into the decision tree. Finally, the output of several gradient boosting decision trees is integrated as the final prediction result. Experimental results on six imbalanced customer datasets show that, compared with EasyEnsemble and other comparison methods, the AUC and Recall value of this method are increased by 4% and 13.6% on average, which is helpful to reduce the losses caused by customer churn.

Catalyzing IVF outcome prediction: exploring advanced machine learning paradigms for enhanced success rate prognostication

This study addresses the research problem of enhancing In-Vitro Fertilization (IVF) success rate prediction by integrating advanced machine learning paradigms with gynecological expertise. The methodology involves the analysis of comprehensive datasets from 2017 to 2018 and 2010–2016. Machine learning models, including Logistic Regression, Gaussian NB, SVM, MLP, KNN, and ensemble models like Random Forest, AdaBoost, Logit Boost, RUS Boost, and RSM, were employed. Key findings reveal the significance of patient demographics, infertility factors, and treatment protocols in IVF success prediction. Notably, ensemble learning methods demonstrated high accuracy, with Logit Boost achieving an accuracy of 96.35%. The implications of this research span clinical decision support, patient counseling, and data preprocessing techniques, highlighting the potential for personalized IVF treatments and continuous monitoring. The study underscores the importance of collaboration between gynecologists and data scientists to optimize IVF outcomes. Prospective studies and external validation are suggested as future directions, promising to further revolutionize fertility treatments and offer hope to couples facing infertility challenges.

Addressing catastrophic forgetting in payload parameter identification using incremental ensemble learning

Collaborative robots (cobots) are increasingly integrated into Industry 4.0 dynamic manufacturing environments that require frequent system reconfiguration due to changes in cobot paths and payloads. This necessitates fast methods for identifying payload inertial parameters to compensate the cobot controller and ensure precise and safe operation. Our prior work used Incremental Ensemble Model (IEM) to identify payload parameters, eliminating the need for an excitation path and thus removing the separate identification step. However, this approach suffers from catastrophic forgetting. This paper introduces a novel incremental ensemble learning method that addresses the problem of catastrophic forgetting by adding a new weak learner to the ensemble model for each new training bag. Moreover, it proposes a new classification model that assists the ensemble model in identifying which weak learner provides the most accurate estimation for new input data. The proposed method incrementally updates the identification model while the cobot navigates any task path, maintaining accuracy on old weak learner even after updating with new data. Validation performed on the Franka Emika cobot showcases the model’s superior accuracy and adaptability, effectively eliminating the problem of catastrophic forgetting.

Detecting Water Stress in Winter Wheat Based on Multifeature Fusion from UAV Remote Sensing and Stacking Ensemble Learning Method

The integration of remote sensing technology and machine learning algorithms represents a new research direction for the rapid and large-scale detection of water stress in modern agricultural crops. However, in solving practical agricultural problems, single machine learning algorithms cannot fully explore the potential information within the data, lacking stability and accuracy. Stacking ensemble learning (SEL) can combine the advantages of multiple single machine learning algorithms to construct more stable predictive models. In this study, threshold values of stomatal conductance (gs) under different soil water stress indices (SWSIs) were proposed to assist managers in irrigation scheduling. In the present study, six irrigation treatments were established for winter wheat to simulate various soil moisture supply conditions. During the critical growth stages, gs was measured and the SWSI was calculated. A spectral camera mounted on an unmanned aerial vehicle (UAV) captured reflectance images in five bands, from which vegetation indices and texture information were extracted. The results indicated that gs at different growth stages of winter wheat was sensitive to soil moisture supply conditions. The correlation between the gs value and SWSI value was high (R2 > 0.79). Therefore, the gs value threshold can reflect the current soil water stress level. Compared with individual machine learning models, the SEL model exhibited higher prediction accuracy, with R2 increasing by 6.67–17.14%. Using a reserved test set, the SEL model demonstrated excellent performance in various evaluation metrics across different growth stages (R2: 0.69–0.87, RMSE: 0.04–0.08 mol m−2 s−1; NRMSE: 12.3–23.6%, MAE: 0.03–0.06 mol m−2 s−1) and exhibited excellent stability and accuracy. This research can play a significant role in achieving large-scale monitoring of crop growth status through UAV, enabling the real-time capture of winter wheat water deficit changes, and providing technical support for precision irrigation.

DSCIMABNet: A novel multi-head attention depthwise separable CNN model for skin cancer detection

Skin cancer is a common type of cancer worldwide. Early diagnosis of skin cancer can reduce the risk of death by increasing treatment success. However, it is challenging for dermatologists or specialists because the symptoms are vague in the early stages and cannot be noticed by the naked eye. This study examines digital diagnostic techniques supported by artificial intelligence, focusing on early skin cancer detection and two methods have been proposed. In the first method, DSCIMABNet deep learning architecture was developed by combining multi-head attention and depthwise separable convolution techniques. This model provides flexibility in learning the dataset's local features, abstract concepts, and long-term relationships. The DSCIMABNet model and modern deep learning models trained on ImageNet are proposed to be combined with the ensemble learning method in the second method. This approach provides a comprehensive feature extraction process that will increase the performance of the classification process with ensemble learning. The proposed approaches are trained and evaluated on the ISIC 2018 dataset with image enhancement applied in preprocessing. In the experimental results, DSCIMABNet achieved 84.28% accuracy, while the proposed hybrid method achieved 99.40% accuracy. Moreover, on the Mendeley dataset (CNN for Melanoma Detection Data), DSCIMABNet achieved 92.58% accuracy, while the hybrid method achieved 99.37% accuracy. This study may significantly contribute to developing new and effective methods for the early diagnosis and treatment of skin cancer.

Correction: Uncertainty Assessment of Surface Water Salinity Using Standalone, Ensemble, and Deep Machine Learning Methods: A Case Study of Lake Urmia

Correction: Uncertainty Assessment of Surface Water Salinity Using Standalone, Ensemble, and Deep Machine Learning Methods: A Case Study of Lake Urmia

Analysis of ensemble machine learning classification comparison on the skin cancer MNIST dataset

This study aims to analyze the performance of various ensemble machine learning methods, such as Adaboost, Bagging, and Stacking, in the context of skin cancer classification using the skin cancer MNIST dataset. We also evaluate the impact of handling dataset imbalance on the classification model’s performance by applying imbalanced data methods such as random under sampling (RUS), random over sampling (ROS), synthetic minority over-sampling technique (SMOTE), and synthetic minority over-sampling technique with edited nearest neighbor (SMOTEENN). The research findings indicate that Adaboost is effective in addressing data imbalance, while imbalanced data methods can significantly improve accuracy. However, the selection of imbalanced data methods should be carefully tailored to the dataset characteristics and clinical objectives. In conclusion, addressing data imbalance can enhance skin cancer classification accuracy, with Adaboost being an exception that shows a decrease in accuracy after applying imbalanced data methods.

Modeling the seasonal wildfire cycle and its possible effects on the distribution of focal species in Kermanshah Province, western Iran.

Predicting environmental disturbances and evaluating their potential impacts on the habitats of various plant and animal species is a suitable strategy for guiding conservation efforts. Wildfires are a type of disturbance that can affect many aspects of an ecosystem and its species. Therefore, through the integration of spatial models and species distribution models (SDMs), we can make informed predictions of the occurrence of such phenomena and their potential impacts. This study focused on five focal species, namely, the brown bear (Ursus arctos), wild goat (Capra aegagrus), wild sheep (Ovis orientalis), wildcat (Felis silvestris), and striped hyena (Hyaena hyaena). This study used MODIS active fire data and ensemble machine learning methods to model the risk of wildfire occurrence in 2023 for spring, summer, and autumn separately. This study also investigated the suitability of habitats for focal species via SDMs. The predicted probability maps for wildfire risk and habitat suitability were converted to binary values via the true skill statistic (TSS) threshold. The overlap of the habitat suitability map and wildfire occurrence areas was analyzed via GAP analysis. The area prone to fire in spring, summer and winter is equal to 9077.32; 10,199.83 and 13,723.49 KM2 were calculated, which indicates an increase in wildfire risk. Proximity to roads is one of the most important factors affecting the possible effects of wildfires in all seasons. Most fire occurrences are concentrated on agricultural lands, which, when integrated with other land use types, have wildfire potential in all seasons. The use of fire to destroy agricultural residues is a critical factor in the occurrence of wildfires. The distribution range of each focal species is considered the most important component of fire susceptibility. Hence, the suitable habitat for Hyaena hyaena in spring, summer, and autumn, with areas of 5.257, 5.856, and 6.889 km2 respectively, is the most affected by the possibility of fire. In contrast, these areas have the lowest values for Ovis orientalis, with 162, 127, and 396 km2 respectively. Therefore, species that are dependent on human-based ecosystems have the highest vulnerability to wildfire. Conservation efforts should focus on familiarizing farmers with methods of destroying agricultural residues as well as the consequences of intentional fires. The findings of this study can be used to mitigate the negative impacts of wildfire and protect the habitat of focal species.

Machine learning discrimination of Gleason scores below GG3 and above GG4 for HSPC patients diagnosis

This study aims to develop machine learning (ML)-assisted models for analyzing datasets related to Gleason scores in prostate cancer, conducting statistical analyses on the datasets, and identifying meaningful features. We retrospectively collected data from 717 hormone-sensitive prostate cancer (HSPC) patients at Yunnan Cancer Hospital. Of these, data from 526 patients were used for modeling. Seven auxiliary models were established using Logistic Regression (LR), Support Vector Machine (SVM), Random Forest (RF), Decision Tree (DT), Extreme gradient boosting tree (XGBoost), Adaptive Boosting (Adaboost), and artificial neural network (ANN) based on 21 clinical biochemical indicators and features. Evaluation metrics included accuracy (ACC), precision (PRE), specificity (SPE), sensitivity (SEN) or regression rate(Recall), and f1 score. Evaluation metrics for the models primarily included ACC, PRE, SPE, SEN or Recall, f1 score, and area under the curve(AUC). Evaluation metrics were visualized using confusion matrices and ROC curves. Among the ensemble learning methods, RF, XGBoost, and Adaboost performed the best. RF achieved a training dataset score of 0.769 (95% CI: 0.759—0.835) and a testing dataset score of 0.755 (95% CI: 0.660—0.760) (AUC: 0.786, 95%CI: 0.722—0.803), while XGBoost achieved a training dataset score of 0.755 (95% CI: 95%CI: 0.711—0.809) and a testing dataset score of 0.745 (95% CI: 0.660—0.764) (AUC: 0.777, 95% CI: 0.726—0.798). Adaboost scored 0.789 on the training dataset (95% CI: 0.782—0.857) and 0.774 on the testing dataset (95% CI: 0.651—0.774) (AUC: 0.799, 95% CI: 0.703—0.802). In terms of feature importance (FI) in ensemble learning, Bone metastases at first visit, prostatic volume, age, and T1-T2 have significant proportions in RF’s FI. fPSA, TPSA, and tumor burden have significant proportions in Adaboost’s FI, while f/TPSA, LDH, and testosterone have the highest proportions in XGBoost. Our findings indicate that ensemble learning methods demonstrate good performance in classifying HSPC patient data, with TNM staging and fPSA being important classification indicators. These discoveries provide valuable references for distinguishing different Gleason scores, facilitating more accurate patient assessments and personalized treatment plans.

Time Series Forecasting of Thermal Systems Dispatch in Legal Amazon Using Machine Learning

This paper analyzes time series forecasting methods applied to thermal systems in Brazil, specifically focusing on diesel consumption as a key determinant. Recognizing the critical role of thermal systems in ensuring energy stability, especially during low rain seasons, this study employs bagged, boosted, and stacked ensemble learning methods for time series forecasting focusing on exploring consumption patterns and trends. By leveraging historical data, the research aims to predict future diesel consumption within Brazil’s thermal energy sector. Based on the bagged ensemble learning approach a mean absolute percentage error of 0.089% and a coefficient of determination of 0.9752 were achieved (average considering 50 experiments), showing it to be a promising model for the short-time forecasting of thermal dispatch for the electric power generation system. The bagged model results were better than for boosted and stacked ensemble learning methods, long short-term memory networks, and adaptive neuro-fuzzy inference systems. Since the thermal dispatch in Brazil is closely related to energy prices, the predictions presented here are an interesting way of planning and decision-making for energy power systems.

Analysis of Lung Disease Prediction using Machine Learning Algorithms

Lung diseases are a notable in global health concern, requiring early diagnosis for better recovery and survival rates. Deep learning strategies, especially CNNs, have shown great promise in self learning lung disease diagnosis from medical images like chest X-rays. Ensemble learning methods using pretrained networks such as VGG16, InceptionV3, and MobileNetV2 have achieved up to 94% accuracy in identifying conditions like COVID-19, pneumonia, and lung opacity. Lightweight CNN models also performed well, with accuracy up to 89.89%. Traditional machine learning algorithms, including Random Forest and Logistic Regression, yielded accuracy rates between 88% and 90%. A hybrid deep learning approach, combining CNN based feature extraction with classifiers like AdaBoost, SVM, and Random Forest, improved classification accuracy by 3.1% and reduced computational complexity by 16.91%. This hybrid method highlights the main feature for integrating deep learning with traditional classifiers to enhance lung disease detection efficiency

Theories and Methods for Indoor Positioning Systems: A Comparative Analysis, Challenges, and Prospective Measures.

In the era of the Internet of Things (IoT), the demand for accurate positioning services has become increasingly critical, as location-based services (LBSs) depend on users' location data to deliver contextual functionalities. While the Global Positioning System (GPS) is widely regarded as the standard for outdoor localization due to its reliability and comprehensive coverage, its effectiveness in indoor positioning systems (IPSs) is limited by the inherent complexity of indoor environments. This paper examines the various measurement techniques and technological solutions that address the unique challenges posed by indoor environments. We specifically focus on three key aspects: (i) a comparative analysis of the different wireless technologies proposed for IPSs based on various methodologies, (ii) the challenges of IPSs, and (iii) forward-looking strategies for future research. In particular, we provide an in-depth evaluation of current IPSs, assessing them through multidimensional matrices that capture diverse architectural and design considerations, as well as evaluation metrics established in the literature. We further examine the challenges that impede the widespread deployment of IPSs and highlight the potential risk that these systems may not be recognized with a single, universally accepted standard method, unlike GPS for outdoor localization, which serves as the golden standard for positioning. Moreover, we outline several promising approaches that could address the existing challenges of IPSs. These include the application of transfer learning, feature engineering, data fusion, multisensory technologies, hybrid techniques, and ensemble learning methods, all of which hold the potential to significantly enhance the accuracy and reliability of IPSs. By leveraging these advanced methodologies, we aim to improve the overall performance of IPSs, thus paving the way for more robust and dependable LBSs in indoor environments.

Predictive Modeling of the Hydrate Formation Temperature in Highly Pressurized Natural Gas Pipelines

In this study, we aim to develop advanced machine learning regression models for the prediction of hydrate temperature based on the chemical composition of sweet gas mixtures. Data were collected in accordance with the BOTAS Gas Network Code specifications, approved by the Turkish Energy Market Regulatory Authority (EMRA), and generated using DNV GasVLe v3.10 software, which predicts the phase behavior and properties of hydrocarbon-based mixtures under various pressure and temperature conditions. We employed linear regression, decision tree regression, random forest regression, generalized additive models, and artificial neural networks to create prediction models for hydrate formation temperature (HFT). The performance of these models was evaluated using the hold-out cross-validation technique to ensure unbiased results. This study demonstrates the efficacy of ensemble learning methods, particularly random forest with an R2 and Adj. R2 of 0.998, for predicting hydrate formation conditions, thereby enhancing the safety and efficiency of gas transport and processing. This research illustrates the potential of machine learning techniques in advancing the predictive accuracy for hydrate formations in natural gas pipelines and suggests avenues for future optimizations through hybrid modeling approaches.

Exploring Technological Innovation in Wave Forecasting Using Machine Learning: A Literature Analysis

In the face of rapid technological advancements, innovations in wave forecasting are increasingly essential for effectively addressing the complex impacts of climate change. This study aims to explore technological developments in wave forecasting that can manage the complexities related to climate change and enhance the accuracy and efficiency of predictions in dynamic marine environments. Employing a qualitative approach through a Systematic Literature Review (SLR) methodology, the research focuses on literature from databases such as Scopus, DOAJ, and Google Scholar, specifically targeting publications from 2014 to 2024. Recent findings reveal that advancements in machine learning technologies, including deep learning, ensemble learning, transfer learning, and data augmentation, have significantly improved the precision and efficiency of wave forecasting models. Techniques like Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) have been particularly effective in capturing complex, non-linear patterns within wave data, enhancing the overall prediction accuracy. Ensemble learning methods have further contributed by increasing the stability and robustness of forecasts. Moreover, transfer learning and data augmentation play vital roles in adapting these models to rapidly changing environmental conditions, making them highly relevant in the context of climate change. These approaches are crucial for models to remain adaptable and responsive to dynamic oceanic conditions influenced by climate variability. The insights derived from this study are expected to provide valuable direction for the future development of machine learning-based wave forecasting models, emphasizing the need for innovative techniques that can accommodate the complexities and uncertainties brought about by climate change.

An Ensemble Machine Learning Model to Enhance Extrapolation Ability of Predicting Coarse Particulate Matter with High Resolutions in China.

Accurate exposure assessment is important for conducting PM10-2.5-related epidemiological studies, which have been limited thus far. In this study, we aimed to develop an ensemble machine learning method to estimate PM10-2.5 concentrations in mainland China during 2013-2020. The study was conducted in two stages. In the first stage, we developed two methods: the indirect method refers to developing models for PM2.5 and PM10 separately and subsequently calculating PM10-2.5 as the difference between them; and the direct method refers to establishing a model between PM10-2.5 measurements and relevant predictors directly. In the second stage, we employed an ensemble method by integrating predictions from both indirect and direct methods. Internal and external cross-validation (CV) were performed to validate the extrapolation capacity of models. The ensemble method demonstrated enhanced extrapolation accuracy in both internal and external CV compared to indirect and direct methods. The predictions produced by the ensemble method captured the spatiotemporal pattern of PM10-2.5, even in the sand and dust storm seasons. Our study introduces an ensemble strategy leveraging the strengths of both indirect and direct methods to estimate PM10-2.5 concentrations, which holds significant potential to support future epidemiological studies to address knowledge gaps in understanding the health effects of PM10-2.5.

Ranking and Combining Latent Structured Predictive Scores without Labeled Data

Combining multiple predictors obtained from distributed data sources to an accurate meta-learner is promising to achieve enhanced performance in lots of prediction problems. As the accuracy of each predictor is usually unknown, integrating the predictors to achieve better performance is challenging. Conventional ensemble learning methods assess the accuracy of predictors based on extensive labeled data. In practical applications, however, the acquisition of such labeled data can prove to be an arduous task. Furthermore, the predictors under consideration may exhibit high degrees of correlation, particularly when similar data sources or machine learning algorithms were employed during their model training. In response to these challenges, this paper introduces a novel structured unsupervised ensemble learning model (SUEL) to exploit the dependency between a set of predictors with continuous predictive scores, rank the predictors without labeled data and combine them to an ensembled score with weights. Two novel correlation-based decomposition algorithms are further proposed to estimate the SUEL model, constrained quadratic optimization (SUEL.CQO) and matrix-factorization-based (SUEL.MF) approaches. The efficacy of the proposed methods is rigorously assessed through both simulation studies and real-world application of risk genes discovery. The results compellingly demonstrate that the proposed methods can efficiently integrate the dependent predictors to an ensemble model without the need of ground truth data.

An Ensemble Learning Method for the Fault Multi-classification of Smart Meters

An Ensemble Learning Method for the Fault Multi-classification of Smart Meters

Improved transfer learning two-branch convolutional neural network image dehazing

In view of the existing image dehazing algorithms, there are problems of incomplete dehazing and image color distortion. A dehazing network combining Transfer learning sub-net and Residual attention sub-net is proposed. First, the pretrained model of the transfer learning subnet is adopted to enhance the feature attributes of the samples. Second, the struct ure of the dual-branch network is constructed, and the residual attention sub-network is used to assist the transfer learning subnetwork to train the parameters of the network model. Finally, the method of tail ensemble learning is used to fuse the features of the dual network to obtain the model parameters of the dehazed image, so as to complete the image restoration task. The experimental results show that the algorithm proposed in the paper improves the PSNR index by 1.87 dB and 4.22 dB on the RESIDE data set and the O-HAZE data set respectively, and the SSIM index on the O-HAZE data set by 6.7%.

Traffic noise prediction model using GIS and ensemble machine learning: a case study at Universiti Teknologi Malaysia (UTM) Campus.

This study represents a pioneering effort to integrate geographic information systems (GIS) and ensemble machine learning methods to predict noise levels on a university campus. Three ensemble models including random forest (RF), gradient boosting (GB), and extreme gradient boosting (XGB) were developed to predict traffic noise based on data collected over a 4-week period at the Universiti Teknologi Malaysia (UTM) campus. Noise measurements were obtained during peak morning hours (7:30 to 9:30 a.m.) on weekdays within the UTM campus in Johor. Additional predictor variables, including data from the digital elevation model (DEM) and land use, were incorporated to capture the complex nonlinear relationships influencing noise levels. The models were optimized through hyperparameter tuning, resulting in high precision, as evidenced by performance metrics such as the coefficient of determination (R2), mean absolute error (MAE), and mean squared error (MSE). The XGB model emerged as the most accurate, with R2 = 0.96, MAE = 0.9, and MSE = 0.3. Noise maps generated using the inverse distance weighting (IDW) interpolation technique highlighted the spatial distribution of noise levels, classified into five classes considering WHO standards. The findings identified distance from roads, the number of light vehicles, and proximity to green areas as the most significant predictors. However, challenges remain in accurately predicting noise levels associated with other predictors. The outcomes of the study indicate the superior performance of the XGB model compared to the GB and RF models. The study recommends several measures to manage and control noise pollution on the UTM campus, including raising awareness, regulating and enforcing vehicle speed limits, reevaluating land use, installing sound insulation systems, and planting trees and vegetation buffer zones around and within educational buildings.