Machine Learning Ensemble Models Research Articles

Evaluation of Groundwater Potential Zones Using GIS‐Based Machine Learning Ensemble Models in the Gidabo Watershed, Ethiopia

AbstractThe main objective of this study is to map and evaluate groundwater potential zones (GWPZs) using advanced ensemble machine learning (ML) models, notably Random Forest (RF) and Support Vector Machine (SVM). GWPZs are identified by considering essential factors such as geology, drainage density, slope, land use/land cover (LULC), rainfall, soil, and lineament density. This is combined with datasets used for training and validating the RF and SVM models, which consisted of 75 potential sites (boreholes and springs), 22 non‐potential sites (bare lands and settlement areas), and 20 potential sites (water bodies). Each dataset is randomly partitioned into two sets: training (70%) and validation (30%). The model's performance is evaluated using the area under the receiver operating characteristic curve (AUC‐ROC). The AUC of the RF model is 0.91, compared to 0.88 for the SVM model. Both models classified GWPZs effectively, but the RF model performed slightly better. The classified GWPZ map shows that high GWPZs are typically located within water bodies, natural springs, low‐lying regions, and forested areas. In contrast, low GWPZs are primarily found in shrubland and grassland areas. This study is vital for decision‐makers as it promotes sustainable groundwater use and ensures water security in the studied area.

Predicting and Analyzing Cardiovascular Disease through Ensemble Learning Approaches

Among the toughest assignments for medical professionals is discovering heart illness indicators as quickly as attainable. Coronary artery disease is an urgent issue and should be treated promptly. The diagnosis of heart illness is complicated by a number of factors affecting health, including high pressure, situated cholesterol levels, inconsistent heartbeat, and several more. Therefore, AI can be helpful in recognising and dealing with ailments at an early stage. This research suggests an ensemble-based method to estimate an individual's risk of heart disease using Deep Learning (DL) and Machine Learning (ML) models. In order to forecast cardiovascular disease, we use six classification methods. A large collection of cardiovascular disease cases that is made open to the world is used to train models. To identify key characteristics related to cardiac illness, we employ Random Forest (RF). The research's results show that the ML ensemble model obtains the highest accuracy of 92.75% in predicting diseases. When compared to conventional machine learning methods like K-nearest neighbor (KNN), Random Forest (RF), and Multi-Layer Perceptrons (MLP), the suggested methodology's uniqueness is evaluated by showing a 5.52% increase in efficiency.

An optimized ensemble model for predicting average localization error of wireless sensor networks

Wireless sensor networks (WSNs) are widely utilized in various applications due to their compact size, cost-effectiveness, and ease of deployment. Nonetheless, one of the biggest problems in WSNs is getting a reasonable estimate of the average location error of a node at the setup in the least amount of time. Wireless sensor networks can undergo changes over time due to various external and internal factors, such as environmental conditions, network congestion, hardware failures, or software updates. When these changes occur, the network may require redesigning, which can incur significant expenses. Traditional WSNs approaches, on the other hand, have been explicitly programmed, which makes it hard for networks to respond dynamically. Therefore, machine learning (ML) techniques can be used to respond appropriately in such scenarios. In this work, we proposed an optimized ML ensemble model for (i) identifying the critical network parameters for node localization when setting up wireless sensor networks with the accuracy needed in a short amount of time and (ii) predicting the average localization error of wireless sensor networks. We used the random forest algorithm with optimized hyperparameters from different optimization techniques to predict average localization error (ALE) using independent features like node density, anchor ratio, transmission range, and iterations.

Machine learning ensemble model prediction of northward shift in potato cyst nematodes (Globodera rostochiensis and G. pallida) distribution under climate change conditions

Potato cyst nematodes (PCNs) are a significant threat to potato production, having caused substantial damage in many countries. Predicting the future distribution of PCN species is crucial to implementing effective biosecurity strategies, especially given the impact of climate change on pest species invasion and distribution. Machine learning (ML), specifically ensemble models, has emerged as a powerful tool in predicting species distributions due to its ability to learn and make predictions based on complex data sets. Thus, this research utilised advanced machine learning techniques to predict the distribution of PCN species under climate change conditions, providing the initial element for invasion risk assessment. We first used Global Climate Models to generate homogeneous climate predictors to mitigate the variation among predictors. Then, five machine learning models were employed to build two groups of ensembles, single-algorithm ensembles (ESA) and multi-algorithm ensembles (EMA), and compared their performances. In this research, the EMA did not always perform better than the ESA, and the ESA of Artificial Neural Network gave the highest performance while being cost-effective. Prediction results indicated that the distribution range of PCNs would shift northward with a decrease in tropical zones and an increase in northern latitudes. However, the total area of suitable regions will not change significantly, occupying 16–20% of the total land surface (18% under current conditions). This research alerts policymakers and practitioners to the risk of PCNs’ incursion into new regions. Additionally, this ML process offers the capability to track changes in the distribution of various species and provides scientifically grounded evidence for formulating long-term biosecurity plans for their control.

Efficient prediction of optical properties in hexagonal PCF using machine learning models

This research explores the use of machine learning (ML) models to forecast optical characteristics in photonic crystal fibers (PCF). Specifically, we focus on a solid core index-guided PCF with a hexagonal cladding arrangement. The primary challenges to PCF propagation analysis and predictions are accuracy, computational error, and time constraints. To address these difficulties, we have specially used ML ensemble models including Decision Tree Regressor (DTR), Random Forest Regressor (RFR), Gradient Boosting Regressor (GBR), eXtreme Gradient Boosting Regression (XGBR), and Bagging Regressor (BR). Model performance is assessed using metrics like Mean Squared Error (MSE) and R-squared (R2) through 10-fold cross-validation. Our key findings show that the GBR model outperforms other models and shows extremely low MSE and outstanding R2 values in predicting effective refractive index (Neff), effective mode area (Aeff), confinement loss, and dispersion. In addition, the study compares the performance of ML models with that of previous works using Artificial Neural Network (ANN), demonstrating improved efficiency in predicting optical characteristics for hexagonal PCFs.

Site-specific production forecast through data-driven and engineering models

The present study investigates the predictive capabilities of two site-specific wind power forecasting models. The two approaches make use of a machine learning ensemble model that combines forecasts from several numerical weather predictions (NWP) to match local observations. In the first approach, the training data consists of historical site power production recordings. This yields a black-box model that includes the wind farm behaviour. The second approach is a combination of data-driven and explicit modelling. In the first step, the machine learning ensemble model forecasts the site wind conditions. Then, this input is fed to a site-specific engineering wake model. This hybrid ”grey-box” approach explicitly resolves some farm flow effects.The study shows that the performance of both methods differs over the wind speed range. The purely data-driven model performs better at medium wind speeds, which also occur more often. The hybrid method predictions agree better during periods of higher wind speeds. In the last part, the study showcases the capabilities of the forecasting methods when participating in the energy market. Additionally, the engineering wake model allows for power maximisation through wind farm control.

Gene network-based and ensemble modeling-based selection of tumor-associated antigens with a predicted low risk of tissue damage for targeted immunotherapy

BackgroundTumor-associated antigens and their derived peptides constitute an opportunity to design off-the-shelf mainline or adjuvant anti-cancer immunotherapies for a broad array of patients. A performant and rational antigen selection pipeline...

Exploring the injury severity of vulnerable road users to truck crashes by ensemble learning

Large numbers of vulnerable road users were killed in truck crashes. In this study, ensemble machine learning models are constructed to predict the injury severity of the vulnerable road user (VRU) to truck (VRU-T) crashes. The study is based on the five years (2017–2021) of VRU-T crash data in the Shandong Province from the Center for Accident Research in Zibo. The injury severity of VRUs is estimated using machine learning ensemble models- Stacking, Voting, Random Forest, and eXtreme Gradient Boosting (XGBoost). Compared to the other three models, the Stacking has excellent predictive performance on the pedestrian and non-motorized datasets. Then, SHapley Additive exPlanations and Partial Dependence Plot box are introduced to analyze risk factors qualitatively and quantitatively. The innovative findings of this study are as follows: (1) as VRUs age, they are more likely to be seriously injured in truck crashes; (2) middle-aged truck drivers and truck drivers with medium driving experience increase the probability of VRUs being severe and fatally injured in truck crashes; (3) crashes involving heavy trucks, under signalized crossing, or on the national and provincial road and urban road have a positive effect on the crash severity for cyclists, and E-Bike riders.

Inferring the Redshift of More than 150 GRBs with a Machine-learning Ensemble Model

Gamma-ray bursts (GRBs), due to their high luminosities, are detected up to a redshift of 10, and thus have the potential to be vital cosmological probes of early processes in the Universe. Fulfilling this potential requires a large sample of GRBs with known redshifts, but due to observational limitations, only 11% have known redshifts (z). There have been numerous attempts to estimate redshifts via correlation studies, most of which have led to inaccurate predictions. To overcome this, we estimated GRB redshift via an ensemble-supervised machine-learning (ML) model that uses X-ray afterglows of long-duration GRBs observed by the Neil Gehrels Swift Observatory. The estimated redshifts are strongly correlated (a Pearson coefficient of 0.93) and have an rms error, namely, the square root of the average squared error 〈Δz 2〉, of 0.46 with the observed redshifts showing the reliability of this method. The addition of GRB afterglow parameters improves the predictions considerably by 63% compared to previous results in peer-reviewed literature. Finally, we use our ML model to infer the redshifts of 154 GRBs, which increase the known redshifts of long GRBs with plateaus by 94%, a significant milestone for enhancing GRB population studies that require large samples with redshift.

A reproducible ensemble machine learning approach to forecast dengue outbreaks

Dengue fever, a prevalent and rapidly spreading arboviral disease, poses substantial public health and economic challenges in tropical and sub-tropical regions worldwide. Predicting infectious disease outbreaks on a countrywide scale is complex due to spatiotemporal variations in dengue incidence across administrative areas. To address this, we propose a machine learning ensemble model for forecasting the dengue incidence rate (DIR) in Brazil, with a focus on the population under 19 years old. The model integrates spatial and temporal information, providing one-month-ahead DIR estimates at the state level. Comparative analyses with a dummy model and ablation studies demonstrate the ensemble model’s qualitative and quantitative efficacy across the 27 Brazilian Federal Units. Furthermore, we showcase the transferability of this approach to Peru, another Latin American country with differing epidemiological characteristics. This timely forecast system can aid local governments in implementing targeted control measures. The study advances climate services for health by identifying factors triggering dengue outbreaks in Brazil and Peru, emphasizing collaborative efforts with intergovernmental organizations and public health institutions. The innovation lies not only in the algorithms themselves but in their application to a domain marked by data scarcity and operational scalability challenges. We bridge the gap by integrating well-curated ground data with advanced analytical methods, addressing a significant deficiency in current practices. The successful transfer of the model to Peru and its consistent performance during the 2019 outbreak in Brazil showcase its scalability and practical application. While acknowledging limitations in handling extreme values, especially in regions with low DIR, our approach excels where accurate predictions are critical. The study not only contributes to advancing DIR forecasting but also represents a paradigm shift in integrating advanced analytics into public health operational frameworks. This work, driven by a collaborative spirit involving intergovernmental organizations and public health institutions, sets a precedent for interdisciplinary collaboration in addressing global health challenges. It not only enhances our understanding of factors triggering dengue outbreaks but also serves as a template for the effective implementation of advanced analytical methods in public health.

Assessing the feasibility of applying machine learning to diagnosing non-effusive feline infectious peritonitis

Feline infectious peritonitis (FIP) is a severe feline coronavirus-associated syndrome in cats, which is invariably fatal without anti-viral treatment. In the majority of non-effusive FIP cases encountered in practice, confirmatory diagnostic testing is not undertaken and reliance is given to the interpretation of valuable, but essentially non-specific, clinical signs and laboratory markers. We hypothesised that it may be feasible to develop a machine learning (ML) approach which may be applied to the analysis of clinical data to aid in the diagnosis of disease. A dataset encompassing 1939 suspected FIP cases was scored for clinical suspicion of FIP on the basis of history, signalment, clinical signs and laboratory results, using published guidelines, comprising 683 FIP (35.2%), and 1256 non-FIP (64.8%) cases. This dataset was used to train, validate and evaluate two diagnostic machine learning ensemble models. These models, which analysed signalment and laboratory data alone, allowed the accurate discrimination of FIP and non-FIP cases in line with expert opinion. To evaluate whether these models may have value as a diagnostic tool, they were applied to a collection of 80 cases for which the FIP status had been confirmed (FIP: n = 58 (72.5%), non–FIP: n = 22 (27.5%)). Both ensemble models detected FIP with an accuracy of 97.5%, an area under the curve (AUC) of 0.969, sensitivity of 95.45% and specificity of 98.28%. This work demonstrates that, in principle, ML can be usefully applied to the diagnosis of non-effusive FIP. Further work is required before ML may be deployed in the laboratory as a diagnostic tool, such as training models on datasets of confirmed cases and accounting for inter-laboratory variation. Nevertheless, these results illustrate the potential benefit of applying ML to standardising and accelerating the interpretation of clinical pathology data, thereby improving the diagnostic utility of existing laboratory tests.

A Machine Learning Ensemble Model for the Detection of Cyberbullying

The pervasive use of social media platforms, such as Facebook, Instagram, and X, has significantly amplified our electronic interconnectedness. Moreover, these platforms are now easily accessible from any location at any given time. However, the increased popularity of social media has also led to cyberbullying.It is imperative to address the need for finding, monitoring, and mitigating cyberbullying posts on social media platforms. Motivated by this necessity, we present this paper to contribute to developing an automated system for detecting binary labels of aggressive tweets.Our study has demonstrated remarkable performance compared to previous experiments on the same dataset. We employed the stacking ensemble machine learning method, utilizing four various feature extraction techniques to optimize performance within the stacking ensemble learning framework. Combining five machine learning algorithms,Decision Trees, Random Forest, Linear Support Vector Classification, Logistic Regression, and K-Nearest Neighbors into an ensemble method, we achieved superior results compared to traditional machine learning classifier models. The stacking classifier achieved a high accuracy rate of 94.00%, outperforming traditional machine learning models and surpassing the results of prior experiments that utilized the same dataset. The outcomes of our experiments showcased an accuracy rate of 0.94% in detection tweets as aggressive or non-aggressive.

Predictive Modeling of Energy Poverty with Machine Learning Ensembles: Strategic Insights from Socioeconomic Determinants for Effective Policy Implementation

This study aims to identify the key predictors of the multidimensional energy poverty index (MEPI) by employing advanced machine learning (ML) ensemble methods. Traditional energy poverty research often relies on conventional statistical techniques, which limits the understanding of complex socioeconomic factors. To address this gap, we propose an approach using three distinct ML ensemble models: extreme gradient boosting (XGBoost)‐random forest (RF), XGBoost‐multiple linear regression (MLR), and XGBoost‐artificial neural network (ANN). These models are applied to a comprehensive dataset encompassing various socioeconomic indicators. The findings demonstrate that the XGBoost‐RF ensemble achieves exceptional accuracy and reliability, with a root mean squared error (RMSE) of 0.041, an R‐squared (R2) of 0.975, and a Pearson correlation coefficient of 0.992. The XGBoost‐MLR ensemble shows superior generalizability, maintaining a consistent R2 of 0.845 across both the testing and training phases. The XGBoost‐ANN model balances complexity with predictive capability, achieving an RMSE of 0.056, an R2 of 0.954 in the testing phase, and an R2 of 0.799 in training. Significantly, the study identifies “Education,” “Food Consumption Score (FCS),” “Household Food Insecurity Access Scale (HFIA),” and “Dietary Diversity Score (DDS)” as critical predictors of MEPI. These results highlight the intricate relationship between energy poverty and factors related to food security and education. By integrating the insights from these ML models with policy initiatives, this study offers a promising new approach to addressing energy poverty. It highlights the importance of education, food security, and socioeconomic factors in crafting effective policy interventions.

Effectiveness of machine learning ensemble models in assessing groundwater potential in Lidder watershed, India

Effectiveness of machine learning ensemble models in assessing groundwater potential in Lidder watershed, India

Stacked machine learning models for predicting species richness and endemism for Mediterranean endemic plants in the Mareotis subsector in Egypt

An effective method for identifying species and evaluating the effects of changes caused by humans on specific species is the application of species distribution modelling (SDM) in desert environments. The fact that many dry lands and deserts throughout the world are situated in inhospitable regions may be the reason why such applications are still infrequently used on plant species in Egypt's Mediterranean region. Henceforth, the current study aims to map species richness and weighted endemism of Mediterranean endemics in the Mareotis subsector in Egypt and determine the environmental variables influencing distribution of these taxa. We produced a map of species distribution range using Ensemble SDMs. Further, stacked machine learning ensemble models derived from Random Forest (RF) and MaxEnt models were applied on 382 Mediterranean endemics distribution data to estimate and map diversity and endemism using two indices: species richness (SR) and weighted endemism index (WEI). The best models for ensemble modelling were chosen based on Kappa values and the Area Under the Receiver Operator Curve (AUC). The results showed that the models had a good predictive ability (Area Under the Curve (AUC) for all SDMs was > 0.75), indicating high accuracy in forecasting the potential geographic distribution of Mediterranean endemics. The main bioclimatic variables that impacted potential distributions of most species were wind speed, elevation and minimum temperature of coldest month. According to our models, six hotspots were determined for Mediterranean endemics in the present study. The highest species richness was recorded in Sallum, Matrouh wadis and Omayed, followed by Burg El-Arab, Ras El-Hekma and Lake Mariut. Indeed, species richness and endemism hotspots are promising areas for conservation planning. This study can help shape policy and mitigation efforts to protect and preserve Mediterranean endemics in the coastal desert of Egypt. These hotspots should be focused on by policy makers and stakeholders and declared as protectorates in the region. The largest number of species per area would be protected by focusing primarily on the hotspots with high species richness.

Interpretable ensemble prediction for anaerobic digestion performance of hydrothermal carbonization wastewater

Hydrothermal carbonization (HTC) is a method to improve fuel quality that can directly treat wet solid waste, but the treatment produces large amounts of wastewater. Hydrothermal carbonation wastewater treatment for methane production by anaerobic digestion can lead to waste utilization and energy saving. However, anaerobic digestion performance prediction of HTC wastewater is challenging due to the complexity of influencing factors. This study applies interpretable machine learning combined with ensemble learning to construct ensemble prediction models for the biogas yield and CH4 concentration. The machine learning ensemble model can integrate the advantages of single models and effectively improve the prediction accuracy of the anaerobic digestion performance of HTC wastewater, with the best R2 reaching 0.836 and 0.820, respectively, which is better than 0.780 and 0.802 of the best single models. The SHapley Additive exPlanations theory is combined with the ensemble models to show that anaerobic digestion reacted time with HTC temperature, pH, and COD has a coupling effect on daily biogas yield and CH4 concentration.

Machine learning-driven scenario-based models for predicting desert dust sources in central playas of Iran

Monitoring and controlling dust sources under climate changes and developing suitable prediction approaches is very important, since they have direct impacts on the environment and human health. The current study was performed with the aim of predicting the dust emission of Yazd province, located in the central playas of Iran, by combining a machine learning ensemble model and the global climate model IPSL-CM6A-LR. The key variables of dust emission including physiographic characteristics, climatic variables and human factors were mapped in order to model occurrence data against background data. After removing 15 autocorrelated points, from 120 pixels located in dust sources, 75 % and 25 % of them were randomly selected as training and test datasets, respectively. To evaluate modeling, in addition to sensitivity, specificity, TSS and Kappa indices, the Partial ROC approach was used. For the final mapping of land susceptibility to dust emission, individual models based on the weighted average of AUCratio values were used to create the ensemble model. The relative importance of dust emission controlling factors was obtained with the consensus of seven machine learning models: distance to dirt roads, altitude, distance to mines and mean temperature of driest quarter. MARS outperformed the other individual models, so that it showed no significant difference with the ensemble model at the thresholds of E = 10 and E = 5 (p-value < 0.05). Statistical analysis of the ensemble model with AUCratio equal to 1.933 showed that currently 28.56 percent of the province, is prone to dust emission and it will increase linearly in the future for scenarios SSP1-2.6 and SSP5-8.5. In both time periods (2041–2070 and 2071–2100), dust emissions will be higher under the SSP5-8.5 scenario compared to the SSP1-2.6 scenario. The results can be used by policymakers for the sustainable management of deserts and thus reduce the impacts of dust on current and future ecosystem health.

Robust genetic machine learning ensemble model for intrusion detection in network traffic

Network security has developed as a critical research subject as a result of the Rapid advancements in the development of Internet and communication technologies over the previous decades. The expansion of networks and data has caused cyber-attacks on the systems, making it difficult for network security to detect breaches effectively. Current Intrusion Detection Systems (IDS) have several flaws, including their inability to prevent attacks on their own, the requirement for a professional engineer to administer them, and the occurrence of false alerts. As a result, a plethora of new attacks are being created, making it harder for network security to properly detect breaches. Despite the best efforts, IDS continues to struggle with increasing detection accuracy while lowering false alarm rates and detecting new intrusions. Therefore, network intrusion detection enhancement by preprocessing and generation of highly reliable algorithms is the main focus nowadays. Machine learning (ML) based IDS systems have recently been implemented as viable solutions for quickly detecting intrusions across the network. In this study, we use a combined data analysis technique with four Robust Machine learning ensemble algorithms, including the Voting Classifier, Bagging Classifier, Gradient Boosting Classifier, and Random Forest-based Bagging algorithm along with the proposed Robust genetic ensemble classifier. For each algorithm, a model is created and tested using a Network Dataset. To assess the performance of both algorithms in terms of their ability to anticipate the anomaly occurrence, graphs of performance rates have been evaluated. The suggested algorithm outperformed other methods as it shows the lowest values of mean square error (MSE) and mean absolute error (MAE). The experiments were conducted on the Network traffic dataset available on Kaggle, on the Python platform, which has limited samples. The proposed method can be applied in the future with more machine learning ensemble classifiers and deep learning techniques.

An explainable machine learning ensemble model to predict the risk of ovarian cancer in BRCA-mutated patients undergoing risk-reducing salpingo-oophorectomy.

It has been estimated that 19,880 new cases of ovarian cancer had been diagnosed in 2022. Most epithelial ovarian cancer are sporadic, while in 15%-25% of cases, there is evidence of a familial or inherited component. Approximately 20%-25% of high-grade serous carcinoma cases are caused by germline mutations in the BRCA1 and BRCA2 genes. However, owing to a lack of effective early detection methods, women with BRCA mutations are recommended to undergo bilateral risk-reducing salpingo-oophorectomy (RRSO) after childbearing. Determining the right timing for this procedure is a difficult decision. It is crucial to find a clinical signature to identify high-risk BRCA-mutated patients and determine the appropriate timing for performing RRSO. In this work, clinical data referred to a cohort of 184 patients, of whom 7.6% were affected by adnexal tumors including invasive carcinomas and intraepithelial lesions after RSSO has been analyzed. Thus, we proposed an explainable machine learning (ML) ensemble approach using clinical data commonly collected in clinical practice to early identify BRCA-mutated patients at high risk of ovarian cancer and consequentially establish the correct timing for RRSO. The ensemble model was able to handle imbalanced data achieving an accuracy value of 83.2%, a specificity value of 85.3%, a sensitivity value of 57.1%, a G-mean value of 69.8%, and an AUC value of 71.1%. In agreement with the promising results achieved, the application of suitable ML techniques could play a key role in the definition of a BRCA-mutated patient-centric clinical signature for ovarian cancer risk and consequently personalize the management of these patients. As far as we know, this is the first work addressing this task from an ML perspective.

Robustness of Optimized Decision Tree-Based Machine Learning Models to Map Gully Erosion Vulnerability

Gully erosion is a worldwide threat with numerous environmental, social, and economic impacts. The purpose of this research is to evaluate the performance and robustness of six machine learning ensemble models based on the decision tree principle: Random Forest (RF), C5.0, XGBoost, treebag, Gradient Boosting Machines (GBMs) and Adaboost, in order to map and predict gully erosion-prone areas in a semi-arid mountain context. The first step was to prepare the inventory data, which consisted of 217 gully points. This database was then randomly subdivided into five percentages of Train/Test (50/50, 60/40, 70/30, 80/20, and 90/10) to assess the stability and robustness of the models. Furthermore, 17 geo-environmental variables were used as potential controlling factors, and several metrics were examined to evaluate the performance of the six models. The results revealed that all of the models used performed well in terms of predicting vulnerability to gully erosion. The C5.0 and RF models had the best prediction performance (AUC = 90.8 and AUC = 90.1, respectively). However, according to the random subdivisions of the database, these models exhibit small but noticeable instability, with high performance for the 80/20% and 70/30% subdivisions. This demonstrates the significance of database refining and the need to test various splitting data in order to ensure efficient and reliable output results.