Decision Tree-based Machine Learning Research Articles

A Meta-Ensemble Machine Learning Strategy to Assess Groundwater Holistic Vulnerability in Coastal Aquifers

Assessing the vulnerability of groundwater in coastal aquifers is crucial for mitigating risks associated with seawater intrusion and anthropogenic impacts. This study introduces an innovative machine learning (ML)-enhanced methodology that synergizes the strengths of two established vulnerability assessment frameworks, DRASTIC and GALDIT. The hybrid approach overcomes the limitations of each framework—DRASTIC’s inadequacies in coastal settings and GALDIT’s limited consideration of agricultural and industrial impacts. Utilizing advanced decision tree-based ML algorithms—Adaptive Boosting (AdaBoost), Extreme Gradient Boosting (XGBoost), Light Gradient Boosting Machine (LGBM), and Random Forest (RF)—this research was conducted in the Azarshahr Plain, NW Iran. Model efficacy was validated using Pearson’s correlation coefficient (r) and distance correlation (DC), with nitrate (NO3–) and total dissolved solids (TDS) serving as proxies for evaluating the DRASTIC and GALDIT models, respectively. The original DRASTIC indices exhibited weak correlations with NO3– (r = 0.24, DC = 0.25), but ML-enhanced models, particularly AdaBoost, showed significant improvements (r = 0.78, DC = 0.79). Similar enhancements were observed with GALDIT, where correlations improved markedly with AdaBoost integration. A sophisticated second-level AdaBoost meta-ensemble was developed to integrate enhanced DRASTIC and GALDIT assessments, achieving superior correlation metrics (r = 0.80, DC = 0.84 for NO3–; r = 0.82, DC = 0.83 for TDS). These results underscore the effectiveness of an integrated ML-based approach in advancing beyond traditional vulnerability assessment methods, providing a more comprehensive, accurate, and robust evaluation of coastal aquifer vulnerability.

Explainable artificial intelligence prediction of defect characterization in composite materials

Non-destructive evaluation (NDE) techniques are integral across diverse applications for void detection within composites. Infrared (IR) thermography (IRT) is a prevalent NDE technique that utilizes reverse heat transfer principles to infer defect characteristics by analyzing temperature distribution. Although the forward heat transfer problem is well-posed, its inverse counterpart lacks uniqueness, posing non-unique solutions. The present study performs simulations using finite element analysis (FEA) in defective (a penny-shaped defect) composites through which the heat transfer flux is modeled. A total of 2100 simulations with various defect positions and sizes (depth, size, and thickness) are executed, and the corresponding surface temperature vs. time and vs. distance diagrams are extracted. The FEA outputs provide ample input data for developing an explainable artificial intelligence (XAI) model to estimate the defect characteristics. A detailed feature engineering task is performed to select the representative information from the diagrams. Explainable decision tree-based machine learning (ML) models with transparent decision paths based on derived features are developed to predict the defect depth, size, and thickness. The ML models’ results suggest superb accuracy (R2 = 0.92 to 0.99) across all three defect characteristics. The provided workflow sets a benchmark applicable to a range of fields, including health monitoring.

Temporal heterogeneity in the performance of machine learning models for PM2.5 concentration estimation

Machine learning (ML) methods have been applied extensively to simulate air pollutant concentrations and assess individual exposure in epidemiological studies. However, there is still a paucity of research on the temporal heterogeneity of ML model performance and the impact of dataset size. To explore the temporal heterogeneity in model performance when estimating daily concentrations of fine particulate matter (PM2.5) across China in 2021, we compared five decision tree-based ML models (Random Forest (RF), Categorical Boosting (CatBoost), Gradient Boost Regression Tree (GBRT), eXtreme Gradient Boosting (XGBoost), and Light Gradient Boosting Machine (LightGBM)) across daily scales within three distinct timeframes. The performance of all models was evaluated using cross-validation. We observed that the performance of ML models varied with time, which showed a significant correlation with PM2.5 concentration. Among the 365 days in 2021, RF model performed best, the annual mean R2 was 0.86, with a minimum of 0.84 and a maximum of up to 0.95. For RF, we chose a cubic polynomial curve to fit the relationship between model performance and PM2.5 concentrations, and based on this, we devised a model selection strategy for different time scales, achieving an accuracy rate of up to 79.45 %, with the selected models having an average R2 of 0.85, and a maximum of up to 0.95. Additionally, we found that increasing the dataset size did not significantly improve model performance. Instead, it resulted in considerably longer runtime and increased memory usage. The methodology and findings of this study hold significant value for advancing the development of more efficient and precise modeling approaches for air pollutant concentrations. Furthermore, this research provides a foundation for regional air pollutant governance and future health-related research.

Evaluating the efficacy of decision tree-based machine learning in classifying intrusive behaviour of network users

Evaluating the efficacy of decision tree-based machine learning in classifying intrusive behaviour of network users

Monitoring saltwater intrusion to estuaries based on UAV and satellite imagery with machine learning models

Saltwater intrusion is a natural mixture process between watershed freshwater and seawater that frequently occurs in estuaries. Station-based monitoring of saltwater intrusion is time-consuming and labor-intensive. To enable quick monitoring of saltwater intrusion, this study developed new remote sensing algorithms for water surface salinity measurement using four decision tree-based machine learning models. These models were built based on simultaneously collected in-situ salinity data from a waterway in the Pearl River Delta and Unmanned Aerial Vehicle (UAV) hyperspectral images. A 10-fold cross-validation was applied to assess the performance of the models, with XGBoost outperforming the other three models (R2=0.93, RMSE = 0.88 psu). Then the developed model was employed for Sentinel-2 multispectral satellite images to invert the estuarine salinity distribution at a larger spatial scale. Results displayed the high performance of the machine learning models proposed in this study for mapping the salinity distribution in river channels, making it an efficient and practical technique for monitoring saltwater intrusion in river channels at a regional scale.

Urban flood susceptibility mapping using frequency ratio and multiple decision tree-based machine learning models

Urban flood susceptibility mapping using frequency ratio and multiple decision tree-based machine learning models

5-EPIFAT trial protocol: a multi-center, randomized, placebo-controlled trial of the efficacy of pharmacotherapy for fatigue using methylphenidate, bupropion, ginseng, and amantadine in advanced cancer patients on active treatment

BackgroundCancer-related fatigue (CRF) is still undertreated in most patients, as evidence for pharmacological treatments is limited and conflicting. Also, the efficacy of the pharmacological agents relative to each other is still unclear. Therefore, medications that may potentially contribute to improving CRF will be investigated in this head-to-head trial. Our main objective is to compare the efficacy of methylphenidate vs. bupropion vs. ginseng vs. amantadine vs. placebo in patients with advanced cancer.MethodsThe 5-EPIFAT study is a 5-arm, randomized, multi-blind, placebo-controlled, multicenter trial that will use a parallel-group design with an equal allocation ratio comparing the efficacy and safety of four medications (Methylphenidate vs. Bupropion vs. Ginseng vs. Amantadine) versus placebo for management of CRF. We will recruit 255 adult patients with advanced cancer who experience fatigue intensity ≥ 4 based on a 0–10 scale. The study period includes a 4-week intervention and a 4-week follow-up with repeated measurements over time. The primary outcome is the cancer-related fatigue level over time, which will be measured by the functional assessment of chronic illness therapy-fatigue (FACIT-F) scale. To evaluate safety, the secondary outcome is the symptomatic adverse events, which will be assessed using the Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events in cancer clinical trials (PRO-CTCAE). Also, a subgroup analysis based on a decision tree-based machine learning algorithm will be employed for the clinical prediction of different agents in homogeneous subgroups.DiscussionThe findings of the 5-EPIFAT trial could be helpful to guide clinical decision-making, personalization treatment approach, design of future trials, as well as the development of CRF management guidelines.Trial registrationIRCT.ir IRCT20150302021307N6. Registered on 13 May 2023.

Interpretable machine learning-based individual analysis of acute kidney injury in immune checkpoint inhibitor therapy.

Acute kidney injury (AKI) is a critical complication of immune checkpoint inhibitor therapy. Since the etiology of AKI in patients undergoing cancer therapy varies, clarifying underlying causes in individual cases is critical for optimal cancer treatment. Although it is essential to individually analyze immune checkpoint inhibitor-treated patients for underlying pathologies for each AKI episode, these analyses have not been realized. Herein, we aimed to individually clarify the underlying causes of AKI in immune checkpoint inhibitor-treated patients using a new clustering approach with Shapley Additive exPlanations (SHAP). We developed a gradient-boosting decision tree-based machine learning model continuously predicting AKI within 7 days, using the medical records of 616 immune checkpoint inhibitor-treated patients. The temporal changes in individual predictive reasoning in AKI prediction models represented the key features contributing to each AKI prediction and clustered AKI patients based on the features with high predictive contribution quantified in time series by SHAP. We searched for common clinical backgrounds of AKI patients in each cluster, compared with annotation by three nephrologists. One hundred and twelve patients (18.2%) had at least one AKI episode. They were clustered per the key feature, and their SHAP value patterns, and the nephrologists assessed the clusters' clinical relevance. Receiver operating characteristic analysis revealed that the area under the curve was 0.880. Patients with AKI were categorized into four clusters with significant prognostic differences (p = 0.010). The leading causes of AKI for each cluster, such as hypovolemia, drug-related, and cancer cachexia, were all clinically interpretable, which conventional approaches cannot obtain. Our results suggest that the clustering method of individual predictive reasoning in machine learning models can be applied to infer clinically critical factors for developing each episode of AKI among patients with multiple AKI risk factors, such as immune checkpoint inhibitor-treated patients.

Combined cortical thickness and blink reflex recovery cycle to differentiate essential tremor with and without resting tremor.

To investigate the performance of structural MRI cortical and subcortical morphometric data combined with blink-reflex recovery cycle (BRrc) values using machine learning (ML) models in distinguishing between essential tremor (ET) with resting tremor (rET) and classic ET. We enrolled 47 ET, 43 rET patients and 45 healthy controls (HC). All participants underwent brain 3 T-MRI and BRrc examination at different interstimulus intervals (ISIs, 100-300 msec). MRI data (cortical thickness, volumes, surface area, roughness, mean curvature and subcortical volumes) were extracted using Freesurfer on T1-weighted images. We employed two decision tree-based ML classification algorithms (eXtreme Gradient Boosting [XGBoost] and Random Forest) combining MRI data and BRrc values to differentiate between rET and ET patients. ML models based exclusively on MRI features reached acceptable performance (AUC: 0.85-0.86) in differentiating rET from ET patients and from HC. Similar performances were obtained by ML models based on BRrc data (AUC: 0.81-0.82 in rET vs. ET and AUC: 0.88-0.89 in rET vs. HC). ML models combining imaging data (cortical thickness, surface, roughness, and mean curvature) together with BRrc values showed the highest classification performance in distinguishing between rET and ET patients, reaching AUC of 0.94 ± 0.05. The improvement in classification performances when BRrc data were added to imaging features was confirmed by both ML algorithms. This study highlights the usefulness of adding a simple electrophysiological assessment such as BRrc to MRI cortical morphometric features for accurately distinguishing rET from ET patients, paving the way for a better classification of these ET syndromes.

Fourier transform infrared spectroscopy and machine learning for Porphyromonas gingivalis detection in oral bacteria.

Porphyromonas gingivalis, a Gram-negative anaerobic bacillus, is the primary pathogen in periodontitis. Herein, we cultivated strains of oral bacteria, including P. gingivalis and the oral commensal bacteria Actinomyces viscosus and Streptococcus mutans, and recorded the infrared absorption spectra of the gases released by the cultured bacteria at a resolution of 0.5cm-1 within the wavenumber range of 500-7500cm-1. From these spectra, we identified the infrared wavenumbers associated with characteristic absorptions in the gases released by P. gingivalis using a decision tree-based machine learning algorithm. Finally, we compared the obtained absorbance spectra of ammonia (NH3) and carbon monoxide (CO) using the HITRAN database. We observed peaks at similar positions in the P. gingivalis gases, NH3, and CO spectra. Our results suggest that P. gingivalis releases higher amounts of NH3 and CO than A. viscosus and S. mutans. Thus, combining Fourier transform infrared spectroscopy with machine learning enabled us to extract the specific wavenumber range that differentiates P. gingivalis from a vast dataset of peak intensity ratios. Our method distinguishes the gases from P. gingivalis from those of other oral bacteria and provides an effective strategy for identifying P. gingivalis in oral bacteria. Our proposed methodology could be valuable in clinical settings as a simple, noninvasive pathogen diagnosis technique.

Application of regression decision tree and machine learning algorithms to examine students’ online learning preferences during COVID-19 pandemic

The emergence of the novel coronavirus (COVID-19) profoundly disrupted the field of education, ushering in an era of widespread online learning adoption. This research paper seeks to investigate the multifaceted factors influencing students' preferences for online learning. Employing data exploration techniques and machine learning algorithms, the study aimed to identify the pivotal variables affecting students' willingness and performance in online educational environments. The research encompassed data collection through designated questionnaires and the application of decision tree-based machine learning algorithms to analyze these diverse factors. Through this approach, seven specific prerequisites were derived, employing multiple linear regression analysis within the decision tree framework, to illuminate the relationships between these factors. Key aspects considered in these prerequisites included factors such as "internet connectivity issues," "COVID-19 pandemic-induced stress," "COVID-19 vaccination status," and "close relatives' COVID-19 infections". Foremost among the reasons for students' reluctance to embrace online learning was the presence of "internet difficulties," including issues like slow connections and frequent disruptions. From the results of this research, it can be concluded that basic computer and internet courses can be beneficial for encouraging online education. Findings of this study underscore the potential benefits of offering basic computer and internet courses as a means to encourage and facilitate effective online education, particularly in the context of the COVID-19 pandemic.

A New Practical Formula for Bond-Slip Stress of Cold-Formed CFSTs Stub-Columns in a Post-fire Based on Four Cycles of Push-Out Tests Using Decision Tree-Based Machine Learning Techniques

A New Practical Formula for Bond-Slip Stress of Cold-Formed CFSTs Stub-Columns in a Post-fire Based on Four Cycles of Push-Out Tests Using Decision Tree-Based Machine Learning Techniques

Hybrid decision tree-based machine learning models for diabetes prediction

Hybrid decision tree-based machine learning models for diabetes prediction

CYBERSECURITY: RESEARCH ON METHODS FOR DETECTING DDOS ATTACKS

This article describes the problem of DDoS attacks, analyzing their nature and consequences. The paper covers common DDoS attack types, such as SYN flood, ICMP flood, UDP flood. Existing methods for detecting attacks from literature are reviewed, including machine learning approaches, including artificial neural networks, support vector machines and decision trees. The paper introduces a decision tree-based machine learning model for the detection of DDoS attacks. The model is trained and tested on a publicly available dataset. The dataset consists of 1,04,345 rows of data, where every row includes 23 features, such as source IP, destination IP, port number, number of bytes transferred from the switch port, etc. A similar set of characteristics can be obtained on a real network hardware using simple calculations, which makes it possible to approximate the model evaluation to real operating conditions. SYN flood, ICMP flood and UDP flood attack types are present in the data, as well as legitimate traffic. To avoid overfitting, only some columns were used, and columns such as IP addresses were discarded. The field “label” in each row of the dataset contains either 0 or 1 where 0 corresponds to legitimate traffic and 1 to malicious one. The problem of DDoS attack detection is therefore formally reduced to the task of binary classification of each row from the dataset. The constructed model achieves an average classification accuracy of 0.94 with a standard deviation at the level of 0.06 in detecting the above mentioned types of attacks. To objectively assess the effectiveness of the model and avoid distortion of the results, stratified 5-fold cross-validation was used. The developed model can be applied in the real world network hardware to filter malicious packets or as a tool for warning the administrator about an attack. This research advances cybersecurity by enhancing DDoS attack detection.

Decision tree ensemble with Bayesian optimization to predict the spatial dynamics of chlorophyll-a concentration: A case study in Bay of Bengal

An accurate prediction of the spatial distribution of phytoplankton biomass, as represented by Chlorophyll-a (CHL-a) concentrations, is important for assessing ecological conditions in the marine environment. This study developed a hyperparameter-optimized decision tree-based machine learning (ML) models to predict the geographical distribution of marine phytoplankton CHL-a in the Bay of Bengal. To predict CHL-a over a large spatial extent, satellite-derived remotely sensed data of ocean color features (CHL-a, colored dissolved organic matter, photosynthetically active radiation, particulate organic carbon) and climatic factors (nighttime sea surface temperature, surface absorbed longwave radiation, sea level pressure) from 2003 to 2022 are used to train and test the models. Results obtained from this study have shown the highest concentrations of CHL-a occurred near the Bay's coastal belts and river estuaries. Analysis revealed that aside from photosynthetically active radiation, organic components exhibited a stronger positive relationship with CHL-a than climatic features, which are correlated negatively. Results showed the chosen decision tree methods to all possess higher R2 and lower root mean square error (RMSE) errors. Furthermore, XGBoost outperforms all other models in predicting the geographic distribution of CHL-a. To assess the model efficacy on seasonal basis, a best performing XGBoost model was validated in the Bay of Bengal region which has shown a good performance in predicting the spatial distribution of Chl-a as well as the pixel values during the summer, winter and monsoon seasons. This study provides the best ML model to researchers for predicting CHL-a in the Bay of Bengal. Further it helps to improve our knowledge of CHL-a spatial dynamics and assist in monitoring marine resources in the Bay of Bengal. It worth noting that the water quality in the Indian Ocean is very dynamic in nature, therefore, additional efforts are needed to test the efficacy of this study model over different seasons and spatial gradients.

Using Clustering, Geochemical Modeling, and a Decision Tree for the Hydrogeochemical Characterization of Groundwater in an In Situ Leaching Uranium Deposit in Bayan-Uul, Northern China

Uranium extraction through the in situ leaching method stands as a pivotal approach in uranium mining. In an effort to comprehensively assess the repercussions of in situ uranium leaching on groundwater quality, this study collected 12 representative groundwater samples within the Bayan-Uul mining area. The basic statistical characteristics of the water samples showed that the concentrations of SO42− and total dissolved solids (TDS) were relatively high. Through the use of cluster analysis, the water samples were categorized into two distinct clusters. Seven samples from wells W-d, W-u, N01, W10-2, W08-1, W10-1, and W13-1, situated at a considerable distance from the mining area, were grouped together. Conversely, five samples from wells W08-2, W13-2, W01-1, W02-2, and the pumping well located in closer proximity to the mining area, formed a separate cluster. A decision tree-based machine learning approach was employed to discern the influence of various hydrochemical indicators in forming these clusters, with results indicating that SO42− exerts the most substantial influence, followed by Ca2+. The mineral saturation indices from geochemical modeling indicated that, as the distance from the mining area increased, the trend of calcium minerals changed from dissolution to precipitation; iron minerals were in a precipitation state, and the precipitation trend was gradually weakening. In light of these findings, it is clear that in situ uranium leaching significantly impacted the groundwater in the vicinity of the mining area. The prolonged consumption of groundwater sourced near the study area, or its use for animal husbandry, poses potential health risks that demand heightened attention.

Towards Automated Vocal Mode Classification in Healthy Singing Voice—An XGBoost Decision Tree-Based Machine Learning Classifier

Towards Automated Vocal Mode Classification in Healthy Singing Voice—An XGBoost Decision Tree-Based Machine Learning Classifier

Spatial Mapping of Flood Susceptibility Using Decision Tree–Based Machine Learning Models for the Vembanad Lake System in Kerala, India

Spatial Mapping of Flood Susceptibility Using Decision Tree–Based Machine Learning Models for the Vembanad Lake System in Kerala, India

Compressive strength prediction of PET fiber-reinforced concrete using Dolphin echolocation optimized decision tree-based machine learning algorithms

Compressive strength prediction of PET fiber-reinforced concrete using Dolphin echolocation optimized decision tree-based machine learning algorithms

Several models combined with ultrasound techniques to predict breast muscle weight in broilers

The weight of breast muscle (WBM) is a highly monitored indicator in broiler breeding that can be obtained after slaughtering. Currently, due to the lack of accurate in vivo phenotypes for both genomic and phenotypic selection, genetic gains in WBM fall short of initial expectations. In this study, 1,006 market-age (42 d) broilers from 3 generations over 2 yr were randomly selected, and the breast width (BW), fossil bone length (FBL), breast muscle thickness (BMT), and live weight (LW) were measured exactly in vivo. Eight models, including multiple linear regression (MLR), ridge regression (RR), least absolute shrinkage and selection operator (LASSO), and elastic net (EN), were fitted to explore the best regression relationships between breast muscle weight and these indicators. Support vector machine (SVM) methods with both linear kernels and radial kernels were used to fit the models, while 2 decision tree-based machine learning algorithms, random forest (RF), and extreme gradient boosting (XGBoost), were used to establish the prediction model. The predictive effects of different combinations of independent variables were compared, leading to the conclusion that the EN model achieves the best predictive power when all 4 live features are used as inputs and is slightly better than the other models (R2=0.7696). This method could be applied in practical production and breeding work, leading to substantial cost savings and enhancements in the breeding process.