Boosting Decision Tree Model Research Articles

Inverse design of high-strength medium-Mn steel using a machine learning-aided genetic algorithm approach

To develop medium-Mn steels with an ultimate tensile strength (UTS) exceeding 2 GPa and excellent ductility, we created a highly accurate UTS prediction machine learning (ML) model using a boosted decision tree model and 1520 dataset of tensile properties of medium-Mn steels with micro-alloying elements. We also optimized the hyper-parameters of a genetic algorithm (GA) using the Shannon diversity index to enhance search efficiency while retaining diversity. In a high-dimensional search space with millions of potential combinations, the ML-GA approach efficiently identified diverse chemical compositions and austenitizing conditions to achieve UTSs above 2 GPa. The k-means clustering method then grouped them into five distinct specimens based on similarities. These five specimens, fabricated using inversly designed chemical compositions and austenitizing temperatures, successfully exhibited UTSs exceeding 2 GPa and greater ductility compared to hot-stamped C steels. These excellent tensile properties were attributed to grain refinement resulting from the low austenitizing temperature and the pinning effect of micro-alloying element carbides, such as TiC and VC.

Economic impact of mortality prediction by predictive model at first and second treatment for bovine respiratory disease.

To evaluate a predictive model's ability to determine cattle mortality following first and second treatment for bovine respiratory disease and to understand the differences in net returns comparing predictive models to the status quo. 2 boosted decision tree models were constructed, 1 using data known at first treatment and 1 with data known at second treatment. Then, the economic impact of each outcome (true positive, true negative, false positive, and false negative) was estimated using various market values to determine the net return per head of using the predictive model to determine which animals should be culled at treatment. This was compared to the status quo to determine the difference in net return. The models constructed for the prediction of mortality performed with moderate accuracy (areas under the curve > 0.7). The economic analysis found that the models at a high specificity (> 90%) could generate a positive net return in comparison to status quo. This study showed that predictive models may be a useful tool to make culling decisions and could result in positive net returns. Bovine respiratory disease is the costliest health condition experienced by cattle on feed. Feedyard record-keeping systems generate vast amounts of data that could be used in predictive models to make management decisions. It is essential to understand the accuracy of predictions made via machine learning. However, the economic impact of implementing predictive models in a feedyard will influence adoption.

On the practical aspects of machine learning based active power loss forecasting in transmission networks

AbstractThe cost for covering active power losses makes a significant item in transmission system operators (TSO) annual budgets, and still it received limited attention in the existing literature. The focus of accurate power loss forecasting and procurement is of high increase during the past 2 years due to spikes in electricity prices, making the cost of covering the active power losses a dominant factor of TSO operational costs. This paper presents practical aspects of the highly accurate models for transmission loss forecast in the day ahead time frame for the Croatian transmission system. The contributions are two‐fold: 1) Practical insights into usable TSO data are provided, filling a critical research gap and a foundational literature review is established on transmission loss forecasting. 2) A novel method utilizing only electricity transit data as input which outperforms existing practices is presented. For this, several algorithms such as gradient boosted decision tree model (XGB), support vector regressors, multiple linear regression and fully connected feedforward artificial neural networks are developed, and implemented and validated on data obtained from the Croatian TSO. The results show that the XGB model outperforms current TSO model by 32% for 4 months of comparison and TSCNET's commercial solution by 25% during a year‐long testing period. The developed XGB model is also implemented as a software tool and put into everyday operation with the Croatian TSO.

Analysis of the Gas Safety Situation Based on the Schweizer-Sklar Rule.

In the coal mining process, a large amount of harmful gases will be produced, which we call "gas". The main component of gas is methane. After the methane concentration reaches a certain limit, an explosion will occur, seriously affecting the safety of the production of coal mines. Gas safety situational awareness is an important basis for gas early warning. In order to take appropriate measures for different levels of risk, the safety situation level is classified into five levels, which can be attributed to the classification problem in machine learning. We propose a gas security situation analysis model based on the Schweizer-Sklar rule (GSSAM-SSR) using the multiweighted Schweizer-Sklar triangular norm array combination rule (SSR). First, the raw data are preprocessed. The model uses K-means clustering with feature preference to determine the label column of the data set and principal component analysis to reduce the dimensionality of the collected high-dimensional gas-related data. Ten basic classification models are constructed, and the best performing models are selected as the basic classifier set by using the preorder method. Then, the SSR combination rule is proposed to combine the basic classifiers. Finally, considering the dynamic and quasi dynamic data of the mine, the basic classifier set and SSR combination rule are combined to construct the GSSAM-SSR model and applied to the gas security situation analysis. Experimental results show that the SSR combination rule achieves the best classification performance, exhibiting an accuracy of 94.43%, which is 2.51% higher than the accuracy of the gradient boosting decision tree model, which was the highest performing basic classifier and higher than the accuracies of the voting method and Bayesian combination rule.

Employing the Interpretable Ensemble Learning Approach to Predict the Bandgaps of the Halide Perovskites.

Halide perovskite materials have broad prospects for applications in various fields such as solar cells, LED devices, photodetectors, fluorescence labeling, bioimaging, and photocatalysis due to their bandgap characteristics. This study compiled experimental data from the published literature and utilized the excellent predictive capabilities, low overfitting risk, and strong robustness of ensemble learning models to analyze the bandgaps of halide perovskite compounds. The results demonstrate the effectiveness of ensemble learning decision tree models, especially the gradient boosting decision tree model, with a root mean square error of 0.090 eV, a mean absolute error of 0.053 eV, and a determination coefficient of 93.11%. Research on data related to ratios calculated through element molar quantity normalization indicates significant influences of ions at the X and B positions on the bandgap. Additionally, doping with iodine atoms can effectively reduce the intrinsic bandgap, while hybridization of the s and p orbitals of tin atoms can also decrease the bandgap. The accuracy of the model is validated by predicting the bandgap of the photovoltaic material MASn1-xPbxI3. In conclusion, this study emphasizes the positive impact of machine learning on material development, especially in predicting the bandgaps of halide perovskite compounds, where ensemble learning methods demonstrate significant advantages.

Hardness-guided machine learning for tungsten alloy strength prediction

Yield strength determines the material's resistance to permanent deformation. However, the traditional estimation method based on the triple relationship of Vickers hardness exhibits limited accuracy when applied to tungsten heavy alloys (WHA) with two-phase structure. This study presents a novel approach using machine learning and hardness data to predict the yield strength of tungsten heavy alloy efficiently and accurately. Among the eight machine learning models used, Gradient boosting decision tree model (GBDT) was the most effective model. The results demonstrate a strong agreement between the GBDT-based predictions and experimental results, with an average error of 6.7 %. We also gave the hardness versus yield strength relationship for the 90WNiFe alloy system. This finding highlights the potential of the GBDT regression model in conjunction with hardness measurements for predicting yield strength with high precision.

Relationships between Resident Activities and Physical Space in Shrinking Cities in China—The Case of Chaoyang City

Shrinking cities suffer from a decreased level of resident activities. As a result, areas with low levels of resident activities may become breeding grounds for social issues. To ease and prevent social issues, it is important to deploy physical space optimisation strategies to effectively guide the distribution of resident activities in shrinking cities. To support the development of such spatial strategies, this paper introduces machine learning-based methods for analysing the nuanced non-linear relationship between resident activities and physical space in shrinking cities. Utilising dual-scale grids, this study calculates multi-source spatial elements, which are subsequently integrated with resident activity data to construct a gradient boosting decision tree model. It then analyses the weight of different spatial elements’ impacts on resident activities and their nonlinear relationships. The model proposed in this study demonstrates good precision in construing the relationship between resident activities and physical space. Based on the research findings, strategies for different types of spatial development in shrinking cities are drawn out. This paper advocates for the application of this analytical approach before conducting spatial planning in shrinking cities to maximise the effectiveness of spatial development in guiding resident activities.

A new FCM-XGBoost system for predicting Pavement Condition Index

Predicting Pavement Condition Index (PCI) is crucial for identifying potential pavement distresses and developing effective pavement maintenance strategies. Here, a new Fuzzy C-Means-eXtreme Gradient Boosting (FCM-XGBoost) PCI prediction system is developed. First, asphalt pavement sections and concrete pavement sections are distinguished, and the Fuzzy C-Means (FCM) clustering algorithm is employed to cluster the asphalt and concrete expressway sections respectively based on multiple factors influencing PCI. Next, an eXtreme Gradient Boosting (XGBoost) model is trained for each cluster of expressway sections. Finally, in the system application stage, the PCI of an expressway section for the next year is predicted using the XGBoost model trained for the specific cluster that the expressway section belongs to. Validation results based on actual pavement condition data indicate that the proposed FCM-XGBoost PCI prediction system can achieve slightly higher accuracy compared with several high performance benchmark models, which include the Random Forest model, the Gradient Boosting Decision Tree model, the Light Gradient Boosting Machine model and the XGBoost model. The developed PCI prediction system contributes to the improvement of PCI prediction approaches and can be potentially applied in deploying effective and cost-efficient pavement maintenance strategies.

Suspended sediment load modeling using Hydro-Climate variables and Machine learning

Machine learning (ML) models have the potential to improve the accuracy of Suspended Sediment Load (SSL) predictions. However, their ability to incorporate climate data to enhance SSL predictions remains underexplored. This study investigates the efficacy of ML models in modeling sediment transport using hydro-climate variables as input data. Boosted decision tree models, including Xgboost, LightGBM, and Catboost, were employed to estimate SSL using hydro-climate variables such as streamflow and precipitation, as well as maximum and minimum temperatures. The models were trained, optimized, and evaluated using either climate/streamflow data or a combination of both, encompassing 47 gauges across the contiguous United States (CONUS) between 1950 and 2022. The results demonstrate that streamflow data alone is insufficient for accurate SSL estimation in all rivers. Consequently, integrating climate data and streamflow data proved effective in predicting SSL at the daily scale across diverse climate zones with commendable accuracy. The models achieved an average Nash-Sutcliffe Efficiency (NSE) of > 0.66, Kling-Gupta Efficiency (KGE) of > 0.67, and Willmott Index (WI) of > 0.83 during the unseen (test) phase. These findings highlight the potential of ML models, particularly when combining climate data with streamflow, to enhance SSL predictions and improve our understanding of sediment transport dynamics.

Prediction of Diabetes Using Data Mining and Machine Learning Algorithms: A Cross-Sectional Study.

This study aimed to develop a model to predict fasting blood glucose status using machine learning and data mining, since the early diagnosis and treatment of diabetes can improve outcomes and quality of life. This crosssectional study analyzed data from 3376 adults over 30 years old at 16 comprehensive health service centers in Tehran, Iran who participated in a diabetes screening program. The dataset was balanced using random sampling and the synthetic minority over-sampling technique (SMOTE). The dataset was split into training set (80%) and test set (20%). Shapley values were calculated to select the most important features. Noise analysis was performed by adding Gaussian noise to the numerical features to evaluate the robustness of feature importance. Five different machine learning algorithms, including CatBoost, random forest, XGBoost, logistic regression, and an artificial neural network, were used to model the dataset. Accuracy, sensitivity, specificity, accuracy, the F1-score, and the area under the curve were used to evaluate the model. Age, waist-to-hip ratio, body mass index, and systolic blood pressure were the most important factors for predicting fasting blood glucose status. Though the models achieved similar predictive ability, the CatBoost model performed slightly better overall with 0.737 area under the curve (AUC). A gradient boosted decision tree model accurately identified the most important risk factors related to diabetes. Age, waist-to-hip ratio, body mass index, and systolic blood pressure were the most important risk factors for diabetes, respectively. This model can support planning for diabetes management and prevention.

Intelligent Classification of Volcanic Rocks Based on Honey Badger Optimization Algorithm Enhanced Extreme Gradient Boosting Tree Model: A Case Study of Hongche Fault Zone in Junggar Basin

Lithology identification is the fundamental work of oil and gas reservoir exploration and reservoir evaluation. The lithology of volcanic reservoirs is complex and changeable, the longitudinal lithology changes a great deal, and the log response characteristics are similar. The traditional lithology identification methods face difficulties. Therefore, it is necessary to use machine learning methods to deeply explore the corresponding relationship between the conventional log curve and lithology in order to establish a lithology identification model. In order to accurately identify the dominant lithology of volcanic rock, this paper takes the Carboniferous intermediate basic volcanic reservoir in the Hongche fault zone as the research object. Firstly, the Synthetic Minority Over-Sampling Technique–Edited Nearest Neighbours (SMOTEENN) algorithm is used to solve the problem of the uneven data-scale distribution of different dominant lithologies in the data set. Then, based on the extreme gradient boosting tree model (XGBoost), the honey badger optimization algorithm (HBA) is used to optimize the hyperparameters, and the HBA-XGBoost intelligent model is established to carry out volcanic rock lithology identification research. In order to verify the applicability and efficiency of the proposed model in volcanic reservoir lithology identification, the prediction results of six commonly used machine learning models, XGBoost, K-nearest neighbor (KNN), gradient boosting decision tree model (GBDT), adaptive boosting model (AdaBoost), support vector machine (SVM) and convolutional neural network (CNN), are compared and analyzed. The results show that the HBA-XGBoost model proposed in this paper has higher accuracy, precision, recall rate and F1-score than other models, and can be used as an effective means for the lithology identification of volcanic reservoirs.

Predicting Risk of Bullying Victimization among Primary and Secondary School Students: Based on a Machine Learning Model.

School bullying among primary and secondary school students has received increasing attention, and identifying relevant factors is a crucial way to reduce the risk of bullying victimization. Machine learning methods can help researchers predict and identify individual risk behaviors. Through a machine learning approach (i.e., the gradient boosting decision tree model, GBDT), the present longitudinal study aims to systematically examine individual, family, and school environment factors that can predict the risk of bullying victimization among primary and secondary school students a year later. A total of 2767 participants (2065 secondary school students, 702 primary school students, 55.20% female students, mean age at T1 was 12.22) completed measures of 24 predictors at the first wave, including individual factors (e.g., self-control, gender, grade), family factors (family cohesion, parental control, parenting style), peer factor (peer relationship), and school factors (teacher-student relationship, learning capacity). A year later (i.e., T2), they completed the Olweus Bullying Questionnaire. The GBDT model predicted whether primary and secondary school students would be exposed to school bullying after one year by training a series of base learners and outputting the importance ranking of predictors. The GBDT model performed well. The GBDT model yielded the top 6 predictors: teacher-student relationship, peer relationship, family cohesion, negative affect, anxiety, and denying parenting style. The protective factors (i.e., teacher-student relationship, peer relationship, and family cohesion) and risk factors (i.e., negative affect, anxiety, and denying parenting style) associated with the risk of bullying victimization a year later among primary and secondary school students are identified by using a machine learning approach. The GBDT model can be used as a tool to predict the future risk of bullying victimization for children and adolescents and to help improve the effectiveness of school bullying interventions.

Development of advanced hybrid computational model for description of molecular separation in liquid phase via polymeric membranes

Separation of molecules using membranes has been computationally studied in this work for understanding the separation process and also assessment of integrating machine learning (ML) techniques to computational fluid dynamics (CFD) techniques. We initially performed CFD computations to simulate mass transfer in a membrane-based molecular separation system, and then exported the mass transfer data for training ML models. Indeed, a hybrid model was developed and employed for membrane process in separating a solute from a solution. We have modeled a data set containing more than 14 thousand data rows, which has two inputs r (m) and z (m) and single response, i.e., C (mol/m3) that is solute’ content in the solution. Decision tree-based models are used in this modeling. Simple Decision Tree (DT), Boosted Decision Tree (BDT) that is DT alongside AdaBoost, and Random Forest (RF) are the selected models for this research. The ML techniques were further optimized via a novel HHO algorithm. Final assessments show the R2 scores of 0.9991, 0.9874, and 0.9997 respectively for BDT, DT, and RF models. Based on this result and other analysis, RF is selected as the best method in this study for simulation of membrane process. This model has an RMSE error rate of 2.22 × 103.

Hardware-Trojan Detection at Gate-Level Netlists Using a Gradient Boosting Decision Tree Model and Its Extension Using Trojan Probability Propagation

Hardware-Trojan Detection at Gate-Level Netlists Using a Gradient Boosting Decision Tree Model and Its Extension Using Trojan Probability Propagation

Many-objective optimization of BEV design parameters based on gradient boosting decision tree models and the NSGA-III algorithm considering the ambient temperature

In this study, extensive efforts have been devoted to optimizing the energy distribution and performance of a battery electric vehicle (BEV). An integrated simulation model based on energy flow test data is built and validated, and a parallel framework to implement automatic batch simulations is developed. On this basis, eXtreme Gradient Boosting (XGBoost), Light Gradient Boosting Machine (LightGBM) and Categorical Boosting (CatBoost) models of the BEV are established and compared, and many-objective optimization is carried out based on the Non-dominated Sorting Genetic Algorithm-III (NSGA-III) algorithm. The results indicate that the developed parallel framework can efficiently perform automatic batch computations of the integrated simulation model for the BEV, and the CatBoost model demonstrates superior prediction performance on both the train and test sets. A uniformly distributed non-dominated set approximating the Pareto Front (PF) is obtained by the many-objective optimization, with the fourth non-dominated solution exhibiting a favorable optimization effect. The electricity consumption per 100 km, half-axis effective work, electricity recovered by battery, energy utilization and recovery efficiency are improved by 9.4 %, 12.2 %, 6.4 %, 96.3 % and 16.0 %, respectively. These findings can provide the theoretical basis, directional guidance and data support for the accurate modeling, batch simulation and many-objective optimization of BEVs.

A fast and non-invasive artificial intelligence olfactory-like system that aids diagnosis of Parkinson's disease.

Several previous studies have shown that skin sebum analysis can be used to diagnose Parkinson's disease (PD). The aim of this study was to develop a portable artificial intelligence olfactory-like (AIO) system based on gas chromatographic analysis of the volatile organic compounds (VOCs) in patient sebum and explore its application value in the diagnosis of PD. The skin VOCs from 121 PD patients and 129 healthy controls were analyzed using the AIO system and three classic machine learning models were established, including the gradient boosting decision tree (GBDT), random forest and extreme gradient boosting, to assist the diagnosis of PD and predict its severity. A 20-s time series of AIO system data were collected from each participant. The VOC peaks at a large number of time points roughly concentrated around 5-12 s were significantly higher in PD subjects. The gradient boosting decision tree model showed the best ability to differentiate PD from healthy controls, yielding a sensitivity of 83.33% and a specificity of 84.00%. However, the system failed to predict PD progression scored by Hoehn-Yahr stage. This study provides a fast, low-cost and non-invasive method to distinguish PD patients from healthy controls. Furthermore, our study also indicates abnormal sebaceous gland secretion in PD patients, providing new evidence for exploring the pathogenesis of PD.

The application of structural and machine learning models to predict the default risk of listed companies in the Iranian capital market.

Inattention of economic policymakers to default risk and making inappropriate decisions related to this risk in the banking system and financial institutions can have many economic, political and social consequences. In this research, it has been tried to calculate the default risk of companies listed in the capital market of Iran. To achieve this goal, two structural models of Merton and Geske, two machine learning models of Random Forest and Gradient Boosted Decision Tree, as well as financial information of companies listed in the Iranian capital market during the years 2016 to 2021 have been used. Another goal of this research is to measure the predictive power of the four models presented in the calculation of default risk. The results obtained from the calculation of the default rate of the investigated companies show that 50 companies listed in the Iranian capital market (46 different companies) have defaulted during the 5-year research period and are subject to the Bankruptcy Article of the Iranian Trade Law. Also, the results obtained from the ROC curves for the predictive power of the presented models show that the structural models of Merton and Geske have almost equal power, but the predictive power of the Random Forest model is a little more than the Gradient Boosted Decision Tree model.

Abstract 17213: Machine Learning-Based Prediction of Type A Aortic Dissection

Background: Existing risk predictors of aortic dissection have certain limitations. We hypothesized that machine learning models trained on clinical, demographic, and anthropometric features can further improve the prediction of patient outcomes. Objective: This study aims to develop a machine learning model that predicts type A aortic dissection and can help clinical decision making. Methods: This cohort study used the Yale Aortic Institute database. The models incorporated variables spanning demographic, anthropometric, medical history, radiological, and laboratory domains. The models were trained and validated using stratified 10-fold cross-validation. Hyperparameters for each algorithm were tuned through grid-search on the training folds. The models were trained to optimize the area under the receiver operator characteristic curve (AUROC) and were assessed in a held-out test set. Results: A total of 2,109 patients were analyzed in our study. Among them, 271 were diagnosed with type A aortic dissection. The models demonstrated strong performance on the held-out test set. Specifically, the extreme gradient boosting decision tree model achieved an AUROC of 0.821, while the random forest model achieved an AUROC of 0.820. Importantly, these models outperformed the prediction of type A aortic dissection when based solely on the ascending aorta diameter, which had an AUROC of 0.549. Besides the ascending aorta diameter, the key predictors were age, weight, height, family history, smoking, bicuspid aortic valve, and hypertension. Conclusion: We developed a machine learning model that provides an individualized prediction of the development of type A aortic dissection. This approach provides an accessible, efficient, and remote tool to identify high-risk patients.

Abstract 15461: Proteomic Profiling Identifies Metabolic and Extracellular Matrix Markers That Predict MACE in Individuals With Type 2 Diabetes: An EXSCEL Substudy

Background: Patients (pts) with type 2 diabetes mellitus (T2DM) are at greater risk for major adverse cardiovascular events (MACE; e.g. stroke, myocardial infarction, and cardiovascular death). Given marked heterogeneity of risk, better multi-marker models are required to improve risk prediction. Methods: We evaluated 5473 pts with T2DM in the EXSCEL trial of exenatide, a GLP1-RA, vs. placebo by profiling ~5000 proteins at baseline and 12 months. Multivariable Cox proportional hazards models (adjusted for age, race, sex, study treatment, BMI, eGFR, BP, HbA1c, CHF and CV disease) were used to test for association between individual proteins with MACE. Pathway analysis using gene set enrichment (GSEA) was performed and LASSO, elastic net, random forest, and gradient boosted decision tree models were used to predict MACE on a held-out test set (N=1094). Protein importance scores and coefficients were analyzed to determine proteins predicting MACE. Linear mixed models (LMM) were performed to assess treatment effect on protein levels. Results: Over a median of 3.2 (IQR 2.2-4.4) years, 813 (6.72%) pts experienced MACE. A total of 1278 individual proteins were associated with MACE (FDR adjusted p <0.05). GSEA identified 8 pathways enriched in these proteins including fatty acid metabolism (p=0.01) and cardiac muscle contraction (p=0.04). LASSO outperformed other models in discrimination for MACE (AUC = 0.75) and prioritized tetranectin (an adipocyte secretory protein associated with insulin resistance and fibrinolysis) and fibulin-1 (a regulator of extracellular matrix) as highly predictive of MACE. Both proteins were highly associated with MACE in Cox proportional hazards models (tetranectin: FDR adjusted p=8.86e-26, HR = 0.11 (95%CI 0.07-0.16), fibulin-1: FDR adjusted p=6.72e-14, HR = 6.49 (95%CI 4.13- 10.19). LMM showed treatment modification in tetranectin levels (p=0.0018) with exenatide. Conclusions: Leveraging a large trial of a GLP1-RA in pts with T2DM, we identified proteomic models highly discriminative of MACE. Models using proteins involved in molecular pathways of fatty acid metabolism and cardiac contraction may have clinical utility in identifying higher risk pts and for understanding beneficial molecular mechanisms of GLP1-RA.

222 High Throughput Screening of Bovine Mastitis from Raw Milk Using Maldi-Tof Ms and Machine Learning

Abstract To prevent pathogenic microorganisms associated with mastitis from entering the food supply, raw milk must be screened. This study investigated whether MALDI-TOF-MS combined with machine learning algorithms is a cost-effective method for screening dairy milk samples at high throughput. Raw milk from animals was analyzed using matrix-assisted laser desorption ionization mass spectrometry. Mass spectra were obtained for a training dataset of composite milk samples (n = 226) collected from a local dairy. The peaks from the mass spectra were then transported into a machine learning model, which detected non-obvious patterns that correspond with the mastitis condition. Finally, the diagnostic accuracy of the new model was evaluated using a separate set of milk samples (scoring set; n = 100). For optimal performance, decision trees, random forest models, gradient boosted trees, and neural network models were investigated. The results show that the gradient boosted decision tree model performs best for mastitis diagnosis, with simultaneous sensitivity and specificity greater than 0.8. The decision tree approach is conceptually straightforward and identifies candidate biomarker molecules indicative of mastitis. The combined MALDI TOF plus machine learning approach is effective for detecting mastitis in dairy cows and is likely a viable option for diagnosing a wide range of animal health issues. The MALDI TOF method is suitable for high-throughput screening because it only requires a microliter of inexpensive reagents.