Boosted Regression Research Articles

Enhancing Prediction of Electron Affinity and Ionization Energy in Liquid Organic Electrolytes for Lithium-Ion Batteries Using Machine Learning

The electron affinity (EA) and ionization energy (IE) of battery electrolyte molecules are pivotal in determining the electrochemical properties of electrode materials, significantly impacting the performance and safety of batteries. In this study, we employed a suite of machine learning algorithms, including linear regression, gradient boosting, CatBoost, XGBoost, and random forest regression, to predict these properties. Our analysis leveraged a comprehensive dataset of 24432 electrolyte molecules extracted from the Joint Center for Energy Storage Research (JCESR) Molecules database, and used RDKit and Mordred tools to generate a series of molecular descriptors. Through meticulous feature engineering and selection using the maximum relevance minimum redundancy (mRMR) algorithm, we identified the most influential molecular features for predicting EA and IE. The CatBoost model emerged as the most accurate, outperforming other models with its ability to handle complex nonlinear relationships and provide robust predictions with lower errors. The model's predictions were further validated through 10-fold cross-validation, demonstrating its generalization capability and resistance to overfitting. The CatBoost model achieved a root mean squared error (RMSE) of 0.239 eV for IE and 0.427 eV for EA, with R-squared (R2) values of 0.944 and 0.879, respectively. SHapley Additive exPlanations (SHAP) value analysis elucidated the contribution of each feature to the model's predictions, highlighting the molecular characteristics that significantly influence EA and IE. Our findings offer valuable insights for the design of novel electrolyte materials with tailored electronic properties, advancing the development of high-performance battery technologies.

Identifying proteomic prognostic markers for Alzheimer's disease with survival machine learning: The Framingham Heart Study.

Protein abundance levels, sensitive to both physiological changes and external interventions, are useful for assessing the Alzheimer's disease (AD) risk and treatment efficacy. However, identifying proteomic prognostic markers for AD is challenging by their high dimensionality and inherent correlations. Our study analyzed 1128 plasma proteins, measured by the SOMAscan platform, from 858 participants 55 years and older (mean age 63 years, 52.9 % women) of the Framingham Heart Study (FHS) Offspring cohort. We conducted regression analysis and machine learning models, including LASSO-based Cox proportional hazard regression model (LASSO) and generalized boosted regression model (GBM), to identify protein prognostic markers. These markers were used to construct a weighted proteomic composite score, the AD prediction performance of which was assessed using time-dependent area under the curve (AUC). The association between the composite score and memory domain was examined in 339 (of the 858) participants with available memory scores, and in a separate group of 430 participants younger than 55 years (mean age 46, 56.7 % women). Over a mean follow-up of 20 years, 132 (15.4 %) participants developed AD. After adjusting for baseline age, sex, education, and APOE ε4 + status, regression models identified 309 proteins (P ≤ 0.2). After applying machine learning methods, nine of these proteins were selected to develop a composite score. This score improved AD prediction beyond the factors of age, sex, education, and APOE ε4 + status across 15-25 years of follow-up, achieving its peak AUC of 0.84 in the LASSO model at the 22-year follow-up. It also showed a consistent negative association with memory scores in the 339 participants (beta = -0.061, P = 0.046), 430 participants (beta = -0.060, P = 0.018), and the pooled 769 samples (beta = -0.058, P = 0.003). These findings highlight the utility of machine learning method in identifying proteomic markers in improving AD prediction and emphasize the complex pathology of AD. The composite score may aid early AD detection and efficacy monitoring, warranting further validation in diverse populations.

A comparative analysis and prediction of carbon emission in India using Machine learning models

<p style="line-height:150%;text-align:justify;text-indent:36.0pt;"><span style="background-color:yellow;font-family:"Times New Roman","serif";font-size:12.0pt;line-height:150%;">Among the most significant issues that will impact and pose a major threat to our future are global warming and climate change. This study focuses on forecasting India's carbon emissions, specifically examining the role of agriculture and agri-related industries, using time-series data from 1990 to 2020. </span><span style="background-color:lime;font-family:"Times New Roman","serif";font-size:12.0pt;line-height:150%;">To achieve this, a machine learning-based approach was proposed, utilizing five advanced models AdaBoost, XGB, Gradient Boosting, Random Forest, and Linear Regression. These models were selected for their ability to process large datasets, identify complex patterns, and provide precise predictions, offering a comprehensive framework for estimating carbon emissions. The approach aims to identify the most accurate predictive model for estimating carbon emissions, facilitating informed policy interventions and strategic planning. </span><span style="background-color:aqua;font-family:"Times New Roman","serif";font-size:12.0pt;line-height:150%;">The Carbon Budget 2023 research highlights that India's carbon emissions increased by 8.2% in 2023, raising significant concerns for environmental sustainability. Using the proposed machine learning models, the study facilitates robust data prediction. Among the five models, XGB demonstrates superior performance with evaluation metrics such as a Mean Absolute Error (MAE) of 19843, Root Mean Square Error (RMSE) of 24022, Mean Squared Error (MSE) of 5770998, R² value of 0.9, and test accuracy of 99%. These findings establish XGB as the most reliable model for forecasting carbon emissions. Consequently, XGB emerges as a powerful tool for accurately estimating emissions and addressing the challenges posed by climate change.</span></p>

Drivers of amphibian species richness in European ponds

Amphibians are commonly occurring inhabitants of most lentic freshwater ecosystems, yet their global populations are in alarming decline. Ponds in particular play a crucial role in supporting amphibian biodiversity. In this study, we identified the main drivers influencing amphibian species richness by conducting a comprehensive ecological characterization in 201 ponds across seven European countries spanning a large latitudinal and longitudinal gradient. The amphibian species richness in each of these ponds was assessed using environmental DNA metabarcoding on water samples. The relative influence of climatic, local abiotic and biotic, and land use variables on variation in species richness across ponds was quantified using boosted regression trees. Our results suggest that local factors, particularly chlorophyll‐a concentration, but also pond area and depth, are the main drivers of amphibian richness, together with climatic variables such as annual mean precipitation and temperature. The highest richness was observed in low‐nutrient, fishless, intermediate‐sized, shallow ponds, located in warmer regions with higher precipitation rates. These potential drivers of amphibian richness should be considered in the planning and implementation of amphibian conservation and management actions.

Predicting thermal stress and failure risk in monoblock divertors using 2D finite difference modeling and gradient boosting regression for fusion energy applications

Abstract This study presents a combined approach using a 2D finite difference method and Gradient Boosting Regressor (GBR) to analyze thermal stress and identify potential failure points in monoblock divertors made of tungsten, copper, and CuCrZr alloy. The model simulates temperature and heat flux distributions under typical fusion reactor conditions, highlighting regions of high thermal gradients and stress accumulation. These stress concentrations, particularly at the interfaces between materials, are key areas for potential failure, such as thermal fatigue and microcracking. Using the GBR model, a predictive maintenance framework is developed to assess failure risk based on thermal stress data, allowing for early intervention. This approach provides insights into the thermomechanical behavior of divertors, contributing to the design and maintenance of more resilient fusion reactor components.

Achieving Excellence in Cyber Fraud Detection: A Hybrid ML+DL Ensemble Approach for Credit Cards

The rapid advancement of technology has increased the complexity of cyber fraud, presenting a growing challenge for the banking sector to efficiently detect fraudulent credit card transactions. Conventional detection approaches face challenges in adapting to the continuously evolving tactics of fraudsters. This study addresses these limitations by proposing an innovative hybrid model that integrates Machine Learning (ML) and Deep Learning (DL) techniques through a stacking ensemble and resampling strategies. The hybrid model leverages ML techniques including Decision Tree (DT), Random Forest (RF), Support Vector Machine (SVM), eXtreme Gradient Boosting (XGBoost), Categorical Boosting (CatBoost), and Logistic Regression (LR) alongside DL techniques such as Convolutional Neural Network (CNN) and Bidirectional Long Short-Term Memory Network (BiLSTM) with attention mechanisms. By utilising the stacking ensemble method, the model consolidates predictions from multiple base models, resulting in improved predictive accuracy compared to individual models. The methodology incorporates robust data pre-processing techniques. Experimental evaluations demonstrate the superior performance of the hybrid ML+DL model, particularly in handling class imbalances and achieving a high F1 score, achieving an F1 score of 94.63%. This result underscores the effectiveness of the proposed model in delivering reliable cyber fraud detection, highlighting its potential to enhance financial transaction security.

A machine learning-based model for predicting the risk of cognitive frailty in elderly patients on maintenance hemodialysis

Elderly patients undergoing maintenance hemodialysis (MHD) face a heightened risk of cognitive frailty (CF), which significantly compromises quality of life. Early identification of at-risk individuals and timely intervention are essential. Nevertheless, current CF risk prediction models fall short in accuracy to adequately fulfill clinical requirements. This study aimed to examine the determinants of CF in elderly patients undergoing MHD and to develop a risk prediction model through machine learning algorithms. The objective is to furnish healthcare professionals with an early prediction tool and to offer insights for personalized CF risk management. A convenience sampling method was employed to select 1,075 elderly MHD patients from various tertiary-level hospitals in Chengdu between October 2023 and March 2024 as the modeling set, and 269 elderly MHD patients from hospitals in Chengdu, Yibin, and Zigong between September 2024 and October 2024 as the external validation set.CF was assessed using the Fried Phenotypic Scale for Frailty (FP) and the Montreal Cognitive Assessment Scale (MOCA). Data on patients’ demographics, sleep, nutrition, depression, and social support were collected. Single-factor and multi-factor logistic regression analyses were conducted to identify the factors influencing CF. Five machine learning algorithms—Support Vector Machine (SVM), Extreme Gradient Boosting (XGBoost), Random Forest (RF), Neural Network (NNET), and Logistic Regression (LR)—were employed to develop risk prediction models. These five models served as base classifiers, and 16 ensemble models were constructed using the Stacking method. The optimal ensemble models were identified and compared with the five individual models, followed by the selection and external validation of the most effective predictive models. Finally, the optimal models were deployed on web platforms utilizing the Streamlit library. CF prevalence was 14.2%. Significant CF risk factors included age, mode of residence, medical payment method, exercise, alcohol consumption, dialysis vascular access, serum albumin classification, serum phosphorus classification, total cholesterol classification, blood urea nitrogen classification, malnutrition score and depression score. The Stacking model showed superior performance (AUC = 0.911), with external validation confirming its accuracy (AUC = 0.832). Machine learning models, particularly Stacking, effectively predict CF risk in elderly MHD patients, providing a valuable tool for clinical intervention.

AI-Enhanced Stock Market Prediction: Evaluating Machine Learning Models for Financial Forecasting in the USA

In the USA, one of the world's largest and most liquid financial markets, the ability to anticipate market trends has deep economic implications. Precise stock market forecasting is highly instrumental for investors and analysts in making better asset allocation decisions, managing risks, and setting investment strategies. This research aimed to analyze the efficiency of some machine learning models in stock market forecast evaluation. This research project concentrated on stock market data from the USA, exploring historical price patterns, trading volumes, and relevant economic indicators to assess the performance of various machine learning models. The dataset used for this research work about predicting stock market trends is an exhaustive collection of historical stock prices, some fundamental financial indicators, and relevant news about the market, gathered from various dependable sources. Historical stock prices are retrieved from financial market databases like Yahoo Finance and Google Finance. These sources have daily records of open, high, low, and close prices, and trading volumes for thousands of publicly traded companies for extended periods, normally running into several years. The analyst selected several strategic models, namely, Random Forest, Gradient Boosting, and Logistic regression. Logistic Regression outperformed the other two models with relatively higher accuracy, while the others are just a little below. The findings of this study have implications well beyond academic curiosity into investment strategies and financial analysis. By leveraging the strengths of the best models, investors can create better-informed trading strategies. These models can also include predictive insights that give a serious edge to the risk management strategy in an investment portfolio. By using the predictive power of models like Random Forest, investors can foresee market fluctuations and adjust their portfolios to reduce potential losses.

Machine Learning Models in Predicting Failure Times Data Using a Novel Version of the Maxwell Model

This work aims to introduce a novel statistical distribution based on Maxwell distribution that can handle both positive and negative data sets with varying failure rates, including decreasing and bathtub-shaped distributions. The novel statistical distribution can be derived via the log transformation approach with an additional exponent parameter, defining the Transformed Log Maxwell (TLMax) distribution. The numerical investigation reveals that the developed TLMax distribution can effectively fit negative and positive data sets. A data set containing failure times for Kevlar 49/epoxy at a pressure of approximately 90% was employed to compare the proposed model against the traditional Maxwell model, and the results obtained indicated that the novel distribution outperformed the comparator. Finally, for the prediction of failure times in the dataset, we employed a machine learning model, including support vector regression (SVR), K-nearest neighbors’ regression (KNN), linear regression (LR), and gradient boosting regression (XGBoost). The findings indicate that the KNN model demonstrates greater prediction robustness than the other models. Beyond practitioners and researchers, this research holds relevance for professionals in physics and chemistry, where the Maxwell distribution is commonly employed.

Research on the Parameter Prediction Model for Fully Mechanized Mining Equipment Selection Based on RF-WOA-XGBoost

Fully mechanized mining equipment is core to the coal mining process. The selection process for this type of equipment is complex and heavily relies on experts’ experience for determining equipment parameters. This paper proposes a fully mechanized mining equipment parameter prediction model based on Extreme Gradient Boosting Regression Trees (XGBoost), which is developed based on the mapping relationships among geological parameters, fully mechanized mining face conditions, and the parameters of fully mechanized mining equipment. Feature selection is performed based on the feature importance ranking obtained through the Random Forest (RF) method, thereby reducing the model complexity. Different optimization algorithms are used to optimize the hyperparameters of XGBoost, and the results show that the Whale Optimization Algorithm (WOA) outperforms other algorithms in terms of convergence speed and optimization effectiveness. By comparing different prediction algorithms, it is found that the WOA-XGBoost model achieves higher prediction accuracy on the test set, with an average absolute error of 0.0458, root mean square error of 0.1610, and a coefficient of determination (R2) of 0.9451. Finally, a RF-WOA-XGBoost-based parameter prediction model for fully mechanized mining equipment is established, which is suitable for lightly inclined mining faces. This model reduces input complexity, improves the selection speed, minimizes reliance on experts, and ensures prediction accuracy, providing an effective reference for the parameter selection of fully mechanized mining equipment.

Hydrological signatures in Brazilian catchments: a machine learning approach for spatial estimates

This study investigates the estimation of hydrological signatures in Brazilian catchments using physical attributes and machine learning algorithms. Using CABra’s dataset with 735 catchments, we tested 163 signatures with regression techniques, including Random Forest, Gradient Boosting, and Support Vector Regression (SVR). A key aspect was the algorithmic approach, providing models with topographic (area, elevation, slope), climatic (precipitation, evapotranspiration, aridity index), soil (silt, clay, organic content), and land cover attributes without assuming correlations. This enabled the models to uncover complex patterns. Cross-validation (k-fold) showed most signatures were satisfactorily predicted (R2 > 0.7), suggesting their application in ungauged basins. Climatic factors, like precipitation, were crucial for predicting most signatures, especially high-flow, while silt content and latitude influenced low-flow and baseflow. Slope and latitude were key for basin dynamics, influencing the flashiness index, and land cover was important for low-flow. CatBoost was the best model. The SHapley Additive exPlanation (SHAP) analysis highlighted the importance of variables with low linear correlation.

Predictive models and determinants of mortality among T2DM patients in a tertiary hospital in Ghana, how do machine learning techniques perform?

BackgroundThe increasing prevalence of type 2 diabetes mellitus (T2DM) in lower and middle – income countries call for preventive public health interventions. Studies from Africa including those from Ghana, consistently reveal high T2DM-related mortality rates. While previous research in the Ho municipality has primarily examined risk factors, comorbidity, and quality of life of T2DM patients, this study specifically investigated mortality predictors among these patients.MethodThe study was retrospective involving medical records of T2DM patients. Data extracted included mortality outcome (dead or alive), sociodemographic characteristics (age, sex, marital status, educational level, occupation and location), family history of diseases (diabetes, cardiovascular disease (CVD), or asthma), lifestyle (smoking and alcohol intake), comorbidities (such as skin infections, sickle cell disease, urinary tract infections, and pneumonia) and complications of diabetes (CVD, nephropathy, neuropathy, foot ulcers, and diabetic ketoacidosis) were analyzed using Stata version 16.0 and Python 3.6.1 programming language. Both descriptive and inferential statistics were done to describe and build predictive models respectively. The performance of machine learning (ML) techniques such as support vector machine (SVM), decision tree, k nearest neighbor (kNN), eXtreme Gradient Boosting (XGBoost) and logistic regression were evaluated using the best-fitting predictive model for T2DM mortality.ResultsOf the 328 participants, 183 (55.79%) were female, and the percentage of mortality was 11.28%. A 100% mortality was recorded among the T2DM patients with sepsis (p-value = 0.012). T2DM in-patients were 3.83 times as likely to die [AOR = 3.83; 95% CI: (1.53–9.61)] if they had nephropathy compared to T2DM in-patients without nephropathy (p-value = 0.004). The full model which included sociodemographic characteristics, family history, lifestyle variables and complications of T2DM had the best prediction of T2DM mortality outcome (ROC = 72.97%). The accuracy for (test and train datasets) were as follows: (90% and 90%), (100% and 100%), (90% and 90%), (90% and 88%) and (88% and 90%) respectively for the various ML classification techniques: logistic regression, Decision tree classifier, kNN classifier, SVM and XGBoost.ConclusionThis study found that all in-patients with sepsis died. Nephropathy was the identified significant predictor of T2DM mortality. Decision tree classifier provided the best classifying potential.

Machine learning algorithms for predictive modeling of dyslipidemia-associated cardiovascular disease risk in pregnancy: a comparison of boosting, random forest, and decision tree regression

BackgroundCardiovascular diseases (CVD) are major contributors to maternal mortality and morbidity during pregnancy and increased atherogenic index of plasma levels is associated with a higher risk of CVD and obesity.MethodsIn this study, we utilized three different machine learning algorithms (boosting, random forest, and decision tree regression) to predict dyslipidemia-associated cardiovascular disease using atherogenic index and lipid profile parameters based on a cross-sectional study datasets of 112 pregnant women aged between 15 and 49 conducted at Aminu Kano Teaching Hospital.ResultsThe results showed that random forest regression outperformed both boosting and decision tree regression, recording the lowest error criteria (MSE = 0.071 and RMSE = 0.266) for evaluating the model. These findings indicated that all the three algorithms have the potential to effectively model the data from atherogenic indices and lipid profile parameters but random forest and boosting were found to outperform decision tree models with respective R2 values of 0.95 and 0.92.ConclusionsOverall, the study highlights the accuracy of machine learning models (random forest, boosting, and decision trees) in predicting dyslipidemia-associated cardiovascular diseases and the findings could contribute to the development of effective strategies for the prevention and treatment of dyslipidemia-associated cardiovascular diseases.

Database Construction for the Virtual Screening of the Ruthenium-Catalyzed Hydrogenation of Ketones.

During the recent development of machine-learning (ML) methods for organic synthesis, the value of "failed experiments" has increasingly been acknowledged. Accordingly, we have developed an exhaustive database comprising 300 entries of experimental data obtained by performing ruthenium-catalyzed hydrogenation reactions using 10 ketones as substrates and 30 phosphine ligands. After evaluating the predictive performance of ML models using the constructed database, we conducted a virtual screening of commercially available phosphine ligands. For the virtual screening, we utilized several models, such as histogram-based gradient boosting and Ridge regression, combined with the Mordred descriptors and MACCSKeys, respectively. The disclosed approach resulted in the identification of high-performance phosphine ligands, and the rationale behind the predictions in the virtual screening was analyzed using SHAP.

Enhancing stroke disease classification through machine learning models via a novel voting system by feature selection techniques.

Heart disease remains a leading cause of mortality and morbidity worldwide, necessitating the development of accurate and reliable predictive models to facilitate early detection and intervention. While state of the art work has focused on various machine learning approaches for predicting heart disease, but they could not able to achieve remarkable accuracy. In response to this need, we applied nine machine learning algorithms XGBoost, logistic regression, decision tree, random forest, k-nearest neighbors (KNN), support vector machine (SVM), gaussian naïve bayes (NB gaussian), adaptive boosting, and linear regression to predict heart disease based on a range of physiological indicators. Our approach involved feature selection techniques to identify the most relevant predictors, aimed at refining the models to enhance both performance and interpretability. The models were trained, incorporating processes such as grid search hyperparameter tuning, and cross-validation to minimize overfitting. Additionally, we have developed a novel voting system with feature selection techniques to advance heart disease classification. Furthermore, we have evaluated the models using key performance metrics including accuracy, precision, recall, F1-score, and the area under the receiver operating characteristic curve (ROC AUC). Among the models, XGBoost demonstrated exceptional performance, achieving 99% accuracy, precision, F1-Score, 98% recall, and 100% ROC AUC. This study offers a promising approach to early heart disease diagnosis and preventive healthcare.

Global occurrence, food web transfer, and human health risks of polycyclic aromatic hydrocarbons in biota.

Polycyclic aromatic hydrocarbons (PAHs) are widespread organic pollutants that pose significant health risks due to their bioaccumulation in the biota. This study examines the global distribution of PAHs in biota, identifies key factors influencing using boosted regression tree (BRT) models, analyzes their transfer through trophic magnification factors (TMF), and evaluates health risks using the EPA risk assessment model. Research on PAHs has grown from 1978 to 2023, peaking in 2021, with 171 out of 241 studies (71.1%) focusing on marine ecosystems. The highest PAH concentrations are observed in the Mediterranean Sea, Red Sea, and North American coastal regions, primarily influenced by industrial and human activities, such as factory emissions and ship transport. BRT analysis shows region factors and feeding habitats significantly influence PAH levels. TMF analysis shows that biodilution is the main mechanism for PAH attenuation, with concentrations decreasing as trophic levels increase. Additionally, health risk assessment further illustrate that toxicity equivalent (TEQ) values are highest in Egypt and Turkey. Across all populations in Egypt, the United States, Turkey, Portugal, and China, as well as children in Portugal and Sweden, there are potential risks from aquatic product consumption (10-6<CRI<10-4), with CRI values positively correlated with liver cancer incidence. While hazard quotients (HQ)<1 suggest overall safety, higher obesity risks are noted, particularly among women and adolescents.

Predictive Modeling of Juvenile Smalltooth Sawfish Habitats: Challenges and Opportunities for Conservation.

Effective conservation of rare species necessitates the identification of critical habitats and their specific features that influence species occurrence. This study focused on smalltooth sawfish (Pristis pectinata), a critically endangered elasmobranch, to explore how predictive spatial modeling can enhance conservation efforts. By leveraging long-term occurrence and relative abundance data from scientific gillnet surveys, along with insitu environmental data, we used boosted regression trees (BRT) to pinpoint key habitat features essential for juvenile sawfish. Our analysis revealed strong correlations between sawfish presence and environmental variables, with a preferential selection of very shallow, warm, and saline waters fringed with mangroves, particularly those with high pneumatophore density. High relative abundances were observed in warmer months, and predictions of presence were consistent around discrete mangrove-lined areas in Everglades National Park throughout all seasons. This study emphasizes the importance of high-quality environmental data in predictive modeling and informs management strategies aimed at protecting the critical habitats necessary for the recovery of this species. Preventing the loss of mangroves in vulnerable regions of the smalltooth sawfish's range-especially near anthropogenic influences such as the Charlotte Harbor Estuary-is crucial for recovery. We also highlight the need for improved data access to facilitate global abundance predictions, thereby enhancing spatial management and conservation efforts for rare species.

Collaborative machine learning-guided overall survival prediction of oral squamous cell carcinoma

Background There is a lack of prognosticators of overall survival (OS) for Oral Squamous Cell Carcinoma (OSCC). Objectives We examined collaborative machine learning (cML) in estimating the OS of OSCC patients. The prognostic significance of the clinicopathological parameters was examined. Methodology Altogether, 9439 OSCC patients were extracted from the Surveillance, Epidemiology, and End Results database (US). Five ML models – voting ensemble, stacked ensemble, extreme gradient boosting, light boosting, and logistic regression were used to predict OS. Three of these ML algorithms were combined to form a cluster of cML models. The performance of the cML was compared with the best performing individual ML algorithm following model training. Results The performance accuracy of the voting ensemble, stacked ensemble, extreme gradient boosting, light boosting, and logistic regression models was 70.2%, 69.9%, 69.1%, 69.4%, and 69.5% respectively, following model training. When the voting ensemble model was compared with cML using temporal validation, the cML showed a comparable performance accuracy. The most significant prognostic factors were age of the patient at diagnosis, T stage, tumor grade, marital status, gender, primary site, surgery, N stage, radiotherapy, ethnicity, chemotherapy, and M stage. Conclusions cML appears to give reliability to the final prediction and thereby may mark a paradigm shift from model individualism to a more cooperative paradigm. This approach may aid in determining an enhanced individualized treatment for OSCC patients.

An interpretation framework of machine learning models: prediction of pavement long-term performance

ABSTRACT Poor interpretation of machine learning models restrains its extensive application to practical engineering. In this regard, this study proposed an interpretation framework of machine learning models used for the prediction of pavement long-term performance, which consists of feature importance based on the SHapley Additive exPlanations (SHAP) method, factor influence analysis based on Accumulated Local Effects (ALE) and Local Conditional Effects, and feature interaction analysis based on H-statistic and second-order ALE. The interpretation framework was validated through its application in the prediction of pavement roughness and rutting. The Gradient Boost Regression Trees model was used as the machine learning model and the generalisation performance of the model exceeded 80%. According to the proposed interpretation framework, it can be concluded that environment-related factors dominated the deterioration of pavement roughness. The influence of precipitation on the pavement roughness followed a logarithmic increase. There was a threshold effect for the influence of traffic load on pavement roughness. In terms of pavement rutting, the traffic load and pavement structure number were the determinants. Among environmental factors, short-wave radiation contributed the most to pavement rutting. Feature interaction was related to the feature level and tended to be more significant at a relatively low feature level.

Modeling of faults in the CEB electrical transmission network by approaches: KNN, Random Forests, logistic regression, SVM, ANN and gradient boosting of supervised learning

The work accumulated in this article presents the results of learning the faults that affect the CEB network. The objective is to predict failures in order to prevent these faults from creating interruptions. The network operating data from 2008 to 2015 are used as materials. The algorithms: SVM, KNN, Random Forest, Gradient Boosting, ANN and Logistic Regression were used as methods to create the models. The results are subjected to evaluation criteria namely: the confusion matrix, the area under the ROC curve and the scores (Accuracy, F1 Score, Precision and recall). A characterization of the faults is carried out. The results of the characterization reveal that there are 19 faults and the most recurrent is the short circuit, which appeared 947 times out of 2427 during the study period. The modeling results are perfect. The True Positives of the confusion matrices are greater than 450 out of 497, for the classes. Some are better than others. The unfavorable is obtained through the KNN with AUC=0.761. Its score, (Accuracy=0.955; F1 Score = 0.957; Precision=0.958; Recall=0.955), confirms this observation. The AUC=0.664, remains even more unfavorable with the SVM modeling but its score, (Accuracy=0.989; F1 Score = 0.988; Precision=0.988; Recall=0.989), exceeds that of KNN. Moreover, for the other models, their AUC exceeds 80% with the more perfect logistic regression giving: AUC=0.991; Accuracy=0.991; F1 Score = 0.991; Precision=0.991; Recall=0.991. These results confirm that, even very random and of various causes, we can predict the defects in the CEB network. However, it is necessary to use more recent data in order to apply these results in future operations.