XGBoost Model Research Articles

Estimating ambient population density using physical features from GIS and machine learning: a study based on Japanese neighborhood

ABSTRACT This research aimed to develop an alternative method for estimating ambient population density at the neighborhood scale by utilizing a simple and universally available dataset. Physical environment data from GIS & OSM databases, basic statistics, and population density data from Mobile Spatial Statistics were combined to train tree-based Machine Learning models. The experiment resulted in an XGBoost model using 16 features capable of estimating ambient population density across three classes of outcome with 75.9% accuracy. The trained models were analyzed and visualized using SHAP and Partial Dependence Plot techniques to reveal feature importance and their threshold values. The study concludes that physical features can be effectively used as predictors of ambient population density and highlights areas for further investigation.

Estimating hair density with XGBoost

AbstractObjectivesHair density estimation is crucial in dermatology and trichology; however, manual counting is time‐consuming and error‐prone. Although automated approaches have been developed using image processing, neural networks, and deep learning, creating a robust and widely applicable method remains challenging. This study explored the use of XGBoost to estimate hair density with the aim of developing a more accurate and versatile approach.MethodsThe study utilized 895 scalp images to extract features and developed an XGBoost model for hair density estimation using 745 images to train the model and testing its performance on 150 images to evaluate the accuracy, error rate, and scatter plot.ResultsThe XGBoost model outperformed previous methods, achieving 89.5% accuracy on the training set and 95.3% accuracy on the test set. This surpassed the results of Kim et al. (52.4%), Urban et al. (79.6%), and Sacha et al. (88.2%) for the test set.ConclusionThe XGBoost algorithm proved to be effective for automated hair density estimation, achieving an accuracy of 95.3% on the test set. This approach, which focusses on scalp coverage and erosion features, can streamline and improve the objectivity of clinical hair analysis.

Predicting the compressive strength of high-performance concrete using an interpretable machine learning model.

Reliable predictions of concrete strength can reduce construction time and labor costs, providing strong support for building construction quality inspection. To enhance the accuracy of concrete strength prediction, this paper introduces an interpretable framework for machine learning (ML) models to predict the compressive strength of high-performance concrete (HPC). By leveraging information from a concrete dataset, an additional six features were added as inputs in the training process of the random forest (RF), AdaBoost, XGBoost and LightGBM models, and the optimal hyperparameters of the models were determined using 5-fold cross-validation and random search methods. Four interpretable ML models for predicting the compressive strength of HPC, including the RF, AdaBoost, XGBoost and LightGBM models, which combine feature derivation and random search, were constructed. In addition, the SHapley Additive exPlanations (SHAP) method was applied to analyze the effects of the input features on the prediction results of the LightGBM model, which combines feature derivation and random search. The results showed that input features such as age, water/cement ratio, slag, and water were the key influences for predicting the compressive strength of HPC. Input features such as the superplastic/cement ratio, slag/cement ratio, and ash/cement ratio had nonsignificant impacts on the predicted compressive strength.

Concrete Creep Prediction Based on Improved Machine Learning and Game Theory: Modeling and Analysis Methods

Understanding the impact of creep on the long-term mechanical features of concrete is crucial, and constructing an accurate prediction model is the key to exploring the development of concrete creep under long-term loads. Therefore, in this study, three machine learning (ML) models, a Support Vector Machine (SVM), Random Forest (RF), and Extreme Gradient Boosting Machine (XGBoost), are constructed, and the Hybrid Snake Optimization Algorithm (HSOA) is proposed, which can reduce the risk of the ML model falling into the local optimum while improving its prediction performance. Simultaneously, the contributions of the input features are ranked, and the optimal model’s prediction outcomes are explained through SHapley Additive exPlanations (SHAP). The research results show that the optimized SVM, RF, and XGBoost models increase their accuracies on the test set by 9.927%, 9.58%, and 14.1%, respectively, and the XGBoost has the highest precision in forecasting the concrete creep. The verification results of four scenarios confirm that the optimized model can precisely capture the compliance changes in long-term creep, meeting the requirements for forecasting the nature of concrete creep.

Solar Irradiance Prediction Method for PV Power Supply System of Mobile Sprinkler Machine Using WOA-XGBoost Model

Solar energy can mitigate the power supply shortage in remote regions for portable irrigation systems. The accurate prediction of solar irradiance is crucial for determining the power capacity of photovoltaic power generation (PVPG) systems for mobile sprinkler machines. In this study, a prediction method is proposed to estimate the solar irradiance of typical irrigation areas. The relation between meteorological parameters and solar irradiance is studied, and four different parameter combinations are formed and considered as inputs to the prediction model. Based on meteorological data provided by ten typical radiation stations uniformly distributed nationwide, an Extreme Gradient Boosting (XGBoost) model optimized using the Whale Optimization Algorithm (WOA) is developed to predict solar radiation. The prediction accuracy and stability of the proposed method are then evaluated for different input parameters through training and testing. The differences between the prediction performances of models trained based on single-station data and mixed data from multiple stations are also compared. The obtained results show that the proposed model achieves the highest prediction accuracy when the maximum temperature, minimum temperature, sunshine hours ratio, relative humidity, wind speed, and extraterrestrial radiation are used as input parameters. In the model testing, the RMSE and MAE of WOA-XGBoost are 2.142 MJ·m−2·d−1 and 1.531 MJ·m−2·d−1, respectively, while those of XGBoost are 2.298 MJ·m−2·d−1 and 1.598 MJ·m−2·d−1. The prediction effectiveness is also verified based on measured data. The WOA-XGBoost model has higher prediction accuracy than the XGBoost model. The model developed in this study can be applied to forecast solar irradiance in different regions. By inputting the meteorological parameter data specific to a given area, this model can effectively produce accurate solar irradiance predictions for that region. This study provides a foundation for the optimization of the configuration of PVPG systems for mobile sprinkler machines.

Prediction Model for the Chloride Ion Permeability Resistance of Recycled Aggregate Concrete Based on Machine Learning

The chloride ion permeability resistance of recycled aggregate concrete (RAC) is influenced by multiple factors, and the prediction model for this resistance based on machine learning is still limited. In the paper, six impact factors (IFs), including the carbonation of recycled coarse aggregates (YN), the replacement ratio of recycled coarse aggregates (r), the bending load level (L), the carbonation time (t) and temperature (T) of RAC, and the replacement ratio of carbonated recycled fine aggregates (f), were considered to conduct a chloride penetration test on RAC. Based on the experimental data, four algorithms, including artificial neural network (ANN), support vector machine (SVM), random forest (RF) and extreme gradient boosting (XGBoost), were adopted to establish the machine learning prediction models and study the relationships between the electric flux of RAC and the IFs. The results showed that the predicted values of all four models were in good agreement with the experimental values, and the XGBoost model had the best prediction performance on the testing set. Based on the XGBoost model, the LIME method was adopted to solve the interpretability problem in the prediction process. The importance ranking of IFs on the electric flux was r > t > f > T > L > YN. A graphical user interface (GUI) was developed based on Python 3.8 software to facilitate the use of machine learning models for the chloride ion permeability resistance of RAC. The research results can provide an accurate prediction of the electric flux of RAC.

Measuring Raven’s Progressive Matrices Combining Eye-Tracking Technology and Machine Learning (ML) Models

Extended testing time in Raven’s Progressive Matrices (RPM) can lead to increased fatigue and reduced motivation, which may impair cognitive task performance. This study explores the application of artificial intelligence (AI) in RPM by combining eye-tracking technology with machine learning (ML) models, aiming to explore new methods for improving the efficiency of RPM testing and to identify the key metrics involved. Using eye-tracking metrics as features, ten ML models were trained, with the XGBoost model demonstrating superior performance. Notably, we further refined the period of interest and reduced the number of metrics, achieving strong performance, with accuracy, precision, and recall all above 0.8, using only 60% of the response time and nine eye-tracking metrics. This study also examines the role of several key metrics in RPM and offers valuable insights for future research.

Detection of Rice Leaf Folder in Paddy Fields Based on Unmanned Aerial Vehicle-Based Hyperspectral Images

Pest infestations significantly impact rice production and threaten food security. Remote sensing offers a vital tool for the non-destructive, rapid detection of rice pests. Existing studies often focus on laboratory conditions at the leaf level, limiting their applicability for precise pesticide application. Therefore, this study aimed to develop a method for detecting rice pests (rice leaf folders) in paddy fields based on unmanned aerial vehicle (UAV) hyperspectral data. Firstly, a UAV imaging system collected hyperspectral images of rice plants in both the jointing and heading stages. A total of 222 field plots for investigating rice leaf folders was established during these two periods. Secondly, 23 vegetation indices were calculated as candidates for identifying rice pests. Then, hyperspectral data and field investigation data from the jointing stage were used to construct a machine learning (extreme gradient boosting, XGBoost) algorithm for detecting rice pests. The results showed that the XGBoost model exhibited the best performance when eight vegetation indices were considered as the selected input features for model construction: the Red-edge Normalized Difference Vegetation Index (red-edge NDVI), Structure Insensitive Pigment Index (SIPI), Enhanced Vegetation Index (EVI), Atmospherically Resistant Vegetation Index (ARVI), Soil-Adjusted Vegetation Index (SAVI), Red-edge Chlorophyll Index (CIred-edge), Pigment-Specific Simple Ratio680 (PSSR680), and Carotenoid Reflectance Index700 (CPI700). The training and testing accuracies reached 87.46% and 86%, respectively. Furthermore, the heading stage application confirmed the model’s feasibility. Thus, the XGBoost model with input features of eight vegetation indices provides an effective and reliable method for detecting rice leaf folders, supporting real-time, precise pesticide use in rice cultivation.

Abstract 4147625: Cardiometabolic Syndrome and Incident Alzheimer’s Disease: The Predicative Value of Age and CMS Using Cox and Machine Learning Models

Background: Cardiometabolic syndrome (CMS) poses a significant public health concern. The study aimed to investigate the predictive value of age and CMS for incident Alzheimer’s disease (AD) in women aged≥50. Methods: A cohort of women aged 50-79 ( n = 63,117) who participated in the Women’s Health Initiative Observational Study (WHIOS) in 1993-1998, without baseline AD and followed through to March 1, 2019, were analyzed. CMS was defined as having ≥3 of five CMS components: large waist circumference, HBP, elevated triglycerides, elevated glucose, and low HDL-cholesterol. AD was classified by physician-diagnoses of incident AD. Hazards ratios (HR) of AD risk associated with CMS by age were analyzed using Cox’s proportional hazards regression analysis. Machine learning (ML)-XGBoost and Lasso Cox models clustered individuals with low, mild, moderate, and severe risk of incident AD. Results: During a median follow-up of 20 years (range: 3.36 to 23.36 years), 8340 developed incident AD. The incident rate (95%CI) of AD was 8.6 (8.1-9.1) per 1000 person-years (PY) in women with CMS, and 7.0 (6.9-7.2) per 1000 PY in those without CMS (p<0.001). Multivariate Cox’s regression analysis indicated that women with CMS versus non-CMS had significantly higher risks of AD in those aged 50-59 [HR (95%CI): 1.37(1.12-1.68)], followed by in those aged 60-69 [1.26 (1.14-1.40)], but nonsignificant in those aged≥70 [1.00 (0.87-1.14)]. Elevated glucose concentration ranked as the top predictor for AD, followed by triglyceride and HBP. Four levels (low to severe) of AD risk were significantly clustered using ML-XGBoost and Lasso Cox models. Conclusions: Women aged 50-69 with CMS vs those without CMS had significantly higher relative risks of AD, but the risk was not significant in those aged≥70. The application of ML prediction models holds potential promise in the early identification of AD risk among U.S. adults.

Factors and machine learning models for predicting successful discontinuation of continuous renal replacement therapy in critically ill patients with acute kidney injury: a retrospective cohort study based on MIMIC-IV database.

For critically ill patients with acute kidney injury (AKI), there remains controversy regarding the predictive factors affecting the discontinuation of continuous renal replacement therapy (CRRT). This study aims to explore factors associated with successful CRRT discontinuation in AKI patients and to develop predictive models for successful discontinuation. We conducted a retrospective study on adult patients with AKI who received CRRT, sourced from the Medical Information Mart for Intensive Care (MIMIC-IV) database. Successful discontinuation of CRRT was defined as no CRRT requirement within 72h after stopping CRRT. Predictive factors for successful discontinuation of CRRT were analyzed. Additionally, we utilized machine learning algorithms to develop predictive models, including logistic regression (LR), decision tree (DT), random forest (RF), XGBoost, and K-nearest neighbor (KNN). A total of 599 patients were included, of whom 475 (79.3%) successfully discontinued CRRT. Urine output, non-renal SOFA score, bicarbonate, systolic blood pressure, and blood urea nitrogen were identified as risk factors for successful CRRT discontinuation. The KNN model exhibited the highest area under the receiver operating characteristic curve (AUC) (0.870), followed by LR (0.739), DT (0.691), RF (0.847), and XGBoost (0.830). When incorporating all available variables, the AUCs for the LR, DT, RF, XGBoost, and KNN models were 0.708, 0.674, 0.875, 0.866, and 0.816, respectively. Considering the performance of the models in both scenarios, the ensemble learning models (RF and XGBoost) were demonstrated superior performance. Our results identified factors associated with successful discontinuation of CRRT in AKI patients. Additionally, we developed promising machine learning models which provided a reference for future research.

Abstract 4136550: In-hospital cardiac arrest can be detected by Innovative smart healthcare syste

Background: Predicting in-hospital cardiac arrest (IHCA) is crucial for preventing patient fatalities. However, it remains challenging due to the irregularities in time series data across various features. Previous studies relied on traditional medical scoring systems, but their accuracy was limited as the models were overly simplistic. Recent research using machine learning methods has shown improved results. In our work, we employ deep learning and time series approaches, which are considered the most powerful models for addressing complex problems in recent years. Method/Research Design: The cohort study took place in a hospital in Taiwan from January 2016 to August 2022, involving 252,000 patients in general wards. Input data encompassed basic information, vital signs, Glasgow Coma Scale scores, medication, etc., to predict IHCA occurrences within 24 hours. Given the immediacy of IHCA events, we utilized left alignment to handle time series. Evaluation metrics included the area under the receiver operating characteristic curve (AUROC). Results: We compared three types of settings. The first was based on scoring systems, where the national early warning system (NEWS) achieved an AUROC of 0.756, and the modified early warning score (MEWS) reached 0.653. In the next setting, based on the data summary within a given time interval, the machine learning models XGBoost and deep learning model TabNet achieved AUROCs of 0.863 and 0.839, respectively. This indicates that machine learning outperforms traditional medical methods. The final setting involved time series analysis, where XGBoost and the Gate Recurrent Unit (GRU) network achieved 0.848 and 0.904, respectively, surpassing all previous settings. Conclusion: Our study demonstrates that learning-based methods outperform rule-based methods in predicting IHCA events. Machine learning algorithms offer more possibilities and have the potential to address complex problems effectively. Moreover, considering time series data rather than summarizing within fixed intervals preserves information and leads to better scores. Leveraging the time series deep learning method, GRU shows promising potential in significantly reducing the risk of IHCA events.

Investigation of Freeway Incident Duration Using Classification and Regression Trees Based on Multisource Data

Targeted contingency measures have proven highly effective at reducing the duration and harm caused by incidents. This study utilized the Classification and Regression Trees (CART) data mining technique to predict and quantify the duration of incidents. To achieve this, multisensor data collected from the Hangzhou freeway in China spanning from 2019 to 2021 was utilized to construct a regression tree with eight levels and 14 leaf nodes. By extracting 14 rules from the tree and establishing contingency measures based on these rules, accurate incident assessment and effective implementation of post-incident emergency plans were achieved. In addition, to more accurately apply the research findings to actual incidents, the CART method was compared with XGBoost, Random Forest (RF), and AFT (accelerated failure time) models. The results indicated that the prediction accuracy of the CART model is better than the other three models. Furthermore, the CART method has strong interpretability. Interactions between explanatory variables, up to seven, are captured in the CART method, rather than merely analyzing the effect of individual variables on the incident duration, aligning more closely with actual incidents. This study has important practical implications for advancing the engineering application of machine learning methods and the analysis of sensor data.

Modeling the Time-Dependent Variation of Road Salt Concentrations Using Analytical and Machine-Learning Approaches to Advance Service Life Predictions for Concrete Structures

The exposure of concrete structures to environmental and climatic conditions is detrimental to their durability. In northern climates, the key contributor to their degradation is corrosion of the reinforcing steel because of chloride ions originating from de-icing salts applied on roadways during the winter season. In consequence, a key input parameter for predicting the time to the initiation of corrosion for concrete elements is the time history of the concentration of chloride ions at their surfaces. To investigate this issue, a specialized mobile monitoring station was deployed along a roadway over several winter seasons to collect data on salting operations, weather conditions, and the temporal variation of chloride ion levels on the roadway. At first, salting operations were monitored, and then exploratory and machine-learning algorithms were applied to develop relationships between weather conditions, road conditions, and chloride ion concentrations. The first proposed model is based on the simulation modeling approach, while the second is based on the machine-learning XGBoost model. The findings demonstrate that both models can predict the variation of salt concentration on the road surface as a function of time after a salting operation. By accounting for the time dependency of surface chloride in service life models, more accurate predictions of corrosion initiation time are possible, since the rate of penetration of chloride ions is highly dependent on wetting/drying cycles throughout the winter.

SNPs and blood inflammatory marker featured machine learning for predicting the efficacy of fluorouracil-based chemotherapy in colorectal cancer.

Fluorouracil-based chemotherapy responses in colorectal cancer (CRC) patients vary widely, highlighting the role of pharmacogenomics in developing better predictive models. We analyzed 379 CRC patients receiving fluorouracil-based chemotherapy, collecting data on fluorouracil metabolism-related SNPs (TYMS, MTHFR, DPYD, RRM1), blood inflammatory markers, and clinical status. Six machine learning models-K-nearest neighbors, support vector machine, gradient boosting decision trees (GBDT), eXtreme Gradient Boosting (XGBoost), LightGBM, and random forest-were compared against multivariate logistic regression and a deep learning model (i.e., multilayer perceptron, MLP). Feature importance analysis highlighted seven predictors: histological grade, N and M staging, monocyte count, platelet-to-lymphocyte ratio, MTHFR rs1801131, and RRM1 rs11030918. In a five-fold cross-validation, XGBoost and GBDT exhibited superior performance, with Area Under Curve (AUC) of 0.88 ± 0.02. XGBoost excelled in identifying favorable prognosis (recall = 0.939). GBDT demonstrated balance in recognizing both categories, with a recall for favorable prognosis of 0.908 and a precision for unfavorable prognosis of 0.863. MLP had a similar AUC (0.87) with high precision for favorable prognosis (recall = 0.946). In external validation, XGBoost model achieved an accuracy of 0.79. An online prognostic tool based on XGBoost was developed, integrating metabolism-related SNPs and inflammatory markers, enhancing CRC treatment precision and supporting tailored chemotherapy.

Development and external validation of a machine learning model to predict diabetic nephropathy in T1DM patients in the real-world.

Studies on machine learning (ML) for the prediction of diabetic nephropathy (DN) in type 1 diabetes mellitus (T1DM) patients are rare. This study focused on the development and external validation of an explainable ML model to predict the risk of DN among individuals with T1DM. This was a retrospective, multicenter study conducted across 19 hospitals in Gansu Province, China (No: 2022-473). In total, 1368 patients were eligible for analysis among 1633 collected T1DM patients from January 2016 to December 2023. Recursive feature elimination using random forest and fivefold cross-validation was conducted to identify key features. Among the 12 initial ML algorithms, the optimal ML model was developed and validated externally in a distinct population, and its predictive outcomes were explained via the SHapley additive exPlanations method, which offered personalized decision insights. Among the 1368 T1DM patients, 324 had DN. The extreme gradient boosting (XGBoost) model, which achieved optimal performance with an AUC of 83% (95% confidence interval [CI]: 76‒89), was selected to predict the risk of DN among T1DM patients. The DN predictive model included variables such as T1DM duration, postprandial glucose (PPG), systolic blood pressure (SBP), glycated hemoglobin (HbA1c), serum creatinine (Scr) and low-density lipoprotein cholesterol (LDL-C). External validation confirmed the reliability of the model, with an AUC of 76% (95% CI: 70‒82). The ML prediction tool has potential for advancing early and precise identification of the risk of DN among T1DM patients. Although successful external validation indicated that the developed model can provide a promising strategy for clinical adoption and help improve patient outcomes through timely and accurate risk assessment, additional prospective data and further validation in diverse populations are necessary.

Collaborative Research for the Development of Localized Solar PV Output Forecasting Models for the Philippines Using Geospatial Data

Abstract. This paper presents a collaborative effort to develop localized solar photovoltaic (PV) power output (PPV) forecasting models for the Philippines using geospatial data. It underlines the importance of solar energy in the country and discusses the opportunities and challenges associated with PPV forecasting. Project SINAG, a two-year research project, aimed to develop solar PV output forecasting models through a collaborative approach with academic institutions, solar energy industries, and government agencies. Actual PPV data from 43 solar PV installations were analyzed alongside meteorological data from the PAGASA weather bureau, ERA5, AHI-8, and FY- 4A. These datasets were filtered based on a one-year period to ensure quality. The study employed SARIMAX, LSTM, and XGBoost models individually and in hybrid models to develop the forecasting models. Model performance was evaluated using root mean square error (RMSE), mean absolute error (MAE), and mean absolute percentage error (MAPE). In a case study in Baguio City, the SARIMAX model exhibited strong seasonal dependence, providing more accurate forecasts in dry seasons than in wet seasons. Additionally, the forecasting accuracy of each model (SARIMAX, LSTM, and XGBoost) varied based on the month and location of the installation, emphasizing the need for local and season-based PPV forecasting models. Despite implementation challenges, such as collaboration arrangements, bureaucratic barriers, and budget constraints, the project produced thirteen research publications and provided data for three student theses. This paper also demonstrated diverse engagements and contributions that emphasize the significance of collaborative research in conducting nationwide-scale data-driven projects.

Machine learning-based assessment of thermal comfort for the elderly in warm environments: Combining the XGBoost algorithm and human body exergy analysis

Many elderly people rarely own or use air conditioners because of low income and economising habits, causing them to live in warm thermal environments when heat waves and hot weather occur. Living in warm conditions worsens thermal discomfort and poses health risks this group. To investigate the thermal comfort and adaptation of the elderly, a total of 38 participants were recruited for two parts of experiments in a climate chamber: Part A collected thermal sensation vote (TSV) and physiological parameters for 30 min at 28, 30, and 32 °C, and Part B presented a 20-min cooling with fans (air velocities of 0.6 and 1.4 m/s) at the same temperature. Furthermore, we constructed a thermal comfort model for the elderly based on human body exergy analysis and the GBDT, AdaBoost, and XGBoost machine-learning algorithms. The results showed that the predicted mean vote considerably overestimated the actual TSV. The TSV and mean skin temperature were decreased by 0.1–0.5 scores and 0.4–0.5 °C by the behavioural adaptation of fan cooling. The predictive results showed that the XGBoost model performed better, with R2 score, mean absolute error (MAE), and mean squared error (MSE) of 81 %, 0.10, and 0.01. Exergy transfer from evaporation (Ex-Esk), mean skin temperature (mtsk), air velocity (va), and convective exergy transfer (Ex-C) contributed more to the feature importance in the SHAP value analysis. The current study has implications for investigating physiological comfort and age-friendly environmental designs for the elderly, providing new perspectives for thermal comfort evaluations.

Diabetes Prediction Based on KNN, XGBoost, SVM and LR model

Abstract. A. Diabetes mellitus is a chronic metabolic disease characterized by high blood sugar levels due to insulin production problems or insulin resistance. Early identification of diabetes is crucial for preventing associated complications and effectively managing the condition. This study explores the application of four machine learning models, i.e., K-Nearest Neighbor (KNN), XGBoost, Support Vector Machine (SVM) and Logistic Regression (LR) in diabetes prediction. the main goal is to assess and contrast these models' efficacy in identifying diabetes risk, thereby helping healthcare professionals make timely diagnostic and treatment decisions. The results show that the logistic regression model with an AUC value of 0.95 performs much better than the other models, demonstrating excellent sensitivity and specificity in diabetes identification. The XGBoost model also demonstrates considerable predictive accuracy with an AUC value of 0.84, highlighting its ability to effectively handle large-scale datasets. Although the SVM and KNN models had slightly lower AUC values of 0.79, they still provided reliable predictive capabilities. These results demonstrate how machine learning may be used to improve diabetes prediction.

Loan Approval Prediction Improved by XGBoost Model Based on Four-Vector Optimization Algorithm

Abstract. This paper discusses an improved XGBoost model based on four-vector optimization algorithm to improve the accuracy of loan approval prediction. Through the analysis of correlation heat maps, we found that there were significant positive and negative correlations among some variables, which laid the foundation for subsequent machine learning analysis. Based on this, we compare the traditional machine learning algorithm with the improved model to evaluate its performance in loan approval forecasting. In the confusion matrix analysis of the training set, the improved XGBoost model demonstrated excellent performance, with all loan approval predictions being correct with 100% accuracy. However, the performance in the test set was slightly different, with 1,182 projects receiving correct loan approval predictions and 99 projects forecasting errors. Of these, 26 projects that should have been predicted to be "unapproved" were incorrectly labeled as "approved," while 73 projects that should have been predicted to be "approved" were incorrectly labeled as "unapproved." These results suggest that we still need to pay attention to the misjudgment of the model in practical application. By synthesizing all model evaluation indicators, we found that the improved XGBoost model based on the four-vector optimization algorithm has a higher accuracy in loan approval prediction than the traditional XGBoost model, with an increase of 1.5%. In addition, the other evaluation indicators also show a trend of significantly better than the traditional model. This study shows that the four-vector optimization algorithm can effectively improve the application effect of XGBoost model in the field of loan approval, and provide more accurate data support and decision-making basis for the financial industry. In the future, we will continue to explore the potential and application prospects of this algorithm in other fields.

Predicting Employee Turnover in the Financial Company: A Comparative Study of CatBoost and XGBoost Models

Abstract: Employee turnover is a significant issue for financial institutions, impacting productivity, increasing recruitment costs, and disrupting critical operations. In this project, this study aimed to predict employee turnover using a dataset containing attributes such as employee satisfaction, performance, and tenure. By framing the task as a binary classification problem, this study employed CatBoost and XGBoost, two advanced regression-based algorithms, to develop predictive models. This paper's analysis demonstrated that CatBoost outperformed XGBoost across all evaluation metrics, including MAE, MSE, RMSE, and R, making it the more effective model for predicting turnover in the financial sector. The study highlights key factors contributing to employee attrition, such as job satisfaction, tenure, and promotion opportunities, offering actionable insights for retention strategies. Additionally, by predicting probabilities rather than binary outcomes, this study aims to make more detailed decisions about employee retention. This research provides valuable tools for financial institutions to mitigate the risk of turnover, retain critical talent, and ensure operational continuity.