Extreme Gradient Boosting Algorithm Research Articles

Nonlinear overall annoyance level modeling and interior sound quality prediction for pure electric vehicle with extreme gradient boosting algorithm

A single psychoacoustic metric cannot reflect pure electric vehicle (PEV) interior overall sound quality (SQ) accurately. There exists a complex nonlinear mapping relationship between psychoacoustic indexes and SQ prediction. In this work, a nonlinear overall annoyance level modeling method using the extreme gradient boosting (XGBoost) algorithm is proposed. Firstly, comprehensive noise samples inside the cabins of PEV and conventional fuel vehicle (FV) at high speed are collected. Subjective and objective SQ evaluations are conducted for FV and PEV interior noise, respectively. Their differences are analyzed by comparison. The correlation among various SQ indexes is studied, revealing a significant multicollinearity disruption. Then, the relative importance ranking is determined by the random forests algorithm, which provides a basis for feature index selection. Finally, the XGBoost-based nonlinear overall annoyance level model for SQ evaluation is established. Using the XGBoost-based model, traditional multiple linear regression (MLR) model, and linear least absolute shrinkage and selection operator (LASSO) model, PEV SQ is predicted. The results show that the XGBoost-based model exhibits high prediction accuracy, improving 75.6% relative to the MLR model and 74.9% relative to the LASSO model. Meanwhile, it has excellent stability and goodness-of-fitting. Thus, it provides an alternative method for PEV SQ evaluation and prediction.

A novel extreme gradient boosting algorithm based model for predicting the scour risk around bridge piers: application to French railway bridges

Bridge failures inevitably cause severe social consequences in terms of the economic losses or even human fatalities. Scour is one of the leading causes for bridge failures. Due to its complexity and multidisciplinary nature, the mechanism of scour has not been completely understood yet. Inspired by the rapid development of Machine Learning (ML) techniques, this paper aims to construct a novel data-driven extreme gradient boosting (XGBoost) algorithm based model to predict the scour risk around bridge piers. Data used in the present study is provided by the French National Railway Company (SNCF). The performance of XGBoost-based model is compared with three commonly used algorithms: support vector machine, random forest and multilayer perceptron. Results show that the XGBoost classifier achieves high accuracy (0.959/0.938), precision (0.970/0.961), recall (0.974/0.956) and low false positive rate (0.085/0.114) for training and test set respectively. Moreover, the classifier obtains an area under the ROC curve (AUC) score equal to 0.974 for test set (a perfect classifier has an AUC equal to 1). This paper presents a cutting-edge application of XGBoost algorithm in the maintenance of railway bridges. The proposed methodology may allow engineers to determine the scour risk of bridge piers in an accurate and rapid way.

A model for Real Estate Price Prediction using Multi-Level Stacking Ensemble Technique

Recent research and economic publications have shown the impact of real estate investment on the over economy of Nigeria. It is therefore crucial to employ machine learning technique to predict the price for real estate properties. Real estate price analysis and prediction will assist in establishment of real estate policies and can also be used to aid real estate property stakeholders to come up with informative decisions without bias or prejudice. Thus, it is imperative to develop a model to improve the accuracy of real estate price prediction. The goal of this research is to develop a model using a multi-level stacking ensemble technique to predict price of real estate property. The dataset utilized for the study was collected from transactions done by real estate firms in Port Harcourt and it consist of a total of 1053 rows with twelve features. The base model used includes Random Forest(RF), Extreme Gradient Boosting Algorithm(XGBoost), Light Gradient Boosting Machine(LightGBM), Decision Tree regression and ElasticNet Regression. Various combinations of the base models were stacked using StackingCVRegressor. The final model was developed by combining the best performing stacked models and evaluated using R-Square, Mean Absolute Error(MAE), Root Mean Square Error(RMSE), Mean Square Error(MSE) and Training time. The proposed model outperformed the various individual base model with R-square of 0.985203, MSE of 0.013438, RMSE of 0.115923, MAE of 0.063411 and training time of 0.599398. The result show that multi-level stacking significant improve the accuracy of a model. Again, it was observed stacking improve the performance accuracy of a model at the cost of computational time. Stacking by using blending function for the proposed model significantly reduced the computational time for training the model to 0.599398 second when compared to using StackingCVRegressor with training time of 107.054931 seconds. Therefore, multi-level stacking ensemble technique can be employed to improve the predictive accuracy of a prediction model. Future work can be done by increasing the dataset and also increasing the number of features.

Diagnostic efficacy of a combined diagnostic model based on extreme gradient boosting algorithm in differentiating the pathological grading of gastric neuroendocrine neoplasms

Objective: To evaluate the diagnostic efficacy of combined diagnostic model based on extreme gradient boosting (XGBoost) algorithm to determine the pathological grading of gastric neuroendocrine neoplasms (NENs). Methods: A total of 81 gastric NENs patients in the First Affiliated Hospital of Zhengzhou University confirmed by definite pathological grading from August 2012 to December 2019 were enrolled. The data of clinical and CT findings were collected. The number of lesions, tumor location, shape, lymph node metastasis, thickness, longitude of tumor and CT values in arterial and venous phase were analyzed. ITK-SNAP software and Python 2.1.0 PyRadiomics software were used to perform the image preprocessing and radiomics features extraction from segmented images. XGBoost algorithm was used to build the CT findings model, radiomics model in arterial phase, radiomics model in venous phase and combined diagnostic model. The diagnostic efficacy of CT imaging model, radiomics model in arterial phase, radiomics model in venous phase and combined diagnostic model were evaluated by accuracy, mean squared error (MSE) and mean absolute error (MAE). Results: The subjects were 28.0 to 78.0 (58.6+10.7) years old, including 56 males (69.1%). The number of lesions, tumor location, shape, lymph node metastasis, thickness and longitude of tumor between G1/G2 and G3 patients showed statistic significances (all P values<0.05), while there were no differences in CT values in arterial and venous phase (both P values>0.05). Six most important features in the combined diagnostic model were A_logarithm_glcm_Imc1, P_squareroot_glcm_Maximum Probability, thickness, longitude, A_wavelet-HHL_glrlm_GrayLevelNonUniformity and P_wavelet-LLL_ngtdm_Contrast, respectively. The accuracy of CT findings model, radiomics model in arterial phase, radiomics model in venous phase and combined diagnostic model were 81.8%, 86.0%, 87.8% and 91.0%, respectively; with MSE were 539.41, 490.08, 429.99 and 371.92, respectively; and MAE were 16.72, 15.25, 14.23 and 12.33, respectively. The MAE value of the combined diagnostic model was lower than those of CT findings model and radiomics model in arterial phase (P<0.001 and 0.004, respectively), while no statistically difference was detected compared to radiomics model in venous phase (P=0.111). Conclusion: The combined diagnostic model based on XGBoost algorithm have a good diagnostic efficiency for the pathological grading of gastric NENs.

XGBXSS: An Extreme Gradient Boosting Detection Framework for Cross-Site Scripting Attacks Based on Hybrid Feature Selection Approach and Parameters Optimization

With the widespread popularity of the Internet and the transformation of the world into a global village, Web applications have been drawn increased attention over the years by companies, organizations, and social media, making it a prime target for cyber-attacks. The cross-site scripting attack (XSS) is one of the most severe concerns, which has been highlighted in the forefront of information security experts' reports. In this study, we proposed XGBXSS, a novel web-based XSS attack detection framework based on an ensemble-learning technique using the Extreme Gradient Boosting algorithm (XGboost) with extreme parameters optimization approach. An enhanced feature extraction method is presented to extract the most useful features from the developed dataset. Furthermore, a novel hybrid approach for features selection is proposed, comprising information gain (IG) fusing with sequential backward selection (SBS) to select an optimal subset reducing the computational costs and maintaining the high-performance of detector' simultaneously. The proposed framework has successfully exceeded several tests on the holdout testing dataset and achieved avant-garde results with accuracy, precision, detection probabilities, F-score, false-positive rate, false-negative rate, and AUC-ROC scores of 99.59%, 99.53 %, 99.01%, 99.27%, 0.18%, 0.98%, and 99.41%, respectively. Moreover, it can bridge the existing research gap concerning previous detectors, with a higher detection rate and lesser computational complexity. It also has the potential to be deployed as a self-reliant system, which is efficient enough to defeat such attacks, including zero-day XSS-based attacks.

A clinical diagnostic model based on an eXtreme Gradient Boosting algorithm to distinguish type 1 diabetes.

BackgroundAccurate classification of type 1 diabetes (T1DM) and type 2 diabetes (T2DM) in the early phase is crucial for individual precision treatment. This study aimed to develop a classification model having fewer and easier to access clinical variables to distinguish T1DM in newly diagnosed diabetes in adults.MethodsClinical and laboratory data were collected from 15,206 adults with newly diagnosed diabetes in this cross-sectional study. This cohort represented 20 provinces and 4 municipalities in China. Types of diabetes were determined based on postprandial C-peptide (PCP) level and glutamic acid decarboxylase autoantibody (GADA) titer. We developed multivariable clinical diagnostic models using the eXtreme Gradient Boosting (XGBoost) algorithm. Classification variables included in the final model were based on their scores of importance. Model performance was evaluated by area under the receiver operating characteristic curve (ROC AUC), sensitivity, and specificity. The performance of models with different variable combinations was compared. Calibration intercept and slope were evaluated for the final model.ResultsAmong the newly diagnosed diabetes cohort, 1,465 (9.63%) persons had T1DM and 13,741 (90.37%) had T2DM. Body mass index (BMI) contributed the most to the model, followed by age of onset and hemoglobin A1c (HbA1c). Compared with models with other clinical variable combinations, a final model that integrated age of onset, BMI and HbA1c had relatively higher performance. The ROC AUC, sensitivity, and specificity for this model were 0.83 (95% CI, 0.80 to 0.85), 0.77, and 0.76, respectively. The calibration intercept and slope were 0.02 (95% CI, –0.03 to 0.06) and 0.90 (95% CI, 0.79 to 1.02), respectively, which suggested a good calibration performance.ConclusionsOur classification model that integrated age of onset, BMI, and HbA1c could distinguish T1DM from T2DM, which provides a useful tool in assisting physicians in subtyping and precising treatment in diabetes.

Computerized Mortality Prediction for Community-acquired Pneumonia at 117 Veterans Affairs Medical Centers.

Rationale: Computerized severity assessment for community-acquired pneumonia could improve consistency and reduce clinician burden. Objectives: To develop and compare 30-day mortality-prediction models using electronic health record data, including a computerized score with all variables from the original Pneumonia Severity Index (PSI) except confusion and pleural effusion ("ePSI score") versus models with additional variables. Methods: Among adults with community-acquired pneumonia presenting to emergency departments at 117 Veterans Affairs Medical Centers between January 1, 2006, and December 31, 2016, we compared an ePSI score with 10 novel models employing logistic regression, spline, and machine learning methods using PSI variables, age, sex and 26 physiologic variables as well as all 69 PSI variables. Models were trained using encounters before January 1, 2015; tested on encounters during and after January 1, 2015; and compared using the areas under the receiver operating characteristic curve, confidence intervals, and patient event rates at a threshold PSI score of 970. Results: Among 297,498 encounters, 7% resulted in death within 30 days. When compared using the ePSI score (confidence interval [CI] for the area under the receiver operating characteristic curve, 0.77-0.78), performance increased with model complexity (CI for the logistic regression PSI model, 0.79-0.80; CI for the boosted decision-tree algorithm machine learning PSI model using the Extreme Gradient Boosting algorithm [mlPSI] with the 19 original PSI factors, 0.83-0.85) and the number of variables (CI for the logistic regression PSI model using all 69 variables, 0.84-085; CI for the mlPSI with all 69 variables, 0.86-0.87). Models limited to age, sex, and physiologic variables also demonstrated high performance (CI for the mlPSI with age, sex, and 26 physiologic factors, 0.84-0.85). At an ePSI score of 970 and a mortality-risk cutoff of <2.7%, the ePSI score identified 31% of all patients as being at "low risk"; the mlPSI with age, sex, and 26 physiologic factors identified 53% of all patients as being at low risk; and the mlPSI with all 69 variables identified 56% of all patients as being at low risk, with similar rates of mortality, hospitalization, and 7-day secondary hospitalization being determined. Conclusions: Computerized versions of the PSI accurately identified patients with pneumonia who were at low risk of death. More complex models classified more patients as being at low risk of death and as having similar adverse outcomes.

Development and validation of an explainable artificial intelligence-based decision-supporting tool for prostate biopsy.

To develop and validate a risk calculator for prostate cancer (PCa) and clinically significant PCa (csPCa) using explainable artificial intelligence (XAI). We used data of 3791 patients to develop and validate the risk calculator. We initially divided the data into development and validation sets. An extreme gradient-boosting algorithm was applied to the development calculator using five-fold cross-validation with hyperparameter tuning following feature selection in the development set. The model feature importance was determined based on the Shapley value. The area under the curve (AUC) of the receiver operating characteristic curve was analysed for each validation set of the calculator. Approximately 1216 (32.7%) and 562 (14.8%) patients were diagnosed with PCa and csPCa. The data of 2843 patients were used for development, whereas the data of 948 patients were used as a test set. We selected the variables for each PCa and csPCa risk calculation according to the least absolute shrinkage and selection operator regression. The AUC of the final PCa model was 0.869 (95% confidence interval [CI] 0.844-0.893), whereas that of the csPCa model was 0.945 (95% CI 0.927-0.963). The prostate-specific antigen (PSA) level, free PSA level, age, prostate volume (both the transitional zone and total), hypoechoic lesions on ultrasonography, and testosterone level were found to be important parameters in the PCa model. The number of previous biopsies was not associated with the risk of csPCa, but was negatively associated with the risk of PCa. We successfully developed and validated a decision-supporting tool using XAI for calculating the probability of PCa and csPCa prior to prostate biopsy.

Radiomic analysis of magnetic resonance imaging predicts brain metastases velocity and clinical outcome after upfront radiosurgery.

BackgroundBrain metastasis velocity (BMV) predicts outcomes after initial distant brain failure (DBF) following upfront stereotactic radiosurgery (SRS). We developed an integrated model of clinical predictors and pre-SRS MRI-derived radiomic scores (R-scores) to identify high-BMV (BMV-H) patients upon initial identification of brain metastases (BMs).MethodsIn total, 256 patients with BMs treated with upfront SRS alone were retrospectively included. R-scores were built from 1246 radiomic features in 2 target volumes by using the Extreme Gradient Boosting algorithm to predict BMV-H groups, as defined by BMV at least 4 or leptomeningeal disease at first DBF. Two R-scores and 3 clinical predictors were integrated into a predictive clinico-radiomic (CR) model.ResultsThe related R-scores showed significant differences between BMV-H and low BMV (BMV-L), as defined by BMV less than 4 or no DBF (P < .001). Regression analysis identified BMs number, perilesional edema, and extracranial progression as significant predictors. The CR model using these 5 predictors achieved a bootstrapping corrected C-index of 0.842 and 0.832 in the discovery and test sets, respectively. Overall survival (OS) after first DBF was significantly different between the CR-predicted BMV-L and BMV-H groups (median OS: 26.7 vs 13.0 months, P = .016). Among patients with a diagnosis-specific graded prognostic assessment of 1.5–2 or 2.5–4, the median OS after initial SRS was 33.8 and 67.8 months for CR-predicted BMV-L, compared to 13.5 and 31.0 months for CR-predicted BMV-H (P < .001 and <.001), respectively.ConclusionOur CR model provides a novel approach showing good performance to predict BMV and clinical outcomes.