Gradient Boosting Decision Tree Algorithm Research Articles

Simplified Pediatric Index of Mortality 3 Score by Explainable Machine Learning Algorithm.

Single-center, retrospective cohort study. Data from the Virtual Pediatric Systems for patients admitted to Cleveland Clinic Children`s PICU between January 2008 and December 2019 was obtained. Light Gradient Boosting Machine Regressor (a gradient boosting decision tree algorithm) was used for building the machine learning models. Variable importance was analyzed by SHapley Additive exPlanations. All of the 11 Pediatric Index of Mortality 3 variables were used as input variables in the machine learning models to predict Pediatric Index of Mortality 3 risk of mortality as the outcome variable. Mean absolute error, root mean squared error, and R-squared were calculated for each of the 11 machine learning models as model performance parameters. Quaternary children's hospital. PICU patients. None. Five-thousand sixty-eight patients were analyzed. The machine learning models were able to maintain similar predictive error until the number of input variables decreased to four. The machine learning model with five input variables (mechanical ventilation in the first hour of PICU admission, very-high-risk diagnosis, surgical recovery from a noncardiac procedure, low-risk diagnosis, and base excess) produced lowest mean root mean squared error of 1.49 (95% CI, 1.05-1.93) and highest R-squared of 0.73 (95% CI, 0.6-0.86) with mean absolute error of 0.43 (95% CI, 0.35-0.5) among all the 11 machine learning models. Explainable machine learning methods were feasible in simplifying the Pediatric Index of Mortality 3 scoring system with similar risk of mortality predictions compared to the original Pediatric Index of Mortality 3 model tested in a single-center dataset.

LDoS attack detection method based on traffic classification prediction

Aiming at the low rate and strong concealment of low-rate Denial of Service (LDoS) attacks, the calculation of traffic Hurst index is combined with traffic classification, and a machine learning LDoS attack detection method based on search sorting is proposed. The method first calculates the segmentation Hurst exponent of each flow, and constructs a traffic similarity matrix as a statistical feature. Then, using the improved model XGBoost of the Gradient Boosting Decision Tree (GBDT), the traffic is classified and predicted. The network angle distinguishes between normal traffic and abnormal Origin-Destination (OD) flows containing LDoS attacks, thereby achieving the purpose of detecting LDoS attacks. The method in this study was validated using the US public network dataset Abilene. The experimental results show that the global LDoS attack traffic detection method based on the Hurst index and GBDT algorithm achieves better detection results under different attack rates.

A study of detection of abnormal network traffic: A comparison of multiple algorithms

AbstractThe detection of abnormal traffic in networks is of great value for maintaining network security. This article gives a brief introduction of abnormal traffic and compares the performance of three algorithms, the support vector domain description algorithm, the gradient boosting decision tree algorithm, and the extreme learning machine‐k‐nearest neighbor (ELM‐KNN) algorithm. Experiments were carried out on the NSL‐KDD dataset. It was found that the accuracies of the three algorithms were 0.8327, 0.8679, and 0.9468, the recall rates were 0.6764, 0.7236, and 0.8997, the F1‐scores were 0.7898, 0.8364, and 0.9578, and the false alarm rates were 0.3236, 0.2764, and 0.1003. The running time of the ELM‐KNN algorithm was far less than the other two algorithms. The experimental results verify the effectiveness of the ELM‐KNN algorithm in abnormal network traffic detection. The ELM‐KNN algorithm can be further promoted and applied in practice.

Volcanic lithology identification based on parameter-optimized GBDT algorithm: A case study in the Jilin Oilfield, Songliao Basin, NE China

The reservoir rocks in the volcanic strata of the Jilin oil field are characterized by great complexity and diversity in composition and structure of lithology. To enhance the rate of lithology identification in subsurface is very laborious. However, lithology identification is often ignored in quantitative studies, though it is the basis for reservoir characterization. In this paper, an ensemble learning algorithm named gradient boosting decision tree (GBDT) was used to establish the classification model for the volcanic lithology identification of the Lower Cretaceous Yingcheng Formation in the Songliao Basin, NE China. At the same time, support vector machine (SVM), logistic regression (LR) and decision tree (DT) classification models were also adopted in contrast with the classification accuracy of GBDT model. Subsequently, the optimal key parameters for each model were determined by employing validation curves and GridSearchCv. These results indicate that the GBDT model is superior to the single classifier and can accurately distinguish the lithologic interface of breccia tuff and rhyolite. Moreover, it also has better recognition ability for thin layer. It was concluded that the ensemble learning algorithm GBDT has significantly enhanced the accuracy of lithology identification and can be used as a lithologic identification technology.

Development of data-driven prediction model for CFRP-steel bond strength by implementing ensemble learning algorithms

Bonding carbon fiber reinforced polymer (CFRP) laminates has been broadly utilized in steel structure rehabilitation. As for the final strengthened capacity, the bond strength between CFRP and steel usually plays a dominant role instead of the CFRP’s mechanical property. However, the bond behavior of the CFRP-steel (CS) interface is very complicated with various failure modes and consequently the bond strength is hard to estimate leading to the CFRP strengthened steel structure insecure. Under this circumstance, in order to accurately predict the bond strength of CS, efficient data-driven models were developed through implementing ensemble learning (EL) algorithms named by “gradient boosting decision tree (GBDT)” and “random forest (RF)” as two representative ones on a collected CS single-shear test database. These models’ performances on bond strength prediction were compared and also three representative machine learning algorithms “artificial neural network (ANN)”, “support vector machine (SVM)” and “classification and regression tree (CART)” are utilized for validating the necessity. The comparison results indicate that the model generated by the GBDT algorithm attains the best accuracy for CS interfacial bond strength prediction (R2 = 0.98) among the ensemble and machine learning algorithms. Through model explaning analysis, the mechanism behind the GBDT based prediction model was also carefully verified. After these tests and analyses, the GBDT based model was proved to have the potential to facilitate the design and evaluation of CFRP strengthened steel structures.

Downscaling the GPM-based satellite precipitation retrievals using gradient boosting decision tree approach over Mainland China

Remote sensing brings unprecedented opportunities to estimate precipitation at regional, continental, and even global scales. Nevertheless, the spatial resolutions of current mainstream satellite precipitation estimates are still too coarse to be directly used in the hydrological and meteorological applications over the medium and small scales. The spatio-temporal distribution of precipitation is often affected by various land surface factors, such as geographical location, vegetational cover, topographic characteristics, and ground temperature. Based on the relationship between precipitation and multi-geospatial factors, this study proposed a new spatial downscaling approach named as gradient boosting decision tree (GBDT) to downscale the annual satellite precipitation estimates of Integrated Multi-satellitE Retrievals for Global Precipitation Measurement (IMERG) from 0.1° to 0.01° gridded resolution over Mainland China from 2015 to 2018. As for the preprocessing of downscaling algorithm, the entire study area of Mainland China was separated into four different climate regions due to the spatial discrepancy of the relationship between precipitation and geospatial factors. To validate the effectiveness of GBDT approach, we compared it with other two types of downscaling methods based on random forests (RF) and support vector machine (SVM), respectively. Our evaluation results show that the geographical location (including both latitude and longitude) seems to be relatively more important and stable for GBDT modelling than other land surface factors across the four climate regions in Mainland China. In terms of large spatial scales, the GBDT and RF algorithms generally are superior to SVM for downscaling the IMERG precipitation, as SVM reduced the considerable accuracy of the downscaled results. Last but not least, GBDT exhibits more robust features in downscaling the satellite precipitation retrievals than RF over complex terrains, where the amount of precipitation has strong spatial heterogeneity.

Factors Identification and Prediction for Mind Wandering Driving Using Machine Learning

Traffic safety is affected by many complex factors. Mind wandering (MW) is a fatal cause affecting driving safety and is hard to be detected and prevented due to its uncertain and complex occurrence mechanism. The aim of this study was to propose a framework for analyzing and predicting MW based on readily available driving status data. The data used in this study are the single-trip information collected by the questionnaire, which includes drivers’ personal characteristics, contextual information in which MW occurs, and in-vehicle environmental factors. After investigating the extent of factors that influence MW, these chosen factors are used to forecast MW. Based on these results, we select factors reliable to be obtained in real life to forecast MW. To verify that the new factors explored are useful in improving the forecast accuracy, the compared analysis is conducted with the results found by our approach and the existing approaches. We compare results obtained by four machine-learning-enabled forecasting approaches on a real-life data set. The result shows that the factors found in this paper can significantly improve forecast accuracy. The confusion matrix, ROC curves, and AUC are conducted, and the performance of the gradient boosting decision tree algorithm is better than other forecast approaches. The importance rankings of most factors obtained by the Gradient Boosting Decision Tree and questionnaire are the same.

P–659 Artificial intelligence (AI) as an assisting tool in generating patient-friendly corifollitropin alfa ovarian stimulation protocol during in vitro fertilization

Abstract Study question How machine learning assisted in generating patient-friendly corifollitropin alfa protocol in normal responders? Summary answer In retrospective experiments, our machine learning model integrated physiological measurements of patients and clinical experience to generate a patient-friendly corifollitropin alfa protocol. What is known already Long-acting corifollitropin alfa can simplify the regimen, minimizing injections during the whole cycle. The previous study has described the patient-friendly protocol using corifollitropin alfa without routine pituitary suppression in normal responder can result in non-compromised clinical outcomes. Some studies showed machine learning can help with making clinical decisions and have the ability to learn from physiological measurements. Those methods effectuate certain points throughout short-acting menotropin protocols, however, there are still no robust AI tools for long-acting corifollitropin alfa protocols. Study design, size, duration 1,309 cycles were collected at Stork Fertility Center from November 2016 to October 2019, and 1,221 cycles were available after data cleaning and applying exclusion criteria, which Anti-Mullerian Hormone (AMH) is lower than 2. The data from electronic medical records (EMRs) consisted of age, AMH, body weight, luteinizing hormone (LH), and estradiol (E2) concentrations measured on revisit. Evaluation is performed by one physician who has more than 20 years of experience in IVF. Participants/materials, setting, methods: The protocol generator consisted of 5 parts: doses of Elonva, trigger type, doses of recombinant follicle-stimulating hormone (rFSH), doses of recombinant luteinizing hormone (rLH), and day of oocyte retrieval. The protocol was predicted by age, AMH, and weight firstly, then fine-tuned by LH and E2 after the first revisit. We used the gradient boosting decision tree algorithm to learn the protocol. The dataset was randomly split into 80% for training and 20% for testing. Main results and the role of chance In classification, the model predicted the dose of Elonva achieved an accuracy of 0.913 and an AUC of 0.946, and trigger type got an accuracy of 0.901 and AUC of 0.852 only using features on stimulation day (SD) 1 and gained 0.012 and 0.056 in accuracy and AUC correspondingly after adding features on the first revisit day. In regression, the mean absolute error (MAE) of rFSH dose, rLH dose, and oocytes retrieved day was 156.30 IU, 232.75 IU, and 0.80 days respectively, and after refining, the MAE dropped to 92.37 IU, 100.07 IU, and 0.46 days. The error of predictions in rFSH and rLH was almost equal to half increments of rFSH (150 IU) and one increment rLH (75 IU). This indicated that our model could provide a better prediction of these clinical decisions with one revisit only. Limitations, reasons for caution The present study was a single-center retrospective, and only analyzed the data from normal responders, whose AMH was equal or greater than 2. Though, the recommendations of our system act as references, the physician will make the final decision. Wider implications of the findings: Our result showed the potential of machine learning in generating protocols is promising. Recommendations generated by our model can provide the junior clinical teams to optimize the clinical plans and learn from the experience of experts. We look forward to applying our machine learning model to different protocols. Trial registration number Not applicable

Analysis and modeling of air conditioner usage behavior in residential buildings using monitoring data during hot and humid season

With split air conditioner (AC) becoming one of the most popular thermal comfort-related appliances in residential buildings in China, many issues, such as high energy consumption, have arisen. One crucial way to address these issues is to study the usage of split AC models, which can be greatly affected by occupant behavior. In this study, we conducted a large-scale field measurement among 34 residential buildings in eight cities distributed in three climate zones using wireless sensors during summer. The AC usage patterns in the three climate zones were analyzed, and models based on artificial neural networks and gradient boosting decision tree algorithms were developed for predicting AC on/off behavior. The results showed that the average usage duration and energy consumption of AC per day in the three climate zones were 7–10 h and 2.78–3.00 kWh, respectively, and that there was a large difference among different households. The AC usage patterns of households in the three climate zones had certain similarities. The models developed in this study showed higher performance and accuracy than the logistic regression-based model, and indoor relative humidity and CO2 concentration were found to significantly improve predicting accuracy. The models can generate an AC operation schedule as an input boundary condition to improve the accuracy of residential building energy consumption simulations in future studies.

A feature extraction method based on the entropy-minimal description length principle and GBDT for common surface water pollution identification

Abstract To effectively prevent river water pollution, water quality monitoring is necessary. However, existing methods for water quality assessment are limited in terms of the characterization of water quality conditions, and few researchers have been able to focus on feature extraction methods relative to water pollution identification, or to obtain accurate water pollution source information. Thus, this study proposed a feature extraction method based on the entropy-minimal description length principle and gradient boosting decision tree (GBDT) algorithm for identifying the type of surface water pollution in consideration of the distribution characteristics and intrinsic association of conventional water quality indicators. To improve the robustness to noise, we constructed the coarse-grained discretization features of each water quality index based on information entropy. The nonlinear correlation between water quality indexes and pollution classes was excavated by the GBDT algorithm, which was utilized to acquire tree transformed features. Water samples collected by a southern city Environmental Monitoring Center were used to test the performance of the proposed algorithm. Experimental results demonstrate that features extracted by the proposed method are more effective than the water quality indicators without feature engineering and features extracted by the principal component analysis algorithm.

A Gradient-Boosted Decision-Tree Algorithm for the Prediction of Short-Term Mortality in Acute Heart Failure Patients

A Gradient-Boosted Decision-Tree Algorithm for the Prediction of Short-Term Mortality in Acute Heart Failure Patients

Short text filtering based on gradient boosting decision tree

With the rapid development of the Internet, event detection from massive data has become a research hotspot. However, the existing event detection methods for social networks rarely consider the noise data in short text data in detail. Therefore, there is a lot of noise in the input of event detection, resulting in a large number of false-positive events in the event detection results, which affects the efficiency and accuracy of event detection. In this paper, three types of features are proposed to mine text features, and the gradient boosting decision tree algorithm is used. Experiments show that the algorithm has good filtering performance and interpretability for short texts in social networks.

Numerical study on the influence of gasoline properties and thermodynamic conditions on premixed laminar flame velocity at multiple conditions

In this paper, the effects of two gasoline key properties, research octane number (RON) and fuel sensitivity S, on the laminar flame velocity (LFV) were studied. Three environmental parameters, temperature, pressure and equivalent ratio ϕ, were also taken into consideration at multiple conditions. An effective hybrid computation method, combining the flame kinetics model and the machine learning (ML) algorithm, was proposed for the parameter study. The flame kinetics model coupled with a well-validation TRF reaction mechanism to model the flame propagation characteristics; four ML algorithms, Random forest (RF), Gradient boosting decision tree (GBDT), Support vector regression (SVR), and Adaboost were adopted to establish the data mapping between investigated parameters with fuel's LFV. The result shows all ML models perform well on the test set, especially for the GBDT algorithm, which with R2 of 0.9984 and RMSE of 0.0084. Based on the power exponential formulas, the LFV correlations concerning temperature, pressure and equivalent ratio were concluded. It also found the influence mechanism of RON and S on LFV was controlled by the equivalence ratio. Besides, the tri-component diagrams of the LFV with the RON-S-ϕ under typical environments were analyzed, and the statistical information in different flame patterns was discussed in detail.

Integration and interplay of machine learning and bioinformatics approach to identify genetic interaction related to ovarian cancer chemoresistance

Although chemotherapy is the first-line treatment for ovarian cancer (OCa) patients, chemoresistance (CR) decreases their progression-free survival. This paper investigates the genetic interaction (GI) related to OCa-CR. To decrease the complexity of establishing gene networks, individual signature genes related to OCa-CR are identified using a gradient boosting decision tree algorithm. Additionally, the genetic interaction coefficient (GIC) is proposed to measure the correlation of two signature genes quantitatively and explain their joint influence on OCa-CR. Gene pair that possesses high GIC is identified as signature pair. A total of 24 signature gene pairs are selected that include 10 individual signature genes and the influence of signature gene pairs on OCa-CR is explored. Finally, a signature gene pair-based prediction of OCa-CR is identified. The area under curve (AUC) is a widely used performance measure for machine learning prediction. The AUC of signature gene pair reaches 0.9658, whereas the AUC of individual signature gene-based prediction is 0.6823 only. The identified signature gene pairs not only build an efficient GI network of OCa-CR but also provide an interesting way for OCa-CR prediction. This improvement shows that our proposed method is a useful tool to investigate GI related to OCa-CR.

A novel predictive model of mixed oil length of products pipeline driven by traditional model and data

Mixed oil will inevitably form during the batch transportation in products pipeline, and accurate calculation of mixed oil length is of great significance for the economic benefit of the oil pipeline network. Traditional models (White Box/WB) are based on underlying engineering principles, however, there are some problems including low accuracy or high computing complexity. Data model (Black Box/BB), developed by historical data and machine learning algorithms, could act as a problem-solving approach while it takes the risk of overfitting due to the lack of understanding of the physical process. In this article, the Austin-Palfrey equation is used as the WB model to convert original features and input them into the BB model built on the Gradient Boosting Decision Tree algorithm (GBDT), and finally a novel predictive model (Grey Box/GB) incorporated prior knowledge is proposed. Experimental results of industrial examples reveal that the new model has higher predictive accuracy and better robustness, which means the dual-driven modeling method is more effective and can even become a new and valuable research direction. • Traditional model of mixed oil length (Austin-Palfrey equation) is used to reconstruct the features of BB model. • GB model is created by coming with traditional model and GBDT algorithm. • Prediction accuracy and robustness of all GB models are better than those of WB model and BB model.

Multifeature pool importance fusion based GBDT (MPIF-GBDT) for short-term electricity load prediction

Feature selection is one of the key factors in predicting. Different feature selection algorithms have their unique preferences for elemental analysis of the data. This results in failing to determine the optimal features when a dataset goes through different feature selection algorithms to get different pools of input features, which in turn affects the prediction quality. To address this problem, the method integrates and fuses the feature importance values of two different feature selection methods. Then the input feature pools are optimized and filtered for the prediction model. Finally, the multifeature pool importance fusion based GBDT (MPIF-GBDT) is developed, which integrates the different feature selection methods and predicts the short-term power load in combination with the gradient boosting decision tree algorithm. In this paper, the tree model feature selection and the Recursive Feature Elimination (RFE) are chosen as feature selection methods. The experimental results show that MPIF-GBDT can significantly improve the accuracy of the prediction compared with the benchmark model.

An Intrusion Detection Method Based on Machine Learning and State Observer for Train-Ground Communication Systems

The communication-based train control (CBTC) system is a typical cyber physical system in urban rail transit. The train-ground communication system is a very important subsystem of the CBTC system and uses the wireless communication protocols to transmit control commands. However, it faces some potential information security risks. To ensure information security of the train-ground communication system, an intrusion detection method based on machine learning and state observer is proposed to detect and recognize various attacks in this paper. The detection system not only detects the anomalies of the wireless network data, but also detects the anomalies of the train physical states. This method includes two layers. The first layer is used to detect and identify wireless network attacks based on machine learning algorithms, such as the random forest algorithm and the gradient boosted decision tree algorithm. The second layer is used to detect the abnormal physical state of train operation based on a state observer. By combining the results of the above two layers, a comprehensive intrusion detection result is given. The simulation results show that the proposed method is effective and practical.

Application of Machine Learning Algorithms to Predict Body Condition Score from Liveweight Records of Mature Romney Ewes

Body condition score (BCS) in sheep (Ovis aries) is a widely used subjective measure of the degree of soft tissue coverage. Body condition score and liveweight are statistically related in ewes; therefore, it was hypothesized that BCS could be accurately predicted from liveweight using machine learning models. Individual ewe liveweight and body condition score data at each stage of the annual cycle (pre-breeding, pregnancy diagnosis, pre-lambing and weaning) at 43 to 54 months of age were used. Nine machine learning (ML) algorithms (ordinal logistic regression, multinomial regression, linear discriminant analysis, classification and regression tree, random forest, k-nearest neighbors, support vector machine, neural networks and gradient boosting decision trees) were applied to predict BCS from a ewe’s current and previous liveweight record. A three class BCS (1.0–2.0, 2.5–3.5, >3.5) scale was used due to high-class imbalance in the five-scale BCS data. The results showed that using ML to predict ewe BCS at 43 to 54 months of age from current and previous liveweight could be achieved with high accuracy (>85%) across all stages of the annual cycle. The gradient boosting decision tree algorithm (XGB) was the most efficient for BCS prediction regardless of season. All models had balanced specificity and sensitivity. The findings suggest that there is potential for predicting ewe BCS from liveweight using classification machine learning algorithms.

Emotion Recognition Based on Skin Potential Signals with a Portable Wireless Device.

Emotion recognition is of great importance for artificial intelligence, robots, and medicine etc. Although many techniques have been developed for emotion recognition, with certain successes, they rely heavily on complicated and expensive equipment. Skin potential (SP) has been recognized to be correlated with human emotions for a long time, but has been largely ignored due to the lack of systematic research. In this paper, we propose a single SP-signal-based method for emotion recognition. Firstly, we developed a portable wireless device to measure the SP signal between the middle finger and left wrist. Then, a video induction experiment was designed to stimulate four kinds of typical emotion (happiness, sadness, anger, fear) in 26 subjects. Based on the device and video induction, we obtained a dataset consisting of 397 emotion samples. We extracted 29 features from each of the emotion samples and used eight well-established algorithms to classify the four emotions based on these features. Experimental results show that the gradient-boosting decision tree (GBDT), logistic regression (LR) and random forest (RF) algorithms achieved the highest accuracy of 75%. The obtained accuracy is similar to, or even better than, that of other methods using multiple physiological signals. Our research demonstrates the feasibility of the SP signal’s integration into existing physiological signals for emotion recognition.

Lung mass density prediction using machine learning based on ultrasound surface wave elastography and pulmonary function testing.

The objective of this study is to predict in vivo lung mass density for patients with interstitial lung disease using different gradient boosting decision tree (GBDT) algorithms based on measurements from lung ultrasound surface wave elastography (LUSWE) and pulmonary function testing (PFT). Age and weight of study subjects (57 patients with interstitial lung disease and 20 healthy subjects), surface wave speeds at three vibration frequencies (100, 150, and 200 Hz) from LUSWE, and predicted forced expiratory volume (FEV1% pre) and ratio of forced expiratory volume to forced vital capacity (FEV1%/FVC%) from PFT were used as inputs while lung mass densities based on the Hounsfield Unit from high resolution computed tomography (HRCT) were used as labels to train the regressor in three GBDT algorithms, XGBoost, CatBoost, and LightGBM. 80% (20%) of the dataset was used for training (testing). The results showed that predictions using XGBoost regressor obtained an accuracy of 0.98 in the test dataset. The obtained results suggest that XGBoost regressor based on the measurements from LUSWE and PFT may be able to noninvasively assess lung mass density in vivo for patients with pulmonary disease.