Gradient Boosting Decision Tree Method Research Articles

Age Determination of LAMOST Red Giant Branch Stars Based on the Gradient Boosting Decision Tree Method

In this study, we estimate the stellar ages of LAMOST DR8 red giant branch (RGB) stars based on the gradient boosting decision tree (GBDT) algorithm. We used 2643 RGB stars extracted from the APOKASC-2 asteroseismological catalog as the training data set. After selecting the parameters ([α/Fe], [C/Fe], T eff, [N/Fe], [C/H], log g) highly correlated with age using GBDT, we apply the same GBDT method to the new catalog of more than 590,000 stars classified as RGB stars. The test data set shows that the median relative error is around 11.6% for the method. We also compare the predicted ages of RGB stars with other studies (e.g., based on APOGEE) and find some systematic differences. The final uncertainty is about 15%–30% compared to the ages of open clusters. Then, we present the spatial distribution of the RGB sample with an age determination, which could recreate the expected result, and discuss systematic biases. All these diagnostics show that one can apply the GBDT method to other stellar samples to estimate atmospheric parameters and age.

Autonomous prediction of rock deformation in fault zones of coal roadways using supervised machine learning

Coal roadway fault zones typically rely on multiple re-excavations and repeated reinforcements for control, posing challenges in achieving precise control of the initial support strength of the roadway. Therefore, predicting the deformation of the surrounding rock during the stability phase of fault zone excavation is a prerequisite for the precise control of the initial support strength. This study employs supervised learning algorithms to extract 14 sets of features as input variables for deformation prediction, focusing on the displacement of the roof and ribs in coal roadway fault zones. By constructing a dataset using field data, the reliability of the four different algorithms for predicting the displacement of the roof and rib in fault zones of roadways is validated. This predicts the deformation in the fault zone surrounding the rocks and ultimately categorises the failure types of the fault zone roadways into four classes. The results indicate that the gradient boosting decision tree method exhibits the highest reliability in predicting the roof and rib displacements, with the maximum prediction and absolute mean errors of the surrounding rock deformation controlled within 30.3 mm. This provides a guiding path for the precise control of the surrounding rock in the early excavation stage of fault-zone roadways.

Breast Tissue Classification Method Based on Machine Learning

Abstract: Early detection and treatment of breast cancer are very necessary, and effective classification of breast tissue is helpful for the diagnosis of breast cancer; so, a classification method named FT_GA_GBDT is proposed. First, the correlations between the features and classification labels of breast tissue samples were determined, and features with higher correlation were analyzed statistically and combined by weight. Thus, feature transformation (FT) is realized. The datasets were then enhanced by calculating the mean and root mean square of the feature attributes of each adjacent odd- and even-row sample with both belonging to the same class. Finally, the genetic algorithm (GA) was used to search the optimal parameters of the gradient boosting decision tree (GBDT) model, and the optimal parameters were substituted into the GBDT to classify the breast tissue. In addition, the K-nearest-neighbor (KNN), support-vector-machine (SVM) and GBDT methods were also used to test the breast tissue classification. Results of 6-fold cross validation on three breast tissue datasets showed that the average Precision, Recall, and F1 score obtained by the FT_GA_GBDT method were better than those obtained by the KNN, SVM and GBDT methods. The results further show that the FT algorithm and searching for the optimal hyper-parameters by the GA were helpful in improving the performance of the breast tissue classification model, which is more obvious when the correlations between features and classification labels are generally not high.

State of health estimation for lithium-ion batteries based on two-stage features extraction and gradient boosting decision tree

To ensure the stable, reliable, and safe operation of lithium-ion batteries, the state of health (SOH) serves as a crucial basis for battery safety monitoring and operational maintenance. However, the complex working environment of batteries leads to the presence of a significant amount of noise data, abnormal data, and data discontinuities, posing challenges for battery feature extraction and SOH estimation. To address this issue, this paper proposes a two-stage transformation-based feature extraction method combined with the gradient boosting decision tree (GBDT) algorithm. The two-stage transformation feature extraction method utilizes singular spectrum analysis (SSA) and fast Fourier transform (FFT) to extract time-domain and frequency-domain health features from the battery data, respectively. The GBDT ensemble learning algorithm is employed to estimate the SOH of lithium-ion batteries, leveraging the capability to effectively uncover and learn relevant features by combining multiple weak learners into a strong model. Furthermore, correlation analysis and ablation experiments are conducted to validate the effectiveness of the two-stage feature extraction method. Additionally, four sets of comparative experiments are designed to demonstrate the superior performance of the GBDT method compared to four other machine learning methods. The experimental results show that the mean absolute error (MAE) and root mean square error (RMSE) of SOH estimation using the proposed method do not exceed 3 % on the NASA and CALCE datasets.

Assessing the effects of the built environment and microclimate on cycling volume

Cycling benefits human health and helps to mitigate environmental issues. However, limited evidence exists regarding how the built environment influences cycling volume under different time and weather conditions. In this paper, we employed a Gradient Boosting Decision tree method to analyze the non-linear and threshold effects of the multiscale built environment and microclimate on cycling volume. Results based on the multisource data of The Netherlands show that 27 out of the 28 variables have a non-linear and threshold effect on cycling volume. Temperature is found to be a dominant factor among all variables. At street level, slope is the most important factor, followed by the green view and sky view indexes. At neighborhood level, population density is the most important factor, followed by residential density, and the density of bus stops. These findings offer useful insights for planning a cycling-friendly urban built environment at different scales.

Quantifying effect of maize tassels on LAI estimation based on multispectral imagery and machine learning methods

A reliable method to estimate the leaf area index (LAI) in a field environment is crucial for precise monitoring of crop-growth status. Currently, the crop canopy information has been widely used to estimate LAI using remote sensing methods. Many studies regard canopy tassels and leaves as integrated objects, no systematic study has yet investigated how tassels affect the accuracy of LAI estimates. Moreover, the estimation accuracy and generalization ability of the number of selected vegetation indices was seldom evaluated. Therefore, this study used deep learning segmentation methods to quantify how maize tassels affect LAI estimates and to evaluate how the number of variables affects LAI estimates. The results showed that the multispectral dataset segmentation tassels had the highest accuracy when using the VGG-encoded U-Net model (class pixel accuracy, CPA = 89.53 %; mean intersection over union, MIoU = 85.97 %). The segmentation accuracy first was increased and then decreased with tassel growth. By quantifying the contribution of tassels to the vegetation index, tassels most strongly affect the modified nonlinear vegetation index (MNLI) constructed from the canopy spectral information. Moreover, removing the tassels in images could significantly improve the accuracy of LAI estimates using the gradient-boosting decision tree method (GBDT). The estimation method obtained the highest accuracy when using nine vegetation indices to estimate the LAI (R2 = 0.816, RMSE = 0.399, rRMSE = 7.4 %). Overall, the proposed method improves the accuracy of LAI estimates, which provides crucial technical support for monitoring the LAI of maize.

Application and Comparison of Machine Learning Methods for Mud Shale Petrographic Identification

Machine learning is the main technical means for lithofacies logging identification. As the main target of shale oil spatial distribution prediction, mud shale petrography is subjected to the constraints of stratigraphic inhomogeneity and logging information redundancy. Therefore, choosing the most applicable machine learning method for different geological characteristics and data situations is one of the key aspects of high-precision lithofacies identification. However, only a few studies have been conducted on the applicability of machine learning methods for mud shale petrography. This paper aims to identify lithofacies using commonly used machine learning methods. The study employs five supervised learning algorithms, namely Random Forest Algorithm (RF), BP Neural Network Algorithm (BPANN), Gradient Boosting Decision Tree Method (GBDT), Nearest Neighbor Method (KNN), and Vector Machine Method (SVM), as well as four unsupervised learning algorithms, namely K-means, DBSCAN, SOM, and MRGC. The results are evaluated using the confusion matrix, which provides the accuracy of each algorithm. The GBDT algorithm has better accuracy in supervised learning, while the K-means and DBSCAN algorithms have higher accuracy in unsupervised learning. Based on the comparison of different algorithms, it can be concluded that shale lithofacies identification poses challenges due to limited sample data and high overlapping degree of type distribution areas. Therefore, selecting the appropriate algorithm is crucial. Although supervised machine learning algorithms are generally accurate, they are limited by the data volume of lithofacies samples. Future research should focus on how to make the most of limited samples for supervised learning and combine unsupervised learning algorithms to explore lithofacies types of non-coring wells.

Improving the capability of water vapor retrieval from Landsat 8 using ensemble machine learning

Precipitable water vapor (PWV) facilitates the exchange of water and energy between the Earth's surface and the surrounding atmosphere. PWV can be retrieved by employing the water vapor absorption channels in Landsat 8. Unfortunately, the conventional split-window covariance-variance ratio (SWCVR) method is susceptible to various factors, resulting in low retrieval accuracy and availability. Therefore, we improve the traditional SWCVR model and propose a new PWV retrieval method using ensemble machine learning to address these limitations. The new method uses Gradient Boosting Decision Tree (GBDT) to establish the model between brightness temperature, GNSS-derived PWV, and related surface parameters. The results of the test set show that the improved SWCVR model has an RMSE, Bias, and availability of 0.4947 g/cm2, 0.0276 g/cm2, and 29.6%, respectively. By contrast, the GBDT model's corresponding values are 0.2870 g/cm2, −0.0094 g/cm2, and 67.9%, respectively. Compared with SWCVR, the GBDT improves the RMSE and availability by 41.99% and 38.3%, respectively. The GBDT algorithm is significantly better than the SWCVR model in low altitude and complex surface coverage areas. From a temporal perspective, the advantages of the GBDT method are more apparent in the summer. Finally, the SWCVR and GBDT models are externally validated using measured PWV data from the sun photometer and radiosonde, and the RMSE is 0.5148 g/cm2 and 0.3775 g/cm2, respectively. Based on the findings, we know that the accuracy of GBDT has significantly improved against the SWCVR.

Tensile Performance Mechanism for Bamboo Fiber-Reinforced, Palm Oil-Based Resin Bio-Composites Using Finite Element Simulation and Machine Learning.

Plant fiber-reinforced composites have the advantages of environmental friendliness, sustainability, and high specific strength and modulus. They are widely used as low-carbon emission materials in automobiles, construction, and buildings. The prediction of their mechanical performance is critical for material optimal design and application. However, the variation in the physical structure of plant fibers, the randomness of meso-structures, and the multiple material parameters of composites limit the optimal design of the composite mechanical properties. Based on tensile experiments on bamboo fiber-reinforced, palm oil-based resin composites, finite element simulations were carried out and the effect of material parameters on the tensile performances of the composites was investigated. In addition, machine learning methods were used to predict the tensile properties of the composites. The numerical results showed that the resin type, contact interface, fiber volume fraction, and multi-factor coupling significantly influenced the tensile performance of the composites. The results of the machine learning analysis showed that the gradient boosting decision tree method had the best prediction performance for the tensile strength of the composites (R2 was 0.786) based on numerical simulation data from a small sample size. Furthermore, the machine learning analysis demonstrated that the resin performance and fiber volume fraction were critical parameters for the tensile strength of composites. This study provides an insightful understanding and effective route for investigating the tensile performance of complex bio-composites.

What key contextual factors contribute to students’ reading literacy among top-performing countries and economies? Statistical and machine learning analyses

Based on self-regulated learning theory, this study explored the key student-level factors that influence students’ reading literacy among high-performing countries and economies in the Programme for International Student Assessment (PISA) in 2018. Data from the top three best-performing countries and economies in PISA 2018 (i.e., three Asian countries and economies: B-S-J-Z, Singapore, and Macau) was analyzed using a combination of statistical and machine learning methods. First, the Sharpley decomposition method, which can decompose the contribution value of each factor to reading literacy, was used to investigate the impacts of different factors on students’ reading literacy and identify the key factors that were the most influential. Thereafter, the relationships between the identified key factors and reading literacy were further examined using the gradient boosting decision tree method with high generalization ability. Results show that metacognitive strategies (especially assessing credibility) are critical in shaping students’ reading literacy in top-performing countries and economies. The findings also highlight the importance of reading interest in student reading literacy development. Exploring the key factors affecting high-performing students’ reading literacy at the student level may offer valuable insights for informing tailored educational policies and pedagogical practices to improve student reading literacy development.

A Data-Driven Oil Production Prediction Method Based on the Gradient Boosting Decision Tree Regression

Accurate prediction of monthly oil and gas production is essential for oil enterprises to make reasonable production plans, avoid blind investment and realize sustainable development. Traditional oil well production trend prediction methods are based on years of oil field production experience and expertise, and the application conditions are very demanding. With the rapid development of artificial intelligence technology, big data analysis methods are gradually applied in various sub-fields of the oil and gas reservoir development. Based on the data-driven artificial intelligence algorithm Gradient Boosting Decision Tree (GBDT), this paper predicts the initial single-layer production by considering geological data, fluid PVT data and well data. The results show that the GBDT algorithm prediction model has great accuracy, significantly improving efficiency and strong universal applicability. The GBDT method trained in this paper can predict production, which is helpful for well site optimization, perforation layer optimization and engineering parameter optimization and has guiding significance for oilfield development.

TOC prediction using a gradient boosting decision tree method: A case study of shale reservoirs in Qinshui Basin

Shale oil and shale gas are important unconventional resources. As the total organic carbon (TOC) of shales is related to the generation potentiality of hydrocarbons, accurate TOC prediction models can improve development efficiency and reduce cost. However, there has been little work on intelligent prediction of TOC. This study focused on hydrocarbon-rich shale from the Linfen area of Qinshui Basin in China. Multiple regression analysis (MRA), ΔlgR, back propagation (BP) neural network, gradient boosting decision tree (GBDT) were selected to predict the TOC of the studied shale reservoirs using density (DEN), uranium (U), acoustic (AC), and resistivity (RT) logs. The predicted TOC was compared with the geochemical-measured TOC. To fairly evaluate these models, the correlation of determination (R2), root-mean-square-error (RMSE) and three-scale analytic hierarchy process (TAHP) were introduced. The results show that MRA, ΔlgR, and BP methods gave low R2, high RMSE values, and low total weight, indicating these methods poorly predicted TOC. These models significantly underestimate the TOC, resulting in evident deviations between the predicted TOC and their real values. In contrast, GBDT provided accurate TOC prediction, with values of R2, RMSE, and total weight of 0.9254, 0.032 and 0.42. The GBDT model can greatly outperform other models by extracting complex relationships between well log data and TOC values. Sensitivity analysis of the GBDT model revealed that the U parameter exhibited the greatest impact on TOC prediction. The results of this work show that the GBDT model is well suited for TOC prediction of unconventional resources.

Automatic assessment of laparoscopic surgical skill competence based on motion metrics

The purpose of this study was to characterize the motion features of surgical devices associated with laparoscopic surgical competency and build an automatic skill-credential system in porcine cadaver organ simulation training. Participants performed tissue dissection around the aorta, dividing vascular pedicles after applying Hem-o-lok (tissue dissection task) and parenchymal closure of the kidney (suturing task). Movements of surgical devices were tracked by a motion capture (Mocap) system, and Mocap-metrics were compared according to the level of surgical experience (experts: ≥50 laparoscopic surgeries, intermediates: 10-49, novices: 0-9), using the Kruskal-Wallis test and principal component analysis (PCA). Three machine-learning algorithms: support vector machine (SVM), PCA-SVM, and gradient boosting decision tree (GBDT), were utilized for discrimination of the surgical experience level. The accuracy of each model was evaluated by nested and repeated k-fold cross-validation. A total of 32 experts, 18 intermediates, and 20 novices participated in the present study. PCA revealed that efficiency-related metrics (e.g., path length) significantly contributed to PC 1 in both tasks. Regarding PC 2, speed-related metrics (e.g., velocity, acceleration, jerk) of right-hand devices largely contributed to the tissue dissection task, while those of left-hand devices did in the suturing task. Regarding the three-group discrimination, in the tissue dissection task, the GBDT method was superior to the other methods (median accuracy: 68.6%). In the suturing task, SVM and PCA-SVM methods were superior to the GBDT method (57.4 and 58.4%, respectively). Regarding the two-group discrimination (experts vs. intermediates/novices), the GBDT method resulted in a median accuracy of 72.9% in the tissue dissection task, and, in the suturing task, the PCA-SVM method resulted in a median accuracy of 69.2%. Overall, the mocap-based credential system using machine-learning classifiers provides a correct judgment rate of around 70% (two-group discrimination). Together with motion analysis and wet-lab training, simulation training could be a practical method for objectively assessing the surgical competence of trainees.

Composition Engineering on the Local Structure and Viscosity of the CaO-SiO2-Al2O3-P2O5-FeO Slag by Machine Learning Methods

Due to the high cost and low accuracy of high-temperature tests, the viscosity data for multicomponent slag systems is difficult to be obtained precisely. Therefore, it is important to fulfill the viscosity database of the multicomponent slag systems via reasonable methods with lower costs. In this study, a viscosity prediction method based on the machine learning method was proposed for the CaO-SiO2-FeO-Al2O3-P2O5 quinary slag system. To provide valid data for the machine learning model, the viscosity predicted by the molecular dynamic method and multiple semi-empirical models were compared to verify the applicability of these methods to the slag system. Different machine learning models were also developed. The results showed that the prediction results from the gradient boosting decision tree method were the most accurate for the CaO-SiO2-FeO-Al2O3-P2O5 quinary slag system. Based on this method, a color-map concerning the numerical effect of Al2O3 and P2O5 contents and slag viscosity is provided, which also provides assistance for the composition engineering to fulfill a certain demand on the viscosity design.

Sales Forecasting for Fashion Products Considering Lost Sales

Sales forecasting for new products is significantly important for fashion retailer companies because prediction with high accuracy helps the company improve management efficiency and customer satisfaction. The low inventory strategy of fashion products and the low stock level in each brick-and-mortar store lead to serious censored demand problems, making forecasting difficult. In this regard, a two layers (TLs) model is proposed in this paper to predict the total sales of new products. In the first layer, the demand is estimated by linear regression (LR). In the second layer, sales are modeled as a function of not only the demand but also the inventory. To solve the TLs model, a gradient-boosting decision tree method (GBDT) is used for feature selection. Considering the heterogeneity in products, a mixed k-mean algorithm is applied for product clustering and a genetic algorithm for parameter estimation in each cluster. The model is tested on real-world data from a Singapore company, and the experimental results show that our model is better than LR, GBDT, support vector regression (SVR) and artificial neural network (ANN) in most cases. Furthermore, two indicators are built: the average conversion rate and the marginal conversion rate, to measure products’ competitiveness and explore the optimal inventory level, respectively, which provide helpful guidance on decision-making for fashion industry managers.

Examining motivations for owning autonomous vehicles: Implications for land use and transportation

Illustrating the associations between built environment characteristics and autonomous vehicle (AV) ownership helps policymakers understand where AVs emerge first and their impacts on society. However, few studies have addressed interest in AV ownership from the spatial perspective. Using regional travel survey data from the Twin Cities, we applied the gradient boosting decision tree method to test three hypotheses (diffusion of innovation, efficiency, and modal substitution) underlying the relationships between respondents' interest in owning AVs and its correlates. Results showed that the innovation-diffusion hypothesis dominates the motivations for owning AVs, followed by preference for efficiency and then modal substitution. However, its associations with built environment variables suggest more of preference for efficiency than of diffusion of innovation and modal substitution. Population density, road connectivity, and land use entropy are negatively associated with the interest. The results provide suggestions to address the externalities of AVs in different areas.

Revisiting commuting, built environment and happiness: New evidence on a nonlinear relationship

Numerous studies have identified the determinants of happiness. However, the literature provides scant evidence on the potential nonlinear relationships between built environment (BE) and commuting duration and happiness. Using survey data of 7837 respondents from 327 Chinese communities, this study applies a gradient boosting decision tree method to investigate the nonlinear relationships of BE and commuting duration with happiness, and how the nonlinear patterns of commuting duration vary across modes. The results suggest that compared with commuting duration (6.71%) and specific BE characteristics (ranging between 4.77% and 8.86%), household income has the largest relative contribution (14.57%). All BE characteristics show a nonlinear relationship with happiness. The commuting duration shows nonlinear relationships with happiness, and the nonlinear patterns vary across commuting modes. These findings identify effective the range of BE and commuting duration in affecting happiness. Therefore, transport practitioners and urban planners should deliberate on land use and transportation policies to promote happiness.

Increase in rear-end collision risk by acute stress-induced fatigue in on-road truck driving.

Increasing road crashes related to occupational drivers’ deteriorating health has become a social problem. To prevent road crashes, warnings and predictions of increased crash risk based on drivers’ conditions are important. However, in on-road driving, the relationship between drivers’ physiological condition and crash risk remains unclear due to difficulties in the simultaneous measurement of both. This study aimed to elucidate the relationship between drivers’ physiological condition assessed by autonomic nerve function (ANF) and an indicator of rear-end collision risk in on-road driving. Data from 20 male truck drivers (mean ± SD, 49.0±8.2 years; range, 35–63 years) were analyzed. Over a period of approximately three months, drivers’ working behavior data, such as automotive sensor data, and their ANF data were collected during their working shift. Using the gradient boosting decision tree method, a rear-end collision risk index was developed based on the working behavior data, which enabled continuous risk quantification. Using the developed risk index and drivers’ ANF data, effects of their physiological condition on risk were analyzed employing a logistic quantile regression method, which provides wider information on the effects of the explanatory variables, after hierarchical model selection. Our results revealed that in on-road driving, activation of sympathetic nerve activity and inhibition of parasympathetic nerve activity increased each quantile of the rear-end collision risk index. The findings suggest that acute stress-induced drivers’ fatigue increases rear-end collision risk. Hence, in on-road driving, drivers’ physiological condition monitoring and ANF-based stress warning and relief system can contribute to promoting the prevention of rear-end truck collisions.

Online voltage consistency prediction of proton exchange membrane fuel cells using a machine learning method

Widely acknowledged by experts, the inconsistency between the cells of the proton exchange membrane fuel cell stack during operation is an important cause of the fuel cell life decay. Existing studies mainly focus on qualitative analysis of the effects of operating parameters on fuel cell stack consistency. However, there is currently almost no quantitative research on predicting the voltage consistency through operating parameters with machine learning methods. To solve this problem, a three-dimensional model of proton exchange membrane fuel cell stack with five single cells is established in this paper. The Computational Fluid Dynamic (CFD) method is used to provide the source data for prediction model. After predicting the voltage consistency with several machine learning methods and comparing the accuracy through simulation data, the integrated regression method based on Gradient Boosting Decision Tree (GBDT) gets the highest score (0.896) and is proposed for quickly predicting the consistency of cell voltage through operating parameters. After verifying the GBDT method with the experimental data from the fuel cell stack of SUNRISE POWER, in which the accuracy score is 0.910, the universality and accuracy of the method is confirmed. The influencing sensitivity of each operating parameter is evaluated and the current density has the greatest influence on the predicted value, which accounts for 0.40. The prediction of voltage consistency under different combination of operating parameters can guide the optimization of structural parameters in the process of the fuel cell design and operating parameters in the process of fuel cell control.

A novel way to determine transient heat flux based on GBDT machine learning algorithm

The traditional transient heat flux measuring instrument has harsh working conditions and high cost, which is not conducive to promoting the wide range of applications in the engineering manufacturing area. A novel transient heat flux measuring method based on Gradient Boosting Decision Tree (GBDT) algorithm is proposed in this paper. The new measurement method consists of a thermal receptor, a thermocouple, and a data processing device. The system can measure the heat flux absorbed by the thermal receptor through collecting the temperature tendency. This turns the complex problem of heat measurement into a simple problem of temperature measurement, which can greatly reduce the cost of measuring equipment. In this work, numerical simulation was also concerned, and related experiments were designed to verify the feasibility of the method. The GBDT method has demonstrated significant predictive accuracy in both cases. The relative errors of the model trained by the simulation data and the experimental data are 0.8% and 2.9%, respectively. The relative error of GBDT measurement method is less than 5.99% and the response time is 0.3s, which can meet most engineering needs. This work has broad application scenarios and commercial prospects.