Boosting Decision Tree Algorithm Research Articles

Effective Breast Cancer Recognition Based on Fine-Grained Feature Selection

Early detection and diagnosis of breast cancer are crucial to improve the survival rates of patients. Hence, pathologists and radiologists need a computer-aided diagnosis system to assist their clinical diagnoses effectively and efficiently. However, most breast cancer recognition models are faced with the sample scarcity problem, which results in serious overfitting and lowers recognition performance. To alleviate the sample scarcity problem, a simple, effective model called “refinement, correlation, adaptive” (RCA) for breast cancer recognition is proposed from the perspective of fine-grained feature selection. An innovative multiview efficient range-based gene selection algorithm is proposed to complete the first-layer feature “refinement,” which contributes to suppressing the noisy information in the original feature space. Then, more-discriminant but low-dimensional information among heterogeneous features is mined through the second-layer cross-modal “correlation” mining. Feature dimensions are reduced to a reasonable value that fits the sample size well and alleviates the overfitting problem. Finally, the last-layer decision-tree-guided “adaptive” feature selection is completed using the gradient boosting decision tree algorithm. The RCA model was validated on two well-known datasets. The experimental results demonstrate that the proposed RCA model can address the sample scarcity problem well. It outperforms state-of-the-art baselines, especially in terms of accuracy and the area of the Kiviat diagram. The largest performance improvements of the metrics are 2.39% and 1121, respectively. Moreover, an online diagnosis system based on the RCA model is proposed. It provides rapid and effective breast cancer recognition, which should make clinical diagnoses more convenient and narrow the gap between theoretical research and practical application.

Comparison of supervised machine learning classification techniques in prediction of locoregional recurrences in early oral tongue cancer

BackgroundThe proper estimate of the risk of recurrences in early-stage oral tongue squamous cell carcinoma (OTSCC) is mandatory for individual treatment-decision making. However, this remains a challenge even for experienced multidisciplinary centers. ObjectivesWe compared the performance of four machine learning (ML) algorithms for predicting the risk of locoregional recurrences in patients with OTSCC. These algorithms were Support Vector Machine (SVM), Naive Bayes (NB), Boosted Decision Tree (BDT), and Decision Forest (DF). Materials and methodsThe study cohort comprised 311 cases from the five University Hospitals in Finland and A.C. Camargo Cancer Center, São Paulo, Brazil. For comparison of the algorithms, we used the harmonic mean of precision and recall called F1 score, specificity, and accuracy values. These algorithms and their corresponding permutation feature importance (PFI) with the input parameters were externally tested on 59 new cases. Furthermore, we compared the performance of the algorithm that showed the highest prediction accuracy with the prognostic significance of depth of invasion (DOI). ResultsThe results showed that the average specificity of all the algorithms was 71% . The SVM showed an accuracy of 68% and F1 score of 0.63, NB an accuracy of 70% and F1 score of 0.64, BDT an accuracy of 81% and F1 score of 0.78, and DF an accuracy of 78% and F1 score of 0.70. Additionally, these algorithms outperformed the DOI-based approach, which gave an accuracy of 63%. With PFI-analysis, there was no significant difference in the overall accuracies of three of the algorithms; PFI-BDT accuracy increased to 83.1%, PFI-DF increased to 80%, PFI-SVM decreased to 64.4%, while PFI-NB accuracy increased significantly to 81.4%. ConclusionsOur findings show that the best classification accuracy was achieved with the boosted decision tree algorithm. Additionally, these algorithms outperformed the DOI-based approach. Furthermore, with few parameters identified in the PFI analysis, ML technique still showed the ability to predict locoregional recurrence. The application of boosted decision tree machine learning algorithm can stratify OTSCC patients and thus aid in their individual treatment planning.

Improving risk prediction in heart failure using machine learning.

Predicting mortality is important in patients with heart failure (HF). However, current strategies for predicting risk are only modestly successful, likely because they are derived from statistical analysis methods that fail to capture prognostic information in large data sets containing multi-dimensional interactions. We used a machine learning algorithm to capture correlations between patient characteristics and mortality. A model was built by training a boosted decision tree algorithm to relate a subset of the patient data with a very high or very low mortality risk in a cohort of 5822 hospitalized and ambulatory patients with HF. From this model we derived a risk score that accurately discriminated between low and high-risk of death by identifying eight variables (diastolic blood pressure, creatinine, blood urea nitrogen, haemoglobin, white blood cell count, platelets, albumin, and red blood cell distribution width). This risk score had an area under the curve (AUC) of 0.88 and was predictive across the full spectrum of risk. External validation in two separate HF populations gave AUCs of 0.84 and 0.81, which were superior to those obtained with two available risk scores in these same populations. Using machine learning and readily available variables, we generated and validated a mortality risk score in patients with HF that was more accurate than other risk scores to which it was compared. These results support the use of this machine learning approach for the evaluation of patients with HF and in other settings where predicting risk has been challenging.

Schedulable capacity forecasting for electric vehicles based on big data analysis

Fast and accurate forecasting of schedulable capacity of electric vehicles (EVs) plays an important role in enabling the integration of EVs into future smart grids as distributed energy storage systems. Traditional methods are insufficient to deal with large-scale actual schedulable capacity data. This paper proposes forecasting models for schedulable capacity of EVs through the parallel gradient boosting decision tree algorithm and big data analysis for multi-time scales. The time scale of these data analysis comprises the real time of one minute, ultra-short-term of one hour and one-day-ahead scale of 24 hours. The predicted results for different time scales can be used for various ancillary services. The proposed algorithm is validated using operation data of 521 EVs in the field. The results show that compared with other machine learning methods such as the parallel random forest algorithm and parallel k-nearest neighbor algorithm, the proposed algorithm requires less training time with better forecasting accuracy and analytical processing ability in big data environment.

Hybrid model based refrigerant charge fault estimation for the data centre air conditioning system

Accurate refrigerant charge fault estimation is important to ensure the efficient operation of air conditioning systems. This paper presents a novel hybrid model based refrigerant charge estimation approach. Firstly, an improved gray box model is presented, which integrates the key characteristic variables of subcooling temperature, superheat temperature, quality and pressure drop. Secondly, three extra variables having highest maximal information coefficients with the prediction residual are used to extend the gray box, and the robust machine learning model is developed using the gradient boosting decision tree algorithm. Then, a hybrid model is presented by combining the improved gray box and machine learning models. Finally, the prediction and generalization capacities of the proposed models under various operation conditions are validated using the experimental data. The results show that the hybrid charge fault estimation model has the best performance. Its overall prediction and generalization MREs are 2.53% and 3.09%, respectively.

Development and validation of a novel prediction model to identify patients in need of specialized trauma care during field triage: design and rationale of the GOAT study

BackgroundAdequate field triage of trauma patients is crucial to transport patients to the right hospital. Mistriage and subsequent interhospital transfers should be minimized to reduce avoidable mortality, life-long disabilities, and costs. Availability of a prehospital triage tool may help to identify patients in need of specialized trauma care and to determine the optimal transportation destination.MethodsThe GOAT (Gradient Boosted Trauma Triage) study is a prospective, multi-site, cross-sectional diagnostic study. Patients transported by at least five ground Emergency Medical Services to any receiving hospital within the Netherlands are eligible for inclusion. The reference standards for the need of specialized trauma care are an Injury Severity Score ≥ 16 and early critical resource use, which will both be assessed by trauma registrars after the final diagnosis is made. Variable selection will be based on ease of use in practice and clinical expertise. A gradient boosting decision tree algorithm will be used to develop the prediction model. Model accuracy will be assessed in terms of discrimination (c-statistic) and calibration (intercept, slope, and plot) on individual participant’s data from each participating cluster (i.e., Emergency Medical Service) through internal-external cross-validation. A reference model will be externally validated on each cluster as well. The resulting model statistics will be investigated, compared, and summarized through an individual participant’s data meta-analysis.DiscussionThe GOAT study protocol describes the development of a new prediction model for identifying patients in need of specialized trauma care. The aim is to attain acceptable undertriage rates and to minimize mortality rates and life-long disabilities.

Pushing Constraints by Rule-Driven Pruning Techniques in Non-Uniform Minimum Support for Predicting Obstructive Sleep Apnea

Boosted Association-Ruled Pruned Decision Tree (ARP-DT), the improved version of the Boosted Decision Tree algorithm, was developed by using association-ruled pre-and post-pruning techniques with referring to the pushed minimum support and minimum confidence constraints as well as the association rules applied. The novelty of the Association-Ruled pruning techniques applied mainly embark on the pre-pruning techniques through researching on the maximum number of decision tree splitting, as well as the post-pruning techniques involving subtree replacement and subtree raising. The applied association rules (ARs) augment the mining of frequent itemset (s) or interesting itemset (s) such that appropriate pre-pruning or subtree pruning techniques can be applied before AdaBoost ensemble is implemented. The ARs applied involve the Adaptive Apriori (AA) augmented rule definitions and theorem as stated in this research focuses on the characteristics of the datasets accessed so as to streamline the rule-driven pruning techniques on the Boosting algorithms developed for predicting Obstructive Sleep Apnea (OSA). There is a significant improvement in the prediction accuracies when comparing the classical boosting algorithms and Boosted ARP-DT being applied to the OSA datasets and those online databases from University of California Irvine (UCI) data repositories.

Detecting Preeclampsia Using Data From a Pregnancy Mobile App [30M

INTRODUCTION: The objective was to create a predictive model for preeclampsia detection when app users do not report diagnosis or are not yet aware of their diagnosis. METHODS: A supervised machine learning algorithm, specifically, a Gradient Boosting Decision Tree algorithm, incorporating data from 77,621 users of a mobile app was used to create a model for the purpose of detecting the presence of preeclampsia. RESULTS: Of the total sample used to train the model, 41% had preeclampsia based on patient report of a diagnosis made by their clinician. The model used 385 different features to model the likelihood of currently having preeclampsia, even if disease status was not yet known or if diagnosis had not yet been reported in-app. Self-report data, as well as in-app calculations based on reported data, were used to create the model. The frequency of reported 'daily data' such as, health symptoms (e.g., fatigue and exhaustion, and blood pressure readings), moods, nutrition, and exercise activities were most influential to the model. Personal demographic information including age and BMI (height and weight) also contributed to the mode. The model's positive predictive value was 0.75, with 0.37 sensitivity. With a positive predictive value of 0.65, sensitivity increased to 0.79. CONCLUSION: A model used to identify the presence of preeclampsia was most influenced by the frequency of data logging for specific features. This model could prove highly informative for patients, especially those without access to regular healthcare, as the presence of a serious condition could be identified, prompting referral to a healthcare provider.

A Machine Learning Method for Predicting Driving Range of Battery Electric Vehicles

It is of great significance to improve the driving range prediction accuracy to provide battery electric vehicle users with reliable information. A model built by the conventional multiple linear regression method is feasible to predict the driving range, but the residual errors between -3.6975 km and 3.3865 km are relatively unfaithful for real-world driving. The study is innovative in its application of machine learning method, the gradient boosting decision tree algorithm, on the driving range prediction which includes a very large number of factors that cannot be considered by conventional regression methods. The result of the machine learning method shows that the maximum prediction error is 1.58 km, the minimum prediction error is -1.41 km, and the average prediction error is about 0.7 km. The predictive accuracy of the gradient boosting decision tree is compared against that of the conventional approaches.

An interpretable machine learning model of biological age

Background: Assessments of biological (rather than chronological) age derived from patient biochemical data have been shown to strongly predict both all-cause and disease-specific mortality. However, these population-based approaches have yet to be translated to the individual. As well as using biological age as a research tool, by being able to better answer the question “why did we get this result?”, clinicians may be able to apply personalised interventions that could improve the long-term health of individual patients. Methods: Here, the boosted decision tree algorithm XGBoost was used to predict biological age using 39 commonly-available blood test results from the US National Health and Nutrition Examination Survey (NHANES) database. Results: Interrogation of the algorithm produced a description of how each marker contributed to the final output in a single individual. Additive explanation plots were then used to determine biomarker ranges associated with a lower biological age. Importantly, a number of markers that are modifiable with lifestyle changes were found to have a significant effect on biological age, including fasting blood glucose, lipids, and markers of red blood cell production. Conclusions: The combination of individualised outputs with target ranges could provide the ability to personalise interventions or recommendations based on an individual’s biochemistry and resulting predicted age. This would allow for the investigation of interventions designed to improve health and longevity in a targeted manner, many of which could be rooted in targeted lifestyle modifications.

Health Data Driven on Continuous Blood Pressure Prediction Based on Gradient Boosting Decision Tree Algorithm

Diseases related to issues with blood pressure are becoming a major threat to human health. With the development of telemedicine monitoring applications, a growing number of corresponding devices are being marketed, such as the use of remote monitoring for the purposes of increasing the autonomy of the elderly and thus encouraging a healthier and longer health span. Using machine learning algorithms to measure blood pressure at a continuous rate is a feasible way to provide models and analysis for telemedicine monitoring data and predicting blood pressure. For this paper, we applied the gradient boosting decision tree (GBDT) while predicting blood pressure rates based on the human physiological data collected by the EIMO device. EIMO equipment-specific signal acquisition includes ECG and PPG. In order to avoid over-fitting, the optimal parameters are selected via the cross-validation method. Consequently, our method has displayed a higher accuracy rate and better performance in calculating the mean absolute error evaluation index than methods, such as the traditional least squares method, ridge regression, lasso regression, ElasticNet, SVR, and KNN algorithm. When predicting the blood pressure of a single individual, calculating the systolic pressure displays an accuracy rate of above 70% and above 64% for calculating the diastolic pressure with GBDT, with the prediction time being less than 0.1 s. In conclusion, applying the GBDT is the best method for predicting the blood pressure of multiple individuals: with the inclusion of data such as age, body fat, ratio, and height, algorithm accuracy improves, which in turn indicates that the inclusion of new features aids prediction performance.

A Novel Reject Inference Model Using Outlier Detection and Gradient Boosting Technique in Peer-to-Peer Lending

Credit scoring is an efficient tool in handling the information asymmetry of peer-to-peer (P2P) lending. Credit scoring models are typically built only with the accepted applicants, which may cause sample bias and further hinder the predictive performances. Reject inference methods utilize the information contained in the rejected samples by inferring their potential status and incorporate them with the accepted samples. In this study, we propose a novel reject inference model (i.e., OD-LightGBM) that combines an outlier detection technique (i.e., isolation forest) and a state-of-the-art gradient boosting decision tree algorithm. The model is evaluated on two real-world P2P lending datasets, and the results of predictive performances demonstrate that our proposed model significantly outperforms the benchmarks in terms of discriminative capability. The analysis of computational cost shows the great potential of our proposed model in handling large-sized problems. The proposed framework remains robust under different parameter settings and provides stable results given various combinations of outlier detection algorithms and classifiers.

Resonant di-Higgs production at gravitational wave benchmarks: a collider study using machine learning

We perform a complementarity study of gravitational waves and colliders in the context of electroweak phase transitions choosing as our template the xSM model, which consists of the Standard Model augmented by a real scalar. We carefully analyze the gravitational wave signal at benchmark points compatible with a first order phase transition, taking into account subtle issues pertaining to the bubble wall velocity and the hydrodynamics of the plasma. In particular, we comment on the tension between requiring bubble wall velocities small enough to produce a net baryon number through the sphaleron process, and large enough to obtain appreciable gravitational wave production. For the most promising benchmark models, we study resonant di-Higgs production at the high-luminosity LHC using machine learning tools: a Gaussian process algorithm to jointly search for optimum cut thresholds and tuning hyperparameters, and a boosted decision trees algorithm to discriminate signal and background. The multivariate analysis on the collider side is able either to discover or provide strong statistical evidence of the benchmark points, opening the possibility for complementary searches for electroweak phase transitions in collider and gravitational wave experiments.

Gradient boosting trees method in the task of software identification

Subject of Research.The paper proposes an approach to the use of gradient boosted decision trees algorithm. For this purpose, CatBoost algorithm developed by Yandex is proposed. Its implementation is aimed at the problem solution of OS Linux software identification in order to reduce the number of system vulnerabilities, which occur due to the installation of unauthorized software by automated system users. We consider an approach to the program signatures formation and further training of CatBoostClassifier classifier model. The subsequent recognition task is set for the identified programs that were not previously involved in the model training process. Method. Free CatBoost software was used for implementation of the gradient boosted decision trees algorithm. CatBoostClassifier multi-classification model was created on its basis. The use of this model allows identifying test sample elf-files.Main Results. The training parameters of the classification model are selected. An experiment is carried out to identify elf-files with the use of ten different featuresof emerging signature programs. The results obtained in the new approach are compared with the results of the previously developed method of identification based on the application of the statistical criterion of Chi-square homogeneity at the significance level p = 0.01. Practical Relevance. The results of the study can be recommended to information security specialists for data media audit. The developed approach gives the possibility to identify violations of the established security policy in the processing of confidential information.

The use of adversaries for optimal neural network training

B-decay data from the Belle experiment at the KEKB collider have a substantial background from e+e−→qq̄ events. To suppress this we employ deep neural network algorithms. These provide improved signal from background discrimination. However, the deep neural network develops a substantial correlation with the ΔE kinematic variable used to distinguish signal from background in the final fit due to its relationship with input variables. The effect of this correlation is reduced by deploying an adversarial neural network. Overall the adversarial deep neural network performs better than a Boosted Decision Tree algorithm and a commercial package, NeuroBayes, which employs a neural net with a single hidden layer.

Combined analysis of double Higgs production via gluon fusion at the HL-LHC in the effective field theory approach

We perform the combined analysis of the double Higgs production via gluon fusion in the $b\bar{b} \gamma\gamma$ and $b\bar{b}\tau^+\tau^-$ decay channels at the High-Luminosity LHC (HL-LHC). To validate our analysis, we reproduce the ATLAS result of the $b\bar{b} \gamma\gamma$ process including all contributions from fakes. For the $b\bar{b}\tau^+\tau^-$ decay channel, we perform the similar analysis to the CMS one. As an improvement, we also perform the multivariate analysis employing the boosted decision tree algorithm. Then, we derive 68% probability contours on anomalous Higgs couplings in the effective field theory (EFT) approach for various analyses. We find that the $b\bar{b}\tau^+\tau^-$ process outperforms the $b\bar{b}\gamma\gamma$ for the measurement of energy-growing operators, while adding the $b\bar{b}\tau^+\tau^-$ process is least beneficial for improving the precision of the Higgs self-coupling (mainly set by the $b\bar{b}\gamma\gamma$ process). We illustrate that the double Higgs production alone can be comparable to the single Higgs process in constraining the modification of the top Yukawa coupling in the positive direction. Focusing on the Higgs self-coupling as a special interest, we derive the precision as a function of various improvable parameters such as tag and mistag rates of tau leptons, heavy flavor jets, photon identification, diphoton mass resolution, and jet energy resolution to take into account future phenomenological studies. As an optimistic benchmark scenario, we illustrate that the 68% and 95% probability intervals of the Higgs self-coupling, $\lambda_3/\lambda_{3}^{SM}$, at the HL-LHC can reach $[0.2,\, 2.3]$ and $[-0.1,\, 3.5] \cup [4.0,\, 6.5]$, respectively, where the correlation among the EFT coefficients is taken into account.

Adaptive neural network based on segmented particle swarm optimization for remote-sensing estimations of vegetation biomass

In this study, the segmented particle swarm optimization (SPSO) algorithm and the concepts of the gradient boosting decision tree algorithm (GBDT) were combined to propose the SPSO adaptive neural network (SANN) method. The purpose of this method is to address the inadequacies of the traditional basis function (BP) and radial basis function (RBF) neural networks when solving problems that involve local optima and overfitting. Experimental results indicated that, overall, the SANN method is accurate in remote-sensing estimations of aquatic vegetation biomass. However, accuracies of estimations were unsatisfactory for certain indicators and sessions when data was taken. The estimations were analyzed using three sets of indicators: (i) root mean square error, average relative error, and total relative error; (ii) correlation coefficient and coefficient of determination, and their scatter plots; and (iii) relative error values and their distributions. The results clearly showed that the SANN method was superior to the BP neural network as well as the stepwise multiple linear regression analysis (SR). However, when the relative errors in biomass estimations by the other two methods were low, the advantages of the SANN method were less pronounced. This was particularly true when the relative errors were <30%, in which case SANN was only marginally better than the other two methods.

Improved Boosted Decision Tree Algorithms by Adaptive Apriori and Post-Pruning for Predicting Obstructive Sleep Apnea

The improved version of Boosted Decision Tree algorithm, named as Boosted Adaptive Apriori post-Pruned Decision Tree (Boosted AApoP-DT), was developed by referring to Adaptive Apriori (AA) properties and by using post-pruning technique. The post-pruning technique used is mainly the error-complexity pruning for the decision trees categorized under Classification and Regression Trees. This technique estimates the re-substitution, cross-validation and generalization error rates before and after the post-pruning. The novelty of the post-pruning technique applied is that it is augmented by AA properties and these depend on the data characteristics in the dataset(s) being accessed. This algorithm is then boosted by using AdaBoost ensemble method. After comparing and contrasting this developed algorithm with the algorithm without being augmented by AA, i.e., Boosted post-Pruned Decision Tree (Boosted poP-DT), and the classical boosted decision tree algorithm, i.e., Boosted DT, there is a stepwise improvement shown when comparison proceeds from Boosted DT to Boosted poP-DT and to Boosted AApoP-DT.

Machine learning approach for student engagement automatic recognition from facial expressions

Digital revolution can significantly improve the quality of education. There have been already discussions for a long time about the advantages, disadvantages and opportunities for transforming traditional classroom activities. Modern students use smartphones and tablets 'from birth', and for the majority of academic subject areas students can often obtain more complete, accurate and up-to-date information from the Internet than from lectures. Is it interesting for students to learn? Are they in time with the professor? Is the presentation clear? How deep are students engaged in learning in the classroom? These issues come to the forefront in the era of digital education. However, it was almost unrealistic to control the level of student engagement until recently: for example, only in the Moscow campuses of the Financial University the classes are held daily from 8.30 to 22.00 in more than 500 classrooms. Existing information systems for student engagement automatic recognition are focused on analyzing individual engagement of students and schoolchildren. We propose a system that constantly analyzes the flow of data from video cameras installed in classrooms, uses machine learning models to identify students' faces, recognize their emotions and determine the level of engagement, and then aggregates engagement data on student groups, faculties, courses, etc. on interactive dashboards. The training dataset consisted of 2,000 faces was used for machine learning model identification with boosted decision trees algorithm (ADABoost). The quality metrics (Accuracy, Precision, Recall, AUC) on a test dataset of 500 students faces were all above 0,81. The system is developed as an elastically scalable cloud service that automatically collects video streams from cameras installed in classrooms and forms the resulting metrics of the students and groups' engagement in the Microsoft Azure cloud.

Predicting Suitable Areas for Growing Cassava Using Remote Sensing and Machine Learning Techniques: A Study in Nakhon-Phanom Thailand

Aim/Purpose: Although cassava is one of the crops that can be grown during the dry season in Northeastern Thailand, most farmers in the region do not know whether the crop can grow in their specific areas because the available agriculture planning guideline provides only a generic list of dry-season crops that can be grown in the whole region. The purpose of this research is to develop a predictive model that can be used to predict suitable areas for growing cassava in Northeastern Thailand during the dry season. Background: This paper develops a decision support system that can be used by farmers to assist them determine if cassava can be successfully grown in their specific areas. Methodology: This study uses satellite imagery and data on land characteristics to develop a machine learning model for predicting suitable areas for growing cassava in Thailand’s Nakhon-Phanom province. Contribution: This research contributes to the body of knowledge by developing a novel model for predicting suitable areas for growing cassava. Findings: This study identified elevation and Ferric Acrisols (Af) soil as the two most important features for predicting the best-suited areas for growing cassava in Nakhon-Phanom province, Thailand. The two-class boosted decision tree algorithm performs best when compared with other algorithms. The model achieved an accuracy of .886, and .746 F1-score. Recommendations for Practitioners: Farmers and agricultural extension agents will use the decision support system developed in this study to identify specific areas that are suitable for growing cassava in Nakhon-Phanom province, Thailand Recommendation for Researchers: To improve the predictive accuracy of the model developed in this study, more land and crop characteristics data should be incorporated during model development. The ground truth data for areas growing cassava should also be collected for a longer period to provide a more accurate sample of the areas that are suitable for cassava growing. Impact on Society: The use of machine learning for the development of new farming systems will enable farmers to produce more food throughout the year to feed the world’s growing population. Future Research: Further studies should be carried out to map other suitable areas for growing dry-season crops and to develop decision support systems for those crops.