Boosted Decision Tree Classifier Research Articles

The quark flavor-violating ALPs in light of B mesons and hadron colliders

The axion-like particle (ALP) may induce flavor-changing neutral currents (FCNCs) when the fermions’ Peccei-Quinn charges are not generation universal. The search for flavor-violating ALP couplings with a bottom quark so far focused on FCNC processes of B mesons at low energies. The recent measurements of B → K + X rare decays place stringent bounds on the quark flavor violations of a light ALP in different decay modes. In this work we propose a novel direct search for bottom flavor-violating interaction of a heavy ALP at the LHC and its upgrades, namely QCD production of an ALP associated with one b jet and one light jet p p → b j a. We consider the decay of the ALP to photons, muons and invisible ALP decays. The Boosted Decision Tree (BDT) algorithm is used to analyze the events and we train the BDT classifier by feeding in the kinematic observables of signal and backgrounds. Finally, we show the complementarity between the search prospects of hadron colliders and the low-energy B meson constraints from B meson mixing and B meson decays to a light ALP.

Anomaly detection in the presence of irrelevant features

Experiments at particle colliders are the primary source of insight into physics at microscopic scales. Searches at these facilities often rely on optimization of analyses targeting specific models of new physics. Increasingly, however, data-driven model-agnostic approaches based on machine learning are also being explored. A major challenge is that such methods can be highly sensitive to the presence of many irrelevant features in the data. This paper presents Boosted Decision Tree (BDT)-based techniques to improve anomaly detection in the presence of many irrelevant features. First, a BDT classifier is shown to be more robust than neural networks for the Classification Without Labels approach to finding resonant excesses assuming independence of resonant and non-resonant observables. Next, a tree-based probability density estimator using copula transformations demonstrates significant stability and improved performance over normalizing flows as irrelevant features are added. The results make a compelling case for further development of tree-based algorithms for more robust resonant anomaly detection in high energy physics.

Suppressing Beam Background and Fake Photons at Belle II using Machine Learning

The Belle II experiment situated at the SuperKEKB energyasymmetric e+ e− collider began operation in 2019. It has since recorded half of the data collected by its predecessor, and reached a world record instantaneous luminosity of 4.7 × 1034 cm-2s-1. For distinguishing decays with missing energy from background events at Belle II, the residual calorimeter energy measured by the electromagnetic calorimeter is an important quantity. Ideally, calorimeter clusters due to beam backgrounds and fake photons should be excluded when the residual calorimeter energy is calculated, so identifying them during the analysis process is key. We present two new boosted decision tree classifiers that have been trained to identify such clusters at Belle II and distinguish them from real photons originating from collision events at the interaction point. We provide results from their application to the B → D*ℓν decay mode, and show that the distribution of residual calorimeter energy for signal events is significantly improved.

Implications of Classification Models for Patients with Chronic Obstructive Pulmonary Disease

Machine learning-based prediction models have the potential to revamp various industries, and one such promising area is healthcare. This study demonstrates the potential impact of machine learning in healthcare, particularly in managing patients with Chronic Obstructive Pulmonary Disease (COPD). The experimental results showcase the remarkable performance of machine learning models, surpassing doctors' predictions for COPD patients. Among the evaluated models, the Gradient Boosted Decision Tree classifier emerges as the top performer, displaying exceptional classification accuracy, precision, recall, and F1-Score compared to doctors' experience. Notably, the comparison between the best machine learning model and doctors' predictions reveals an interesting pattern: machine learning models tend to be more conservative, resulting in an increased probability of patient recovery.

IGRLDTI: An Improved Graph Representation Learning Method for Predicting Drug-Target Interactions over Heterogeneous Biological Information Network.

The task of predicting drug-target interactions (DTIs) plays a significant role in faciliating the development of novel drug discovery. Compared with laboratory-based approaches, computational methods proposed for DTI prediction are preferred due to their high-efficiency and low-cost advantages. Recently, much attention has been attracted to apply different graph neural network (GNN) models to discover underlying DTIs from hetergeneous biological information network (HBIN). Although GNN-based prediction methods achieve better performance, they are prone to encounter the over-smoothing simulation when learning the latent representations of drugs and targets with their rich neighborhood information in HBIN, and thereby reduce the discriminative ability in DTI prediction. In this work, an improved graph representation learning method, namely iGRLDTI, is proposed to address the above issue by better capturing more discriminative representations of drugs and targets in a latent feature space. Specifically, iGRLDTI first constructs a HBIN by integrating the biological knowledge of drugs and targets with their interactions. After that, it adopts a node-dependent local smoothing strategy to adaptively decide the propagation depth of each biomolecule in HBIN, thus significantly alleviating over-smoothing by enhancing the discriminative ability of feature represeantions of drugs and targets. Finally, a Gradient Boosting Decision Tree classifier is used by iGRLDTI to predict novel DTIs. Experimental results demonstrate that iGRLDTI yields better performance that several state-of-the-art computational methods on the benchmark dataset. Besides, our case study indicates that iGRLDTI can successfully identify novel DTIs with more distinguishable features of drugs and targets. Python codes and dataset are available at https://github.com/stevejobws/iGRLDTI/. Supplementary data are available at Bioinformatics online.

A feature extraction method based on noise reduction for circRNA-miRNA interaction prediction combining multi-structure features in the association networks.

A large number of studies have shown that circular RNA (circRNA) affects biological processes by competitively binding miRNA, providing a new perspective for the diagnosis, and treatment of human diseases. Therefore, exploring the potential circRNA-miRNA interactions (CMIs) is an important and urgent task at present. Although some computational methods have been tried, their performance is limited by the incompleteness of feature extraction in sparse networks and the low computational efficiency of lengthy data. In this paper, we proposed JSNDCMI, which combines the multi-structure feature extraction framework and Denoising Autoencoder (DAE) to meet the challenge of CMI prediction in sparse networks. In detail, JSNDCMI integrates functional similarity and local topological structure similarity in the CMI network through the multi-structure feature extraction framework, then forces the neural network to learn the robust representation of features through DAE and finally uses the Gradient Boosting Decision Tree classifier to predict the potential CMIs. JSNDCMI produces the best performance in the 5-fold cross-validation of all data sets. In the case study, seven of the top 10 CMIs with the highest score were verified in PubMed. The data and source code can be found at https://github.com/1axin/JSNDCMI.

Assessing the effects of therapeutic combinations on SARS-CoV-2 infected patient outcomes: A big data approach.

The COVID-19 pandemic has demonstrated the need for efficient and comprehensive, simultaneous assessment of multiple combined novel therapies for viral infection across the range of illness severity. Randomized Controlled Trials (RCT) are the gold standard by which efficacy of therapeutic agents is demonstrated. However, they rarely are designed to assess treatment combinations across all relevant subgroups. A big data approach to analyzing real-world impacts of therapies may confirm or supplement RCT evidence to further assess effectiveness of therapeutic options for rapidly evolving diseases such as COVID-19. Gradient Boosted Decision Tree, Deep and Convolutional Neural Network classifiers were implemented and trained on the National COVID Cohort Collaborative (N3C) data repository to predict the patients' outcome of death or discharge. Models leveraged the patients' characteristics, the severity of COVID-19 at diagnosis, and the calculated proportion of days on different treatment combinations after diagnosis as features to predict the outcome. Then, the most accurate model is utilized by eXplainable Artificial Intelligence (XAI) algorithms to provide insights about the learned treatment combination impacts on the model's final outcome prediction. Gradient Boosted Decision Tree classifiers present the highest prediction accuracy in identifying patient outcomes with area under the receiver operator characteristic curve of 0.90 and accuracy of 0.81 for the outcomes of death or sufficient improvement to be discharged. The resulting model predicts the treatment combinations of anticoagulants and steroids are associated with the highest probability of improvement, followed by combined anticoagulants and targeted antivirals. In contrast, monotherapies of single drugs, including use of anticoagulants without steroid or antivirals are associated with poorer outcomes. This machine learning model by accurately predicting the mortality provides insights about the treatment combinations associated with clinical improvement in COVID-19 patients. Analysis of the model's components suggests benefit to treatment with combination of steroids, antivirals, and anticoagulant medication. The approach also provides a framework for simultaneously evaluating multiple real-world therapeutic combinations in future research studies.

The Identification of Elderly People with High Fall Risk Using Machine Learning Algorithms.

Falling is an important public health issue, and predicting the fall risk can reduce the incidence of injury events in the elderly. However, most of the existing studies may have additional human and financial costs for community workers and doctors. Therefore, it is socially important to identify elderly people who are at high fall risk through a reasonable and cost-effective method. We evaluated the potential of multifractal, machine learning algorithms to identify the elderly at high fall risk. We developed a 42-point calibration model of the human body and recorded the three-dimensional coordinate datasets. The stability of the motion trajectory is calculated by the multifractal algorithm and used as an input dimension to compare the performance of the six classifiers. The results showed that the instability of the faller group was significantly greater than that of the no-faller group in the male and female cohorts (p < 0.005), and the Gradient Boosting Decision Tree classifier showed the best performance. The findings could help elderly people at high fall risk to identify individualized risk factors and facilitate tailored fall interventions.

Clinical decision support system for early detection of Alzheimer's disease using an enhanced gradient boosted decision tree classifier.

Alzheimer's disease (AD) is one of the most common forms of dementia contributing to more than 70% of the cases. The factors accounting for the cause and progression of neurodegenerative diseases like AD are primarily genetic, in addition to life style and environmental factors. Early and accurate diagnoses of AD empower practitioners to take timely clinical decisions and preventive actions. This being the motivation, the work proposes a novel pattern matching and scoring method on genetic material towards devising an effective classifier. We propose a distinctive disease causing gene sequence pattern identification using suffix trees as a base detection model with an accuracy of 91.5% in linear time complexity. A scoring mechanism is implemented to assign scores to genes based on the severity of the disease causing and disease resistant Single Nucleotide Polymorphisms associated with the genes. These scores are then used as a remarkable feature in the gradient boosted decision tree classifier to enhance the classification of AD versus healthy control. The efficiency of the proposed gene powered EGBDT classifier is evaluated on ADNI benchmark data set with the prediction accuracy of 94.16% and is found to be efficient compared to the recent works in the literature.

Measurement of the atmospheric νe and νμ energy spectra with the ANTARES neutrino telescope

This letter presents a combined measurement of the energy spectra of atmospheric νe and νμ in the energy range between ∼100 GeV and ∼50 TeV with the ANTARES neutrino telescope. The analysis uses 3012 days of detector livetime in the period 2007–2017, and selects 1016 neutrinos interacting in (or close to) the instrumented volume of the detector, yielding shower-like events (mainly from νe+ν‾e charged current plus all neutrino neutral current interactions) and starting track events (mainly from νμ+ν‾μ charged current interactions). The contamination by atmospheric muons in the final sample is suppressed at the level of a few per mill by different steps in the selection analysis, including a Boosted Decision Tree classifier. The distribution of reconstructed events is unfolded in terms of electron and muon neutrino fluxes. The derived energy spectra are compared with previous measurements that, above 100 GeV, are limited to experiments in polar ice and, for νμ, to Super-Kamiokande.

A Framework for AI-Assisted Detection of Patent Ductus Arteriosus from Neonatal Phonocardiogram

The current diagnosis of Congenital Heart Disease (CHD) in neonates relies on echocardiography. Its limited availability requires alternative screening procedures to prioritise newborns awaiting ultrasound. The routine screening for CHD is performed using a multidimensional clinical examination including (but not limited to) auscultation and pulse oximetry. While auscultation might be subjective with some heart abnormalities not always audible it increases the ability to detect heart defects. This work aims at developing an objective clinical decision support tool based on machine learning (ML) to facilitate differentiation of sounds with signatures of Patent Ductus Arteriosus (PDA)/CHDs, in clinical settings. The heart sounds are pre-processed and segmented, followed by feature extraction. The features are fed into a boosted decision tree classifier to estimate the probability of PDA or CHDs. Several mechanisms to combine information from different auscultation points, as well as consecutive sound cycles, are presented. The system is evaluated on a large clinical dataset of heart sounds from 265 term and late-preterm newborns recorded within the first six days of life. The developed system reaches an area under the curve (AUC) of 78% at detecting CHD and 77% at detecting PDA. The obtained results for PDA detection compare favourably with the level of accuracy achieved by an experienced neonatologist when assessed on the same cohort.

Privacy-Preserving Boosting in the Local Setting

In machine learning, boosting is one of the most popular methods that is designed to combine multiple base learners into a superior one. The well-known Boosted Decision Tree classifier has been widely adopted in data mining and pattern recognition. With the emerging challenge in privacy, the personal images, browsing history, and financial reports, which are held by individuals and entities are more likely to contain sensitive information. The privacy concern is intensified when the data leaves the hand of owners and is used for further mining. Such privacy issues demand that the machine learning algorithms should be privacy-aware. Recently, Local Differential Privacy has been proposed as an effective privacy protection approach, which allows data owners to perturb the data before any release. In this paper, we propose a distributed privacy-preserving boosting algorithm that can be applied to various types of classifiers. By adopting LDP as a building block, the proposed boosting algorithm leverages the aggregation of the perturbed data shares to build the base learner, which ensures that privacy is well preserved for the participated data owners. Our experiments demonstrate that the proposed algorithm effectively boosts various classifiers and the boosted classifiers maintain a high utility.

Probing exotic charged Higgs decays in the Type-II 2HDM through top rich signal at a future 100 TeV pp collider

The exotic decay modes of non-Standard Model Higgs bosons are efficient in probing the hierarchical Two Higgs Doublet Models (2HDM). In particular, the decay mode H±→HW± serves as a powerful channel in searching for charged Higgses. In this paper, we analyze the reach for H±→HW±→ toverline{t}W at a 100 TeV pp collider, and show that it extends the reach of the previously studied ττW final states once above the top threshold. Top tagging technique is used, in combination with a boosted decision tree classifier. At the low tan β region, almost the entire hierarchical Type-II 2HDM parameter space can be probed via the combination of all exotic decay channels.

A gradient boosted decision tree-based sentiment classification of twitter data

People communicate their views, arguments and emotions about their everyday life on social media (SM) platforms (e.g. Twitter and Facebook). Twitter stands as an international micro-blogging service that features a brief message called tweets. Freestyle writing, incorrect grammar, typographical errors and abbreviations are some noises that occur in the text. Sentiment analysis (SA) centered on a tweet posted by the user, and also opinion mining (OM) of the customers review is another famous research topic. The texts are gathered from users’ tweets by means of OM and automatic-SA centered on ternary classifications, namely positive, neutral and negative. It is very challenging for the researchers to ascertain sentiments as a result of its limited size, misspells, unstructured nature, abbreviations and slangs for Twitter data. This paper, with the aid of the Gradient Boosted Decision Tree classifier (GBDT), proposes an efficient SA and Sentiment Classification (SC) of Twitter data. Initially, the twitter data undergoes pre-processing. Next, the pre-processed data is processed using HDFS MapReduce. Now, the features are extracted from the processed data, and then efficient features are selected using the Improved Elephant Herd Optimization (I-EHO) technique. Now, score values are calculated for each of those chosen features and given to the classifier. At last, the GBDT classifier classifies the data as negative, positive, or neutral. Experiential results are analyzed and contrasted with the other conventional techniques to show the highest performance of the proposed method.

SpineCloud: image analytics for predictive modeling of spine surgery outcomes

Purpose: Data-intensive modeling could provide insight on the broad variability in outcomes in spine surgery. Previous studies were limited to analysis of demographic and clinical characteristics. We report an analytic framework called "SpineCloud" that incorporates quantitative features extracted from perioperative images to predict spine surgery outcome. Approach: A retrospective study was conducted in which patient demographics, imaging, and outcome data were collected. Image features were automatically computed from perioperative CT. Postoperative 3- and 12-month functional and pain outcomes were analyzed in terms of improvement relative to the preoperative state. A boosted decision tree classifier was trained to predict outcome using demographic and image features as predictor variables. Predictions were computed based on SpineCloud and conventional demographic models, and features associated with poor outcome were identified from weighting terms evident in the boosted tree. Results: Neither approach was predictive of 3- or 12-month outcomes based on preoperative data alone in the current, preliminary study. However, SpineCloud predictions incorporating image features obtained during and immediately following surgery (i.e., intraoperative and immediate postoperative images) exhibited significant improvement in area under the receiver operating characteristic (AUC): ( to 0.83) at 3 months and ( to 0.82) at 12 months. Conclusions: Predictive modeling of lumbar spine surgery outcomes was improved by incorporation of image-based features compared to analysis based on conventional demographic data. The SpineCloud framework could improve understanding of factors underlying outcome variability and warrants further investigation and validation in a larger patient cohort.

Avocado: Photometric Classification of Astronomical Transients with Gaussian Process Augmentation

Abstract Upcoming astronomical surveys such as the Large Synoptic Survey Telescope (LSST) will rely on photometric classification to identify the majority of the transients and variables that they discover. We present a set of techniques for photometric classification that can be applied even when the training set of spectroscopically confirmed objects is heavily biased toward bright, low-redshift objects. Using Gaussian process regression to model arbitrary light curves in all bands simultaneously, we “augment” the training set by generating new versions of the original light curves covering a range of redshifts and observing conditions. We train a boosted decision tree classifier on features extracted from the augmented light curves, and we show how such a classifier can be designed to produce classifications that are independent of the redshift distributions of objects in the training sample. Our classification algorithm was the best-performing among the 1094 models considered in the blinded phase of the Photometric LSST Astronomical Time-Series Classification Challenge, scoring 0.468 on the organizers’ logarithmic-loss metric with flat weights for all object classes in the training set, and achieving an AUC of 0.957 for classification of SNe Ia. Our results suggest that spectroscopic campaigns used for training photometric classifiers should focus on typing large numbers of well-observed, intermediate-redshift transients, instead of attempting to type a sample of transients that is directly representative of the full data set being classified. All of the algorithms described in this paper are implemented in the avocado software package (https://www.github.com/kboone/avocado).

A Boosted Decision Tree Model for Predicting Loan Default in P2P Lending Communities

Loan Default Prediction For Social Lending Is An Emerging Area Of Research In Predictive Analytics. The Need For Large Amount Of Data And Few Available Studies In The Current Loan Default Prediction Models For Social Lending Suggest That Other Viable And Easily Implementable Models Should Be Investigated And Developed. In View Of This, This Study Developed A Data Mining Model For Predicting Loan Default Among Social Lending Patrons, Specifically The Small Business Owners, Using Boosted Decision Tree Model. The United States Small Business Administration (Usba) Publicly-Available Loan Administration Dataset Of 27 Features And 899164 Data Instances Was Used In 80:20 Ratios For The Training And Testing Of The Model. 16 Data Features Were Finally Used As Predictors After Data Cleaning And Feature Engineering. The Gradient Boosting Decision Tree Classifier Recorded 99% Accuracy Compared To The Basic Decision Tree Classifier Of 98%. The Model Is Further Evaluated With (A) Receiver Operating Characteristics (Roc) And Area Under Curve (Auc), (B) Cumulative Accuracy Profile (Cap), And (C) Cumulative Accuracy Profile (Cap) Under Auc. Each Of These Model Performance Evaluation Metrics, Especially Roc-Auc, Showed The Relationship Between The True Positives And False Positives That Implies The Model Is A Good Fit.

Molecular Subtypes Recognition of Breast Cancer in Dynamic Contrast-Enhanced Breast Magnetic Resonance Imaging Phenotypes from Radiomics Data.

Background and Objective Breast cancer is a major cause of mortality among women if not treated in early stages. Recognizing molecular markers from DCE-MRI directly to distinguish the four molecular subtypes without invasive biopsy is helpful for guiding treatment plans for breast cancer, which provides a fast way to consequential treatment plan decision in early time and best opportunity for patients. Methods This study presents an approach of molecular subtypes recognition from breast cancer image phenotypes by radiomics. An improved region growth algorithm with dynamic threshold without user interaction is proposed for cancer lesion segmentation, which gives the precise border of lesion other than area with background. The lesions are extracted automatically based on radiologists' annotation which guarantees the lesion is segmented correctly. Various features are extracted on lesions data including texture, morphology, dynamic kinetics, and statistics features carried out on a large patient cohort, which are used to validate the relationship between image phenotypes and the molecular subtypes. A new algorithm of multimodel-based recursive feature elimination is applied on the radiomics data generated by the feature extraction process. This method obtains the feature subset with stable performance for different classification models, and the gradient boosting decision tree model gets the best results of both classification performance and imbalance performance on molecular subtypes. Result From the experimental results, 69 optimal features from 143 original features are found by the multimodel-based recursive feature elimination algorithms and the gradient boosting decision tree classifier obtains a good performance with accuracy 0.87, precise 0.88, recall 0.87, and F1-score 0.87. The dataset with 637 patients in this paper has serious imbalance problem on different molecular subtypes, and the the robust features that are generated by multimodel-based recursive feature eliminiation algorithm make the gradient boosting decision tree classifier have good behaviors. The recognition precision for the four molecular subtypes of luminal A, luminal B, HER-2, and basal-like are 0.91, 0.89, 0.83, and 0.87, respectively. Conclusions The improved lesion segmentation method gives more precise lesion edge, which not only saves the time of automatic extraction of lesion region of interest without threshold setting for each case, but also prevents the segmentation error by manual and prejudice from different radiologists. The feature selection algorithm of multimodel-based recursive feature elimination has the ability to find robust and optimal features that distinguish the four molecular subtypes from image phenotypes. The gradient boosting decision tree classifier rather plays a main role in recognition than other models used in this paper.

Heart Rate Variability during Periods of Low Blood Pressure as a Predictor of Short-Term Outcome in Preterms.

Efficient management of low blood pressure (BP) in preterm neonates remains challenging with a considerable variability in clinical practice. The ability to assess preterm wellbeing during episodes of low BP will help to decide when and whether hypotension treatment should be initiated. This work aims to investigate the relationship between heart rate variability (HRV), BP and the short-term neurological outcome in preterm infants less than 32 weeks gestational age (GA). The predictive power of common HRV features with respect to the outcome is assessed and shown to improve when HRV is observed during episodes of low mean arterial pressure (MAP) - with a single best feature leading to an AUC of 0.87. Combining multiple features with a boosted decision tree classifier achieves an AUC of 0.97. The work presents a promising step towards the use of multimodal data in building an objective decision support tool for clinical prediction of short-term outcome in preterms who suffer episodes of low BP.

A rejection inference technique based on contrastive pessimistic likelihood estimation for P2P lending

Abstract The majority of current credit-scoring models are built solely on accepted samples and thus cause sample bias. Sample bias is particularly severe in the peer-to-peer (P2P) lending domain due to its comparatively high rejection rate. Reject inference solves sample bias by inferring the possible outcomes of rejected samples and incorporating them into credit score modeling. This study addresses the problem of reject inference in a specific P2P lending domain from the perspective of semi-supervised learning. A novel reject inference method (CPLE-LightGBM) is proposed by combining the contrastive pessimistic likelihood estimation framework and an advanced gradient boosting decision tree classifier (LightGBM). The performance of the proposed CPLE-LightGBM method is validated on multiple datasets, and results demonstrate the efficiency of our proposal. Analysis of the influence of rejection rate on predictive accuracy reveals the usefulness of sampling in rejected datasets.