Tree-based Machine Learning Techniques Research Articles

Designing a Network Intrusion Detection System Based on Machine Learning for Software Defined Networks

Software-defined Networking (SDN) has recently developed and been put forward as a promising and encouraging solution for future internet architecture. Managed, the centralized and controlled network has become more flexible and visible using SDN. On the other hand, these advantages bring us a more vulnerable environment and dangerous threats, causing network breakdowns, systems paralysis, online banking frauds and robberies. These issues have a significantly destructive impact on organizations, companies or even economies. Accuracy, high performance and real-time systems are essential to achieve this goal successfully. Extending intelligent machine learning algorithms in a network intrusion detection system (NIDS) through a software-defined network (SDN) has attracted considerable attention in the last decade. Big data availability, the diversity of data analysis techniques, and the massive improvement in the machine learning algorithms enable the building of an effective, reliable and dependable system for detecting different types of attacks that frequently target networks. This study demonstrates the use of machine learning algorithms for traffic monitoring to detect malicious behavior in the network as part of NIDS in the SDN controller. Different classical and advanced tree-based machine learning techniques, Decision Tree, Random Forest and XGBoost are chosen to demonstrate attack detection. The NSL-KDD dataset is used for training and testing the proposed methods; it is considered a benchmarking dataset for several state-of-the-art approaches in NIDS. Several advanced preprocessing techniques are performed on the dataset in order to extract the best form of the data, which produces outstanding results compared to other systems. Using just five out of 41 features of NSL-KDD, a multi-class classification task is conducted by detecting whether there is an attack and classifying the type of attack (DDoS, PROBE, R2L, and U2R), accomplishing an accuracy of 95.95%.

Predicting the price of Vietnamese shrimp products exported to the US market using machine learning

Accurately predicting the price of exported fishery products is an important task for fisheries because it will enable market trends to be determined, leading to the development of high-quality fishery products. In this study, we predicted prices in selected base periods (2, 3, 6, and 12 months) to investigate how historical data influenced the Vietnamese export price. A dataset (from May 1995 to May 2019) was collected from the US Department of Agriculture (USDA). We initially hypothesized that the dependent variable, Vietnamese export price, was affected by 33 independent variables, but ultimately used 15 key variables, which were chosen on the basis of Akaike information criterion (AIC) to train the models. A tree-based machine learning technique, including the random forest and gradient boosting tree algorithms, was applied for predictions. It was found that the random forest algorithm performed well for historical data for periods of more than 6 months, while the gradient boosting tree algorithm was better over short durations of less than 6 months.

Development of a Clinical and Genetic Prediction Model for Early Intestinal Resection in Patients with Crohn’s Disease: Results from the IMPACT Study

Early intestinal resection in patients with Crohn’s disease (CD) is necessary due to a severe and complicating disease course. Herein, we aim to predict which patients with CD need early intestinal resection within 3 years of diagnosis, according to a tree-based machine learning technique. The single-nucleotide polymorphism (SNP) genotype data for 337 CD patients recruited from 15 hospitals were typed using the Korea Biobank Array. For external validation, an additional 126 CD patients were genotyped. The predictive model was trained using the 102 candidate SNPs and seven sets of clinical information (age, sex, cigarette smoking, disease location, disease behavior, upper gastrointestinal involvement, and perianal disease) by employing a tree-based machine learning method (CatBoost). The importance of each feature was measured using the Shapley Additive Explanations (SHAP) model. The final model comprised two clinical parameters (age and disease behavior) and four SNPs (rs28785174, rs60532570, rs13056955, and rs7660164). The combined clinical–genetic model predicted early surgery more accurately than a clinical-only model in both internal (area under the receiver operating characteristic (AUROC), 0.878 vs. 0.782; n = 51; p < 0.001) and external validation (AUROC, 0.836 vs. 0.805; n = 126; p < 0.001). Identification of genetic polymorphisms and clinical features enhanced the prediction of early intestinal resection in patients with CD.

Identifying and understanding determinants of high healthcare costs for breast cancer: a quantile regression machine learning approach

BackgroundTo identify and rank the importance of key determinants of high medical expenses among breast cancer patients and to understand the underlying effects of these determinants.MethodsThe Oncology Care Model (OCM) developed by the Center for Medicare & Medicaid Innovation were used. The OCM data provided to Mount Sinai on 2938 breast-cancer episodes included both baseline periods and three performance periods between Jan 1, 2012 and Jan 1, 2018. We included 11 variables representing information on treatment, demography and socio-economics status, in addition to episode expenditures. OCM data were collected from participating practices and payers. We applied a principled variable selection algorithm using a flexible tree-based machine learning technique, Quantile Regression Forests.ResultsWe found that the use of chemotherapy drugs (versus hormonal therapy) and interval of days without chemotherapy predominantly affected medical expenses among high-cost breast cancer patients. The second-tier major determinants were comorbidities and age. Receipt of surgery or radiation, geographically adjusted relative cost and insurance type were also identified as important high-cost drivers. These factors had disproportionally larger effects upon the high-cost patients.ConclusionsData-driven machine learning methods provide insights into the underlying web of factors driving up the costs for breast cancer care management. Results from our study may help inform population health management initiatives and allow policymakers to develop tailored interventions to meet the needs of those high-cost patients and to avoid waste of scarce resource.

Predictors of death and heart transplantation in biopsy-proven myocarditis: a machine-learning approach

Abstract Background Risk stratification for death and heart transplantation (HTx) in myocarditis is complex. A random forest (RF) is a tree-based machine learning technique (MLT) which is being increasingly used for clinical data analysis; it allows the detection of complex relationships between the outcome of interest and the covariates, overcoming the limits of traditional statistical analysis (i.e. regression approaches). Purpose To assess the potential role of clinical and diagnostic features at presentation as predictors of death and HTx in biopsy (Bx)-proven myocarditis using RF. Methods From January 1993 to August 2019, we consecutively enrolled 357 patients with Bx-proven myocarditis (65% male, median age 39 years, interquartile range (IQR) 26–51). An RF approach for survival data was used. Variables included in the analysis were: histology type by Bx, NYHA, type of presentation (infarct-like, arrhythmia, heart failure), viral genome detection on Bx, serum antiheart (AHA), antiintercalated disk (AIDA), anticardiac endothelial cells (AECA), antinuclear (ANA) autoantibodies, immunosuppressive therapy, cardiac catheterisation (left ventricular enddiastolic volume (LVEDV), mean capillary wedge pressure, right and left ventricular enddiastolic pressure) and 2-D echocardiographic measures (LVEDV, left ventricular ejection fraction (LVEF) at presentation and at follow-up, right ventricular fractional area change (FAC%), right ventricular diastolic area). Results The median follow-up time was of 1352 days (IQR 423.25–2535.75). At the end of follow-up, 42 patients were dead or transplanted. The 1-year, 5-year, and 10-year survival probabilities were of 0.928, 0.854, and 0.817, respectively. The most relevant predictors of death or HTx identified by the RF algorithm (according to the variable importance measure) were histological type, NYHA, clinical presentation, LVEF, and FAC%. Among the circulating auto-antibodies AECA were found to be the most important. Histological type was the strongest predictor of death/HT (100% relative importance, (RI)), giant cell myocarditis having a lower survival probability compared to other types. The next stronger predictors were advanced (III-IV) NYHA and heart failure presentation with lower survival probabilities (90% and 84% RI respectively). AECA-positive patients had lower survival probability compared to AECA negative ones (20% RI). The RF algorithm revealed an excellent predictive performance in the correct identification of all alive patients, with only 5 dead patients being misclassified (balanced accuracy 94%). Conclusions Autoimmune features, i.e Giant cell myocarditis and AECA, as well as severity of heart failure and of left ventricular disfunction at presentation were the strongest predictors of dismal prognosis. Our RF approach provides a new automated powerful tool for accurate risk stratification for death/HTx in Bx-proven myocarditis. Funding Acknowledgement Type of funding source: Public Institution(s). Main funding source(s): Budget Integrato per la Ricerca dei Dipartimenti (BIRD, year 2019), Padova University, Padova, Italy (project Title: Myocarditis: genetic background, predictors of dismal prognosis and of response to immunosuppressive therapy.)

Predicting the Vulnerability of Women to Intimate Partner Violence in South Africa: Evidence from Tree-based Machine Learning Techniques.

Intimate partner violence (IPV) is a pervasive social challenge with severe health and demographic consequences. Global statistics indicate that more than a third of women have experienced IPV at some point in their lives. In South Africa, IPV is considered a significant contributor to the country's broader problem with violence and a leading cause of femicide. Consequently, IPV has been the major focus of legislation and research across different disciplines. The present article aims to contribute to the growing scholarly literature by predicting factors that are associated with the risk of experiencing IPV. We used the 2016 South African Demographic and Health Survey dataset and restricted our analysis to 1,816 ever-married women who had complete information on the variables that were used to generate IPV. Prior research has mainly used regression analysis to identify correlates of IPV; however, while regression analysis can test a priori specified effects, it cannot capture unspecified inter-relationship across factors. To address this limitation, we opted for machine learning methods, which identify hidden and complex patterns and relationships in the data. Our results indicate that the fear of the husband is the most critical factor in determining the experience of IPV. In other words, the risk of IPV in South Africa is associated more with the husband or partner's characteristics than the woman's. The models developed in this study can be used to develop interventions by different stakeholders such as social workers, policymakers, and or other interested partners.

Designing predictive maintenance systems using decision tree-based machine learning techniques

PurposeThe after-sale service industry is estimated to contribute over 8 percent to the US GDP. For use in this considerably large service management industry, this article provides verification in the application of decision tree-based machine learning algorithms for optimal maintenance decision-making. The motivation for this research arose from discussions held with a large agricultural equipment manufacturing company interested in increasing the uptime of their expensive machinery and in helping their dealer network.Design/methodology/approachWe propose a general strategy for the design of predictive maintenance systems using machine learning techniques. Then, we present a case study where multiple machine learning algorithms are applied to a particular example situation for an illustration of the proposed strategy and evaluation of its performance.FindingsWe found progressive improvements using such machine learning techniques in terms of accuracy in predictions of failure, demonstrating that the proposed strategy is successful.Research limitations/implicationsThis approach is scalable to a wide variety of applications to aid in failure prediction. These approaches are generalizable to many systems irrespective of the underlying physics. Even though we focus on decision tree-based machine learning techniques in this study, the general design strategy proposed can be used with all other supervised learning techniques like neural networks, boosting algorithms, support vector machines, and statistical methods.Practical implicationsThis approach is applicable to many different types of systems that require maintenance and repair decision-making. A case is provided for a cloud data storage provider. The methods described in the case can be used in any number of systems and industrial applications, making this a very scalable case for industry practitioners. This scalability is possible as the machine learning techniques learn the correspondence between machine conditions and outcome state irrespective of the underlying physics governing the systems.Social implicationsSustainable systems and operations require allocating and utilizing resources efficiently and effectively. This approach can help asset managers decide how to sustainably allocate resources by increasing uptime and utilization for expensive equipment.Originality/valueThis is a novel application and case study for decision tree-based machine learning that will aid researchers in developing tools and techniques in this area as well as those working in the artificial intelligence and service management space.

Gradient Boosting for Quantitative Finance

In this paper, we discuss how tree-based machine learning techniques can be used in the context of derivatives pricing. Gradient boosted regression trees are employed to learn the pricing map for a couple of classical, time-consuming problems in quantitative finance. In particular, we illustrate this methodology by reducing computation times for pricing exotic derivatives products and American options. Once the gradient boosting model is trained, it is used to make fast predictions of new prices. We show that this approach leads to speed-ups of several orders of magnitude, while the loss of accuracy is very acceptable from a practical point of view. In addition to the predictive performance of these methods, we acknowledge the importance of interpretability of pricing models. For both applications, we therefore look under the hood of the gradient boosting model and elaborate on how the price is constructed and interpreted.

Advanced wind speed prediction using convective weather variables through machine learning application

High precision and reliable wind speed forecasting is a challenge for meteorologists. We used multiple nonparametric tree-based machine learning techniques, for predicting the maximum wind speed at 10 m using selected convective weather variables. Analysis is based on 127 convective storms from 2005 to 2013. The study evaluated two error models - the Bayesian Additive Regression Trees (BART) and the Quantile Regression Forests (QRF) - and compares them in terms of point estimates and prediction intervals. The error model performances were evaluated based on different error metrics evaluating both the bias and random error of point estimates and the prediction intervals using ensemble verification statistics. The study showed that error modeling based on QRF is superior to BART, especially in terms of point estimate and prediction interval results. Wind speed prediction through QRF was successfully verified using systematic and random error metrics, and ensemble verification statistics of the corresponding prediction intervals. The model generated realizations of wind speed that successfully encapsulated the reference wind speed and notably reduced systematic and random error. The predicted wind speed from QRF can potentially support emergency preparedness efforts associated with severe weather impacts.

Electricity Price Forecasting with Dynamic Trees: A Benchmark Against the Random Forest Approach

Dynamic Trees are a tree-based machine learning technique specially designed for online environments where data are to be analyzed sequentially as they arrive. Our purpose is to test this methodology for the very first time for Electricity Price Forecasting (EPF) by using data from the Iberian market. For benchmarking the results, we will compare them against another tree-based technique, Random Forest, a widely used method that has proven its good results in many fields. The benchmark includes several versions of the Dynamic Trees approach for a very short term EPF (one-hour ahead) and also a short term (one-day ahead) approach but only with the best versions. The numerical results show that Dynamic Trees are an adequate method, both for very short and short term EPF—even improving upon the performance of the Random Forest method. The comparison with other studies for the Iberian market suggests that Dynamic Trees is a proper and promising method for EPF.

Novel Risk Assessment Tool for Immunoglobulin Resistance in Kawasaki Disease: Application Using a Random Forest Classifier.

Resistance to intravenous immunoglobulin (IVIG) therapy is a risk factor for coronary lesions in patients with Kawasaki disease (KD). Risk-adjusted initial therapy may improve coronary outcome in KD, but identification of high risk patients remains a challenge. This study aimed to develop a new risk assessment tool for IVIG resistance using advanced statistical techniques. Data were retrospectively collected from KD patients receiving IVIG therapy, including demographic characteristics, signs and symptoms of KD and laboratory results. A random forest (RF) classifier, a tree-based machine learning technique, was applied to these data. The correlation between each variable and risk of IVIG resistance was estimated. Data were obtained from 767 patients with KD, including 170 (22.1%) who were refractory to initial IVIG therapy. The predictive tool based on the RF algorithm had an area under the receiver operating characteristic curve of 0.916, a sensitivity of 79.7% and a specificity of 87.3%. Its misclassification rate in the general patient population was estimated to be 15.5%. RF also identified markers related to IVIG resistance such as abnormal liver markers and percentage neutrophils, displaying relationships between these markers and predicted risk. The RF classifier reliably identified KD patients at high risk for IVIG resistance, presenting clinical markers relevant to treatment failure. Evaluation in other patient populations is required to determine whether this risk assessment tool relying on RF has clinical value.