Decision Tree Learning Algorithm Research Articles

Using a decision tree learning algorithm to classify injury status in adolescent males following return-to-activity criteria

Using a decision tree learning algorithm to classify injury status in adolescent males following return-to-activity criteria

Application of machine learning algorithms for code smell prediction using object-oriented software metrics

Code smells are generally not considered as bugs; instead, they point out certain shortcomings in the software design or code. Identification of code smell is a necessary step for improving the software quality and reducing the maintenance effort. In this study, we introduce a bad smell prediction technique based on object-oriented software metrics that use Decision Tree (DT) and Random Forest (RF) machine learning algorithm. An open-source project, namely JHOTDRAW, was used as our dataset, for which values of object-oriented software metrics were calculated. Two feature selection methods-Random Forest Importance (RFI) and Information Gain (IG) were applied to extract the most relevant attributes for the prediction of code smells, namely, Feature envy, Dispersed coupling, refused parent bequest, and God class. The random-search algorithm was used to tune the parameters of Random Forest and Decision Tree. Results show that the best classification accuracy for Decision Tree was obtained at 99.13% for refused parent bequest code smells. Results also show that after using the Random Forest classifier, refused parent bequest smell was predicted with the best accuracy of 99.14%. Finally, in this research study, a set of code smell prediction rules were extracted using Decision Tree.

Measuring soil coverage using image feature descriptors and the decision tree learning algorithm

The quantification and identification of ground cover plays a key role in erosion modelling, weed measurement, plant disease identification and other environmental applications. Currently, a variety of methods are used to mechanically classify digital images for ground cover. Only a few of these methods can distinguish green vegetation, straw/dormant vegetation, and exposed soil using only the Red-Green-Blue (RGB) spectrum. This research presents an approach to classifying ground cover using standard JPEG images and readily available Matlab (2018b) functions. The approach uses block segmentation, as opposed to pixel-wise or object-based segmentation, and compares multiple machine learning approaches with varying pixel block size and classification acceptance thresholds. The most successful classification approach found through this study was the decision tree algorithm with a 70-pixel block size and 60% classification acceptance threshold. Images were reduced to three feature descriptors: colour, texture, and oriented gradients to represent the respective RGB spectrum for an image. Both the training set and test set images used in this research came from field and greenhouse studies done between 2016 and 2019. The produced classifications were compared to manual coverage classifications using Samplepoint, a grid-based method, with R-squared values of 0.86 for green vegetation, 0.87 for straw/dormant vegetation, and 0.96 for exposed soil, respectively. This method showed strong performance for images containing exposed soil and either green vegetation or straw/dormant vegetation. The method was less effective for images with large quantities of both green vegetation and straw/dormant vegetation likely due to their similar shape.

Treant: training evasion-aware decision trees

Despite its success and popularity, machine learning is now recognized as vulnerable to evasion attacks, i.e., carefully crafted perturbations of test inputs designed to force prediction errors. In this paper we focus on evasion attacks against decision tree ensembles, which are among the most successful predictive models for dealing with non-perceptual problems. Even though they are powerful and interpretable, decision tree ensembles have received only limited attention by the security and machine learning communities so far, leading to a sub-optimal state of the art for adversarial learning techniques. We thus propose Treant, a novel decision tree learning algorithm that, on the basis of a formal threat model, minimizes an evasion-aware loss function at each step of the tree construction. Treant is based on two key technical ingredients: robust splitting and attack invariance, which jointly guarantee the soundness of the learning process. Experimental results on publicly available datasets show that Treant is able to generate decision tree ensembles that are at the same time accurate and nearly insensitive to evasion attacks, outperforming state-of-the-art adversarial learning techniques.

Remote Sensing of Boreal Wetlands 2: Methods for Evaluating Boreal Wetland Ecosystem State and Drivers of Change

The following review is the second part of a two part series on the use of remotely sensed data for quantifying wetland extent and inferring or measuring condition for monitoring drivers of change on wetland environments. In the first part, we introduce policy makers and non-users of remotely sensed data with an effective feasibility guide on how data can be used. In the current review, we explore the more technical aspects of remotely sensed data processing and analysis using case studies within the literature. Here we describe: (a) current technologies used for wetland assessment and monitoring; (b) the latest algorithmic developments for wetland assessment; (c) new technologies; and (d) a framework for wetland sampling in support of remotely sensed data collection. Results illustrate that high or fine spatial resolution pixels (≤10 m) are critical for identifying wetland boundaries and extent, and wetland class, form and type, but are not required for all wetland sizes. Average accuracies can be up to 11% better (on average) than medium resolution (11–30 m) data pixels when compared with field validation. Wetland size is also a critical factor such that large wetlands may be almost as accurately classified using medium-resolution data (average = 76% accuracy, stdev = 21%). Decision-tree and machine learning algorithms provide the most accurate wetland classification methods currently available, however, these also require sampling of all permutations of variability. Hydroperiod accuracy, which is dependent on instantaneous water extent for single time period datasets does not vary greatly with pixel resolution when compared with field data (average = 87%, 86%) for high and medium resolution pixels, respectively. The results of this review provide users with a guideline for optimal use of remotely sensed data and suggested field methods for boreal and global wetland studies.

Predicting Prostephanus truncatus (Horn) (Coleoptera: Bostrichidae) populations and associated grain damage in smallholder farmers’ maize stores: A machine learning approach

Prostephanus truncatus is a notorious pest of stored-maize grain and its spread throughout sub-Saharan Africa has led to increased levels of grain storage losses. The current study developed models to predict the level of P. truncatus infestation and associated damage of maize grain in smallholder farmer stores. Data were gathered from grain storage trials conducted in Hwedza and Mbire districts of Zimbabwe and correlated with weather data for each site. Insect counts of P. truncatus and other common stored grain insect pests had a strong correlation with time of year with highest recorded numbers from January to May. Correlation analysis showed insect-generated grain dust from boring and feeding activity to be the best indicator of P. truncatus presence in stores (r = 0.70), while a moderate correlation (r = 0.48) was found between P. truncatus numbers and storage insect parasitic wasps, and grain damage levels significantly correlated with the presence of Tribolium castaneum (r = 0.60). Models were developed for predicting P. truncatus infestation and grain damage using parameter selection algorithms and decision-tree machine learning algorithms with 10-fold cross-validation. The P. truncatus population size prediction model performance was weak (r = 0.43) due to the complicated sampling and detection of the pest and eight-week long period between sampling events. The grain damage prediction model had a stronger correlation coefficient (r = 0.93) and is a good estimator for in situ stored grain insect damage. The models were developed for use under southern African climatic conditions and can be improved with more input data to create more precise models for building decision-support tools for smallholder maize-based production systems.

Decision tree-based diagnosis of coronary artery disease: CART model

Background and ObjectiveAs the most common cardiovascular defect, coronary artery disease (CAD), also called ischemic heart disease, is one of the substantial causes of death globally. Several diagnosis approaches such as baseline electrocardiography, echocardiography, magnetic resonance imaging, and coronary angiography are suggested for screening the suspected patients that may suffer from CAD. However, applying such methods may have health side effects and/or expensive costs. MethodsAs an alternative to the available diagnosis tools/methods, this research involves a decision tree learning algorithm called classification and regression tree (CART) for a simple and reliable diagnosis of CAD. Several CART models are developed based on the recently CAD dataset published in the literature. ResultsUtilizing all the features of the dataset (55 independent parameters), it was found that only 40 independent parameters influence the CAD diagnosis and consequently development of the predictive model. Based on the feature importance obtained from the first CART model, three new CART models are then developed using 18, 10, and 5 selected features. Except for the five-feature CART model, the outcomes of developed CART models demonstrate the maximum achievable accuracy, sensitivity, and specificity for CAD diagnosis (100%), while comparing the predictions with the reported targets. The error analysis reveals that the literature models including sequential minimal optimization (SMO), bagging SMO, Naïve Bayes (NB), artificial neural network (ANN), C4.5, J48, Bagging, and ANN in conjunction with the genetic algorithm (GA) do not outperform the CART methodology in classifying patients as normal or CAD. ConclusionsHence, the robustness of the tree-based algorithm in accurate and fast predictions is confirmed, implying the proposed classification technique can be successfully utilized to develop a coherent decision-making system for the CAD diagnosis.

Credit card fraud detection using naive Bayesian and C4.5 decision tree classifiers

Growing problem of card payment fraudulent abuse is a main focus of banks and payment Service Providers (PSPs). This study is using naive Bayes, C4.5 decision tree and bagging ensemble machine learning algorithms to predict outcome of regular and fraud transactions. Performance of algorithms is evaluated through: precision, recall, PRC area rates. Performance of machine learning algorithms PRC rates between 0,999 and 1,000 expressing that these algorithms are quite good in distinguishing binary class 0 in our data set. Amongst all algorithms best performing PRC class 1 rate has Bagging with C4.5 decision tree as base learner with rate of 0,825. For prediction of fraud transactions with success of 92,74% correctly predicted with C4.5 decision tree algorithm.

PENERAPAN METODE ITERATIVE DICHOTOMIZER 3 (ID 3) UNTUK MENENTUKAN BEASISWA BERPRESTASI PADA SMP PGRI CARINGIN SUKABUMI

Scholarships are assistance from the government to students / students who are less able or have the ability in the academic and non-academic fields that are given individually to reduce the burden in terms of material. Frequently stalling time in selection, the number of students who apply for scholarships, the number of students whose homes are far from school, the number of students who race to come early as one of the criteria eligible to receive scholarships as well as the most scholarship applicants feel disadvantaged by unfavorable decisions. Iterative Dichotomizer 3 (ID3) algorithm is the most basic decision tree learning algorithm (decision tree learning algorithm). This algorithm conducts a thorough search on all possible decisions. In this research, it will be analyzed the application of the iterative dichotomizer 3 method in the case of determining achievement scholarships. In order to make decisions quickly and accurately. From 708 scholarship candidates including 28 eligible and 680 scholarship recipients, 136 scholarship recipients were obtained from ID3 algorithm with 3 eligible and 133 who had not, and obtained an accuracy rate of 97.75% so that it could be concluded that good and can help the school.

Attribute Selection Based on Constraint Gain and Depth Optimal for a Decision Tree.

Uncertainty evaluation based on statistical probabilistic information entropy is a commonly used mechanism for a heuristic method construction of decision tree learning. The entropy kernel potentially links its deviation and decision tree classification performance. This paper presents a decision tree learning algorithm based on constrained gain and depth induction optimization. Firstly, the calculation and analysis of single- and multi-value event uncertainty distributions of information entropy is followed by an enhanced property of single-value event entropy kernel and multi-value event entropy peaks as well as a reciprocal relationship between peak location and the number of possible events. Secondly, this study proposed an estimated method for information entropy whose entropy kernel is replaced with a peak-shift sine function to establish a decision tree learning (CGDT) algorithm on the basis of constraint gain. Finally, by combining branch convergence and fan-out indices under an inductive depth of a decision tree, we built a constraint gained and depth inductive improved decision tree (CGDIDT) learning algorithm. Results show the benefits of the CGDT and CGDIDT algorithms.

Detection of critical safety events on freeways in clear and rainy weather using SHRP2 naturalistic driving data: Parametric and non-parametric techniques

A better understanding of what happened before and during a crash event could lead to development of countermeasures that reduce crash risk on freeways. This paper focused on detecting near-crashes on freeways by comparing environmental conditions and vehicle kinematics signatures of near-crash events to normal driving using data collected from the Second Strategic Highway Research Program (SHRP2) – Naturalist Driving Study. The study considered near-crash events as surrogate measures of crash risk only in rainy and clear weather conditions due to a limited number of near-crashes in other weather conditions. A time chunking technique was used with different aggregation levels to monitor changes in vehicle kinematics on a timescale. Both parametric and non-parametric techniques were used to detect near-crashes on freeways. The Binary Logistic Regression model was used as a parametric detection model, while Decision Tree , K-Nearest Neighbors (K-NN), and Deep Learning Artificial Neural Network were used as non-parametric detection models. The results showed that the logistic regression model provided a good fit of the input data and can detect near-crashes with outstanding discrimination. However, the time zone of interest identified by non-parametric models led to higher accurancy than that identified by the logistic regression model. In addition, Decision Tree and Deep Learning ANN machine learning algorithms showed higher detection accuracy of near-crashes compared to the K-NN algorithm.

Encrypted DNP3 Traffic Classification Using Supervised Machine Learning Algorithms

The Distributed Network Protocol (DNP3) is predominately used by the electric utility industry and, consequently, in smart grids. The Peekaboo attack was created to compromise DNP3 traffic, in which a man-in-the-middle on a communication link can capture and drop selected encrypted DNP3 messages by using support vector machine learning algorithms. The communication networks of smart grids are a important part of their infrastructure, so it is of critical importance to keep this communication secure and reliable. The main contribution of this paper is to compare the use of machine learning techniques to classify messages of the same protocol exchanged in encrypted tunnels. The study considers four simulated cases of encrypted DNP3 traffic scenarios and four different supervised machine learning algorithms: Decision tree, nearest-neighbor, support vector machine, and naive Bayes. The results obtained show that it is possible to extend a Peekaboo attack over multiple substations, using a decision tree learning algorithm, and to gather significant information from a system that communicates using encrypted DNP3 traffic.

A New Enhanced Learning Approach to Automatic Image Classification Based on Salp Swarm Algorithm

In this paper we propose a new image classification technique. According to this note that most research focuses on extraction of features in the frequency domain, location, and reduction of feature dimensions, in this research we focused on learning step in image classification. The main aim is to use the heuristic methods to increase the function of the estimator of the learning algorithm and continue to achieve the desired state, as well as categorization without user interference and automatically performed by the model produced from the above steps. So, in this paper, a new learning approach based on the Salp Swarm Algorithm was proposed that was implemented and evaluated on learning algorithm Decision Tree, K-Nearest Neighbors and Naïve Bayes. The results demonstrate the improvement of the performance of learning algorithms in all the achieved criteria by using the SSA algorithm in comparison with traditional learning algorithms. In the accuracy, sensitivity, classification error and F1 criterion, the best performance of the proposed model is using the Decision Tree learning method with values of 99.17%, 100%, 0.83% and 95.65% respectively. In the specificity and precision criterion, the best performance of the proposed model is based on K-Nearest Neighbors learning method with values of 100%.

A study on predicting loan default based on the random forest algorithm

Recently, with the advance of electronic commerce and big data technology, P2P online lending platforms have brought opportunities to businessmen, but at the same time, they are also faced with the risk of user loan default, which is related to the sustainable and healthy development of platforms. Therefore, based on the Random Forest algorithm, this paper builds a loan default prediction model in view of the real-world user loan data on Lending Club. The SMOTE method is adopted to cope with the problem of imbalance class in the dataset, and then a series of operations such as data cleaning and dimensionality reduction are carried out. The experimental results show that: Random Forest algorithm outperforms than logistic regression, decision tree and other machine learning algorithms in predicting default samples.

Advancing Cyber-Physical Systems Resilience: The Effects of Evolving Disruptions

Highly interconnected cyber-physical systems (CPSs) and complex networks can enable greater production capabilities and quality of life through their cyber-augmented characteristics. The high level of interconnectedness within CPSs, however, also increases their vulnerabilities to intelligent malware and coordinated attacks. A disrupted part of a CPS can not only cease to function properly, it can also increase the vulnerability of other parts of the CPS due to the interconnections. In this research, we expand on our previous work, the Collaborative Response against Disruption Propagation (CRDP) model and explore the effects of intelligently coordinated attacks on CPSs. This work addresses a severe limitation of the CRDP model: The assumption that disruptions have random targets. In this new work, the adversarial disruption network initially has only limited information about the target CPS network and assumes random targets. Over time, the adversarial disruption network is assumed to utilize decision tree learning algorithm to intelligently learn about the CPS’s response capabilities over time. Then, the disruption network utilizes this learning to improve its attacks by the defense targeting and response protocols. The novel adversarial disruption learning and targeting protocols are simulated and validated with a random network model. The experiments show that the developed protocols defending the disrupted network can significantly stress the CPS’s resilience capabilities. Further research is needed to understand how the role of cyber-collaborative protocols can advance the CPS resilience.

Predicting lithium treatment response in bipolar patients using gender-specific gene expression biomarkers and machine learning

Background: We sought to test the hypothesis that transcriptome-level gene signatures are differentially expressed between male and female bipolar patients, prior to lithium treatment, in a patient cohort who later were clinically classified as lithium treatment responders. Methods: Gene expression study data was obtained from the Lithium Treatment-Moderate dose Use Study data accessed from the National Center for Biotechnology Information’s Gene Expression Omnibus via accession number GSE4548. Differential gene expression analysis was conducted using the Linear Models for Microarray and RNA-Seq (limma) package and the Decision Tree and Random Forest machine learning algorithms in R. Results: Using quantitative gene expression values reported from patient blood samples, the RBPMS2 and LILRA5 genes classify male lithium responders with an area under the receiver operator characteristic curve (AUROC) of 0.92 and the ABRACL, FHL3, and NBPF14 genes classify female lithium responders AUROC of 1. A Decision Tree rule for establishing male versus female samples, using gene expression values were found to be: if RPS4Y1 ≥ 9.643, patient is a male and if RPS4Y1 < 9.643, patient is female with a probability=100%. Conclusions: We developed a pre-treatment gender- and gene-expression-based predictive model selective for classifying male lithium responders with a sensitivity of 96% using 2-genes and female lithium responders with sensitivity=92% using 3-genes.

Classifying the severity of basal stem rot disease in oil palm plantations using WorldView-3 imagery and machine learning algorithms

ABSTRACTBasal stem rot (BSR) disease caused by Ganoderma boninense is a major disease in oil palm plantations and there is no effective fungicide to control this disease. Several researchers have applied remote sensing for BSR studies, but, until now, WorldView-3 imagery has not been used to classify the severity of BSR disease symptoms. The objectives of this study were to predict disease severity with WorldView-3 imagery using supervised learning algorithms, and to describe the characteristic symptoms of BSR disease in oil palm at different disease severity levels that can be identified by WorldView-3 imagery. Field observation data were collected for 1923 oil palm trees with various levels of infection, namely healthy trees, and unhealthy trees with three levels of symptoms from mild to severe. Decision tree, random forest, and support vector machine learning algorithms were applied. The overall accuracy was low, but the accuracy was improved by about 1%–5% after outliers were removed from the data. Outliers were removed based on box plots of the distribution of reflectance value of band 4. Band 4 was selected by Jeffries-Matusita distance analysis and ‘stepclass’ algorithm. The separation of four levels of BSR disease described in this study was affected by the criteria used to describe the BSR disease symptoms in the field observation step. A redefined method and new criteria for BSR disease symptoms that can be identify using WorldView-3 imagery are described.

Horn-ICE learning for synthesizing invariants and contracts

We design learning algorithms for synthesizing invariants using Horn implication counterexamples (Horn-ICE), extending the ICE-learning model. In particular, we describe a decision-tree learning algorithm that learns from nonlinear Horn-ICE samples, works in polynomial time, and uses statistical heuristics to learn small trees that satisfy the samples. Since most verification proofs can be modeled using nonlinear Horn clauses, Horn-ICE learning is a more robust technique to learn inductive annotations that prove programs correct. Our experiments show that an implementation of our algorithm is able to learn adequate inductive invariants and contracts efficiently for a variety of sequential and concurrent programs.

Educational data mining for students’ academic performance analysis in selected Ethiopian universities

Universities are working in a very dynamic and powerfully viable environment today. Due to the advent of information technology, they gather large volumes of data related to their students in electronic form in various formats like records, files, documents, images, sound, videos, scientific data and many new data formats. This study focuses on predicting performance of student at an early stage of the degree program, in order to help the university not only to focus more on bright students but also to initially identify students with low academic achievement and find ways to support them. The knowledge is hidden among the educational data set and it is extractable through data mining techniques. The aim of this paper is to use data mining methodologies to design and develop a Data Mining model to predict academic performance of students at the end of first year degree program in selected Ethiopian higher educational institutions (universities).The data of different undergraduate students has been mined with decision tree classifiers. A model is built using C4.5 Decision tree learning algorithm – generates five classification rule set classifiers (predictors) in an experiment. The experiment using a test data set produces 81.4% accuracy.Keywords: Educational Data, Educational Data mining, Decision tree, Classification rule, C4.5

Predicting lithium treatment response in bipolar patients using gender-specific gene expression biomarkers and machine learning

Background: We sought to test the hypothesis that transcriptome-level gene signatures are differentially expressed between male and female bipolar patients, prior to lithium treatment, in a patient cohort who later were clinically classified as lithium treatment responders. Methods: Gene expression study data was obtained from the Lithium Treatment-Moderate dose Use Study data accessed from the National Center for Biotechnology Information’s Gene Expression Omnibus via accession number GSE4548. Differential gene expression analysis was conducted using the Linear Models for Microarray and RNA-Seq (limma) package and the Decision Tree and Random Forest machine learning algorithms in R. Results: Using quantitative gene expression values reported from patient blood samples, the RBPMS2 and LILRA5 genes classify male lithium responders with an area under the receiver operator characteristic curve (AUROC) of 0.92 and the ABRACL, FHL3, and NBPF14 genes classify female lithium responders AUROC of 1. A Decision Tree rule for establishing male versus female samples, using gene expression values were found to be: if RPS4Y1 ≥ 9.643, patient is a male and if RPS4Y1 < 9.643, patient is female with a probability=100%. Conclusions: We developed a pre-treatment gender- and gene-expression-based predictive model selective for classifying male lithium responders with a sensitivity of 96% using 2-genes and female lithium responders with sensitivity=92% using 3-genes.