Optimal Decision Tree Research Articles

A diagnostic strategy for Parkinsonian syndromes using quantitative indices of DAT SPECT and MIBG scintigraphy: an investigation using the classification and regression tree analysis

PurposeWe aimed to evaluate the diagnostic performances of quantitative indices obtained from dopamine transporter (DAT) single-photon emission computed tomography (SPECT) and 123I-metaiodobenzylguanidine (MIBG) scintigraphy for Parkinsonian syndromes (PS) using the classification and regression tree (CART) analysis.MethodsWe retrospectively enrolled 216 patients with or without PS, including 80 without PS (NPS) and 136 with PS [90 Parkinson’s disease (PD), 21 dementia with Lewy bodies (DLB), 16 progressive supranuclear palsy (PSP), and 9 multiple system atrophy (MSA). The striatal binding ratio (SBR), putamen-to-caudate ratio (PCR), and asymmetry index (AI) were calculated using DAT SPECT. The heart-to-mediastinum uptake ratio (H/M) based on the early (H/M [Early]) and delayed (H/M [Delay]) images and cardiac washout rate (WR) were calculated from MIBG scintigraphy. The CART analysis was used to establish a diagnostic decision tree model for differentiating PS based on these quantitative indices.ResultsThe sensitivity, specificity, positive predictive value, negative predictive value, and accuracy were 87.5, 96.3, 93.3, 92.9, and 93.1 for NPS; 91.1, 78.6, 75.2, 92.5, and 83.8 for PD; 57.1, 95.9, 60.0, 95.4, and 92.1 for DLB; and 50.0, 98.0, 66.7, 96.1, and 94.4 for PSP, respectively. The PCR, WR, H/M (Delay), and SBR indices played important roles in the optimal decision tree model, and their feature importance was 0.61, 0.22, 0.11, and 0.05, respectively.ConclusionThe quantitative indices showed high diagnostic performances in differentiating NPS, PD, DLB, and PSP, but not MSA. Our findings provide useful guidance on how to apply these quantitative indices in clinical practice.

Optimal Decision Tree for Cycle Time Prediction and Allowance Determination

The due-date quotation is a key performance indicator for managing customer orders which would influence customer acceptance and/or the future potential lateness penalty. The production cycle time and allowance time are added and used as the due date of order. The objective is to maximize the hit rate which is the percentage of the orders fulfilled within the time limit of quoted due date. Under the framework of supervised machine learning, we explore the new developments in feature selection and the optimal decision tree to predict cycle time by using mixed-integer optimization. Cycle time allowance could be added to the predicted cycle time or incorporated in an optimization problem as a managerial decision variable. Case studies are used to demonstrate the effectiveness of this approach, and their performances are comparable to the other popular ensemble tree approaches, such as random forests and gradient boosting.

Developing a Decision Model Based on Decision Tree and Particle Swarm Optimization Algorithms to Identify Optimal Locations for Solar Power Plants Construction in Iran

Developing a Decision Model Based on Decision Tree and Particle Swarm Optimization Algorithms to Identify Optimal Locations for Solar Power Plants Construction in Iran

How fair can we go in machine learning ? Assessing the boundaries of accuracy and fairness

Fair machine learning works have been focusing on the development of equitable algorithms that address discrimination of certain groups. Yet, many of these fairness-aware approaches aim to obtain a unique solution to the problem, which leads to a poor understanding of the statistical limits of bias mitigation interventions. We present the first methodology that allows to explore those limits within a multi-objective framework that seeks to optimize any measure of accuracy and fairness and provides a Pareto front with the best feasible solutions. In this work, we focus our study on decision tree classifiers since they are widely accepted in machine learning, are easy to interpret and can deal with non-numerical information naturally. We conclude experimentally that our method can optimize decision tree models by being fairer with a small cost of the classification error. We believe that our contribution will help stakeholders of sociotechnical systems to assess how far they can go being fair and accurate, thus serving in the support of enhanced decision making where machine learning is used.

HYPER HEURISTIC EVOLUTIONARY APPROACH FOR CONSTRUCTING DECISION TREE CLASSIFIERS

Decision tree models have earned a special status in predictive modeling since these are considered comprehensible for human analysis and insight. Classification and Regression Tree (CART) algorithm is one of the renowned decision tree induction algorithms to address the classification as well as regression problems. Finding optimal values for the hyper parameters of a decision tree construction algorithm is a challenging issue. While making an effective decision tree classifier with high accuracy and comprehensibility, we need to address the question of setting optimal values for its hyper parameters like the maximum size of the tree, the minimum number of instances required in a node for inducing a split, node splitting criterion and the amount of pruning. The hyper parameter setting influences the performance of the decision tree model. As researchers, we know that no single setting of hyper parameters works equally well for different datasets. A particular setting that gives an optimal decision tree for one dataset may produce a sub-optimal decision tree model for another dataset. In this paper, we present a hyper heuristic approach for tuning the hyper parameters of Recursive and Partition Trees (rpart), which is a typical implementation of CART in statistical and data analytics package R. We employ an evolutionary algorithm as hyper heuristic for tuning the hyper parameters of the decision tree classifier. The approach is named as Hyper heuristic Evolutionary Approach with Recursive and Partition Trees (HEARpart). The proposed approach is validated on 30 datasets. It is statistically proved that HEARpart performs significantly better than WEKA’s J48 algorithm in terms of error rate, F-measure, and tree size. Further, the suggested hyper heuristic algorithm constructs significantly comprehensible models as compared to WEKA’s J48, CART and other similar decision tree construction strategies. The results show that the accuracy achieved by the hyper heuristic approach is slightly less as compared to the other comparative approaches.

Somatotype identification of middle-aged women based on decision tree algorithm

PurposeBody type classification has a great influence on plate making and garment sizing system, and the accuracy of body type classification method will greatly affect the fit of garment production. The purpose of this paper is to use the decision tree algorithm to study body classification rules, develop a decision tree body recognition model and judge the body shape of middle-aged women in Xinjiang.Design/methodology/approachFirst, perform dimensionless processing on the collected data of 256 middle-aged women in Xinjiang, and the dimensionless data were used for K-means body clustering; Then, quantitatively analyze the effectiveness of different classification clusters based on the silhouette coefficients. Second, the decision tree algorithm is used to divide the classified sample data into a training set and a test set at a ratio of 70/30, and select the best node and the best branch based on the Gini coefficient to construct a classification tree. Last, the overall optimal decision tree is generated by means of hyperparameter pruning.FindingsThe body shape of middle-aged women in Xinjiang can be divided into three types: standard body, plump body and obese body. The decision tree model has an excellent effect on body classification of middle-aged women in Xinjiang (precision (macro), 95.46%; precision (micro), 95.95%; recall (macro), 95.46%; recall (micro), 95.95%; F1 (macro), 95.46%; F1 (micro), 95.95%).Originality/valueFor scientific research, this paper is conducive to increasing the regional body type theory and stimulating the establishment of a garment sizing subdivision system in Xinjiang. In terms of production practice, this paper not only establishes a model for judging the shape of middle-aged women in Xinjiang, but also provides reference data for intermediates of various sizes. In addition, to facilitate pattern-making and the establishment of a subdivision system for the size of middle-aged women's garments in Xinjiang, this paper provides the grading values of various body control parts of middle-aged women in Xinjiang.

Soil Mapping Based on Globally Optimal Decision Trees and Digital Imitations of Traditional Approaches

Most digital soil mapping (DSM) approaches aim at complete statistical model extraction. The value of the explicit rules of soil delineation formulated by soil-mapping experts is often underestimated. These rules can be used for expert testing of the notional consistency of soil maps, soil trend prediction, soil geography investigations, and other applications. We propose an approach that imitates traditional soil mapping by constructing compact globally optimal decision trees (EVTREE) for the covariates of traditionally used soil formation factor maps. We evaluated our approach by regional-scale soil mapping at a test site in the Belgorod region of Russia. The notional consistency and compactness of the decision trees created by EVTREE were found to be suitable for expert-based analysis and improvement. With a large sample set, the accuracy of the predictions was slightly lower for EVTREE (59%) than for CART (67%) and much lower than for Random Forest (87%). With smaller sample sets of 1785 and 1000 points, EVTREE produced comparable or more accurate predictions and much more accurate models of soil geography than CART or Random Forest.

Optimization of Decision Tree with PSO on Sharia Cooperative Customer Funding

Credit is a service for Sharia Cooperatives to provide funding to customers who are paid in installments. The accuracy of the customer paying the payments is a determining factor for the smooth operation of the Sharia Cooperative. In addition to timeliness, the ability of customers to pay installments is also the most critical factor for credit returns. Bad credit is a significant threat for Sharia Cooperatives where customers cannot pay payments according to agreed agreements. That makes the prediction of the smooth operation of Sharia Cooperative customers needed. One way that can be done to make predictions is to apply data mining techniques. To make a prediction, several attributes of the customer are used, such as gender, age, status, residency status, number of dependents owned, education, employment, remittance, ceiling, type of loan, credit period, method of payment and collat- eral used. The research will use the J48 as one of the Decision Tree algorithms with particle swarm optimization techniques to improve algorithm performance. The accuracy of J48 with PSO is lower than without PSO.

Learning optimal decision trees using constraint programming

Decision trees are among the most popular classification models in machine learning. Traditionally, they are learned using greedy algorithms. However, such algorithms pose several disadvantages: it is difficult to limit the size of the decision trees while maintaining a good classification accuracy, and it is hard to impose additional constraints on the models that are learned. For these reasons, there has been a recent interest in exact and flexible algorithms for learning decision trees. In this paper, we introduce a new approach to learn decision trees using constraint programming. Compared to earlier approaches, we show that our approach obtains better performance, while still being sufficiently flexible to allow for the inclusion of constraints. Our approach builds on three key building blocks: (1) the use of AND/OR search, (2) the use of caching, (3) the use of the CoverSize global constraint proposed recently for the problem of itemset mining. This allows our constraint programming approach to deal in a much more efficient way with the decompositions in the learning problem.

New bounds for energy complexity of Boolean functions

For a Boolean function f:{0,1}n→{0,1} computed by a Boolean circuit C over a finite basis B, the energy complexity of C (denoted by ECB(C)) is the maximum over all inputs {0,1}n of the number gates of the circuit C (excluding the inputs) that output a one. Energy Complexity of a Boolean function over a finite basis B denoted by ECB(f)=defminC⁡ECB(C) where C is a Boolean circuit over B computing f.We study the case when B={∧2,∨2,¬}, the standard Boolean basis. It is known that any Boolean function can be computed by a circuit (with potentially large size) with an energy of at most 3n(1+ϵ(n)) for a small ϵ(n) (which we observe is improvable to 3n−1). We show several new results and connections between energy complexity and other well-studied parameters of Boolean functions.•For all Boolean functions f, EC(f)≤O(DT(f)3) where DT(f) is the optimal decision tree depth of f.•We define a parameter positive sensitivity (denoted by psens), a quantity that is smaller than sensitivity (Cook et al. 1986, [3]) and defined in a similar way, and show that for any Boolean circuit C computing a Boolean function f, EC(C)≥psens(f)/3.•For a monotone function f, we show that EC(f)=Ω(KW+(f)) where KW+(f) is the cost of monotone Karchmer-Wigderson game of f.•Restricting the above notion of energy complexity to Boolean formulas, we show EC(F)=Ω(L(F)−Depth(F)) where L(F) is the size and Depth(F) is the depth of a formula F.

Prediction of the Abundance of Artemia parthenogenetica in a Hypersaline Wetland Using Decision Tree Model

The hypersaline wetland of Meighan located in western Iran is an important habitat for Artemia parthenogenetica. The habitat condition of this native zooplankton is facing with various problems in the wetland, so its abundance has been reduced in the wetland in the recent years. The study aimed to optimize decision tree model with an optimizer (greedy stepwise) to predict the species abundance in 10 different sampling sites over one-year study period (2017–2018). The model output was the species abundance categorized into 4 classes (poor: 5–20; fair: 21–50; good: 51–100; very good:101–255 individuals) and measured with abiotic variables. The optimizer method improved the model performance leading to easy interpretation of the model. According to the model’s prediction, high abundance of species in the wetland is associated with high concentration of specific conductivity, dissolved oxygen and total dissolved solids. In contrast, increased concentration of chloride, total suspended solids, nitrate and precipitation might decrease the abundance of zooplankton. Chi-square test showed a significant difference between the species abundance and spatio-temporal patterns in the wetland (x2 = 160.2, p = 0.001) so that warm seasons (spring and summer) had more contribution to the zooplankton sampling than cold seasons (autumn and winter).

Depth-Optimized Delay-Aware Tree (DO-DAT) for Virtual Network Function Placement

With the constant increase in demand for data connectivity, network service providers are faced with the task of reducing their capital and operational expenses while ensuring continual improvements to network performance. Although Network Function Virtualization (NFV) has been identified as a solution, several challenges must be addressed to ensure its feasibility. In this letter, we present a machine learning-based solution to the Virtual Network Function (VNF) placement problem. This letter proposes the Depth-Optimized Delay-Aware Tree (DO-DAT) model by using the particle swarm optimization technique to optimize decision tree hyper-parameters. Using the Evolved Packet Core (EPC) as a use case, we evaluate the performance of the model and compare it to a previously proposed model and a heuristic placement strategy.

Learning Optimal Decision Trees Using Caching Branch-and-Bound Search

Several recent publications have studied the use of Mixed Integer Programming (MIP) for finding an optimal decision tree, that is, the best decision tree under formal requirements on accuracy, fairness or interpretability of the predictive model. These publications used MIP to deal with the hard computational challenge of finding such trees. In this paper, we introduce a new efficient algorithm, DL8.5, for finding optimal decision trees, based on the use of itemset mining techniques. We show that this new approach outperforms earlier approaches with several orders of magnitude, for both numerical and discrete data, and is generic as well. The key idea underlying this new approach is the use of a cache of itemsets in combination with branch-and-bound search; this new type of cache also stores results for parts of the search space that have been traversed partially.

Efficient Inference of Optimal Decision Trees

Inferring a decision tree from a given dataset is a classic problem in machine learning. This problem consists of building, from a labelled dataset, a tree where each node corresponds to a class and a path between the tree root and a leaf corresponds to a conjunction of features to be satisfied in this class. Following the principle of parsimony, we want to infer a minimal tree consistent with the dataset. Unfortunately, inferring an optimal decision tree is NP-complete for several definitions of optimality. For this reason, the majority of existing approaches rely on heuristics, and the few existing exact approaches do not work on large datasets. In this paper, we propose a novel approach for inferring an optimal decision tree with a minimum depth based on the incremental generation of Boolean formulas. The experimental results indicate that it scales sufficiently well and the time it takes to run grows slowly with the size of datasets.

Damaged buildings recognition of post-earthquake high-resolution remote sensing images based on feature space and decision tree optimization

Multi-Agent system has broad application in real world, whose security performance, however, is barely considered. Reinforcement learning is one of the most important methods to resolve Multi-Agent problems. At present, certain progress has been made in applying Multi-Agent reinforcement learning to robot system, man-machine match, and automatic, etc. However, in the above area, an agent may fall into unsafe states where the agent may find it difficult to bypass obstacles, to receive information from other agents and so on. Ensuring the safety of Multi-Agent system is of great importance in the above areas where an agent may fall into dangerous states that are irreversible, causing great damage. To solve the safety problem, in this paper we introduce a Multi-Agent Cooperation Q-Learning Algorithm based on Constrained Markov Game. In this method, safety constraints are added to the set of actions, and each agent, when interacting with the environment to search for optimal values, should be restricted by the safety rules, so as to obtain an optimal policy that satisfies the security requirements. Since traditional Multi-Agent reinforcement learning algorithm is no more suitable for the proposed model in this paper, a new solution is introduced for calculating the global optimum state-action function that satisfies the safety constraints. We take advantage of the Lagrange multiplier method to determine the optimal action that can be performed in the current state based on the premise of linearizing constraint functions, under conditions that the state-action function and the constraint function are both differentiable, which not only improves the efficiency and accuracy of the algorithm, but also guarantees to obtain the global optimal solution. The experiments verify the effectiveness of the algorithm.

COMPARISON BETWEEN DECISION TREE AND OPTIMAL POWER FLOW TECHNIQUES APPLIED TO VOLTAGE CORRECTIVE CONTROL IN ELECTRIC SYSTEMS

This article presents a comparison of using techniques of optimal load flow and decision tree in making corrective voltage control actions for electrical power systems under insecure operation condition, that is, with voltage constraints violated. The decision tree technique is used in conjunction with a technique of sensitivity analysis in such a way that control actions are taken to be more effective to eliminate the voltage constraints violation in each case of contingency analyzed. A comparison is made between the control actions suggested by the optimal load flow strategy and those suggested by the decision tree for the same operating condition, discussing the achievement of implementing these actions in a real power system. The analysis methodology proposed here is tested in a real power system of a Brazilian utility, which operates in the State of Amapá - Brazil. The results showed how both methodologies can be used as operational tools in power system supervision and control centers.

Representation of Knowledge by Decision Trees for Decision Tables with Multiple Decisions

Abstract In this paper, we study decisions trees for decision tables with multiple decisions as a means for knowledge representation. To this end, we consider three methods to design decision trees and evaluate the number of nodes, and local and global misclassification rates of constructed trees. The considered methods are based on a dynamic programming algorithm for bi-objective optimization of decision trees. The goal of this study is to construct trees with reasonable number of nodes and at the same time reasonable accuracy. Previously, it was mentioned that the consideration of only the global misclassification rate of the decision tree is not enough and it is necessary to study also the local misclassification rate. The reason is that even if the global misclassification rate related to the whole tree is enough small, the local misclassification rate related to the terminal nodes of the tree can be too big. One of the considered methods allows us to construct the decision trees with moderate number of nodes as well as moderate global and local misclassification rates. These decision trees can be used for the knowledge representation.

Student Network Behavior Analysis and Relevance Research Based on Optimal Decision Tree Algorithms

Based on the optimal decision tree algorithm, this paper proposes a student network behavior analysis model. Through training and predicting the academic performance of Beihang Grade 2013 undergraduates, it is found that students’ online behavior has a profound impact on students’ academic performance. To maintain a good learning state, students must strictly limit the time spent on the Internet in idle time, effectively control the time spent on the internet, and ensure that their daily sleep is not affected by the Internet behavior. The regularity of students’ life is positively correlated with their achievement ranking.

Learning Explainable Decision Rules via Maximum Satisfiability

Decision trees are a popular choice for providing explainable machine learning, since they make explicit how different features contribute towards the prediction. We apply tools from constraint satisfaction to learn optimal decision trees in the form of sparse k-CNF (Conjunctive Normal Form) rules. We develop two methods offering different trade-offs between accuracy and computational complexity: one offline method that learns decision trees using the entire training dataset and one online method that learns decision trees over a local subset of the training dataset. This subset is obtained from training examples near a query point. The developed methods are applied on a number of datasets both in an online and an offline setting. We found that our methods learn decision trees which are significantly more accurate than those learned by existing heuristic approaches. However, the global decision tree model tends to be computationally more expensive compared to heuristic approaches. The online method is faster to train and finds smaller decision trees with an accuracy comparable to that of the k-nearest-neighbour method.

Ensemble of Rule Learner and Sequential Minimum Optimization Algorithm for Intrusion Detection System

An intrusion detection system is a process which automates analyzing activities in network or a computer system. It is used to detect nasty code, hateful activities, intruders and uninvited communications over the Internet. The general intrusion detection system is struggling with some problems like false positive rate, false negative rate, low classification accuracy and slow speed. Now-a-days, this has turned an attention of many researchers to handle these issues. Recently, ensemble of different base classifier is widely used to implement intrusion detection system. In ensemble method of machine learning, the proper selection of base classifier is a challenging task. In this paper, machine learning ensemble have designed and implemented for the intrusion detection system. The ensemble of Partial Decision Tree and Sequential Minimum optimization algorithm to train support vector machine have used for intrusion detection system. Partial Decision Tree rule learner is simplicity and it generates rules fast. Sequential Minimum optimization algorithm is easy to use and is better scaling with training set size with less computational time. Due to these advantages of both classifiers, they jointly used with different methods of ensemble. We make use of all types of methods of ensemble. The performances of base classifiers have evaluated in term of false positive, accuracy and true positive. Performance results display that proposed majority voting method of ensemble using Partial Decision Tree rule learner and Sequential Minimum optimization algorithm based Support Vector Machine offers highest classification among different ensemble classifiers on training dataset. This method of ensemble exhibits highest true positive and lowest false positive rates. It is also observed that stacking of both PART and SMO exhibits lowest and same classification accuracy on test dataset