Crisp Decision Trees Research Articles

A New K ‐Ary Crisp Decision Tree Induction with Continuous Valued Attributes

The simplicity and interpretability of decision tree induction makes it one of the more widely used machine learning methods for data classification. However, for continuous valued (real and integer) attribute data, there is room for further improvement in classification accuracy, complexity, and tree scale. We propose a new K-ary partition discretization method with no more than K−1 cut points based on Gaussian membership functions and the expected class number. A new K-ary crisp decision tree induction is also proposed for continuous valued attributes with a Gini index, combining the proposed discretization method. Experimental results and non-parametric statistical tests on 19 real-world datasets showed that the proposed algorithm outperforms four conventional approaches in terms of both classification accuracy, tree scale, and particularly tree depth. Considering the number of nodes, the proposed methods decision tree tends to be more balanced than in the other four methods. The complexity of the proposed algorithm was relatively low.

A computational intelligence technique for the effective diagnosis of diabetic patients using principal component analysis (PCA) and modified fuzzy SLIQ decision tree approach

Knowledge inference systems are built to identify hidden and logical patterns in huge data. Decision trees play a vital role in knowledge discovery but crisp decision tree algorithms have a problem with sharp decision boundaries which may not be implicated to all knowledge inference systems. A fuzzy decision tree algorithm overcomes this drawback. Fuzzy decision trees are implemented through fuzzification of the decision boundaries without disturbing the attribute values. Data reduction also plays a crucial role in many classification problems. In this research article, it presents an approach using principal component analysis and modified Gini index based fuzzy SLIQ decision tree algorithm. The PCA is used for dimensionality reduction, and modified Gini index fuzzy SLIQ decision tree algorithm to construct decision rules. Finally, through PID data set, the method is validated in the simulation experiment in MATLAB.

Flood‐type classification in mountainous catchments using crisp and fuzzy decision trees

AbstractFloods are governed by largely varying processes and thus exhibit various behaviors. Classification of flood events into flood types and the determination of their respective frequency is therefore important for a better understanding and prediction of floods. This study presents a flood classification for identifying flood patterns at a catchment scale by means of a fuzzy decision tree. Hence, events are represented as a spectrum of six main possible flood types that are attributed with their degree of acceptance. Considered types are flash, short rainfall, long rainfall, snow‐melt, rainfall on snow and, in high alpine catchments, glacier‐melt floods. The fuzzy decision tree also makes it possible to acknowledge the uncertainty present in the identification of flood processes and thus allows for more reliable flood class estimates than using a crisp decision tree, which identifies one flood type per event. Based on the data set in nine Swiss mountainous catchments, it was demonstrated that this approach is less sensitive to uncertainties in the classification attributes than the classical crisp approach. These results show that the fuzzy approach bears additional potential for analyses of flood patterns at a catchment scale and thereby it provides more realistic representation of flood processes.

Two new mathematical models for prediction of early mortality risk in coronary artery bypass graft surgery

The aim of this study was to develop new models for prediction of short-term mortality risk in on-pump coronary artery bypass grafting (CABG) surgery using decision tree (DT) methods. Between September 2005 and April 2006, 948 consecutive patients underwent CABG surgery at Rajaie Heart Center. Potential risk factors were reviewed and univariate and multivariate analysis for short-term mortality were performed. The whole dataset was divided into mutually exclusive subsets. An entropy error fuzzy decision tree (EEFDT) and an entropy error crisp decision tree (EECDT) were implemented using 650 (68.6%) patient data and tested with 298 (31.4%) patient data. Ten times hold-out cross validation was done and the area under the receiver operative characteristic curve (AUC) was reported as model performance. The results were compared with the logistic regression (LR) model and EuroSCORE. The overall short-term mortality rate was 3.8%, and was statistically higher in women than men (P<.001). The final EEFDT selected 19 variables and resulted in a tree with 39 nodes, 20 conditional rules, and AUC of 0.90±0.008. The final EECDT selected 15 variables and resulted in a tree with 35 nodes, 18 conditional rules, and AUC of 0.86±0.008. The LR model selected 10 variables and resulted in an AUC of 0.78±0.008; the AUC for EuroSCORE was 0.77±0.003. There were no differences in the discriminatory power of EEFDT and EECDT (P=.066) and their performance was superior to LR and EuroSCORE. EEFDT, EECDT, LR, and EuroSCORE had clinical acceptance but the performance and accuracy of the DTs were superior to the other models.

Parameter Optimization Based on Genetic Algorithm in the Research of Equivalent Pruning Effect on Fuzzy Decision Tree

for a parameter α that played an important role in the course of fuzzy decision tree creation ,there exists a specific interval, within which gradual increasing of the parameter α may have the same effect as the crisp decision tree post pruning on the size and test accuracy, and there exists a optimum value of α within this specific interval, When α gets the value, can make the fuzzy decision tree reach its performance optimal. To obtain α optimum value , this paper proposed a method of optimizing parameter based on genetic algorithm and proved the validity of the method through comparing with the relative experiment.

Interpretability-based fuzzy decision tree classifier a hybrid of the subtractive clustering and the multi-objective evolutionary algorithm

The fuzzy decision tree (FDT) is a powerful, top---down, hierarchical search scheme to extract human interpretable classification rules. Furthermore, the FDT is considered an approach to model a system by making use of a descriptive language with fuzzy logic and fuzzy predicates. There exist two contradictory requirements with fuzzy modeling, namely, accuracy and interpretability. In this study, the subtractive clustering and a multi-objective evolutionary algorithm are used to develop a novel fuzzy modeling scheme based on the FDT classifier to construct an accurate and interpretable system that is defined as the interpretability-based fuzzy decision tree classifier. Two interpretability measures, namely, complexity interpretability and semantics interpretability are considered to reach acceptable accuracy. These measures are optimized as different objectives within the multi-objective framework. Results obtained in several benchmark classification problems are encouraging because they show the ability of the developed scheme while yielding accuracy comparable to that achieved by other methods like neural networks and the crisp decision tree (C4.5). The experimental results demonstrate the superiority of the developed scheme.

HILK++: an interpretability-guided fuzzy modeling methodology for learning readable and comprehensible fuzzy rule-based classifiers

This work presents a methodology for building interpretable fuzzy systems for classification problems. We consider interpretability from two points of view: (1) readability of the system description and (2) comprehensibility of the system behavior explanations. The fuzzy modeling methodology named as Highly Interpretable Linguistic Knowledge (HILK) is upgraded. Firstly, a feature selection procedure based on crisp decision trees is carried out. Secondly, several strong fuzzy partitions are automatically generated from experimental data for all the selected inputs. For each input, all partitions are compared and the best one according to data distribution is selected. Thirdly, a set of linguistic rules are defined combining the previously generated linguistic variables. Then, a linguistic simplification procedure guided by a novel interpretability index is applied to get a more compact and general set of rules with a minimum loss of accuracy. Finally, partition tuning based on two efficient search strategies increases the system accuracy while preserving the high interpretability. Results obtained in several benchmark classification problems are encouraging because they show the ability of the new methodology for generating highly interpretable fuzzy rule-based classifiers while yielding accuracy comparable to that achieved by other methods like neural networks and C4.5. The best configuration of HILK will depend on each specific problem under consideration but it is important to remark that HILK is flexible enough (thanks to the combination of several algorithms in each modeling stage) to be easily adaptable to a wide range of problems.

A fuzzy numeric inference strategy for classification and regression problems

This paper introduces a novel Fuzzy Numeric Inference Strategy (FNIS) which induces fuzzy trees that can be applied to data domains where the outcome can be either numeric or discrete. The methodology applies the principles of fuzzy theory to pre-generated crisp decision trees in order to soften the sharp decision boundaries that are inherent in such induction techniques. Introducing fuzziness around a tree node allows classification knowledge to be represented more naturally and in-line with human thinking thus creating more robust trees when handling imprecise or conflicting information. The FNIS methodology first extrapolates rules from crisp decision trees. Each attribute is then fuzzified using a Genetic Algorithm (GA) to determine the size of the fuzzy partitions around each tree node automatically. A fuzzy decision tree is then created using a one-to-one mapping and a genetically optimised pre-selected fuzzy inference technique is used to combine all information throughout the tree. FNIS uses two strategies for defuzzification, depending on the type of the outcome variable. For discrete values the traditional centre of gravity approach is adopted, whilst for predicting numeric outcomes a novel method of defuzzification is proposed. CHAID is a successfully proven algorithm for inducing decision trees which can solve both classification and regression problems. It is used to illustrate the creation of fuzzy trees through the proposed strategy. A series of experiments is carried out to compare the performance of crisp trees with FNIS induced fuzzy trees, using real world datasets. The results are shown to compare favourably with other fuzzy and crisp decision tree algorithms. The fuzzy trees are also shown to be more robust leading to improved classification/prediction over crisp trees.

Fuzzifying Gini Index based decision trees

Crisp decision tree algorithms face the problem of having sharp decision boundaries which may not be found in all real life classification problems. A fuzzy decision tree algorithm Gini Index based (G-FDT) is proposed in this paper to fuzzify the decision boundary without converting the numeric attributes into fuzzy linguistic terms. Gini Index is used as split measure for choosing the most appropriate splitting attribute at each node. The performance of G-FDT algorithm is compared with Gini Index based crisp decision tree algorithm (SLIQ) using several real life datasets taken from the UCI machine learning repository. G-FDT algorithm outperforms its crisp counterpart in terms of classification accuracy. The size of the G-FDT is significantly less compared to SLIQ.

On constructing a fuzzy inference framework using crisp decision trees

This paper proposes a framework which consists of a novel fuzzy inference algorithm to generate fuzzy decision trees from induced crisp decision trees. Fuzzy theoretical techniques are used to fuzzify crisp decision trees in order to soften sharp decision boundaries at decision nodes inherent in this type of trees. A framework for the investigation of various types of membership functions and fuzzy inference techniques is proposed. Once the decision tree has been fuzzified, all branches throughout the tree will fire, resulting in a membership grade being generated at each branch. Five different fuzzy inference mechanisms are used to investigate the degree of interaction between membership grades on each path in the decision tree, which ultimately leads to a final crisp classification. A genetic algorithm is used to optimize and automatically determine the set of fuzzy regions for all branches and simultaneously the degree in which the inference parameters will be applied. Comparisons between crisp trees and the fuzzified trees suggest that the later fuzzy tree is significantly more robust and produces a more balanced classification. In addition, the results obtained from five real-world data sets show that there is a significant improvement in the accuracy of the fuzzy trees when compared with crisp trees.

Fuzziness As A Recognition Problem:Using Decision Tree Learning AlgorithmsFor Inducing Fuzzy Membership Functions

In this article we establish a new method for inducing fuzzy set membership degrees based on empirical training data. The approach is founded on the notion of Redundant Decision Trees (RDT), a generalisation of regular crisp Decision Trees (DT). RDTs suffice in capturing the attribute tests required for recognising crisp concepts, from which the related fuzzy concepts may be unambiguously derived. Potential applications of this method include categorisation and the semiautomatic construction and the statistical evaluation of fuzzy concepts. In addition, since the definition of the membership degrees is effectively based on a robust DT machine learning algorithm, the induced fuzzy membership functions generalise. Thus, with certain assumptions, they output sensible membership degrees of previously unseen objects. In addition to introducing and analysing the basic definitions and algorithms, we briefly evaluate their applicability with examples and present some remarks concerning the scope of the approach.

A Fuzzy Hierarchical Classification System for Olfactory Signals

A fuzzy logic-based system to classify olfactory signals is presented. The odour samples are obtained from an electronic nose that contains conducting polymer sensors with partially overlapping sensitivities to odours. The sensor responses are represented by means of the coefficients of their Fast Fourier Transform (FFT). A feature reduction method is applied to reduce the feature space dimension. Then, an Unsupervised Fuzzy Divisive Hierarchical Clustering (UFDHC) method is used to establish the optimal number of clusters in the data set, as well as the optimal cluster structure. The output of UFDHC is a binary hierarchy of fuzzy classes that are adopted to build a supervised fuzzy hierarchical classifier. At each level of the hierarchy a separating hyperplane of the two corresponding fuzzy classes is determined. The hyperplane identifies two crisp decision regions, which will be refined at the next level of the hierarchy. In this way, we obtain a hierarchy of regions, which defines a crisp decision tree. Each region is, therefore, related to a specific expected output of the system. Two small-scale applications demonstrate the effectiveness and the good recognition performance of the proposed method.

Generation of Fuzzy Rules from Decision Trees

The paper introduces two ways to develop fuzzy rules, using decision trees, from data with continuous valued inputs and outputs. A key problem is dealing with continuous outputs. Output classes are created, then a crisp decision tree is created using a set of fuzzy output classes and letting each training example to partially belong to classes. Alternatively, a discrete set of fuzzy outputs classes is created that includes a selected group of overlaps, such as class A.75/class B.25. Training examples are then provided to a standard decision tree learning program, such as C4.5. In both cases, fuzzy rules are extracted from the resulting decision tree. Output classes must be created for a case in which examples belong to discrete but overlapping classes. We discuss tradeoffs of the two approaches to output class creation. An example of system performance uses a discrete set of overlapping classes on the Box-Jenkins gas furnace prediction problem and a function approximation problem. The learned rules provide effective control and function approximation.