Fuzzy Decision Tree Induction Research Articles

A fuzzy content-based group recommender system with dynamic selection of the aggregation functions

Recommender systems are currently software tools that are focused on providing users with the best choices in an overloaded search space of possible options. Hence, group recommender systems have recently become an important trend in recommendation, because they aim at recommending a special type of items so-called social items, that tend to be consumed in groups such as TV programs, travel packages, etc. Among the different types of algorithms applied for group recommender systems, this paper is focused on content-based group recommender systems, as a novel group recommendation paradigm that exploits item features in the recommendation generation process. Specifically, our goal is to introduce a new content-based group recommendation approach, based on the recommendation aggregation paradigm whose main novelty is the development of a dynamic selection process of the aggregation scheme. Such an approach is centered on the identification of group's characteristics that are matching with the most appropriate function to use in the individual recommendation aggregation step. To perform such a matching, it is proposed a fuzzy decision tree induction process. The experimental evaluation shows that this scheme improves the recommendation performance of previous content-based group recommendation approaches, as well as it serves a starting point for further research based on this dynamic selection paradigm.

Information granulation-based fuzzy partition in decision tree induction

In this study, an interval information granulation-based fuzzy partition (InterIG-FP) method is established to define fuzzy items in the framework of fuzzy decision tree induction. The proposed method first employs the principle of justifiable granularity to build information granules in terms of different classes on different condition attributes, where the number of information granules is equal to the number of classes. Then, the average values of the representative samples determined by each information granule are utilized to define the membership functions of fuzzy items for each condition attribute. Finally, an interval information granulation-based fuzzy decision tree (InterIG-FDT) is constructed based on the predefined fuzzy items in a top-down recursive way. The experiments illustrate the effectiveness, comparability and immunity capability of the proposed method. The effectiveness is verified by some comparative analysis with several traditional no-fuzzy decision trees. Furthermore, under the same constructing framework, some fuzzy decision trees with different fuzzy partition methods are studied to show the comparability on classification accuracies and the immunity capability on noise data, respectively.

Fuzzy decision trees embedded with evolutionary fuzzy clustering for locating users using wireless signal strength in an indoor environment

Location estimation is one of the critical requirement for developing smart environment products. Due to huge utilization and accessibility of WiFi infrastructure facility in indoor environments, researchers widely studied this technology to locate users accurately to provide several services instantly. In this research work, a hybrid algorithm namely fuzzy decision tree (FDT) with evolutionary fuzzy clustering methods is adopted for optimal user localization in a closed environment. Here we consider the wireless signal strengths received from the smart phones as predictors and the location of the user as the classification label. The required data for the current research is collected from the physical facility available at an office location in USA. The classification results obtained are promising enough to show that the evolutionary clustering approaches provide good fuzzy clusters for FDT induction with better accuracy.

Dynamic programming based fuzzy partition in fuzzy decision tree induction

Fuzzy decision trees are one of the most popular extensions of decision trees for symbolic knowledge acquisition by fuzzy representation. Among the majority of fuzzy decision trees learning methods, the number of fuzzy partitions is given in advance, that is, there are the same amount of fuzzy items utilized in each condition attribute. In this study, a dynamic programming-based partition criterion for fuzzy items is designed in the framework of fuzzy decision tree induction. The proposed criterion applies an improved dynamic programming algorithm used in scheduling problems to establish an optimal number of fuzzy items for each condition attribute. Then, based on these fuzzy partitions, a fuzzy decision tree is constructed in a top-down recursive way. A comparative analysis using several traditional decision trees verify the feasibility of the proposed dynamic programming based fuzzy partition criterion. Furthermore, under the same framework of fuzzy decision trees, the proposed fuzzy partition solution can obtain a higher classification accuracy than some cases with the same amount of fuzzy items.

Fuzzy Decision Trees in Medical Decision Making Support Systems

Fuzzy decision trees represent classification knowledge more naturally to the way of human thinking and are more robust in tolerating imprecise, conflict, and missing information. Decision Making Support Systems are used widely in clinical medicine because decisions play an important role in diagnostic processes. Decision trees are a very suitable candidate for induction of simple decision-making models with the possibility of automatic learning. The goal of this paper is to demonstrate a new approach for predictive data mining models in clinical medicine. This approach is based on induction of fuzzy decision trees. This approach allows us to build decision-making modesl with different properties (ordered, stability etc.). Three new types of fuzzy decision trees (non-ordered, ordered and stable) are considered in the paper. Induction of these fuzzy decision trees is based on cumulative information estimates. Results of experimental investigation are presented. Predictive data mining is becoming an essential instrument for researchers and clinical practitioners in medicine. Using new approaches based on fuzzy decision trees allows to increase the prediction accuracy. Decision trees are a very suitable candidate for induction using simple decision-making models with the possibility of automatic and AI learning.

DIAGNOSIS OF BREAST LESIONS USING THE LOCAL CHAN-VESE MODEL, HIERARCHICAL FUZZY PARTITIONING AND FUZZY DECISION TREE INDUCTION

Breast cancer is one of the leading causes of death among women. Mammography remains today the best technology to detect breast cancer, early and efficiently, to distinguish between benign and malignant diseases. Several techniques in image processing and analysis have been developed to address this problem. In this paper, we propose a new solution to the problem of computer aided detection and interpretation for breast cancer. In the proposed approach, a Local Chan-Vese (LCV) model is used for the mass lesion segmentation step to isolate a suspected abnormality in a mammogram. In the classification step, we propose a two-step process: firstly, we use the hierarchical fuzzy partitioning (HFP) to construct fuzzy partitions from data, instead of using the only human information, available from expert knowledge, which are not sufficiently accurate and confronted to errors or inconsistencies. Secondly,fuzzy decision tree induction are proposed to extract classification knowledge from a set of feature-based examples. Fuzzy decision trees are first used to determine the class of the abnormality detected (well-defined mass, ill-defined mass, architectural distortion, or speculated masses), then, to identify the Severity of the abnormality, which can be benign or malignant. The proposed system is tested by using the images from Mammographic Image Analysis Society[MIAS] database. Experimental results show the efficiency of the proposed approach, resulting in an accuracy rate of 87, a sensitivity of 82.14%, and good specificity of 91.42

On the estimation of optimal number of clusters for the induction of fuzzy decision trees

Fuzzy decision tree (FDT) induction is a powerful methodology to extract human interpretable fuzzy classification rules. As far as our knowledge goes there is no recent comparative study of fuzzy cluster validity indices with an objective of using it for estimating the optimal number of clusters for each of the continuous attributes during the process of induction of FDT. In this paper, we study the performance of the FDT with optimal number of partitions for each node appearing in the FDT. By obtaining optimal number of fuzzy clusters, we capture the intrinsic structure of the attribute values during the formation of fuzzy partitions, which in turn improves the classification accuracy of FDT. Extensive computational experiments are conducted on FDT developed using Fuzzy ID3 and eight fuzzy cluster validity indices over 30 publicly available pattern classification datasets. Non-parametric statistical tests are conducted to test the null hypothesis.

On the estimation of optimal number of clusters for the induction of fuzzy decision trees

Fuzzy decision tree (FDT) induction is a powerful methodology to extract human interpretable fuzzy classification rules. As far as our knowledge goes there is no recent comparative study of fuzzy cluster validity indices with an objective of using it for estimating the optimal number of clusters for each of the continuous attributes during the process of induction of FDT. In this paper, we study the performance of the FDT with optimal number of partitions for each node appearing in the FDT. By obtaining optimal number of fuzzy clusters, we capture the intrinsic structure of the attribute values during the formation of fuzzy partitions, which in turn improves the classification accuracy of FDT. Extensive computational experiments are conducted on FDT developed using Fuzzy ID3 and eight fuzzy cluster validity indices over 30 publicly available pattern classification datasets. Non-parametric statistical tests are conducted to test the null hypothesis.

A Study on the Approximation of Clustered Data to Parameterized Family of Fuzzy Membership Functions for the Induction of Fuzzy Decision Trees

Abstract This paper investigates the Triangular, Trapezoidal and Gaussian approximation methods for the purpose of induction of Fuzzy Decision Trees (FDT). The generation of FDT is done using a Fuzzy ID3 induction algorithm. In this work three fuzzy partitioning techniques which form the basis for our investigation are given attention, namely Fuzzy C Means clustering (FCM), Grid partitioning and Subtractive clustering (Subclust). Our contribution lies in studying the effect of various approximations on the generation of FDT giving specific attention to the classification accuracy of FDT. In this study we show that the accuracy levels of FDT generated using FCM clustered raw data, bypassing the approximation step, is acceptable and this method has several advantages too. Several computational experiments are conducted and non parametric statistical tests are performed to find if any significant differences exist between the method of bypassing the approximation step and the other methods which include approximation. Ten FDTs are developed and used in this study. These FDT’s differ in their fuzzy partitioning techniques and the approximation methods used.

VR-BFDT: A variance reduction based binary fuzzy decision tree induction method for protein function prediction

In protein function prediction (PFP) problem, the goal is to predict function of numerous well-sequenced known proteins whose function is not still known precisely. PFP is one of the special and complex problems in machine learning domain in which a protein (regarded as instance) may have more than one function simultaneously. Furthermore, the functions (regarded as classes) are dependent and also are organized in a hierarchical structure in the form of a tree or directed acyclic graph. One of the common learning methods proposed for solving this problem is decision trees in which, by partitioning data into sharp boundaries sets, small changes in the attribute values of a new instance may cause incorrect change in predicted label of the instance and finally misclassification. In this paper, a Variance Reduction based Binary Fuzzy Decision Tree (VR-BFDT) algorithm is proposed to predict functions of the proteins. This algorithm just fuzzifies the decision boundaries instead of converting the numeric attributes into fuzzy linguistic terms. It has the ability of assigning multiple functions to each protein simultaneously and preserves the hierarchy consistency between functional classes. It uses the label variance reduction as splitting criterion to select the best “attribute-value” at each node of the decision tree. The experimental results show that the overall performance of the proposed algorithm is promising.

Inducting Fuzzy Decision Tree Based on Discrete Attributes through Uncertainty Reduction

In this paper, we give a fuzzy decision tree (simply FDT) induction algorithm, named FDTAmbig, to handle the classification with discrete attributes through the uncertainty reduction. In FDTAmbig, the uncertainty is measured with classification ambiguity. FDTAmbig selects the attribute which will cause the further reduction of uncertainty as the expanded attribute for each decision node. The experimental result shows that FDTAmbig has the better generalization capability in comparison with the FDT induced with classification entropy (FDTEntr).

Interval-valued fuzzy decision trees with optimal neighbourhood perimeter

This research proposes a new model for constructing decision trees using interval-valued fuzzy membership values. Most existing fuzzy decision trees do not consider the uncertainty associated with their membership values, however, precise values of fuzzy membership values are not always possible. In this paper, we represent fuzzy membership values as intervals to model uncertainty and employ the look-ahead based fuzzy decision tree induction method to construct decision trees. We also investigate the significance of different neighbourhood values and define a new parameter insensitive to specific data sets using fuzzy sets. Some examples are provided to demonstrate the effectiveness of the approach.

Fuzzy Decision Tree

Decision trees have been widely used in machine learning. However, due to some reasons, data collecting in real world contains a fuzzy and uncertain form. The decision tree should be able to handle such fuzzy data. This paper presents a method to construct fuzzy decision tree. It proposes a fuzzy decision tree induction method in iris flower data set, obtaining the entropy from the distance between an average value and a particular value. It also presents an experiment result that shows the accuracy compared to former ID3.

COMPARING DIFFERENT STOPPING CRITERIA FOR FUZZY DECISION TREE INDUCTION THROUGH IDFID3

Fuzzy Decision Tree (FDT) classifiers combine decision trees with approximate reasoning offered by fuzzy representation to deal with language and measurement uncertainties. When a FDT induction algorithm utilizes stopping criteria for early stopping of the tree's growth, threshold values of stopping criteria will control the number of nodes. Finding a proper threshold value for a stopping criterion is one of the greatest challenges to be faced in FDT induction. In this paper, we propose a new method named Iterative Deepening Fuzzy ID3 (IDFID3) for FDT induction that has the ability of controlling the tree’s growth via dynamically setting the threshold value of stopping criterion in an iterative procedure. The final FDT induced by IDFID3 and the one obtained by common FID3 are the same when the numbers of nodes of induced FDTs are equal, but our main intention for introducing IDFID3 is the comparison of different stopping criteria through this algorithm. Therefore, a new stopping criterion named Normalized Maximum fuzzy information Gain multiplied by Number of Instances (NMGNI) is proposed and IDFID3 is used for comparing it against the other stopping criteria. Generally speaking, this paper presents a method to compare different stopping criteria independent of their threshold values utilizing IDFID3. The comparison results show that FDTs induced by the proposed stopping criterion in most situations are superior to the others and number of instances stopping criterion performs better than fuzzy information gain stopping criterion in terms of complexity (i.e. number of nodes) and classification accuracy. Also, both tree depth and fuzzy information gain stopping criteria, outperform fuzzy entropy, accuracy and number of instances in terms of mean depth of generated FDTs.

Induction of fuzzy decision trees and its refinement using gradient projected-neuro-fuzzy decision tree

Fuzzy decision tree (FDT) induction is a powerful methodology to extract human interpretable classification rules. Due to the greedy nature of FDT, there is a chance of FDT resulting in poor classification accuracy. To improve the accuracy of FDT, Bhatt and Gopal (2006) have proposed a back propagation strategy, where the interpretability of derived fuzzy rules is affected, as the certainty factor of the rules does not lie within the theoretical bounds of 0 and 1. To retain the human interpretability of fuzzy rules, and to make rules consistent with fuzzy set theory, we restrict the values of certainty factor to lie within theoretical bounds using the concept of gradient projection over neuro fuzzy decision tree and the model is named as Gradient Projected-Neuro-Fuzzy Decision Tree (GP-N-FDT). Here, the parameters of FDT developed using Fuzzy ID3 algorithm are fine tuned using GP-N-FDT strategy to improve the classification accuracy.

INDUCING FUZZY REGRESSION TREE FORESTS USING ARTIFICIAL IMMUNE SYSTEMS

Fuzzy decision forests aim to improve the predictive power of single fuzzy decision trees by allowing multiple views of the same domain to be modelled. Such forests have been successfully created for classification problems where the outcome field is discrete; however predicting a continuous output value is more challenging in combining the output from multiple fuzzy decision trees. This paper presents a new approach to creating fuzzy regression tree forests based upon the induction of multiple fuzzy regression decision trees from one training sample, where each tree will represent a different view of the data domain. The singular fuzzy regression trees are induced using a proven algorithm known as Elgasir which fuzzifies crisp CHAID decision trees using trapezoidal membership functions for fuzzification and applies Takagi-Sugeno inference to obtain the final predicted values. A modified version of Artificial Immune System Network model (opt-aiNet) is then used for the simultaneous optimization of the membership functions across all trees within the forest. A strength of the proposed method is that data does not require fuzzification before forest induction this reducing pre-processing time and the need for subjective human experts. Five problem sets from the UCI repository and KEEL repository are used to evaluate the approach. The experimental results have shown that fuzzy regression tree forests reduce the error rate compared with single fuzzy regression trees.

Maximum Ambiguity-Based Sample Selection in Fuzzy Decision Tree Induction

Sample selection is to select a number of representative samples from a large database such that a learning algorithm can have a reduced computational cost and an improved learning accuracy. This paper gives a new sample selection mechanism, i.e., the maximum ambiguity-based sample selection in fuzzy decision tree induction. Compared with the existing sample selection methods, this mechanism selects the samples based on the principle of maximal classification ambiguity. The major advantage of this mechanism is that the adjustment of the fuzzy decision tree is minimized when adding selected samples to the training set. This advantage is confirmed via the theoretical analysis of the leaf-nodes' frequency in the decision trees. The decision tree generated from the selected samples usually has a better performance than that from the original database. Furthermore, experimental results show that generalization ability of the tree based on our selection mechanism is far more superior to that based on random selection mechanism.

Fuzzy SLIQ Decision Tree Based on Classification Sensitivity

The determination of membership function is fairly critical to fuzzy decision tree induction. Unfortunately, generally used heuristics, such as SLIQ, show the pathological behavior of the attribute tests at split nodes inclining to select a crisp partition. Hence, for induction of binary fuzzy tree, this paper proposes a method depending on the sensitivity degree of attributes to all classes of training examples to determine the transition region of membership function. The method, properly using the pathological characteristic of common heuristics, overcomes drawbacks of G-FDT algorithm proposed by B. Chandra, and it well remedies defects brought on by the pathological behavior. Moreover, the sensitivity degree based algorithm outperforms G-FDT algorithm in respect to classification accuracy.

Induction of multiple fuzzy decision trees based on rough set technique

The integration of fuzzy sets and rough sets can lead to a hybrid soft-computing technique which has been applied successfully to many fields such as machine learning, pattern recognition and image processing. The key to this soft-computing technique is how to set up and make use of the fuzzy attribute reduct in fuzzy rough set theory. Given a fuzzy information system, we may find many fuzzy attribute reducts and each of them can have different contributions to decision-making. If only one of the fuzzy attribute reducts, which may be the most important one, is selected to induce decision rules, some useful information hidden in the other reducts for the decision-making will be losing unavoidably. To sufficiently make use of the information provided by every individual fuzzy attribute reduct in a fuzzy information system, this paper presents a novel induction of multiple fuzzy decision trees based on rough set technique. The induction consists of three stages. First several fuzzy attribute reducts are found by a similarity based approach, and then a fuzzy decision tree for each fuzzy attribute reduct is generated according to the fuzzy ID3 algorithm. The fuzzy integral is finally considered as a fusion tool to integrate the generated decision trees, which combines together all outputs of the multiple fuzzy decision trees and forms the final decision result. An illustration is given to show the proposed fusion scheme. A numerical experiment on real data indicates that the proposed multiple tree induction is superior to the single tree induction based on the individual reduct or on the entire feature set for learning problems with many attributes.

Fuzzy rule induction in a set covering framework

Classes of algorithms and their corresponding knowledge representations for the induction of fuzzy logic classification rules include, for example, clustering and fuzzy decision trees. This paper introduces a new class of induction algorithms based on fuzzy set covering principles. We present a set covering framework for concept learning using fuzzy sets, and develop an algorithm, FUZZYBEXA, based on this approach to induce fuzzy classification rules from data. Unlike the induction of fuzzy decision trees that follow a divide-and-conquer strategy, this algorithm performs a separate-and-conquer general-to-specific search of the instance space. We show that the description language allows a partial ordering of candidate hypotheses leading to a lattice of conjunctions to be searched. Properties of the lattice allow the development of new heuristics to guide the search for good concept descriptions and to terminate the search early enough in the induction process. The operation of the algorithm is illustrated and then compared with other well-known crisp and fuzzy machine learning algorithms. The results show that highly accurate and comprehensible rules are induced, and that this methodology is an important new tool in the arsenal of fuzzy machine learning algorithms.