Fuzzy Rule-based Classification Systems Research Articles

Construction of fuzzy classification systems by fitness sharing based genetic search and boosting based ensemble

This paper concentrates on the development of precise fuzzy rule-based classification systems for high-dimensional and multi-class problems. The approach begins with the extraction on potential fuzzy if-then rules using fitness sharing based genetic algorithms, this ensures effective searching for productive niches, thereby evolving and maintaining a diverse, cooperative population. Subsequently, for the purpose of combining the obtained fuzzy rules and eliminating their conflicts, an adaboost ensemble method is utilized, enhancing the accuracy of the fuzzy classification systems.Experiments have been conducted on 10 UCI datasets and 3 well-known image classification problems. The features for these image tasks were derived from the activation values of the final convolutional layer in pre-trained convolutional neural networks. These datasets, which were chosen to evaluate the effectiveness of the proposed approach, exhibit significant variation in terms of dimensionality and the number of class labels. Comparative analyses are carried out with conventional fuzzy rule-based classification methods, and the results demonstrate that the classification systems can be developed for complex problems, while maintaining a high-level of prediction accuracy.

On the Suitability of Fuzzy Rule-Based Classification Systems with Noisy Data

On the Suitability of Fuzzy Rule-Based Classification Systems with Noisy Data

Learning positive-negative rule-based fuzzy associative classifiers with a good trade-off between complexity and accuracy

Nowadays, the call for transparency in Artificial Intelligence models is growing due to the need to understand how decisions derived from the methods are made when they ultimately affect human life and health. Fuzzy Rule-Based Classification Systems have been used successfully as they are models that are easily understood by models themselves. However, complex search spaces hinder the learning process, and in most cases, lead to problems of complexity (coverage and specificity). This problem directly affects the intention to use them to enable the user to analyze and understand the model. Because of this, we propose a fuzzy associative classification method to learn classifiers with an improved trade-off between accuracy and complexity. This method learns the most appropriate granularity of each variable to generate a set of simple fuzzy association rules with a reduced number of associations that consider positive and negative dependencies to be able to classify an instance depending on the presence or absence of certain items. The proposal also chooses the most interesting rules based on several interesting measures and finally performs a genetic rule selection and adjustment to reach the most suitable context of the selected rule set. The quality of our proposal has been analyzed using 23 real-world datasets, comparing them with other proposals by applying statistical analysis. Moreover, the study carried out on a real biomedical research problem of childhood obesity shows the improved trade-off between the accuracy and complexity of the models generated by our proposal.

Z-number-valued rule-based classification system

The fuzzy rule-based classification system (FRBCS) is a popular tool for classification problems due to its interpretability and comprehensibility. As an extension of fuzzy numbers, the concept of Z-number is a more appropriate formal structure to describe uncertain and partially reliable information. A Z-number is an ordered pair of fuzzy numbers, where the second fuzzy number describes the reliability of the first one. As a result of its representation capability, it can receive better classification results. However, there is still a gap in the application of Z-numbers to classification problems due to their high computation complexity. To take advantage of the Z-number, we design a simple way to make Z-numbers apply to classification problems. Use the second fuzzy number to adjust the first fuzzy number to fit the training data. Then we create a kind of Z-number-valued if–then rule by extending the fuzzy if–then rule. In addition, a Z-number-valued rule-based classification system (ZRBCS) is developed, including two main processes: rule generation and new pattern classification. The developed system can cover more information than the classic fuzzy rule-based system, which can improve classification effects. The proposed ZRBCS is compared with classical FRBCS with/without certain degrees and three classical classification algorithms. According to statistical tests, ZRBCS is superior to FRBCS and two other algorithms.

Deep Fuzzy Rule-Based Classification System With Improved Wang–Mendel Method

Wang–Mendel (WM) fuzzy system is an effective and interpretable model for solving tabular data classification problem. However, original WM fuzzy system is weak in handling dataset with high dimensionality or large volume. Meanwhile, its capability of characterizing data is narrow, which results from lacking hierarchical transformation of features like deep learning-based models. In this article, we propose a deep fuzzy rule-based classification system (DFRBCS) based on improved WM method, in which fuzzy technique and deep learning strategy are combined to make a desirable tradeoff between model’s interpretability and prediction accuracy. We first redefine the consequent part of fuzzy rule in WM fuzzy system with class probability vector, which endows the improved WM fuzzy system with capacity for serving as building block of deep model. The model structure of DFRBCS is designed in layer-by-layer manner, where raw features can be transformed hierarchically. For every layer in DFRBCS, it contains many improved WM fuzzy systems whose input spaces are generated by shuffling and sliding window operation on concatenated outputs of fuzzy systems in previous layer. Comparative experiments are conducted on 45 real-world datasets with various sizes and dimensionality between our method, five baseline models, and the other deep fuzzy classifiers (D-TSK-FC, HID-TSK-FC, FCCI-TSK, DSA-FC, and MEEFIS). The experimental results show that DFRBCS is competitive in classification performance and promising in model’s interpretability.

Evaluation of fuzzy membership functions for linguistic rule-based classifier focused on explainability, interpretability and reliability

Objective:Decision support systems focus on their interpretability with a different caution. The majority of approaches utilize reliable optimization techniques to achieve high recognition accuracy. However, accuracy improvement is done at the cost of the readability of the decision support system. This paper proposes a novel technique of linguistic rule extraction focused on: explainability, interpretability, and reliability, called EasIeR. Methods:Proposed solution performs a direct analysis of the learning set without omitting real data imperfections. The proposed algorithm processes the imprecision of data with linguistic values described by fuzzy membership functions and the uncertainty of knowledge employing a fuzzy belief function. It describes the learning set with the most appropriate fuzzy if-then rules. For this purpose, a new method of evaluation of membership functions in a set of rules and their iterative refinement is proposed. Results:EasIeR achieves the best results of trade-off criteria between the generalization quality and the interpretability of the knowledge base when compared with state-of-art fuzzy rule-based classification approaches. Achieving the simplest rule base and competitive generalization quality is verified by cross-validation experiments with benchmark data sets. Significance:The new method of rule evaluation allows for better determination of their quality in decision support, and the selection algorithm quickly provides a clear and easy to interpret knowledge base. This rule evaluation technique will be proficient for all fuzzy rule-based classification systems.

N-Dimensional Admissibly Ordered Interval-Valued Overlap Functions and Its Influence in Interval-Valued Fuzzy-Rule-Based Classification Systems

Overlap functions are a type of aggregation functions that are not required to be associative, generally used to indicate the overlapping degree between two values. They have been successfully used as a conjunction operator in several practical problems, such as fuzzy-rule-based classification systems (FRBCSs) and image processing. Some extensions of overlap functions were recently proposed, such as general overlap functions and, in the interval-valued context, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</i> -dimensional interval-valued overlap functions. The latter allow them to be applied in <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</i> -dimensional problems with interval-valued inputs, such as interval-valued classification problems, where one can apply interval-valued FRBCSs (IV-FRBCSs). In this case, the choice of an appropriate total order for intervals, such as an admissible order, can play an important role. However, neither the relationship between the interval order and the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</i> -dimensional interval-valued overlap function (which may or may not be increasing for that order) nor the impact of this relationship in the classification process have been studied in the literature. Moreover, there is not a clear preferred <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</i> -dimensional interval-valued overlap function to be applied in an IV-FRBCS. Hence, in this article, we: first, present some new results on admissible orders, which allow us to introduce the concept of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</i> -dimensional admissibly ordered interval-valued overlap functions, that is, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</i> -dimensional interval-valued overlap functions that are increasing with respect to an admissible order; second, develop a width-preserving construction method for this kind of function, derived from an admissible order and an <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</i> -dimensional overlap function, discussing some of its features; finally, analyze the behavior of several combinations of admissible orders and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</i> -dimensional (admissibly ordered) interval-valued overlap functions when applied in IV-FRBCSs. All in all, the contribution of this article resides in pointing out the effect of admissible orders and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</i> -dimensional admissibly ordered interval-valued overlap functions, both from a theoretical and applied points of view, the latter when considering classification problems.

Attributes regrouping by genetic algorithm in fuzzy inference systems

Fuzzy Rule-Based Classification Systems (FRBCS) are a very popular and useful tool due to their interpretable classification models based on linguistic variables. However, a high number of attributes leads to an explosion of the number of classification rules. In this paper, we are interested in ensemble methods applied to FRBCS. An attributes regrouping method based on association rules and called SIFRA was proposed to solve this problem. It determines the relevant groups of related attributes that will be treated separately by FRBCSs. We propose a Genetic Algorithm-based method to select final groups of related attributes determined by SIFRA. In our approach, the genetic algorithm is used to filter the groups of related attributes and conserve the best ones, while maintaining a good classification rate and a reduced number of rules. We lead experiments to compare our contribution with previous methods in the same context; the experimental results are satisfactory.

Efficient inference models for classification problems with a high number of fuzzy rules

In data science there are problems that are not visible until you work with a sufficiently large number of data. This is the case, for example, with the design of the inference engine in fuzzy rule-based classification systems. The most common way to implement the winning rule inference method is to use sequential processing that reviews each of the rules in the rule set, to determine the best one and return the associated class. This implementation produces fast response times when the set of rules is small and is applied to a small set of examples. In this paper we explore new versions to implement this inference method, avoiding analyzing all the rules and focusing the analysis on the neighborhood of rules around the example. We study experimentally the conditions where each of them should be applied. Finally, we propose an implementation that combines all the studied versions offering good accuracy results and a significant reduction in the response time.

An accurate fuzzy rule-based classification systems for heart disease diagnosis

Physicians and healthcare providers need to better understand the thought processes and methods used in clinical decision-making. This allows physicians to diagnose and detect diseases early, especially heart disease that causes death. The diversity and availability of healthcare data encourage clinicians to use healthcare applications in the diagnosis process. Most of these applications use machine learning techniques to make accurate and fast decisions. On the other hand, Explainability in healthcare applications increase the level of clinician confidence and reduces the risk of making wrong decisions, thus expands the scope and efficiency of healthcare applications. In this paper, we propose a novel data-driven method based on fuzzy clustering and linguistic modifiers to design a fuzzy rule-based classification system for heart disease diagnosis. The proposed system provides an interpretable knowledge base to explain the decision-making process. Regarding the experiment, we have used Cleveland, Hungarian and Va long beach heart disease datasets to compare the proposed method with five known machine learning methods for predicting heart disease: Artificial neural network, Support Vector Machine, K-Nearest Neighbor, Naïve Bayes, and Random Forest. The findings show that the proposed model is superior in terms of balancing interpretability and precision.

A new interval type-2 fuzzy reasoning method for classification systems based on normal forms of a possibility-based fuzzy measure

To deal with two sources of uncertainty, cluster centers and reasoning aggregation operators, in the inference engine, this paper represents a new fuzzy reasoning method for an interval type-2 fuzzy classification system including cluster-based rules.In the reasoning, a new Possibility-based fuzzy measure is introduced to consider the uncertainty of cluster centers determined by the iterative clustering algorithms (because of the NP-Hardness of clustering models) in an interval type-2 fuzzy rule-based classification system (IT2 FRBCS). Then, the diversity of aggregation operators which are generally applied in reasoning process to combine the matching degrees of a test or unseen pattern with all clusters of an output class is regarded as the other source of reasoning uncertainty extending the crisp matching degrees into the intervals. The disjunctive and conjunctive normal forms of the OR operator for presented fuzzy measure are used as the lower and upper bounds of this interval, respectively. To evaluate the performance of the proposed system, a comparison against some other systems is conducted for 10 data sets. Further, it is applied to a practical problem of classification of generation units’ bidding behavior in the Iranian wholesale electricity market. Experimental results display the superior performance of the proposed system.

Towards interval uncertainty propagation control in bivariate aggregation processes and the introduction of width-limited interval-valued overlap functions

Overlap functions are a class of aggregation functions that measure the overlapping degree between two values. They have been successfully applied as a fuzzy conjunction operation in several problems in which associativity is not required, such as image processing and classification. Interval-valued overlap functions were defined as an extension to express the overlapping of interval-valued data, and they have been usually applied when there is uncertainty regarding the assignment of membership degrees, as in interval-valued fuzzy rule-based classification systems. In this context, the choice of a total order for intervals can be significant, which motivated the recent developments on interval-valued aggregation functions and interval-valued overlap functions that are increasing to a given admissible order, that is, a total order that refines the usual partial order for intervals. Also, width preservation has been considered on these recent works, in an intent to avoid the uncertainty increase and guarantee the information quality, but no deeper study was made regarding the relation between the widths of the input intervals and the output interval, when applying interval-valued functions, or how one can control such uncertainty propagation based on this relation. Thus, in this paper we: (i) introduce and develop the concepts of width-limited interval-valued functions and width limiting functions, presenting a theoretical approach to analyze the relation between the widths of the input and output intervals of bivariate interval-valued functions, with special attention to interval-valued aggregation functions; (ii) introduce the concept of (a,b)-ultramodular aggregation functions, a less restrictive extension of one-dimension convexity for bivariate aggregation functions, which have an important predictable behaviour with respect to the width when extended to the interval-valued context; (iii) define width-limited interval-valued overlap functions, taking into account a function that controls the width of the output interval and a new notion of increasingness with respect to a pair of partial orders (≤1,≤2); (iv) present and compare three construction methods for these width-limited interval-valued overlap functions, considering a pair of orders (≤1,≤2), which may be admissible or not, showcasing the adaptability of our developments.

A fuzzy association rule-based classifier for imbalanced classification problems

Imbalanced classification problems are attracting the attention of the research community because they are prevalent in real-world problems and they impose extra difficulties for learning methods. Fuzzy rule-based classification systems have been applied to cope with these problems, mostly together with sampling techniques. In this paper, we define a new fuzzy association rule-based classifier, named FARCI, to tackle directly imbalanced classification problems. Our new proposal belongs to the algorithm modification category, since it is constructed on the basis of the state-of-the-art fuzzy classifier FARC–HD. Specifically, we modify its three learning stages, aiming at boosting the number of fuzzy rules of the minority class as well as simplifying them and, for the sake of handling unequal fuzzy rule lengths, we also change the matching degree computation, which is a key step of the inference process and it is also involved in the learning process. In the experimental study, we analyze the effectiveness of each one of the new components in terms of performance, F-score, and rule base size. Moreover, we also show the superiority of the new method when compared versus FARC–HD alongside sampling techniques, another algorithm modification approach, two cost-sensitive methods and an ensemble.

Business Intelligence in Airline Passenger Satisfaction Study—A Fuzzy-Genetic Approach with Optimized Interpretability-Accuracy Trade-Off

The main objective and contribution of this paper is the application of our knowledge-discovery business-intelligence technique (fuzzy rule-based classification systems) characterized by genetically optimized interpretability-accuracy trade-off (using multi-objective evolutionary optimization algorithms) to decision support related to airline passenger satisfaction problems. Recently published and accessible at Kaggle’s repository airline passengers satisfaction data set containing 259,760 records is used in our experiments. A comparison of our approach with an alternative method (using SAS-system’s accuracy-oriented prediction tools to determine the attribute importance hierarchy) is also performed showing the advantages of our method in terms of: (i) discovering the actual hierarchy of attribute significance for passenger satisfaction and (ii) knowledge-discovery system’s interpretability-accuracy trade-off optimization. The main results and findings of our work include: (i) an introduction of the modern fuzzy-genetic business-intelligence solution characterized both by high interpretability and high accuracy to the airline passenger satisfaction decision support, (ii) an analysis of the effect of possible "overlapping" of some input attributes over the other ones in order to discover the real hierarchy of influence of particular input attributes upon the airline passengers satisfaction, and (iii) an extended cross-validation experiment confirming high effectiveness of our approach for different learning-test splits of the data set considered.

A wrapper methodology to learn interval-valued fuzzy rule-based classification systems

Learning an interval-valued fuzzy rule-based classification system is a challenge as its success directly depends on the interval-valued fuzzy partition used. In fact, the learning of an interval-valued fuzzy system usually starts by creating a partition composed of numerical fuzzy sets, which are used to build an initial fuzzy classifier. Then, it is augmented with interval-valued fuzzy sets whose shape is subsequently optimized to improve the system’s performance. However, as in this methodology the fuzzy rules are learned using numerical fuzzy sets, the benefits of the interval-valued fuzzy sets may not be fully exploited. In this paper we define a new learning methodology that avoids building the initial fuzzy classifier but directly learns interval-valued fuzzy rules. To do so, we define a wrapper methodology to learn the interval-valued fuzzy partitions such that they lead to an interval-valued fuzzy rule-based classification system as accurate as possible. Moreover, our new method allows one to represent each membership function using the most proper type of fuzzy set for the sake of modeling the uncertainty in the best possible manner. Consequently, the antecedents of the rules can be formed of only numerical fuzzy sets, only interval-valued fuzzy sets or a mixture of both. The quality of the proposal is compared versus four state-of-the-art fuzzy classifiers like FARC-HD, IVTURS, FURIA and FARC-HD using an inference based on a generalization of the Choquet integral. We also compare our new approach besides its numerical fuzzy counterpart to clearly show the benefits of the usage of interval-valued fuzzy sets. Specifically, the average accuracy rate of our new method is 81.17%, which is at least 0.66% better than the remainder state-of-the-art fuzzy classifiers.

Intrusion detection system using a new fuzzy rule-based classification system based on genetic algorithm

Intrusion can compromise the integrity, confidentiality, or availability of a computer system. Intrusion Detection System (IDS) is a type of security software designed to monitor network traffic and identify network intrusions. In this paper, A Fuzzy Rule – Based classification system is used to detect intrusion in a computer network. In order to improve the classification rate, a new method is proposed based on Genetic Algorithm (GA) for rule weights specification. The proposed method is tested on KDD99 dataset. Experimental results show the proposed method improves the performance of the fuzzy rule-based classification systems in terms of detection rate and false alarm rate.

A Modern Data-Mining Approach Based on Genetically Optimized Fuzzy Systems for Interpretable and Accurate Smart-Grid Stability Prediction

The main objective and contribution of this paper was/is the application of our knowledge-based data-mining approach (a fuzzy rule-based classification system) characterized by a genetically optimized interpretability-accuracy trade-off (by means of multi-objective evolutionary optimization algorithms) for transparent and accurate prediction of decentral smart grid control (DSGC) stability. In particular, we aim at uncovering the hierarchy of influence of particular input attributes upon the DSGC stability. Moreover, we also analyze the effect of possible "overlapping" of some input attributes over the other ones from the DSGC-stability perspective. The recently published and available at the UCI Database Repository Electrical Grid Stability Simulated Data Set and its input-aggregate-based concise version were used in our experiments. A comparison with 39 alternative approaches was also performed, demonstrating the advantages of our approach in terms of: (i) interpretable and accurate fuzzy rule-based DSGC-stability prediction and (ii) uncovering the hierarchy of DSGC-system’s attribute significance.

General interval-valued overlap functions and interval-valued overlap indices

Overlap functions are aggregation functions that express the overlapping degree between two values. They have been used both as a conjunction in several practical problems (e.g., image processing and decision making), and to generate overlap indices between two fuzzy sets, which can be used to construct fuzzy confidence values to be applied in fuzzy rule based classification systems. Some generalizations of overlap functions were recently proposed, such as n-dimensional and general overlap functions, which allowed their application in n-dimensional problems. More recently, the concept of interval-valued overlap functions was presented, mainly to deal with uncertainty in providing membership functions. In this paper, we introduce: (i) the concept of n-dimensional interval-valued overlap functions, studying their representability, (ii) the definition of general interval-valued overlap functions, providing their characterization and some construction methods. Moreover, we also define the concept of interval-valued overlap index, as well as some constructing methods. In addition, we show an illustrative example where those new concepts are applied.

Infrared Thermography Based Defects Testing of Solar Photovoltaic Panel with Fuzzy Rule-Based Evaluation

Infrared Thermography has been used as a tool for predictive and preventive maintenance of Photovoltaic panels. International Electrotechnical Commission provides some guidelines for using thermography to detect defects in Photovoltaic panels. However, the proposed guidelines focus only on the location of the hot spot than diagnosing the types of faults. The long-term reliability and efficiency of panels can be affected by progressive defects such as discolouring and delamination. This paper proposed the new Thermal Pixel Counting algorithm to detect the above faults based on three thermal profile index values. The real-time experimental testing was carried out using FLIR T420bx® thermal imager and results have been provided to validate the proposed method. In this work, the fuzzy rule-based classification system is proposed to automate the classification process. Fuzzy reasoning method based on a single winner rule fuzzy classifier is designed with modified rule weights by particular grade. The performance of the proposed classifier is compared with the conventional fuzzy classifier and neural network model.

A parallel fuzzy rule-base based decision tree in the framework of map-reduce

Decision trees are commonly used for learning and extracting classification rules from data. The fuzzy rule based decision tree (FRDT) is very representative owing to its better robustness and generalization. However, FRDT cannot work well on the analysis of large-scale data sets. One solution for this problem is parallel computing. A proved effective parallel computing model is Map-Reduce. Ensemble learning is an effective strategy which can significantly improve the generalization ability of machine learning systems. The objective of this paper is to develop a fuzzy rule-base based decision tree on the strategies of parallel computing and ensemble learning. First, we implement a parallel fusing fuzzy rule based classification system via Map-Reduce (MR-FFRCS) to display how to extract fuzzy rules from data in parallel and how to evaluate the fuzzy rules in an ensemble learning way. Then, taking MR-FFRCS as a fundamental module, we propose a parallel fuzzy rule-base based decision tree (MR-FRBDT) to improve the original FRDT algorithm. The experimental studies mainly focus on feasibility and parallelism. Compared with FRDT on 23 UCI benchmark data sets, the proposed MR-FRBDT algorithm with fewer parameters is effective and has the ability to handle large-scale data sets. Furthermore, some numerical experiments conducted on several large-scale data sets verify the parallel performance on reducing computing time and avoiding memory restrictions.