Fuzzy Rule-based Classification Systems Research Articles

CF-integrals: A new family of pre-aggregation functions with application to fuzzy rule-based classification systems

This paper introduces the family of CF-integrals, which are pre-aggregations functions that generalizes the Choquet integral considering a bivariate function F that is left 0-absorbent. We show that CF-integrals are 1→-pre-aggregation functions, studying in which conditions they are idempotent and/or averaging functions. This characterization is an important issue of our approach, since we apply these functions in the Fuzzy Reasoning Method (FRM) of a fuzzy rule-based classification system and, in the literature, it is possible to observe that non-averaging aggregation functions usually provide better results. We carry out a study with several subfamilies of CF-integrals having averaging or non-averaging characteristics. As expected, the proposed non-averaging CF-integrals obtain more accurate results than the averaging ones, thus, offering new possibilities for aggregating accurately the information in the FRM. Furthermore, it allows us to enhance the results of classical FRMs like the winning rule and the additive combination.

CHI-BD: A fuzzy rule-based classification system for Big Data classification problems

The previous Fuzzy Rule-Based Classification Systems (FRBCSs) for Big Data problems consist in concurrently learning multiple Chi et al. FRBCSs whose rule bases are then aggregated. The problem of this approach is that different models are obtained when varying the configuration of the cluster, becoming less accurate as more computing nodes are added. Our aim with this work is to design a new FRBCS for Big Data classification problems (CHI-BD) which is able to provide exactly the same model as the one that would be obtained by the original Chi et al. algorithm if it could be executed with this quantity of data. In order to do so, we take advantage of the suitability of the Chi et al. algorithm for the MapReduce paradigm, solving the problems of the previous approach, which lead us to obtain the same model (i.e., classification accuracy) regardless of the number of computing nodes considered.

RST-BatMiner: A fuzzy rule miner integrating rough set feature selection and Bat optimization for detection of diabetes disease

Fuzzy classification rules are more interpretable and cope better with pervasive uncertainty and vagueness with respect to crisp rules. Because of this fact, fuzzy classification rules are extensively used in classification and decision support systems for disease diagnosis. But, most of the rule mining techniques failed to generate accurate and comprehensive fuzzy rules. This paper presents a hybrid decision support system based on Rough Set Theory (RST) and Bat optimization Algorithm (BA) called RST-BatMiner. It consists of two stages. In the first stage, redundant features have been removed from the data set through RST-based QUICK-REDUCT approach. In the second stage, for each class BA is invoked to generate fuzzy rules by minimizing proposed fitness function. Further, an Ada-Boosting technique is applied to the rules generated by BA to increase the accuracy rate of generated fuzzy rules. Moreover, to generate comprehensive fuzzy rules, a new ≠ (not equal) operator along with = (equal) operator is introduced into BA encoding scheme. The proposed RST-BatMiner builds consolidated fuzzy ruleset by learning the rules associated with each class separately. The proposed RST-BatMiner is experimented on six bench-mark datasets namely Pima Indians Diabetes, Wisconsin Breast Cancer, Cleveland Heart disease, iris, wine and glass, in order to validate its generalization capability. These experimental results show that except for wine dataset the proposed RST-BatMiner yields high accuracy and comprehensible ruleset when compared to other state-of-the-art bio-inspired based fuzzy rule miners and Fuzzy Rule Based Classification Systems (FRBCS) in the literature. In the case of wine dataset, the proposed RST-BatMiner yields second highest accuracy along with comprehensible ruleset.

Consensus via penalty functions for decision making in ensembles in fuzzy rule-based classification systems

The aim of this paper is to propose a consensus method via penalty functions for decision making in ensembles of fuzzy rule-based classification systems (FRBCSs). For that, we first introduce a method based on overlap indices for building confidence and support measures, which are usually used to evaluate the degree of certainty or interest of a certain association rule. Those overlap indices (a generalizations of the Zadeh's consistency index between two fuzzy sets) are built using overlap functions, which are a special kind of non necessarily associative aggregation functions proposed for applications related to the overlap problem and/or when the associativity property is not demanded. Then, we introduce a new FRM for the FRBCS, considering different overlap indices, which generalizes the classical methods. By considering several overlap indices and aggregation functions, we generate fuzzy rule-based ensembles, providing different results. For the decision making related to the selection of the best class, we introduce a consensus method for classification, based on penalty functions. We also present theoretical results related to the developed methods. A detailed example of a generation of fuzzy rule-based ensembles based on the proposed approach, and the decision making by consensus via penalty functions, is presented.

An Ensemble Classifier Adopting Random Subspace Method based on Fuzzy Partial Mining

Objectives: Ensemble classification with fuzzy partial mining is a novel approach. The random subspace ensemble classifier contains several classifiers working on original attribute space. The aim of this paper is to examine the appropriateness of the random subspace ensemble method for fuzzy partial mining classification and thereby develop an algorithm Ensemble Classification on Fuzzy Utility Mining (ECFUM) by using a skill utility measure in addition to Support and Confidence. Methods/Statistical Analysis: The algorithm show high accuracy with ensemble classifier than solitary classifiers. The classifier is trained on random subspace method which is suitable when there is more number of attributes for the classification, where in many of the fuzzy rule based classification systems suffer increase in dimensionality. Findings: The unique integration of ensemble classification with fuzzy partial weighted mining generates Fuzzy Association Rules and Class Association Rules. Fuzzy association rules have been generated which holds the attributes association. Class association rules have been generated which holds the target class for the attribute association. The resultant classifier produced, shows credible results with better accuracy. ECFUM generates more number of hidden interesting rules compared to traditional associative classifiers. These hidden rules play a major role in later prediction of the algorithm. Improvements: Future work concentrates on the role of infinite sampling on class association rule with higher order confidence precedence to standardize the predictive power of the algorithm ECFUM.

An Expert System for Diagnosis of Sleep Disorder Using Fuzzy Rule-Based Classification Systems

Sleep disorder is an anomaly that could cause problems for someone’ sleeping pattern. Nowadays, it becomes an issue since people are getting busy with their own business and have no time to visit the doctors. Therefore, this research aims to develop a system used for diagnosis of sleep disorder using Fuzzy Rule-Based Classification System (FRBCS). FRBCS is a method based on the fuzzy set concepts. It consists of two steps: (i) constructing a model/knowledge involving rulebase and database, and (ii) prediction over new data. In this case, the knowledge is obtained from experts whereas in the prediction stage, we perform fuzzification, inference, and classification. Then, a platform implementing the method is built with a combination between PHP and the R programming language using the “Shiny” package. To validate the system that has been made, some experiments have been done using data from a psychiatric hospital in West Java, Indonesia. Accuracy of the result and computation time are 84.85% and 0.0133 seconds, respectively.

Why Linguistic Fuzzy Rule Based Classification Systems perform well in Big Data Applications?

The significance of addressing Big Data applications is beyond all doubt. The current ability of extracting interesting knowledge from large volumes of information provides great advantages to both corporations and academia. Therefore, researchers and practitioners must deal with the problem of scalability so that Machine Learning and Data Mining algorithms can address Big Data properly. With this end, the MapRe-duce programming framework is by far the most widely used mechanism to implement fault-tolerant distributed applications. This novel framework implies the design of a divide-and-conquer mechanism in which local models are learned separately in one stage (Map tasks) whereas a second stage (Reduce) is devoted to aggregate all sub-models into a single solution. In this paper, we focus on the analysis of the behavior of Linguistic Fuzzy Rule Based Classification Systems when embedded into a MapRe-duce working procedure. By retrieving different information regarding the rules learned throughout the MapReduce process, we will be able to identify some of the capabilities of this particular paradigm that allowed them to provide a good performance when addressing Big Data problems. In summary, we will show that linguistic fuzzy classifiers are a robust approach in case of scalability requirements.

EEG rhythm/channel selection for fuzzy rule-based alertness state characterization

The aim of the paper is to automatically select the optimal EEG rhythm/channel combinations capable of classifying human alertness states. Four alertness states were considered, namely ‘engaged’, ‘calm’, ‘drowsy’ and ‘asleep’. The features used in the automatic selection are the energies associated with the conventional rhythms, \(\delta , \theta , \alpha , \beta\) and \(\gamma\), extracted from overlapping windows of the different EEG channels. The selection process consists of two stages. In the first stage, the optimal brain regions, represented by sets of EEG channels, are selected using a simple search technique based on support vector machine (SVM), extreme learning machine (ELM) and LDA classifiers. In the second stage, a fuzzy rule-based alertness classification system (FRBACS) is used to identify, from the previously selected EEG channels, the optimal features and their supports. The IF–THEN rules used in FRBACS are constructed using a novel differential evolution-based search algorithm particularly designed for this task. Each alertness state is represented by a set of IF–THEN rules whose antecedent parts contain EEG rhythm/channel combination. The selected spatio-frequency features were found to be good indicators of the different alertness states, as judged by the classification performance of the FRBACS that was found to be comparable to those of the SVM, ELM and LDA classifiers. Moreover, the proposed classification system has the advantage of revealing simple and easy to interpret decision rules associated with each of the alertness states.

CC-integrals: Choquet-like Copula-based aggregation functions and its application in fuzzy rule-based classification systems

This paper introduces the concept of Choquet-like Copula-based aggregation function (CC-integral) and its application in fuzzy rule-based classification systems. The standard Choquet integral is expanded by distributing the product operation. Then, the product operation is generalized by a copula. Unlike the generalization of the Choquet integral by t-norms using its standard form (i.e., without distributing the product operator), which results in a pre-aggregation function, the CC-integral satisfies all the conditions required for an aggregation function. We build some examples of CC-integrals considering different examples of copulas, including t-norms, overlap functions and copulas that are neither t-norms nor overlap functions. We show that the CC-integral based on the minimum t-norm, when applied in fuzzy rule-based classification systems, obtains a performance that is, with a high level of confidence, better than that which adopts the winning rule (maximum). We concluded that the behavior of CC-integral is similar to the best Choquet-like pre-aggregation function. Consequently, the CC-integrals introduced in this paper can enlarge the scope of the applications by offering new possibilities for defining fuzzy reasoning methods with a similar gain in performance.

Interpretable and accurate medical data classification – a multi-objective genetic-fuzzy optimization approach

In medical decision support systems, both the accuracy (i.e., the ability to adequately represent the decision making processes) as well as the transparency and interpretability (i.e., the ability to provide a domain user with compact and understandable explanation and justification of the proposed decisions) play essential roles. This paper presents an approach for automatic design of fuzzy rule-based classification systems (FRBCSs) from medical data using multi-objective evolutionary optimization algorithms (MOEOAs). Our approach generates, in a single run, a collection of solutions (medical FRBCSs) characterized by various levels of accuracy-interpretability trade-off. We propose a new complexity-related interpretability measure and we address the semantics-related interpretability issue by means of efficient implementation of the so-called strong fuzzy partitions of attribute domains. We also introduce a special-coding-free representation of the rule base and original genetic operators for its processing as well as we implement our ideas in the context of well-known and one of the presently most advanced MOEOAs, i.e., Non-dominated Sorting Genetic Algorithm II (NSGA-II). An important part of the paper is devoted to a broad comparative analysis of our approach and as many as 26 alternative techniques arranged in 32 experimental set-ups and applied to three well-known benchmark medical data sets (Breast Cancer Wisconsin (Original), Pima Indians Diabetes, and Heart Disease (Cleveland)) available from the UCI repository of machine learning databases (http://archive.ics.uci.edu/ml). A number of useful in medical applications performance measures including accuracy, sensitivity, specificity, and several interpretability measures are employed. The results of such a broad comparative analysis demonstrate that our approach significantly outperforms the alternative methods in terms of the interpretability of the obtained FRBCSs while remaining either competitive or superior in terms of their accuracy. It is worth stressing that the overwhelming majority of the existing medical classification methods concentrate almost exclusively on the accuracy issues.

An integer programming approach for fuzzy rule-based classification systems

Fuzzy rule-based classification systems (FRBCSs) have been successfully employed as a data mining technique where the goal is to discover the hidden knowledge in a data set in the form of interpretable rules and develop an accurate classification model. In this paper, we propose an exact approach to learn fuzzy rules from a data set for a FRBCS. First, we propose a mixed integer programming model that extracts optimal fuzzy rules from a data set. The model’s embedded feature selection allows absence of insignificant features in a fuzzy rule in order to enhance its accuracy and coverage. In order to build a comprehensive Rule Base (RB), we use this model in an iterative procedure that finds multiple rules by converting the obtained optimal solutions into a set of taboo constraints that prevents the model from re-finding the previously obtained rules. Furthermore, it changes the search direction by temporarily removing the correctly predicted patterns from the training set aiming to find the optimal rules that predict uncovered patterns in the training set. This procedure ensures that most of the patterns in the training set are covered by the RB. Next, another mixed integer programming model is developed to maximize predictive accuracy of the classifier by pruning the RB and removing redundant rules. The predictive accuracy of the proposed model is tested on the benchmark data sets and compared with the state-of-the-art algorithms from the literature by non-parametric statistical tests.

Fuzzy rule based classification systems for big data with MapReduce: granularity analysis

Due to the vast amount of information available nowadays, and the advantages related to the processing of this data, the topics of big data and data science have acquired a great importance in the current research. Big data applications are mainly about scalability, which can be achieved via the MapReduce programming model.It is designed to divide the data into several chunks or groups that are processed in parallel, and whose result is "assembled" to provide a single solution. Among different classification paradigms adapted to this new framework, fuzzy rule based classification systems have shown interesting results with a MapReduce approach for big data. It is well known that the performance of these types of systems has a strong dependence on the selection of a good granularity level for the Data Base. However, in the context of MapReduce this parameter is even harder to determine as it can be also related with the number of Maps chosen for the processing stage. In this paper, we aim at analyzing the interrelation between the number of labels of the fuzzy variables and the scarcity of the data due to the data sampling in MapReduce. Specifically, we consider that as the partitioning of the initial instance set grows, the level of granularity necessary to achieve a good performance also becomes higher. The experimental results, carried out for several Big Data problems, and using the Chi-FRBCS-BigData algorithms, support our claims.

DECO3R: A Differential Evolution-based algorithm for generating compact Fuzzy Rule-based Classification Systems

In this paper a novel Genetic Fuzzy Rule-based Classification System, named DECO3R (Differential Evolution based Cooperative and Competing learning of Compact FRBCS), is proposed. DECO3R follows the genetic cooperative - competitive learning (GCCL) approach and uses Differential Evolution as its learning algorithm. In this frame, every chromosome encodes a single fuzzy rule. The proposed AdaBoost-based Fuzzy Token Competition (FTC) method is employed to deal with the cooperation - competition problem, an integral part to all GCCL algorithms. DECO3R learns clear, precise and predictive rules where the fuzzy sets in the premise part are consecutive. The experimental component analysis demonstrates that DE as a learning algorithm outperforms a simple Genetic Algorithm. Additionally, the novel FTC method exceeds the performance of other similar techniques. The experimental comparative analysis highlights the robust performance of DECO3R compared to other rule learning algorithms, both in terms of accuracy and of structural complexity.

Preaggregation Functions: Construction and an Application

In this paper, we introduce the notion of preaggregation function. Such a function satisfies the same boundary conditions as an aggregation function, but, instead of requiring monotonicity, only monotonicity along some fixed direction (directional monotonicity) is required. We present some examples of such functions. We propose three different methods to build preaggregation functions. We experimentally show that in fuzzy rule-based classification systems, when we use one of these methods, namely, the one based on the use of the Choquet integral replacing the product by other aggregation functions, if we consider the minimum or the Hamacher product t-norms for such construction, we improve the results obtained when applying the fuzzy reasoning methods obtained using two classical averaging operators such as the maximum and the Choquet integral.

A reinforcement learning algorithm for adjusting antecedent parameters and weights of fuzzy rules in a fuzzy classifier

This paper proposes a new fuzzy classifier based on reinforcement learning. A fuzzy rule based classification system is a special type of fuzzy modeling where its output is a discrete crisp value. The main challenging issue in designing fuzzy classifiers is constructing fuzzy rule base. Here, each fuzzy rule is considered as an agent who has to select the suitable class between candidate classes. It is considered a weight for each candidate class in each rule. These weights are adjusted using the proposed reinforcement learning algorithm. For each sample of training data, if the final result is true, the winner rule (agent) is rewarded and some other rules are punished based on the criteria which are defined in this paper. If the result is false, the winner rule is punished and the rules with high firing strength that have selected correct class are rewarded. Moreover, the input membership functions of rules are adjusted regarding the defined criteria which depend on punishment frequency of rules. The proposed approach is assessed on some UCI datasets. We compare our ideas in comparison with conventional reward and punishment scheme and multi-layer perceptron network. The experimental results show that our proposed approach outperforms both mentioned approaches in the terms of quality of classification and precision.

Competitive interaction reasoning: A bio-inspired reasoning method for fuzzy rule based classification systems

In designing fuzzy rule based classification systems (FRBCSs), complex fuzzy rule extraction techniques and tuning membership functions are frequently used to enhance classification accuracy. However, these approaches not only decrease system's transparency which is the hallmark of fizzy design, but also are oftentimes computationally expensive and require multiple parameters to be optimized using a large amount of training data. In this paper, inspired by the so-called competitive behavior of mini-columns in the brain neuronal circuitry, we proposed a new reasoning method for fuzzy classifiers referred to as Competitive Interaction Reasoning (CIR) that employs the cumulative information provided by all fuzzy rules and adjusts the decision boundaries as if the membership functions are directly modified. This mechanism is mathematically implemented by a linear transformation and resembles the competitive interaction observed in brain neuronal columns. Cross-rule competition weights are optimized using Hebbian reinforcement learning. Using a large number of simulations on benchmark data sets, we show that the proposed CIR significantly improves classification accuracy without compromising interpretability of the fuzzy classifier. In addition, CIR can further facilitate formation of the fuzzy rules and incorporation of the expert knowledge by confining the destructive effects of noisy rules or expert inconsistencies. Experiments on 23 well-known benchmark data sets confirm high performance of CIR in comparison with a number of popular classifiers.

Enhancing evolutionary fuzzy systems for multi-class problems: Distance-based relative competence weighting with truncated confidences (DRCW-TC)

Classification problems with multiple classes suppose a challenge in Data Mining tasks. There is a difficulty inherent to the learning process when trying to find the most adequate discrimination functions among the different concepts within the dataset. Using Fuzzy Rule Based Classification Systems in general, and Evolutionary Fuzzy Systems in particular, provide the advantage of describing smoother borderline areas, thanks to the linguistic label-based representation.In multi-classification, the pairwise learning approach (One-vs-One) has gained a notorious attention. However, there is certain dependence between the goodness of the confidence degrees or scores of binary classifiers, and the final performance shown by the global model. Regarding this fact, the problem of non-competent classifiers is of special relevance. It occurs when a binary classifier outputs a positive score for a couple of classes unrelated with the input example, which may degrade the final accuracy. Precisely, the previously exposed properties of fuzzy classifiers make them more prone to the former condition.In this paper, we propose an extension of the distance-based combination strategy to overcome this non-competence problem. It is based on the truncation of the confidence degrees of the classes prior to the distance-based tuning. This allows taking advantage of the good classification abilities of Evolutionary Fuzzy Systems, while diminishing the adverse effect of the aforementioned non-competence. Experimental results, using FARC-HD with overlap functions as the fuzzy learning algorithm, show that this new adaptation of the Distance-based Relative Competence Weighting model outperforms both the OVO and standard distance-based approaches, and it is competitive with robust classifiers such as Support Vector Machines.

Capacities and overlap indexes with an application in fuzzy rule-based classification systems

In this work, we introduce a method for constructing capacities using overlap indexes between the fuzzy sets which are generated from the inputs of the considered problem. We also use these capacities to aggregate information by means of the Choquet integral in a fuzzy rule-based classifier. We observe that with these capacities the obtained results are better than those obtained with other measures.

Review of Fuzzy Rule Based Classification systems

Fuzzy Rule Based Classification systems (FRBCs) have received significant attention among the researchers due to the good behaviour in the real time databases. An important issue in the design of fuzzy rule-based classification system is the optimized generation of fuzzy if-then rules and the membership functions. The inductive learning of fuzzy rule classifier suffers in rule generation and rule optimization when the search space or variables becomes high. This creates the new idea of making the fuzzy system with precise rules leading to less scalability and improved accuracy. Accordingly, different approaches have been presented in the literature for optimal finding of fuzzy rules using optimization algorithms. Among the different techniques available in the literature, choosing the type, number of membership functions and defining parameters of membership function are still challenging tasks. In this paper, the optimization algorithms for optimal design of membership function and optimal rule generation are reviewed.

Detection of SQL Injection Using a Genetic Fuzzy Classifier System

SQL Injection (SQLI) is one of the most popular vulnerabilities of web applications. The consequences of SQL injection attack include the possibility of stealing sensitive information or bypassing authentication procedures. SQL injection attacks have different forms and variations. One difficulty in detecting malicious attacks is that such attacks do not have a specific pattern. A new fuzzy rule-based classification system (FBRCS) can tackle the requirements of the current stage of security measures. This paper proposes a genetic fuzzy system for detection of SQLI where not only the accuracy is a priority, but also the learning and the flexibility of the obtained rules. To create the rules having high generalization capabilities, our algorithm builds on initial rules, data-dependent parameters, and an enhancing function that modifies the rule evaluation measures. The enhancing function helps to assess the candidate rules more effectively based on decision subspace. The proposed system has been evaluated using a number of well-known data sets. Results show a significant enhancement in the detection procedure