Fuzzy Rule-based Classification Research Articles

Surrogate-Assisted Multi-objective Genetic Algorithms for Fuzzy Rule-Based Classification

In this paper, we present a surrogate-assisted multi-objective genetic algorithm to mine a small number of linguistically interpretable fuzzy rules for high-dimensional classification task in the realm of data mining. However, the difficulties like (1) handling of high-dimensional problems by fuzzy rule-based systems (i.e., the exponential increase in the number of fuzzy rules with the number of input variables), (2) the deterioration in the comprehensibility of fuzzy rules when they involve many antecedent conditions, and (3) the optimization of multiple objectives in fuzzy rule-based system may stand as pertinent issues. Hence, to combat with the aforesaid issues, we design the problem as a combinatorial optimization problem with three objectives: to maximize the number of correctly classified training patterns, minimize the number of fuzzy rules, and minimize the total number of antecedent conditions. We optimize these objectives of the fuzzy rule-based system by using a multi-objective genetic algorithm. Further to reduce the number of required evaluations to approximate the Pareto front of computationally expensive multi-objective genetic algorithm for fuzzy rule-based system, a radial basis neural network surrogate model is adapted. This approach searches for non-dominated rule sets with respect to these three objectives. The performance of the surrogate-assisted model is evaluated through a few benchmark datasets obtained from knowledge extraction based on evolutionary learning data repository. The experimental outcome confirm that this model is competitive compared to the non-surrogate-assisted model. However, the performance of the model has drawn a clear edge over rule mining approaches like Decision Table, JRip, OneR, PART, and ZeroR.

Fuzzy classifiers with information granules in feature space and logic-based computing

Fuzzy classifiers have been studied in the area of fuzzy sets for a long time resulting in a number of architectures. In this study, we thoroughly investigate and critically assess fuzzy rule-based classifiers. A topology of the classifier is discussed along with a discussion of the role of fuzzy set technology in the construction of condition and conclusion parts of the classification rules. Some optimization mechanisms utilized in the adjustment of information granules forming the rules are presented. Performance of the fuzzy classifiers is quantified in terms of their accuracy and an area under curve (AUC) determined for the receiver operating characteristics (ROC). The performance of the classifier is evaluated vis-à-vis a collection of triangular norms used in the construction of the fuzzy classifiers. Experimental studies involve synthetic and publicly available data. Furthermore, comparative studies include the experiments with the commonly used non-fuzzy classifiers.

Self-organising fuzzy logic classifier

In this paper, we present a self-organising nonparametric fuzzy rule-based classifier. The proposed approach identifies prototypes from the observed data through an offline training process and uses them to build a 0-order AnYa type fuzzy rule-based system for classification. Once primed offline, it is able to continuously learn from the streaming data afterwards to follow the changing data pattern by updating the system structure and meta-parameters recursively. The meta-parameters of the proposed approach are derived from data directly. By changing the level of granularity, the proposed approach can make a trade-off between performance and computational efficiency, and, thus, the classifier is able to address a wide variety of problems with specific needs. The classifier also supports different types of distance measures. Numerical examples based on benchmark datasets demonstrate the high performance of the proposed approach and its ability of handling high-dimensional, complex, large-scale problems.

CHI-PG: A fast prototype generation algorithm for Big Data classification problems

Abstract The growing amount of available data has become a serious challenge to data mining and machine learning techniques. Well-known classification methods that have been widely applied so far are no longer feasible in Big Data environments. For this reason, prototype reduction techniques (both selection and generation) come up as a candidate solution to build a reduced version of the dataset that speeds up the execution of algorithms such as k-Nearest Neighbors and overcome their memory constraints. However, these solutions generally have a quadratic O(N2) time complexity and share similar limitations to those encountered in data mining and machine learning algorithms in terms of time and memory requirements. In order to overcome these limitations, we introduce a new distributed MapReduce prototype generation method called CHI-PG that provides a linear O(N) time complexity and ensures constant accuracy regardless of the degree of parallelism. This approach builds prototypes by applying a simple scheme based on the rule generation process of the Chi et al. Fuzzy Rule-Based Classification System and takes advantage of the suitability of this classifier for the MapReduce paradigm. The empirical study shows that our new approach significantly improves the execution time of a state-of-the-art distributed prototype reduction algorithm (MRPR) without decreasing (and even improving) classification accuracy and reduction rates. Moreover, CHI-PG has been shown to be a candidate solution to the time and memory constraints of k-Nearest Neighbors when tackling large-scale datasets.

On Distributed Fuzzy Decision Trees for Big Data

Fuzzy decision trees (FDTs) have shown to be an effective solution in the framework of fuzzy classification. The approaches proposed so far to FDT learning, however, have generally neglected time and space requirements. In this paper, we propose a distributed FDT learning scheme shaped according to the MapReduce programming model for generating both binary and multiway FDTs from big data. The scheme relies on a novel distributed fuzzy discretizer that generates a strong fuzzy partition for each continuous attribute based on fuzzy information entropy. The fuzzy partitions are, therefore, used as an input to the FDT learning algorithm, which employs fuzzy information gain for selecting the attributes at the decision nodes. We have implemented the FDT learning scheme on the Apache Spark framework. We have used ten real-world publicly available big datasets for evaluating the behavior of the scheme along three dimensions: 1) performance in terms of classification accuracy, model complexity, and execution time; 2) scalability varying the number of computing units; and 3) ability to efficiently accommodate an increasing dataset size. We have demonstrated that the proposed scheme turns out to be suitable for managing big datasets even with a modest commodity hardware support. Finally, we have used the distributed decision tree learning algorithm implemented in the MLLib library and the Chi-FRBCS-BigData algorithm, a MapReduce distributed fuzzy rule-based classification system, for comparative analysis.

CF-integrals: A new family of pre-aggregation functions with application to fuzzy rule-based classification systems

This paper introduces the family of CF-integrals, which are pre-aggregations functions that generalizes the Choquet integral considering a bivariate function F that is left 0-absorbent. We show that CF-integrals are 1→-pre-aggregation functions, studying in which conditions they are idempotent and/or averaging functions. This characterization is an important issue of our approach, since we apply these functions in the Fuzzy Reasoning Method (FRM) of a fuzzy rule-based classification system and, in the literature, it is possible to observe that non-averaging aggregation functions usually provide better results. We carry out a study with several subfamilies of CF-integrals having averaging or non-averaging characteristics. As expected, the proposed non-averaging CF-integrals obtain more accurate results than the averaging ones, thus, offering new possibilities for aggregating accurately the information in the FRM. Furthermore, it allows us to enhance the results of classical FRMs like the winning rule and the additive combination.

Multicriteria Design of Cost-Conscious Fuzzy Rule-Based Classifiers

Many real-world classification systems must comply with a series of inherent restrictions to the problem at hand such as response times, power consumptions or computational costs. This poses a fundamental limitation to traditional performance-driven classifiers and learning algorithms by restraining their applicability in cost-sensitive scenarios. Because of this, fuzzy systems are leveraged to learn cost-conscious multi-stage classifiers through multiobjective optimization to find a set of optimal tradeoffs between accuracy and any related cost. This approach allows find a suitable balance between all objectives regardless of the scenario. Experimental evaluations were done for Sound Environment Classification in modern battery-powered hearing aids by jointly optimising classification accuracy and computational costs.

T-operators in hesitant fuzzy sets and their applications to fuzzy rule-based classifier

Extending of the T-operators on hesitant fuzzy sets (HFSs) is important in the theory and applications. In this paper, we used two comparative operators for expressing monotonicity of the T-operators on hesitant fuzzy elements (HFEs). Hence, properties of some T-operators for Xu-Xia partial order (≤ℍ(m)) and Xia-Xu order (⪯) are studied on typical hesitant fuzzy elements (THFEs). Finally, an application of T-operators on HFEs has been explained in fuzzy rule-based systems for a practical problem in economics. Also, for a conceptual comparison, a benchmark data under a hesitant fuzzy information environment has been used.

CHI-BD: A fuzzy rule-based classification system for Big Data classification problems

The previous Fuzzy Rule-Based Classification Systems (FRBCSs) for Big Data problems consist in concurrently learning multiple Chi et al. FRBCSs whose rule bases are then aggregated. The problem of this approach is that different models are obtained when varying the configuration of the cluster, becoming less accurate as more computing nodes are added. Our aim with this work is to design a new FRBCS for Big Data classification problems (CHI-BD) which is able to provide exactly the same model as the one that would be obtained by the original Chi et al. algorithm if it could be executed with this quantity of data. In order to do so, we take advantage of the suitability of the Chi et al. algorithm for the MapReduce paradigm, solving the problems of the previous approach, which lead us to obtain the same model (i.e., classification accuracy) regardless of the number of computing nodes considered.

A distributed approach to multi-objective evolutionary generation of fuzzy rule-based classifiers from big data

In the last years, multi-objective evolutionary algorithms (MOEAs) have been extensively used to generate sets of fuzzy rule-based classifiers (FRBCs) with different trade-offs between accuracy and interpretability. Since the computation of the accuracy for each chromosome evaluation requires the scan of the overall training set, these approaches have proved to be very expensive in terms of execution time and memory occupation. For this reason, they have not been applied to very large datasets yet. On the other hand, just for these datasets, interpretability of classifiers would be very desirable. In the last years the advent of a number of open source cluster computing frameworks has however opened new interesting perspectives. In this paper, we exploit one of these frameworks, namely Apache Spark, and propose the first distributed multi-objective evolutionary approach to learn concurrently the rule and data bases of FRBCs by maximizing accuracy and minimizing complexity. During the evolutionary process, the computation of the fitness is divided among the cluster nodes, thus allowing the designer to distribute both the computational complexity and the dataset storing. We have performed a number of experiments on ten real-world big datasets, evaluating our distributed approach in terms of both classification rate and scalability, and comparing it with two well-known state-of-art distributed classifiers. Finally, we have evaluated the achievable speedup on a small computer cluster. We present that the distributed version can efficiently extract compact rule bases with high accuracy, preserving the interpretability of the rule base, and can manage big datasets even with modest hardware support.

Consensus via penalty functions for decision making in ensembles in fuzzy rule-based classification systems

The aim of this paper is to propose a consensus method via penalty functions for decision making in ensembles of fuzzy rule-based classification systems (FRBCSs). For that, we first introduce a method based on overlap indices for building confidence and support measures, which are usually used to evaluate the degree of certainty or interest of a certain association rule. Those overlap indices (a generalizations of the Zadeh's consistency index between two fuzzy sets) are built using overlap functions, which are a special kind of non necessarily associative aggregation functions proposed for applications related to the overlap problem and/or when the associativity property is not demanded. Then, we introduce a new FRM for the FRBCS, considering different overlap indices, which generalizes the classical methods. By considering several overlap indices and aggregation functions, we generate fuzzy rule-based ensembles, providing different results. For the decision making related to the selection of the best class, we introduce a consensus method for classification, based on penalty functions. We also present theoretical results related to the developed methods. A detailed example of a generation of fuzzy rule-based ensembles based on the proposed approach, and the decision making by consensus via penalty functions, is presented.

Multiobjective Evolutionary Optimization of Type-2 Fuzzy Rule-Based Systems for Financial Data Classification

Classification techniques are becoming essential in the financial world for reducing risks and possible disasters. Managers are interested in not only high accuracy, but in interpretability and transparency as well. It is widely accepted now that the comprehension of how inputs and outputs are related to each other is crucial for taking operative and strategic decisions. Furthermore, inputs are often affected by contextual factors and characterized by a high level of uncertainty. In addition, financial data are usually highly skewed toward the majority class. With the aim of achieving high accuracies, preserving the interpretability, and managing uncertain and unbalanced data, this paper presents a novel method to deal with financial data classification by adopting type-2 fuzzy rule-based classifiers (FRBCs) generated from data by a multiobjective evolutionary algorithm (MOEA). The classifiers employ an approach, denoted as scaled dominance, for defining rule weights in such a way to help minority classes to be correctly classified. In particular, we have extended PAES-RCS, an MOEA-based approach to learn concurrently the rule and data bases of FRBCs, for managing both interval type-2 fuzzy sets and unbalanced datasets. To the best of our knowledge, this is the first work that generates type-2 FRBCs by concurrently maximizing accuracy and minimizing the number of rules and the rule length with the objective of producing interpretable models of real-world skewed and incomplete financial datasets. The rule bases are generated by exploiting a rule and condition selection (RCS) approach, which selects a reduced number of rules from a heuristically generated rule base and a reduced number of conditions for each selected rule during the evolutionary process. The weight associated with each rule is scaled by the scaled dominance approach on the fuzzy frequency of the output class, in order to give a higher weight to the minority class. As regards the data base learning, the membership function parameters of the interval type-2 fuzzy sets used in the rules are learned concurrently to the application of RCS. Unbalanced datasets are managed by using, in addition to complexity, selectivity and specificity as objectives of the MOEA rather than only the classification rate. We tested our approach, named IT2-PAES-RCS, on 11 financial datasets and compared our results with the ones obtained by the original PAES-RCS with three objectives and with and without scaled dominance, the FRBCs, fuzzy association rule-based classification model for high-dimensional dataset (FARC-HD) and fuzzy unordered rules induction algorithm (FURIA), the classical C4.5 decision tree algorithm, and its cost-sensitive version. Using nonparametric statistical tests, we will show that IT2-PAES-RCS generates FRBCs with, on average, accuracy statistically comparable with and complexity lower than the ones generated by the two versions of the original PAES-RCS. Further, the FRBCs generated by FARC-HD and FURIA and the decision trees computed by C4.5 and its cost-sensitive version, despite the highest complexity, result to be less accurate than the FRBCs generated by IT2-PAES-RCS. Finally, we will highlight how these FRBCs are easily interpretable by showing and discussing one of them.

Design of fuzzy rule based classifier using the monkey algorithm

Ilya A. Hodashinsky - Professor, Department of Complex Information Security of Computer Systems, Tomsk State University of Control Systems and Radioelectronics (TUSUR) Address: 40, Prospect Lenina, Tomsk, 634050, Russian FederationE-mail: hodashn@rambler.ruSergey S. Samsonov - Student, Department of Complex Information Security of Computer Systems, Tomsk State University of Control Systems and Radioelectronics (TUSUR) Address: 40, Prospect Lenina, Tomsk, 634050, Russian FederationE-mail: samsonicx@mail.ru This article presents an approach for building fuzzy rule based classifiers. A fuzzy rule-based classifier consists of IF-THEN rules with fuzzy antecedents (IF-part) and the class marks in consequents (THEN-part). Antecedent parts of the rules break down the input feature space into a set of fuzzy areas, and consequents define the classifier exit, marking these areas with a class mark. Two main phases of building the classifier are selected: generating the fuzzy rule base and optimizing the rule antecedent parameters. The classifier structure was formed by an algorithm for generating the rule base by extreme features found in the training sample. The peculiarity of this algorithm is that it is generated according to one classification rule for each class. The rule base formed by this algorithm has as low as practicable size in classification of a given data set. The optimization of parameters of antecedents of the fuzzy rules is implemented using the monkey algorithm adapted for these purposes, which is based on observations of monkey migration in the highlands. In the process of the algorithm work, three operations are performed: climb process, watch jump process and somersault process. One of the algorithm’s advantages in solution of high-dimension optimization problems is calculation of the pseudo-gradient of the objective function. Irrespective of the dimension at each iteration of the algorithm execution only two values of the objective function are to be calculated. The effectiveness of fuzzy rule-based classifiers built with the use of the proposed algorithms was checked on actual data from the KEEL-dataset repository. A comparative analysis was conducted using the known analog algorithms “D-MOFARC” and “FARC-HD”. The number of rules used by the classifiers built with the use of the algorithms so developed is much lower than the number of rules in analog classifiers with a comparable classification accuracy, that points to the highest interpretability of the classifiers built with the use of the proposed approach.[1] This work was supported by the Ministry of Education and Science of the Russian Federation, agreement no. 8.9628.2017/BP

An Expert System for Diagnosis of Sleep Disorder Using Fuzzy Rule-Based Classification Systems

Sleep disorder is an anomaly that could cause problems for someone’ sleeping pattern. Nowadays, it becomes an issue since people are getting busy with their own business and have no time to visit the doctors. Therefore, this research aims to develop a system used for diagnosis of sleep disorder using Fuzzy Rule-Based Classification System (FRBCS). FRBCS is a method based on the fuzzy set concepts. It consists of two steps: (i) constructing a model/knowledge involving rulebase and database, and (ii) prediction over new data. In this case, the knowledge is obtained from experts whereas in the prediction stage, we perform fuzzification, inference, and classification. Then, a platform implementing the method is built with a combination between PHP and the R programming language using the “Shiny” package. To validate the system that has been made, some experiments have been done using data from a psychiatric hospital in West Java, Indonesia. Accuracy of the result and computation time are 84.85% and 0.0133 seconds, respectively.

Design of Fuzzy Rule-based Classifiers through Granulation and Consolidation

Abstract This paper addresses the issue how to strike a good balance between accuracy and compactness in classification systems - still an important question in machine learning and data mining. The fuzzy rule-based classification approach proposed in current paper exploits the method of rule granulation for error reduction and the method of rule consolidation for complexity reduction. The cooperative nature of those methods - the rules are split in a way that makes efficient rule consolidation feasible and rule consolidation itself is capable of further error reduction - is demonstrated in a number of experiments with nine benchmark classification problems. Further complexity reduction, if necessary, is provided by rule compression.

An advanced computing in fuzzy rule-based preprocessing design of image filters’ system for removing impulse noises

In this study, advanced computing in fuzzy rule-based preprocessing classifier is used to categorize five distribution types that are identified as noise by noise detection from the service of next era cloud-empowered computing and techniques. The proposed method extracts useful local information from the corrupted image that is supported by filter processing and resulted in more image details to preserve. The fuzzy rule-based preprocessing classifier goes through a four-phase detection procedure to determine the condition of central pixels for the local image window by using the similarity between the neighboring pixels. A filters’ system is set up with our proposed fuzzy rule-based preprocessing classifier with several effective filters to verify its performance. Simulation results are compared with other individual filters by objective numerical measurements and subjectively visual inspection to indicate that our proposed method performs significantly better in terms of noise suppression and detail preservation.

Gene selection for designing optimal fuzzy rule base classifier by estimating missing value

DNA microarray technology, a high throughput technology evaluates the expression of thousands of genes simultaneously under different experimental conditions. Analysis of the gene expression data reveals that not all but few important genes are responsible for the diseases. However, the DNA microarray data set usually contain multiple missing value and therefore, selection of important genes using the incomplete data set may be erroneous, resulting misclassification in disease prediction. In the paper we propose an integrated framework, which first imputes the missing value and then in order to achieve maximum accuracy in classifying the patients a classifier has been designed to select the genes using the complete microarray data set.Here functionally similar genes are employed to estimate the missing value unlike the existing gene expression value based distance similarity measure. However, the functionally similar genes may differ in their protein production capacity and so the degree of similarity between the genes varies from gene to gene. The problem has been dealt by proposing a novel method to impute the missing value using the concept of fuzzy similarity. After imputing the missing value, the continuous gene expression matrix is discretized using fuzzy sets to distinguish the activation levels of different genes. The proposed fuzzy importance factor (FIf) of each gene represents its activation level or protein production capacity both in the disease and normal class. The importance of each gene is evaluated while optimizing the number of rules in the fuzzy classifier depending on the FIf. The methodology we propose has been demonstrated using nine different cancer data sets and compared with the state of the art methods. Analysis of experimental results reveals that the proposed framework able to classify the diseased and normal patients with improved accuracy.

EEG rhythm/channel selection for fuzzy rule-based alertness state characterization

The aim of the paper is to automatically select the optimal EEG rhythm/channel combinations capable of classifying human alertness states. Four alertness states were considered, namely ‘engaged’, ‘calm’, ‘drowsy’ and ‘asleep’. The features used in the automatic selection are the energies associated with the conventional rhythms, \(\delta , \theta , \alpha , \beta\) and \(\gamma\), extracted from overlapping windows of the different EEG channels. The selection process consists of two stages. In the first stage, the optimal brain regions, represented by sets of EEG channels, are selected using a simple search technique based on support vector machine (SVM), extreme learning machine (ELM) and LDA classifiers. In the second stage, a fuzzy rule-based alertness classification system (FRBACS) is used to identify, from the previously selected EEG channels, the optimal features and their supports. The IF–THEN rules used in FRBACS are constructed using a novel differential evolution-based search algorithm particularly designed for this task. Each alertness state is represented by a set of IF–THEN rules whose antecedent parts contain EEG rhythm/channel combination. The selected spatio-frequency features were found to be good indicators of the different alertness states, as judged by the classification performance of the FRBACS that was found to be comparable to those of the SVM, ELM and LDA classifiers. Moreover, the proposed classification system has the advantage of revealing simple and easy to interpret decision rules associated with each of the alertness states.

CC-integrals: Choquet-like Copula-based aggregation functions and its application in fuzzy rule-based classification systems

This paper introduces the concept of Choquet-like Copula-based aggregation function (CC-integral) and its application in fuzzy rule-based classification systems. The standard Choquet integral is expanded by distributing the product operation. Then, the product operation is generalized by a copula. Unlike the generalization of the Choquet integral by t-norms using its standard form (i.e., without distributing the product operator), which results in a pre-aggregation function, the CC-integral satisfies all the conditions required for an aggregation function. We build some examples of CC-integrals considering different examples of copulas, including t-norms, overlap functions and copulas that are neither t-norms nor overlap functions. We show that the CC-integral based on the minimum t-norm, when applied in fuzzy rule-based classification systems, obtains a performance that is, with a high level of confidence, better than that which adopts the winning rule (maximum). We concluded that the behavior of CC-integral is similar to the best Choquet-like pre-aggregation function. Consequently, the CC-integrals introduced in this paper can enlarge the scope of the applications by offering new possibilities for defining fuzzy reasoning methods with a similar gain in performance.

Pre-Clinical Study on the Detection of Simulated Epileptic Seizures

Wearable devices have promoted the application of Human Activity Recognition to the development of techniques for the assessment or diagnosing of illnesses and seizures, among other applications. For instance, the use of tri-axial accelerometry (3DACM) to detect abnormal and sudden movements has been introduced in the epileptic seizure recognition. In a previous research, Fuzzy Rule Based Classifiers (FRBC) have been found valid for the detection of epileptic convulsions; however, Ant Colony Systems learned FRBC performed with a high variability depending on the training data. In this study, we cope with this problem by the selection of a suitable partitioning method that has been extended to generate Fuzzy partitions. The comparison with the previous obtained results shows the fuzzy partitioning does not improve the overall performance in terms of error but highly reduces the variability in the performance of the obtained models, which allows us to obtain general models.