Learning Fuzzy Rules Research Articles

Multi-task learning for intelligent data processing in granular computing context

Classification is a popular task in many application areas, such as decision making, rating, sentiment analysis and pattern recognition. In the recent years, due to the vast and rapid increase in the size of data, classification has been mainly undertaken in the way of supervised machine learning. In this context, a classification task involves data labelling, feature extraction, feature selection and learning of classifiers. In traditional machine learning, data is usually single-labelled by experts, i.e., each instance is only assigned one class label, since experts assume that different classes are mutually exclusive and each instance is clear-cut. However, the above assumption does not always hold in real applications. For example, in the context of emotion detection, there could be more than one emotion identified from the same person. On the other hand, feature selection has typically been done by evaluating feature subsets in terms of their relevance to all the classes. However, it is possible that a feature is only relevant to one class, but is irrelevant to all the other classes. Based on the above argumentation on data labelling and feature selection, we propose in this paper a framework of multi-task learning. In particular, we consider traditional machine learning to be single task learning, and argue the necessity to turn it into multi-task learning to allow an instance to belong to more than one class (i.e., multi-task classification) and to achieve class specific feature selection (i.e., multi-task feature selection). Moreover, we report two experimental studies in terms of fuzzy multi-task classification and rule learning based multi-task feature selection. The results show empirically that it is necessary to undertake multi-task learning for both classification and feature selection.

Soft-Boosted Self-Constructing Neural Fuzzy Inference Network

This correspondence paper proposes an improved version of the self-constructing neural fuzzy inference network (SONFIN), called soft-boosted SONFIN (SB-SONFIN). The design softly boosts the learning process of the SONFIN in order to decrease the error rate and enhance the learning speed. The SB-SONFIN boosts the learning power of the SONFIN by taking into account the numbers of fuzzy rules and initial weights which are two important parameters of the SONFIN, SB-SONFIN advances the learning process by: 1) initializing the weights with the width of the fuzzy sets rather than just with random values and 2) improving the parameter learning rates with the number of learned fuzzy rules. The effectiveness of the proposed soft boosting scheme is validated on several real world and benchmark datasets. The experimental results show that the SB-SONFIN possesses the capability to outperform other known methods on various datasets.

Accuracy-Based Learning Classifier Systems for Multistep Reinforcement Learning: A Fuzzy Logic Approach to Handling Continuous Inputs and Learning Continuous Actions

Despite their proven effectiveness, many Michigan learning classifier systems (LCSs) cannot perform multistep reinforcement learning in continuous spaces. To meet this technical challenge, some LCSs have been designed to learn fuzzy logic rules. They can be largely classified into strength-based and accuracy-based systems. The latter is gaining more research attention in the last decade. However, existing accuracy-based learning systems either address primarily single-step learning problems or require the action space to be discrete. In this paper, a new accuracy-based learning fuzzy classifier system (LFCS) is developed to explicitly handle continuous state input and continuous action output during multistep reinforcement learning. Several technical improvements have been achieved while developing the new learning algorithm. Particularly, we have successfully extended ${Q}$ -learning like credit assignment methods to continuous spaces. To enable direct learning of stochastic strategies for action selection, we have also proposed to use a new fuzzy logic system with stochastic action outputs. Moreover, fine-grained learning of fuzzy rules has been achieved effectively in our algorithm by using a natural gradient learning method. It is the first time that these techniques are utilized substantially in any accuracy-based LFCSs. Meanwhile, in comparison with several recently proposed learning algorithms, our algorithm is shown to perform highly competitively on four benchmark learning problems and a robotics problem. The practical usefulness of our algorithm is also demonstrated by improving the performance of a wireless body area network.

S-FRULER: Scalable fuzzy rule learning through evolution for regression

In genetic fuzzy systems (GFS) the size of the problem has a huge influence in the performance of the obtained models, since i) the fuzzy rule bases learned suffer from exponential rule explosion when the number of variables increases, and ii) the convergence time increments with the number of examples. In this paper we present S-FRULER, a scalable distributed version of FRULER which is a GFS that learns simple and linguistic TSK-1 knowledge bases for regression problems. S-FRULER obtains models with high accuracy and low complexity, whilst reducing the algorithm runtime. S-FRULER focuses on splitting the problem into smaller partitions and incorporates a feature selection process for reducing the number of variables used in each partition. Each partition is then solved independently using the FRULER algorithm. Afterwards, an aggregation function obtains the final linguistic TSK fuzzy rule base from the information generated in each partition. S-FRULER has been validated in terms of scalability, precision and complexity using 10 large-scale datasets and has been compared with three state of the art GFSs. Experimental results show that S-FRULER scales well while achieving simple models with a linguistic approach and a precision comparable with approximative models. Moreover, S-FRULER has been applied to a bioinformatics problem that was proposed as a benchmark for scalability of regression problems, obtaining good results in both accuracy and complexity.

Ordinal classification based on the sequential covering strategy

Ordinal classification is a supervised learning problem. The distinctive feature of ordinal classification is that there is an order relationship among the categories to learn. In this paper, we present a fuzzy rule learning algorithm based on the sequential covering strategy applied to ordinal classification. This proposal modifies a nominal classification algorithm, called NSLV, to adapt it to this kind of problems. To take into account the order relationship among the categories, a new fitness function and a new concept of negative examples for a rule are proposed. Moreover, we introduce a new rule evaluation model for ordinal classification problems. Experimental results show that the proposed algorithm offers a better performance compared to other ordinal algorithms.

FRULER: Fuzzy Rule Learning through Evolution for Regression

The use of Takagi–Sugeno–Kang (TSK) fuzzy systems in regression problems is widely extended due to the precision of the obtained models. Moreover, the use of simple linear TSK models is usually referred as a good choice in many real problems since it provides a straightforward functional relationship between the output and input variables. In this paper we present FRULER (Fuzzy RUle Learning through Evolution for Regression), a new genetic fuzzy system for automatically learning accurate and simple linguistic TSK fuzzy rule bases for regression problems. FRULER achieves a low complexity of the learned models while keeping a high accuracy, by following three stages: instance selection, multi-granularity fuzzy discretization of the input variables, and evolutionary learning of the rule base combined with Elastic Net regularization to obtain the consequents of the rules. Each of these stages was validated using 28 real-world datasets. FRULER was also experimentally compared with three state of the art genetic fuzzy systems, showing the most accurate and simple models even when compared with approximative approaches.

Finding Intensity of Disease by Fuzzy Learning Rules (FLR) based Intelligent Decisive Support System for Efficient Disease Prediction

Experimental analysis is frequently done by doctor‟s expertise and involvement. But still, cases are testified of incorrect diagnosis and action. Patients are demanded to take some tests for examination. In many belongings, not all the tests contribute towards efficient analysis of a disease. Three classifiers similar Naive Bayes, Ordering by clustering and decision tree are used to calculate the analysis of patients with the same accuracy as acquired before the discount of some attributes. Fuzzy learning rules (FLR) are generally applied for finding the intensity of disease in data sets. Fuzzy learning rules better compared to other three methods. We propose a fuzzy learning algorithm to determine relationships between data resources based on their disease attributes, as well as to characterize knowledge through the connotation of disease covered by those properties. The algorithm addresses the significant problem of important a suitable number of clusters for suitably catching all the diseases of the knowledge domain. Using fuzzy rule-based classification system, the proposed system proves to improve the classification accuracy.

A Comparative Study on the Performance of Evolutionary Fuzzy and Crisp Rule Based Classification Methods in Congestion Prediction

Accurate estimation of the future state of the traffic is an attracting area for researchers in the field of Intelligent Transportation Systems (ITS). This kind of predictions can lead to traffic managers and drivers to act in consequence, reducing the economic and social impact of a possible congestion. Due to the inter-urban traffic information nature, the task of predicting the future state of the traffic requires, in most cases, a non-linear patterns search in the input data. In recent years, a wide variety of models has been used to solve this problem in the most accurate way. Due to that, models generated to provide information about the future state of the road are, usually, incomprehensible to a human operator, making impossible to give him/her an explanation about the causes of the prediction. Given the capacity of rule based systems to explain the reasoning followed to classify a new pattern, the advantages and disadvantages of such approaches are explored in this work.To conduct such task, datasets recorded from the California Department of Transportation are created. A 9-kilometer section of the I5 highway of Sacramento is used for this research. Two different types of datasets are built for the experimentation. One of them contains the entire information recorded. The other one contains with a simplified version of the information, considering only the first, middle and last monitored points of the road. Twelve prediction horizons, from 5 to 60minutes, were considered for prediction. An experimental comparative study involving 16 state of the art techniques is performed. Techniques tested include those that fall within the categories of Evolutionary Crisp Rule Learning (ECRL) and Evolutionary Fuzzy Rule Learning (EFRL). These methods were selected since they offer to the final user, not only a prediction, but also a legible model about the way in which the decision was taken. Techniques are compared in terms of accuracy and complexity of the models generated.

Fuzzy rule weight modification with particle swarm optimisation

The most challenging problem in developing fuzzy rule-based classification systems is the construction of a fuzzy rule base for the target problem. In many practical applications, fuzzy sets that are of particular linguistic meanings, are often predefined by domain experts and required to be maintained in order to ensure interpretability of any subsequent inference results. However, learning fuzzy rules using fixed fuzzy quantity space without any qualification will restrict the accuracy of the resulting rules. Fortunately, adjusting the weights of fuzzy rules can help improve classification accuracy without degrading the interpretability. There have been different proposals for fuzzy rule weight tuning through the use of various heuristics with limited success. This paper proposes an alternative approach using Particle Swarm Optimisation in the search of a set of optimal rule weights, entailing high classification accuracy. Systematic experimental studies are carried out using common benchmark data sets, in comparison to popular rule based learning classifiers. The results demonstrate that the proposed approach can boost classification performance, especially when the size of the initially built rule base is relatively small, and is competitive to popular rule-based learning classifiers.

An interpretability improvement for fuzzy rule bases obtained by the iterative rule learning approach

Interpretability is one of the key concepts in many of the applications using the fuzzy rule-based approach. It is well known that there are many different criteria around this concept, the complexity being one of them. In this paper, we focus our efforts in reducing the complexity of the fuzzy rule sets. One of the most interesting approaches for learning fuzzy rules is the iterative rule learning approach. It is mainly characterized by obtaining rules covering few examples in final stages, being in most cases useless to represent the knowledge. This behavior is due to the specificity of the extracted rules, which eventually creates more complex set of rules. Thus, we propose a modified version of the iterative rule learning algorithm in order to extract simple rules relaxing this natural trend. The main idea is to change the rule extraction process to be able to obtain more general rules, using pruned searching spaces together with a knowledge simplification scheme able to replace learned rules. The experimental results prove that this purpose is achieved. The new proposal reduces the complexity at both, the rule and rule base levels, maintaining the accuracy regarding to previous versions of the algorithm.

Learning fuzzy controllers in mobile robotics with embedded preprocessing

The automatic design of controllers for mobile robots usually requires two stages. In the first stage, sensorial data are preprocessed or transformed into high level and meaningful values of variables which are usually defined from expert knowledge. In the second stage, a machine learning technique is applied to obtain a controller that maps these high level variables to the control commands that are actually sent to the robot. This paper describes an algorithm that is able to embed the preprocessing stage into the learning stage in order to get controllers directly starting from sensorial raw data with no expert knowledge involved. Due to the high dimensionality of the sensorial data, this approach uses Quantified Fuzzy Rules (QFRs), that are able to transform low-level input variables into high-level input variables, reducing the dimensionality through summarization. The proposed learning algorithm, called Iterative Quantified Fuzzy Rule Learning (IQFRL), is based on genetic programming. IQFRL is able to learn rules with different structures, and can manage linguistic variables with multiple granularities. The algorithm has been tested with the implementation of the wall-following behavior both in several realistic simulated environments with different complexity and on a Pioneer 3-AT robot in two real environments. Results have been compared with several well-known learning algorithms combined with different data preprocessing techniques, showing that IQFRL exhibits a better and statistically significant performance. Moreover, three real world applications for which IQFRL plays a central role are also presented: path and object tracking with static and moving obstacles avoidance.

Cost-Sensitive Learning of Fuzzy Rules for Imbalanced Classification Problems Using FURIA

This paper is intended to verify that cost-sensitive learning is a competitive approach for learning fuzzy rules in certain imbalanced classification problems. It will be shown that there exist cost matrices whose use in combination with a suitable classifier allows for improving the results of some popular data-level techniques. The well known FURIA algorithm is extended to take advantage of this definition. A numerical study is carried out to compare the proposed cost-sensitive FURIA to other state-of-the-art classification algorithms, based on fuzzy rules and on other classical machine learning methods, on 64 different imbalanced datasets.

Intrusion Detection System based on Learning Fuzzy Rules and Membership Functions using Genetic Algorithms

With the rapid expansion of Internet in recent years, computer systems are facing increased number of security threats. Despite numerous technological innovations for information assurance, it is still very difficult to protect computer systems. Therefore, unwanted intrusions take place when the actual software systems are running. Different soft computing based approaches have been proposed to detect computer network attacks. Hybrid methods proved more effective and accurate, this paper tries to introduce how to use dynamic fuzzy rules and genetic algorithm in intrusion detection systems.

IVTURS: A Linguistic Fuzzy Rule-Based Classification System Based On a New Interval-Valued Fuzzy Reasoning Method With Tuning and Rule Selection

Interval-valued fuzzy sets have been shown to be a useful tool to deal with the ignorance related to the definition of the linguistic labels. Specifically, they have been successfully applied to solve classification problems, performing simple modifications on the fuzzy reasoning method to work with this representation and making the classification based on a single number. In this paper, we present IVTURS, which is a new linguistic fuzzy rule-based classification method based on a new completely interval-valued fuzzy reasoning method. This inference process uses interval-valued restricted equivalence functions to increase the relevance of the rules in which the equivalence of the interval membership degrees of the patterns and the ideal membership degrees is greater, which is a desirable behavior. Furthermore, their parametrized construction allows the computation of the optimal function for each variable to be performed, which could involve a potential improvement in the system's behavior. Additionally, we combine this tuning of the equivalence with rule selection in order to decrease the complexity of the system. In this paper, we name our method IVTURS-FARC, since we use the FARC-HD method to accomplish the fuzzy rule learning process. The experimental study is developed in three steps in order to ascertain the quality of our new proposal. First, we determine both the essential role that interval-valued fuzzy sets play in the method and the need for the rule selection process. Next, we show the improvements achieved by IVTURS-FARC with respect to the tuning of the degree of ignorance when it is applied in both an isolated way and when combined with the tuning of the equivalence. Finally, the significance of IVTURS-FARC is further depicted by means of a comparison by which it is proved to outperform the results of FARC-HD and FURIA, which are two high performing fuzzy classification algorithms.

A feature construction approach for genetic iterative rule learning algorithm

This paper presents a proposal that introduces the use of feature construction in a fuzzy rule learning algorithm. This is done by means of the combination of two different approaches together with a new learning strategy. The first of these two approaches consists of using relations in the antecedent of fuzzy rules while the second one employs functions in the antecedent of that rules. Thus, the method we propose tries to integrate these two models so that, using a learning strategy that allows us to start learning more general rules and finish the process learning more specific ones, we are able to increase the amount of information extracted from the initial variables. The experimental results show that the proposed method obtains a good trade-off among accuracy, interpretability and time needed to get the model in relation to the rest of algorithms using feature construction involved in the comparison.

Knowledge acquisition based on learning of maximal structure fuzzy rules

There are a lot of systems that make decisions or classifications on the basis of a number of rules. This set of rules that govern such a system is called the rule base. When a new system of this kind is being developed, setting up its rule base is a time-consuming and expensive process because the rule base contains the knowledge of the outside world, which could be acquired from experts or produced from previous experiences. In this latter case, machine-learning algorithms can help. In fact, many methods have been proposed to generate rules from training instances. The aim of this paper is to present a new fuzzy learning algorithm to generate IF-THEN rules, for classifying instances in one application domain. This algorithm can improve the results offered by a previously presented algorithm. In addition, the more common classification problems of the original algorithm are presented and a measure to determine the conflicts among generated rules is introduced. Moreover, we study the classification stage of that inductive fuzzy learning algorithm and an improvement is suggested to obtain better classification results.

Fuzzy rule-based similarity model enables learning from small case bases

The concept of similarity plays a fundamental role in case-based reasoning. However, the meaning of “similarity” can vary in situations and is largely domain dependent. This paper proposes a novel similarity model consisting of linguistic fuzzy rules as the knowledge container. We believe that fuzzy rules representation offers a more flexible means to express the knowledge and criteria for similarity assessment than traditional similarity metrics. The learning of fuzzy similarity rules is performed by exploiting the case base, which is utilized as a valuable resource with hidden knowledge for similarity learning. A sample of similarity is created from a pair of known cases in which the vicinity of case solutions reveals the similarity of case problems. We do pair-wise comparisons of cases in the case base to derive adequate training examples for learning fuzzy similarity rules. The empirical studies have demonstrated that the proposed approach is capable of discovering fuzzy similarity knowledge from a rather low number of cases, giving rise to the competence of CBR systems to work on a small case library.

Crop parameters estimation by fuzzy inference system using X-band scatterometer data

Learning fuzzy rule based systems with microwave remote sensing can lead to very useful applications in solving several problems in the field of agriculture. Fuzzy logic provides a simple way to arrive at a definite conclusion based upon imprecise, ambiguous, vague, noisy or missing input information. In the present paper, a subtractive based fuzzy inference system is introduced to estimate the potato crop parameters like biomass, leaf area index, plant height and soil moisture. Scattering coefficient for HH- and VV-polarizations were used as an input in the Fuzzy network. The plant height, biomass, and leaf area index of potato crop and soil moisture measured at its various growth stages were used as the target variables during the training and validation of the network. The estimated values of crop/soil parameters by this methodology are much closer to the experimental values. The present work confirms the estimation abilities of fuzzy subtractive clustering in potato crop parameters estimation. This technique may be useful for the other crops cultivated over regional or continental level.

A Fuzzy Rule Learning and Classification Method using Cellular Evolutionary

A Fuzzy Rule Learning and Classification Method using Cellular Evolutionary

ON DESIGNING FUZZY RULE-BASED MULTICLASSIFICATION SYSTEMS BY COMBINING FURIA WITH BAGGING AND FEATURE SELECTION

In this work, we conduct a study considering a fuzzy rule-based multiclassification system design framework based on Fuzzy Unordered Rule Induction Algorithm (FURIA). This advanced method serves as the fuzzy classification rule learning algorithm to derive the component classifiers considering bagging and feature selection. We develop an exhaustive study on the potential of bagging and feature selection to design a final FURIA-based fuzzy multiclassifier dealing with high dimensional data. Several parameter settings for the global approach are tested when applied to twenty one popular UCI datasets. The results obtained show that FURIA-based fuzzy multiclassifiers outperform the single FURIA classifier and are competitive with C4.5 multiclassifiers and random forests.