Fuzzy Rule Induction Research Articles

FRRI: A novel algorithm for fuzzy-rough rule induction

Interpretability is the next frontier in machine learning research. In the search for white box models — as opposed to black box models, like random forests or neural networks — rule induction algorithms are a logical and promising option, since the rules can easily be understood by humans. Fuzzy and rough set theory have been successfully applied to this archetype, almost always separately. As both approaches offer different ways to deal with imprecise and uncertain information, often with the use of an indiscernibility relation, it is natural to combine them. The QuickRules [20] algorithm was a first attempt at using fuzzy rough set theory for rule induction. It is based on QuickReduct, a greedy algorithm for building decision superreducts. QuickRules already showed an improvement over other rule induction methods. However, to evaluate the full potential of a fuzzy rough rule induction algorithm, one needs to start from the foundations. Accordingly, the novel rule induction algorithm, Fuzzy Rough Rule Induction (FRRI), we introduce in this paper, uses an approach that has not yet been utilised in this setting. We provide background and explain the workings of our algorithm. Furthermore, we perform a computational experiment to evaluate the performance of our algorithm and compare it to other state-of-the-art rule induction approaches. We find that our algorithm is more accurate while creating small rulesets consisting of relatively short rules.

Landslide spatial prediction utilizing fuzzy unordered rules induction ensemble models: a case study in Thai Nguyen, Vietnam

Landslide spatial prediction utilizing fuzzy unordered rules induction ensemble models: a case study in Thai Nguyen, Vietnam

A Novel Classification Method Using the Takagi–Sugeno Model and a Type-2 Fuzzy Rule Induction Approach

The main purpose of this research was to introduce a classification method, which combines a rule induction procedure with the Takagi–Sugeno inference model. This proposal is a continuation of our previous research, in which a classification process based on interval type-2 fuzzy rule induction was introduced. The research goal was to verify if the Mamdani fuzzy inference used in our previous research could be replaced with the first-order Takagi–Sugeno inference system. In the both cases to induce fuzzy rules, a new concept of a fuzzy information system was defined in order to deal with interval type-2 fuzzy sets. Additionally, the introduced rule induction assumes an optimization procedure concerning the footprint of uncertainty of the considered type-2 fuzzy sets. A key point in the concept proposed is the generalization of the fuzzy information systems’ attribute information to handle uncertainty, which occurs in real data. For experimental purposes, the classification method was tested on different classification benchmark data and very promising results were achieved. For the data sets: Breast Cancer Data, Breast Cancer Wisconsin, Data Banknote Authentication, HTRU 2 and Ionosphere, the following F-scores were achieved, respectively: 97.6%, 96%, 100%, 87.8%, and 89.4%. The results proved the possibility of applying the Takagi–Sugeno model in the classification concept. The model parameters were optimized using an evolutionary strategy.

Android malware detection using PMCC heatmap and Fuzzy Unordered Rule Induction Algorithm (FURIA)

Many smart mobile devices, including smartphones, smart televisions, smart watches, and smart vacuums, have been powered by Android devices. Therefore, mobile devices have become the prime target for malware attacks due to their rapid development and utilization. Many security practitioners have adopted different approaches to detect malware. However, its attacks continuously evolve and spread, and the number of attacks is still increasing. Hence, it is important to detect Android malware since it could expose a great threat to the users. However, in machine learning intelligence detection, too many insignificant features will decrease the percentage of the detection’s accuracy. Therefore, there is a need to discover the significant features in a minimal amount to assist with machine learning detection. Consequently, this study proposes the Pearson correlation coefficient (PMCC), a coefficient that measures the linear relationship between all features. Afterwards, this study adopts the heatmap method to visualize the PMCC value in the color of the heat version. For machine learning classification algorithms, we used a type of fuzzy logic called lattice reasoning. This experiment used real 3799 Android samples with 217 features and achieved the best accuracy rate of detection of more than 98% by using Unordered Fuzzy Rule Induction (FURIA).

Landslide susceptibility mapping at sin Ho, Lai Chau province, Vietnam using ensemble models based on fuzzy unordered rules induction algorithm

Landslide susceptibility map is considered as one of the important steps in assessing vulnerability of an area to landslide hazard. In this study, the main objective is to propose ensemble machine learning models: BF, DF and RSSF which are a combination of Fuzzy Unordered Rules Induction algorithm (F) and three optimization techniques namely Bagging, Decorate, and Random Subspace, respectively for landslide susceptibility mapping. In addition, two other single models namely F and Support Vector Machines (SVM) were also applied for the comparison of performance of the proposed models. For this purpose, the Sin Ho district, Lao Cai Province, Vietnam was selected as the study area. For the development of models, database of 850 present and historical landslides of this province including ten landslide affecting input parameters namely slope, curvature, elevation, aspect, Topographic Wetness Index (TWI), deep division, river density, fault density, aquifer, and geology were used. Validation of the models was done using various popular statistical indicators including Area Under the Receiver Operating Characteristics (AUC) curve. The results show that the BF model (AUC = 0.923) is the best model for accurate landslide susceptibility mapping (LSM) in comparison to other models namely DF (AUC = 0.899), RSSF (AUC = 0.893), SVM (AUC = 0.840), and F (AUC = 0.862). The study revealed that LSM map constructed using BF model can be used for better land use planning and proper landslide hazard management.

Composite Monte Carlo decision making under high uncertainty of novel coronavirus epidemic using hybridized deep learning and fuzzy rule induction

In the advent of the novel coronavirus epidemic since December 2019, governments and authorities have been struggling to make critical decisions under high uncertainty at their best efforts. In computer science, this represents a typical problem of machine learning over incomplete or limited data in early epidemic Composite Monte-Carlo (CMC) simulation is a forecasting method which extrapolates available data which are broken down from multiple correlated/casual micro-data sources into many possible future outcomes by drawing random samples from some probability distributions. For instance, the overall trend and propagation of the infested cases in China are influenced by the temporal–spatial data of the nearby cities around the Wuhan city (where the virus is originated from), in terms of the population density, travel mobility, medical resources such as hospital beds and the timeliness of quarantine control in each city etc. Hence a CMC is reliable only up to the closeness of the underlying statistical distribution of a CMC, that is supposed to represent the behaviour of the future events, and the correctness of the composite data relationships. In this paper, a case study of using CMC that is enhanced by deep learning network and fuzzy rule induction for gaining better stochastic insights about the epidemic development is experimented. Instead of applying simplistic and uniform assumptions for a MC which is a common practice, a deep learning-based CMC is used in conjunction of fuzzy rule induction techniques. As a result, decision makers are benefited from a better fitted MC outputs complemented by min–max rules that foretell about the extreme ranges of future possibilities with respect to the epidemic.

A Hybrid Dynamic Wavelet-Based Modeling Method for Blood Glucose Concentration Prediction in Type 1 Diabetes.

Background:Diabetes mellitus (DM) is a chronic disease that affects public health. The prediction of blood glucose concentration (BGC) is essential to improve the therapy of type 1 DM (T1DM).Methods:Having considered the risk of hyper- and hypo-glycemia, we provide a new hybrid modeling approach for BGC prediction based on a dynamic wavelet neural network (WNN) model, including a heuristic input selection. The proposed models include a hybrid dynamic WNN (HDWNN) and a hybrid dynamic fuzzy WNN (HDFWNN). These wavelet-based networks are designed based on dominant wavelets selected by the genetic algorithm-orthogonal least square method. Furthermore, the HDFWNN model structure is improved using fuzzy rule induction, an important innovation in the fuzzy wavelet modeling. The proposed networks are tested on real data from 12 T1DM patients and also simulated data from 33 virtual patients with an UVa/ Padova simulator, an approved simulator by the US Food and Drug Administration.Results:A comparison study is performed in terms of new glucose-based assessment metrics, such as gFIT, glucose-weighted form of ESODn (gESODn), and glucose-weighted R2 (gR2). For real patients’ data, the values of the mentioned indices are accomplished as gFIT = 0.97 ± 0.01, gESODn = 1.18 ± 0.38, and gR2 = 0.88 ± 0.07. HDFWNN, HDWNN and jump NN method showed the prediction error (root mean square error [RMSE]) of 11.23 ± 2.77 mg/dl, 10.79 ± 3.86 mg/dl and 16.45 ± 4.33 mg/dl, respectively.Conclusion:Furthermore, the generalized estimating equation and post hoc tests show that proposed models perform better compared with other proposed methods.

Fuzzy jump wavelet neural network based on rule induction for dynamic nonlinear system identification with real data applications.

AimFuzzy wavelet neural network (FWNN) has proven to be a promising strategy in the identification of nonlinear systems. The network considers both global and local properties, deals with imprecision present in sensory data, leading to desired precisions. In this paper, we proposed a new FWNN model nominated “Fuzzy Jump Wavelet Neural Network” (FJWNN) for identifying dynamic nonlinear-linear systems, especially in practical applications.MethodsThe proposed FJWNN is a fuzzy neural network model of the Takagi-Sugeno-Kang type whose consequent part of fuzzy rules is a linear combination of input regressors and dominant wavelet neurons as a sub-jump wavelet neural network. Each fuzzy rule can locally model both linear and nonlinear properties of a system. The linear relationship between the inputs and the output is learned by neurons with linear activation functions, whereas the nonlinear relationship is locally modeled by wavelet neurons. Orthogonal least square (OLS) method and genetic algorithm (GA) are respectively used to purify the wavelets for each sub-JWNN. In this paper, fuzzy rule induction improves the structure of the proposed model leading to less fuzzy rules, inputs of each fuzzy rule and model parameters. The real-world gas furnace and the real electromyographic (EMG) signal modeling problem are employed in our study. In the same vein, piecewise single variable function approximation, nonlinear dynamic system modeling, and Mackey–Glass time series prediction, ratify this method superiority. The proposed FJWNN model is compared with the state-of-the-art models based on some performance indices such as RMSE, RRSE, Rel ERR%, and VAF%.ResultsThe proposed FJWNN model yielded the following results: RRSE (mean±std) of 10e-5±6e-5 for piecewise single-variable function approximation, RMSE (mean±std) of 2.6–4±2.6e-4 for the first nonlinear dynamic system modelling, RRSE (mean±std) of 1.59e-3±0.42e-3 for Mackey–Glass time series prediction, RMSE of 0.3421 for gas furnace modelling and VAF% (mean±std) of 98.24±0.71 for the EMG modelling of all trial signals, indicating a significant enhancement over previous methods.ConclusionsThe FJWNN demonstrated promising accuracy and generalization while moderating network complexity. This improvement is due to applying main useful wavelets in combination with linear regressors and using fuzzy rule induction. Compared to the state-of-the-art models, the proposed FJWNN yielded better performance and, therefore, can be considered a novel tool for nonlinear system identification.

A fuzzy logic approach to reliable real-time recognition of facial emotions

This paper represents our newly developed software for emotion recognition from facial expressions. Besides allowing emotion recognition from image files and recorded video files, it uses webcam data to provide real-time, continuous, and unobtrusive facial emotional expressions. It uses FURIA algorithm for unordered fuzzy rule induction to offer timely and appropriate feedback based on learners’ facial expressions. The main objective of this study was first to validate the use of webcam data for a real-time and accurate analysis of facial expressions in e-learning environments. Second, transform these facial expressions to detected emotional states using the FURIA algorithm. We measured the performance of the software with ten participants, provided them with the same computer-based tasks, requested them a hundred times to mimic specific facial expressions, and recorded all sessions on video. We used the recorded video files to feed our newly developed software. We then used two experts’ opinions to annotate and rate participants’ recorded behaviours and to validate the software’s results. The software provides accurate and reliable results with the overall accuracy of 83.2%, which is comparable to the recognition by humans. This study will help to increase the quality of e-learning.

Glass breaks detection system using deep auto-encoders with fuzzy rules induction algorithm

Main uses of glass windows in commercial and residential buildings are prevalent. While a glass-based material has its advantages, it also poses security risks. Therefore, glass break detectors play an important role in security protection for offices and residential buildings. Conventional vibration-based and acoustic-based glass break detectors are designed to detect predetermined temporal and frequency feature thresholds of glass breakage sound signals. This leads to the inability to differentiate glass break from environmental sounds (such as the sound of striking objects, heavy sounds and shouted sounds) that are similar in their amplitude threshold and frequency pattern. Machine learning based acoustic audio classification has been popular in security surveillance applications. Researchers are interested in this research area, and different approaches have been proposed for anomaly event detection (such as gunshots, glass breakage sounds, etc.). This paper proposes a new design of a glass break detection algorithm based on Fuzzy Deep Auto-encoder Neural Network. The algorithm reduces false alarms and improves detection accuracy. Experimental results indicate that proposed fuzzy deep auto-encoder network system attained 95.5% correct detection for the proposed audio dataset.

A visualization cybersecurity method based on features' dissimilarity

Phishing attacks on websites are a serious problem that has seen a recent dramatic increase due to the higher volume of online financial transactions and advancements in computer network technology. One of the main challenges with existing intelligent phishing detection approaches is that despite their promising detection rates they do not provide novice users with alerting mechanisms in order to enrich users’ experience and knowledge of deceptive techniques. This paper proposes a new anti-phishing technique that not only detects phishing websites accurately, but also offers to novice users an alerting mechanism with rich rules. The key to success in the proposed anti-phishing technique are the features that have been developed by using a hybrid feature analysis. These provide visual cues in the web browser when phishing attacks occur. The rich rules are derived using a fuzzy rule induction approach and they can be utilized by the novice users to understand the security issues of the phishing problem. To evaluate the proposed technique, several experiments have been conducted using feature selection methods and classification algorithms (Furia, SMO, AdaBoost, Naïve Bayes, C4.5) against distinctive feature sets derived from a real phishing dataset. The results show that there are six features, which are not redundant, and when processed using Furia generate effective phishing detection models. More importantly, detection of these features is the basis of an alerting tool for pinpointing possible phishing attacks.

Induction of accurate and interpretable fuzzy rules from preliminary crisp representation

This paper proposes a novel approach for building transparent knowledge-based systems by generating accurate and interpretable fuzzy rules. The learning mechanism reported here induces fuzzy rules via making use of only predefined fuzzy labels that reflect prescribed notations and domain expertise, thereby ensuring transparency in the knowledge model adopted for problem solving. It works by mapping every coarsely learned crisp production rule in the knowledge base onto a set of potentially useful fuzzy rules, which serves as an initial step towards an intuitive technique for similarity-based rule generalisation. This is followed by a procedure that locally selects a compact subset of the emerging fuzzy rules, so that the resulting subset collectively generalises the underlying original crisp rule. The outcome of this local procedure forms the input to a global genetic search process, which seeks for a trade-off between accuracy and complexity of the eventually induced fuzzy rule base while maintaining transparency. Systematic experimental results are provided to demonstrate that the induced fuzzy knowledge base is of high performance and interpretability.

An adaptive neuro-fuzzy inference system (ANFIS) approach for measuring country sustainability performance

With the increasing demand for sustainable development, many international institutions and governments are seeking a balance between the environment, society and economy. With the aim of understanding and monitoring sustainability performance, various sustainability assessment methods have been developed. Fuzzy logic theory has been widely used for sustainability assessment. Good as these approaches are, there are criticisms that most studies use pre-defined simple linear membership functions (triangular or trapezoidal) and fuzzy rules, which are largely derived from experts' knowledge. However, sustainability is a very complex, multi-criteria issue, which contains various complex non-linear relationships. Moreover, it is time-consuming to find out the optimal membership functions and rules based on the expert knowledge. Therefore, it becomes necessary to explore a new approach for induction of membership functions and fuzzy rules. This paper introduces the adaptive neuro-fuzzy inference system (ANFIS) approach for country level sustainability assessment. The membership functions and fuzzy rules are generated from 128 training samples. The assessment results are close to the SAFE, Sustainability Assessment by Fuzzy Evaluation, model. Furthermore, three different types of non-linear membership functions, including Gaussian, bell-shaped and sigmoidal, are tested. The Gaussian membership function is the best one for country sustainability assessment. This study explores sustainability assessment, and results show that, by using appropriate training data, the ANFIS method is effective to measure the countries' sustainability performance. Using ANFIS, assessment accuracy can be further improved through appropriate selection of training samples using alternative data from UN-Habitat, or World Bank, or even new data sets.

Multiobjective Evolutionary Optimization of Type-2 Fuzzy Rule-Based Systems for Financial Data Classification

Classification techniques are becoming essential in the financial world for reducing risks and possible disasters. Managers are interested in not only high accuracy, but in interpretability and transparency as well. It is widely accepted now that the comprehension of how inputs and outputs are related to each other is crucial for taking operative and strategic decisions. Furthermore, inputs are often affected by contextual factors and characterized by a high level of uncertainty. In addition, financial data are usually highly skewed toward the majority class. With the aim of achieving high accuracies, preserving the interpretability, and managing uncertain and unbalanced data, this paper presents a novel method to deal with financial data classification by adopting type-2 fuzzy rule-based classifiers (FRBCs) generated from data by a multiobjective evolutionary algorithm (MOEA). The classifiers employ an approach, denoted as scaled dominance, for defining rule weights in such a way to help minority classes to be correctly classified. In particular, we have extended PAES-RCS, an MOEA-based approach to learn concurrently the rule and data bases of FRBCs, for managing both interval type-2 fuzzy sets and unbalanced datasets. To the best of our knowledge, this is the first work that generates type-2 FRBCs by concurrently maximizing accuracy and minimizing the number of rules and the rule length with the objective of producing interpretable models of real-world skewed and incomplete financial datasets. The rule bases are generated by exploiting a rule and condition selection (RCS) approach, which selects a reduced number of rules from a heuristically generated rule base and a reduced number of conditions for each selected rule during the evolutionary process. The weight associated with each rule is scaled by the scaled dominance approach on the fuzzy frequency of the output class, in order to give a higher weight to the minority class. As regards the data base learning, the membership function parameters of the interval type-2 fuzzy sets used in the rules are learned concurrently to the application of RCS. Unbalanced datasets are managed by using, in addition to complexity, selectivity and specificity as objectives of the MOEA rather than only the classification rate. We tested our approach, named IT2-PAES-RCS, on 11 financial datasets and compared our results with the ones obtained by the original PAES-RCS with three objectives and with and without scaled dominance, the FRBCs, fuzzy association rule-based classification model for high-dimensional dataset (FARC-HD) and fuzzy unordered rules induction algorithm (FURIA), the classical C4.5 decision tree algorithm, and its cost-sensitive version. Using nonparametric statistical tests, we will show that IT2-PAES-RCS generates FRBCs with, on average, accuracy statistically comparable with and complexity lower than the ones generated by the two versions of the original PAES-RCS. Further, the FRBCs generated by FARC-HD and FURIA and the decision trees computed by C4.5 and its cost-sensitive version, despite the highest complexity, result to be less accurate than the FRBCs generated by IT2-PAES-RCS. Finally, we will highlight how these FRBCs are easily interpretable by showing and discussing one of them.

A fuzzy rule induction algorithm for discovering classification rules

© 2016 - IOS Press and the authors. All rights reserved. Recently, the topic of data mining has attracted considerable attention from both academia and industry. Data mining is the process of extracting useful knowledge from large amounts of data. Among the types of knowledge to be mined, classification knowledge is the most widely exploited in engineering applications. A variety of methods exist for inductive learning of crisp classification knowledge. This paper presents a new inductive learning algorithm called FuzzyRULES that extracts fuzzy classification rules from a database of examples. The use of fuzzy sets and fuzzy logic methods not only provides a powerful, flexible approach to handling vagueness and uncertainty, but also increases the expressive power and comprehensibility of the induced classification knowledge. An example involving the induction of process planning rules is used to illustrate the operation of FuzzyRULES. The algorithm has also been compared against conventional crisp and fuzzy rule induction algorithms on several benchmark data sets. The results obtained have shown that FuzzyRULES induces more compact and more accurate classification rule sets.

Rotation forest fuzzy rule-based classifier ensemble for spatial prediction of landslides using GIS

In this paper, we present a novel classifier ensemble method, namely Rotation Forest fuzzy rule-based Classifier Ensemble (RFCE), for spatial prediction of landslides. An area located in the Uttarakhand State in India has been taken as a case study. RFCE is a hybrid method of rotation forest ensemble and fuzzy unordered rules induction algorithm classifier. Both are the current state-of-the-art techniques which have been seldom used for spatial prediction of landslides. A dataset has been generated for this study including landslide inventory map (930 landslide locations) and fifteen landslide influencing factors such as slope angle, slope aspect, elevation curvature, plan curvature, profile curvature, soil types, land cover, rainfall, distance to lineaments, distance to roads, distance to rivers, lineament density, road density, and river density. Performance of the RFCE model has been evaluated using statistical index-based evaluations and receiver operating characteristic curve technique. Additionally, other ensemble techniques, such as AdaBoost, Bagging, MultiBoost, and other classifiers, namely Support Vector Machines (SVM) and Fuzzy Unordered Rules Induction Algorithm (FURIA), have been taken into account for comparison. Overall, the performance of the RFCE model is well within spatial prediction of landslides (AUC = 0.896). It improved significantly the predictive capability of a base classifier of the FURIA model (AUC = 0.865). Its performance is even better than the AdaBoost model (AUC = 0.874) and almost same as the MultiBoost model (AUC = 0.901) and the SVM model (AUC = 0.901). However, it falls behind the Bagging model (AUC = 0.912). The present study indicates that RFCE is an encouraging method that can be used for spatial prediction of landslides.

A novel algorithm for fuzzy rule induction in data mining

Rule induction as a method of constructing classifiers is of particular interest to data mining because it generates models in the form of If-Then rules which are more expressive and easier for humans to comprehend and check. Several induction algorithms have been developed to learn classification rules. However, most of these algorithms are based on ‘crisp’ data and produce ‘crisp’ models. This paper presents FuzzySRI, a novel algorithm based on the techniques of fuzzy sets and fuzzy logic for inducing fuzzy classification rules. The algorithm possesses the clear knowledge representation capability of rule induction methods and the ability of fuzzy techniques to handle vague information. Experimental results show that FuzzySRI can outperform other fuzzy and non-fuzzy learning systems in terms of predictive accuracy, comprehensibility, and computational efficiency. It is also shown that FuzzySRI can be successfully applied to an industrial application concerning the automatic identification of machine faults.

Web usage mining for predicting final marks of students that use Moodle courses

AbstractThis paper shows how web usage mining can be applied in e‐learning systems in order to predict the marks that university students will obtain in the final exam of a course. We have also developed a specific Moodle mining tool oriented for the use of not only experts in data mining but also of newcomers like instructors and courseware authors. The performance of different data mining techniques for classifying students are compared, starting with the student's usage data in several Cordoba University Moodle courses in engineering. Several well‐known classification methods have been used, such as statistical methods, decision trees, rule and fuzzy rule induction methods, and neural networks. We have carried out several experiments using all available and filtered data to try to obtain more accuracy. Discretization and rebalance pre‐processing techniques have also been used on the original numerical data to test again if better classifier models can be obtained. Finally, we show examples of some of the models discovered and explain that a classifier model appropriate for an educational environment has to be both accurate and comprehensible in order for instructors and course administrators to be able to use it for decision making. © 2010 Wiley Periodicals, Inc. Comput Appl Eng Educ 21: 135–146, 2013

FUZZY QUALITATIVE LINK ANALYSIS FOR ACADEMIC PERFORMANCE EVALUATION

Many approaches have been developed for academic performance evaluation using various fuzzy techniques. Initial methods rely greatly on experts' specification of analytical parameters, without making use of valuable information embedded in collected data. Given this insight, fuzzy rule induction has recently been studied as a data-driven alternative. Despite its efficiency and reported performance, the fuzzy subsethood metric representing the strength of relations between system variables is only used at a coarse level, with the underlying semantics being unfortunately distorted via a local re-scaling scheme. To overcome this problem, a novel fuzzy classification system is introduced in this paper, in which the existing measure is used to its full potential via the methodology of qualitative link analysis. With a network representation where variables and their relations are encoded as graph nodes and edges, the classification of a new instance conceptually becomes a problem of link-based similarity estimation that can be effectively resolved using the proposed fuzzy qualitative model. This new approach has been evaluated against the existing rule-based method, revealing significant advantages of the present work.

Fuzzy data mining for segmentation and profiling

The paper presents fuzzy data mining for churn behavior analysis in telecom. Fuzzy data mining is used to identify the clusters in the given large dataset using the modified single pass fuzzy clustering algorithm. Subsequently, fuzzy rule induction is used to reveal variables that provide maximal separation between churners and non-churners in different clusters. This study provides evidence against extra resources for handling large data. The preliminary results are encouraging. The quality of clusters and rules derived for classification are almost identical on both the proposed approach and the original Fuzzy C-Means algorithm.