Compact Rule Sets Research Articles

From explanation to intervention: Interactive knowledge extraction from Convolutional Neural Networks used in radiology.

Deep Learning models such as Convolutional Neural Networks (CNNs) are very effective at extracting complex image features from medical X-rays. However, the limited interpretability of CNNs has hampered their deployment in medical settings as they failed to gain trust among clinicians. In this work, we propose an interactive framework to allow clinicians to ask what-if questions and intervene in the decisions of a CNN, with the aim of increasing trust in the system. The framework translates a layer of a trained CNN into a measurable and compact set of symbolic rules. Expert interactions with visualizations of the rules promote the use of clinically-relevant CNN kernels and attach meaning to the rules. The definition and relevance of the kernels are supported by radiomics analyses and permutation evaluations, respectively. CNN kernels that do not have a clinically-meaningful interpretation are removed without affecting model performance. By allowing clinicians to evaluate the impact of adding or removing kernels from the rule set, our approach produces an interpretable refinement of the data-driven CNN in alignment with medical best practice.

Adaptive fuzzy-evidential classification based on association rule mining

As one of the most promising classification approaches, association classification (AC) integrates data classification and association discovery techniques for generating a compact set of classification association rules. Recently, the fuzzy set and evidence theories are successively applied into AC in order to improve the classification performance in terms of accuracy and interpretability. However, from the perspectives of applicability and universality, there still exists two important issues in the current AC framework. On one hand, several key parameters, such as the number of fused rules in classification and the minimum support threshold in association discovery, are difficult to be accurately predefined in practice. On the other hand, the fixed grid-based fuzzy partition is not benefit to adapt for those datasets with large number of features. In this paper, an association rule-based adaptive fuzzy-evidential classification framework (AR-AFEC) is developed for overcoming the above limitations. To do so, an optimal rule fusion strategy and a dynamic minimum support threshold setting scheme are proposed for adaptively learning the parameters during classification and association mining respectively. In addition, an entropy-based trapezoidal fuzzy partition technique is proposed to adaptively obtain the fuzzy sets defined on each continuous feature domain. Experiments on 26 benchmark datasets and a human activity recognition application demonstrate that the proposal can achieve better accuracy than some state-of-the-art rule-based classification approaches, using less rules with more general structure.

A novel cluster heads election method based on group sparse TSK fuzzy system for energy-efficient wireless sensor network

Selecting appropriate Cluster Heads (CHs) can significantly enhance the lifetime of the wireless sensor networks (WSNs). Fuzzy logic is an effective approach for CH election. However, existing fuzzy-logic-based CH election methods usually require a large number of fuzzy rules, making the CH election procedure inefficiency. In this study, a data-driven CH election method is proposed based on a compact set of fuzzy rules, which are learned by group sparse Takagi-Sugeno-Kang (GS-TSK) fuzzy system. Specifically, five linguistic variables were first used as features to describe the status of sensor nodes. After that, a compact set of fuzzy rules were learned by GS-TSK, and they were then used to predict the chance of each sensor node becoming a CH. Based on the selected CHs, the clusters are generated. Simulation results show that the GS-TSK can select CHs with fewer rules more accurately. Besides, by using the proposed DD-FLC, an average improvement of WSN was shown in terms of first node dead (FND), 10% of nodes dead (10PND), quarter of nodes dead (QND), half of nodes dead (HND).

Rule Induction in Knowledge Graphs Using Linear Programming

We present a simple linear programming (LP) based method to learn compact and interpretable sets of rules encoding the facts in a knowledge graph (KG) and use these rules to solve the KG completion problem. Our LP model chooses a set of rules of bounded complexity from a list of candidate first-order logic rules and assigns weights to them. The complexity bound is enforced via explicit constraints. We combine simple rule generation heuristics with our rule selection LP to obtain predictions with accuracy comparable to state-of-the-art codes, even while generating much more compact rule sets. Furthermore, when we take as input rules generated by other codes, we often improve interpretability by reducing the number of chosen rules, while maintaining accuracy.

Design of flexible pavements through fuzzy inference system with genetic algorithm optimized rule base

In this paper, a novel method for the design of flexible pavements is elaborated. The method is based on fuzzy inference system with genetic algorithm (GA) aided optimized rule base. The model is founded on layered fuzzy antecedent and consequent conjunctive rules. The data for the model consists of 300 flexible pavement design instances that breaks up in to 25% of the data drawn from research and real field applications and 75% of data generated in spread sheets compliant with Indian road congress (IRC) code guidelines. In the first step, the inputs and outputs were fuzzified and around 110 rules were generated using training data set. GA was implemented to find optimal and a compact rule set. GA was able to garner 35 rules that are adequate to predict the thickness of base course, sub base and surface course with high accuracy. The model with optimized rules was validated using test data set. The results of the evaluation are encouraging with low values of RMSE ranging between 3.6–11 for GSB, binder course (BC) and surface course (SC). The coefficient of determination is also high and between 0.85–0.90 indicating accuracy in prediction. Correlation coefficient values stood at an average of 0.92 indicating closeness between predicted and actual values of thickness of courses.

ReG-Rules: An Explainable Rule-Based Ensemble Learner for Classification

The learning of classification models to predict class labels of new and previously unseen data instances is one of the most essential tasks in data mining. A popular approach to classification is ensemble learning, where a combination of several diverse and independent classification models is used to predict class labels. Ensemble models are important as they tend to improve the average classification accuracy over any member of the ensemble. However, classification models are also often required to be explainable to reduce the risk of irreversible wrong classification. Explainability of classification models is needed in many critical applications such as stock market analysis, credit risk evaluation, intrusion detection, etc. Unfortunately, ensemble learning decreases the level of explainability of the classification, as the analyst would have to examine many decision models to gain insights about the causality of the prediction. The aim of the research presented in this paper is to create an ensemble method that is explainable in the sense that it presents the human analyst with a conditioned view of the most relevant model aspects involved in the prediction. To achieve this aim the authors developed a rule-based explainable ensemble classifier termed Ranked ensemble G-Rules (ReG-Rules) which gives the analyst an extract of the most relevant classification rules for each individual prediction. During the evaluation process ReG-Rules was evaluated in terms of its theoretical computational complexity, empirically on benchmark datasets and qualitatively with respect to the complexity and readability of the induced rule sets. The results show that ReG-Rules scales linearly, delivers a high accuracy and at the same time delivers a compact and manageable set of rules describing the predictions made.

Disruption‐Free Load Balancing for Aerial Access Network

A fundamental issue of 6G networks with aerial access networks (AAN) as a core component is that user devices will send high‐volume traffic via AAN to backend servers. As such, it is critical to load balance such traffic such that it will not cause network congestion or disruption and affect users’ experience in 6G networks. Motivated by the success of software‐defined networking‐based load balancing, this paper proposes a novel system called Tigris, to load balance high‐volume AAN traffic in 6G networks. Different from existing load balancing solutions in traditional networks, Tigris tackles the fundamental disruption-resistant challenge in 6G networks for avoiding disruption of continuing flows and the control-path update challenge for limiting the throughput of updating load balancing instructions. Tigris achieves disruption‐free and low‐control‐path‐cost load balancing for AAN traffic by developing an online algorithm to compute disruption‐resistant, per‐flow load balancing policies and a novel bottom‐up algorithm to compile the per‐flow policies into a highly compact rule set, which remains disruption‐resistant and has a low control‐path cost. We use extensive evaluation to demonstrate the efficiency and efficacy of Tigris to achieve zero disruption of continuing AAN flows and an extremely low control‐path update overhead, while existing load balancing techniques in traditional networks such as ECMP cause high load variance and disrupt almost 100% continuing AAN flows.

RS-HeRR: a rough set-based Hebbian rule reduction neuro-fuzzy system

Interpretabilty is one of the desired characteristics in various classification task. Rule-based system and fuzzy logic can be used for interpretation in classification. The main drawback of rule-based system is that it may contain large complex rules for classification and sometimes it becomes very difficult in interpretation. Rule reduction is also difficult for various reasons. Removing important rules may effect in classification accuracy. This paper proposes a hybrid fuzzy-rough set approach named RS-HeRR for the generation of effective, interpretable and compact rule set. It combines a powerful rule generation and reduction fuzzy system, called Hebbian-based rule reduction algorithm (HeRR) and a novel rough-set-based attribute selection algorithm for rule reduction. The proposed hybridization leverages upon rule reduction through reduction in partial dependency as well as improvement in system performance to significantly reduce the problem of redundancy in HeRR, even while providing similar or better accuracy. RS-HeRR demonstrates these characteristics repeatedly over four diverse practical classification problems, such as diabetes identification, urban water treatment monitoring, sonar target classification, and detection of ovarian cancer. It also demonstrates excellent performance for highly biased datasets. In addition, it competes very well with established non-fuzzy classifiers and outperforms state-of-the-art methods that use rough sets for rule reduction in fuzzy systems.

Development of information and analytical tools based on adaptive classifier Cascade ARTMAP

The article deals with methods of teaching incremental classifiers capable to adapt to incoming online streaming data, using the example of environmental monitoring systems. It is proposed to use Cascade ARTMAP neural network architecture, compatible with symbolic representation based on IF- THEN rules. The knowledge gained during Cascade ARTMAP network training can be transformed into a compact set of decisive rules for classification of source data that can be easily interpreted by experts in the field of environmental assessment.

Managing a pool of rules for credit card fraud detection by a Game Theory based approach

In the automatic credit card transaction classification there are two phases: in the Real-Time (RT) phase the system decides quickly, based on the bare transaction information, whether to authorize the transaction; in the subsequent Near-Real-Time (NRT) phase, the system enacts a slower ex-post evaluation, based on a larger information context. The classification rules in the NRT phase trigger alerts on suspicious transactions, which are transferred to human investigators for final assessment. The management criteria used to select the rules, to be kept operational in the NRT pool, are traditionally based mostly on the performance of individual rules, considered in isolation; this approach disregards the non-additivity of the rules (aggregating rules with high individual precision does not necessarily make a high-precision pool). In this work, we propose to apply, to the rule selection for the NRT phase, an approach which assigns a normalized score to the individual rule, quantifying the rule influence on the overall performance of the pool. As a score we propose to use a power-index developed within Coalitional Game Theory, the Shapley Value (SV), summarizing the performance in collaboration. Such score has two main applications: (1) it can be used, within the periodic rule assessment process, to support the decision of whether to keep or drop the rule from the pool; (2) it can be used to select the k top-ranked rules, so as to work with a more compact rule set. Using real-world credit card fraud data containing approximately 300 rules and 3×105 transactions records, we show that: (1) this score fares better – in granting the performance of the pool – than the one assessing the rules in isolation; (2) that the same performance of the whole pool can be achieved keeping only one tenth of the rules — the top-k SV-ranked rules. We observe that the latter application can be reframed in terms of Feature Selection (FS) task for a classifier: we show that our approach is comparable w.r.t benchmark FS algorithms, but argue that it presents an advantage for the management, consisting in the assignment of a normalized score to the individual rule. This is not the case for most FS algorithms, which only focus in yielding a high-performance feature-set solution.

Multitask TSK Fuzzy System Modeling by Jointly Reducing Rules and Consequent Parameters

Existing multitask Takagi-Sugeno-Kang (TSK) fuzzy modeling methods always produce high complex fuzzy models with numerous redundant rules and consequent parameters. To this end, we propose a novel multitask TSK fuzzy modeling method called mtSparseTSK, which learns a compact set of fuzzy rules and shared consequent parameters across tasks in a unified procedure. Specifically, we consider the fuzzy rule reduction and consequent parameter selection across tasks by devising novel group sparsity regularizations in the learning criterion of the model. We also integrate the intertask relations in the proposed TSK model for multitask learning. We fully utilize the block structure in the TSK fuzzy models in formulating a joint block sparse optimization problem and develop a procedure for alternating direction method of multipliers (ADMMs) to find the optimal solution of the problem. Experiments on the synthetic and real-world datasets demonstrate the distinctive performance of the proposed methods over the existing ones on multitask fuzzy system modeling.

Using interpretable fuzzy rule-based models for the estimation of soil organic carbon from VNIR/SWIR spectra and soil texture

In this paper, the use of a novel evolutionary fuzzy rule-based system (FRBS) for the prediction of Soil Organic Carbon from visible, near-infrared, and short-wave infrared (VNIR/SWIR) spectra and the textural information as additional predictor is examined. Compared to other techniques, the proposed model generates a compact set of rules with a high interpretation degree, mapping local input to local output regions. This is achieved through an evolutionary learning procedure which is applied to establish linguistic rules and assist in the interpretation of the association between spectra and the target property. The rule base may be also decomposed into texture-specific sets of rules, allowing a more detailed analysis on a per textural class basis. These intrinsic properties enable the development of spectral prototype signatures and sparse feature utilization histograms at different levels of aggregation, i.e. per textural class and/or output region. The proposed model is applied to the LUCAS topsoil database comprised of roughly 18,000 mineral samples across 23 European Union member-states. We first demonstrate the enhanced interpretation capabilities of our fuzzy approach, which can assist in the extraction of fruitful knowledge governing the association between soil properties and VNIR/SWIR spectra. The model is then compared with other contemporary approaches, namely PLS, SVM, and Cubist. The results indicate that our approach produced compact and interpretable results with fair prediction accuracies (equivalent with the best approach).

Fuzzy Neural Networks based on Fuzzy Logic Neurons Regularized by Resampling Techniques and Regularization Theory for Regression Problems

This paper presents a novel learning algorithm for fuzzy logic neuron based on neural networks and fuzzy systems able to generate accurate and transparent models. The learning algorithm is based on ideas from Extreme Learning Machine [36], to achieve a low time complexity, and regularization theory, resulting in sparse and accurate models. A compact set of incomplete fuzzy rules can be extracted from the resulting network topology. Experiments considering regression problems are detailed. Results suggest the proposed approach as a promising alternative for pattern recognition with a good accuracy and some level of interpretability.

Fuzzy Neural Networks based on Fuzzy Logic Neurons Regularized by Resampling Techniques and Regularization Theory for Regression Problems

This paper presents a novel learning algorithm for fuzzy logic neuron based on neural networks and fuzzy systems able to generate accurate and transparent models. The learning algorithm is based on ideas from Extreme Learning Machine [36], to achieve a low time complexity, and regularization theory, resulting in sparse and accurate models. A compact set of incomplete fuzzy rules can be extracted from the resulting network topology. Experiments considering regression problems are detailed. Results suggest the proposed approach as a promising alternative for pattern recognition with a good accuracy and some level of interpretability.

SDN Programming for Heterogeneous Switches with Flow Table Pipelining

High-level programming is one of the critical building blocks of the effective use of software-defined networking (SDN). Existing solutions, however, either (1) cannot utilize the state-of-the-art switches with flow table pipelining, a key technique to prevent flow rule set explosion or (2) force programmers to manually organize and manage hardware flow table pipelines, which is time-consuming and error-prone. This paper presents a high-level SDN programming framework to address these issues. The framework can automatically (1) generate rule sets for heterogeneous switches with different flow table pipelining designs and (2) update installed rules when the network state changes. As a result, the framework can not only generate efficient rule sets for switches but also provide programmers a centralized, intuitive, and hence easy-to-use programming API. Experiments show that the framework can generate compact rule sets that are 29–116 times smaller than those generated by other open-source SDN controllers. Besides, the framework is 5 times faster to recover from network link failures in comparison to other controllers.

Comparison of rule induction, decision trees and formal concept analysis approaches for classification

Rule-based learning algorithms have higher transparency and easiness to interpret in comparison with neural networks and deep learning algorithms. These properties make it possible to effectively use such algorithms to solve descriptive tasks of data mining. The choice of an algorithm depends also on its ability to solve predictive tasks. The article compares the quality of the solution of the problems with binary and multiclass classification based on the experiments with six datasets from the UCI Machine Learning Repository. The authors investigate three algorithms: Ripper (rule induction), C4.5 (decision trees), In-Close (formal concept analysis). The results of the experiments show that In-Close demonstrates the best quality of classification in comparison with Ripper and C4.5, however the latter two generate more compact rule sets.

Parallel algorithms for the automated discovery of declarative process models

The aim of process discovery is to build a process model from an event log without prior information about the process. The discovery of declarative process models is useful when a process works in an unpredictable and unstable environment since several allowed paths can be represented as a compact set of rules. One of the tools available in the literature for discovering declarative models from logs is the Declare Miner, a plug-in of the process mining tool ProM. Using this plug-in, the discovered models are represented using Declare, a declarative process modeling language based on ltl for finite traces. However, the high execution times of the Declare Miner when processing large sets of data hampers the applicability of the tool to real-life settings. Therefore, in this paper, we propose a new approach for the discovery of Declare models based on the combination of an Apriori algorithm and a group of algorithms for Sequence Analysis to enhance the time performance of the plug-in. The approach has been developed in a way that it is easy to be parallelized using two different partitioning methods: the search space partitioning, in which different groups of candidate constraints are processed in parallel, and the database partitioning, in which different chunks of the log are processed at the same time. The approach has been implemented in ProM in its sequential version and in two multi-threading implementations leveraging these two partitioning methods. All the new variants of the plug-in have been evaluated using a large set of synthetic and real-life event logs.

Extended fuzzy hyperline-segment neural network with classification rule extraction

The hyperline segment layer of fuzzy hyperline segment neural network (FHLSNN) consists of number of n- dimensional hyperline segments. Each hyperline segment has the two end points defined in terms of continuous attribute values and an associated membership function. The membership function processes the continuous attribute values and gives the membership of each input pattern to the hyperline segments. These end points and membership function are defined only in terms of continuous attributes and cannot process discrete attributes. But the real world data comprises both continuous and discrete attributes. Due to this original FHLSNN and its ancestor versions cannot be applied to these types of data. Also, justification of the classification decision given by these fuzzy min–max neural networks is required to be obtained to make them more applicable to the real world applications. The objective of the proposed extended fuzzy hyperline segment neural network (EFHLSNN) is to solve these two problems. In the EFHLSNN, each hyperline segment has two end points defined not only in terms of continuous attributes but also the set of binary strings defined for discrete attributes. The membership function and expansion condition of the original FHLSNN are modified to process both continuous and discrete attribute values. The proposed EFHLSNN model is applied to eight different benchmark datasets. The experimental results show that the proposed model gives very good accuracy and the compact rule set that justifies the classification decision of the EFHLSNN and thus, giving rise to good rule fidelity.

Do activity lifecycles affect the validity of a business rule in a business process?

Traditional process mining techniques offer limited possibilities to analyze business processes working in low-predictable and dynamic environments. Recently, to close this gap, declarative process models have been introduced to represent process mining results since they allow for describing complex behaviors as a compact set of business rules. However, in this context, activities of a business process are still considered as atomic/instantaneous events. This is a strong limitation for these approaches because often, in realistic environments, process activities are not instantaneous but executed across a time interval and pass through a sequence of states of a lifecycle. This paper investigates how the existing techniques for the discovery of declarative process models can be adapted when the business process under analysis contains non-atomic activities. In particular, we base our proposed approach on the use of discriminative rule mining to determine how the characteristics of the activity lifecycles in a business process influence the validity of a business rule in that process. The approach has been implemented as a plug-in of the process mining tool ProM and validated on synthetic logs and on a real-life log recorded by an incident and problem management system called VINST in use at Volvo IT Belgium.

On fuzzy feature selection in designing fuzzy classifiers for high-dimensional data

Generating fuzzy rules for high-dimensional data has been a serious challenge in designing fuzzy rule-based classification systems. For data sets with low dimensions, there are some efficient methods to generate a compact set of short fuzzy rules. However, when the dimensions go up, the number of rules increases exponentially. One solution for lowering the dimensions is feature selection which selects a subset of more effective features. In this regard, a fuzzy feature selection approach is proposed in this paper which tries to choose more relevant features; those which can distinguish the distinct classes well. Our method employs the training patterns in the subspace of some predefined fuzzy sets on each feature and applies their compatibility degrees to evaluate that feature. Since each feature is evaluated individually, this method can be applied efficiently on high-dimensional data. Using the selected features to generate rules in fuzzy rule-based classifiers, this paper also presents a novel criterion to assess each generated rule. This criterion measures the capability of each fuzzy rule in discriminating the positive and negative patterns. To illustrate the scalability of our fuzzy feature selection method beside to the efficiency of generated fuzzy rules, they are applied on some benchmark data sets and the results are compared to some other methods in the literature. The experimental results justify the feasibility of our approach to work with high-dimensional data and its acceptable performance in terms of designing CPU time and classification accuracy.