Rule Learning System Research Articles

From fault detection to anomaly explanation: A case study on predictive maintenance

Predictive Maintenance applications are increasingly complex, with interactions between many components. Black-box models are popular approaches based on deep-learning techniques due to their predictive accuracy. This paper proposes a neural-symbolic architecture that uses an online rule-learning algorithm to explain when the black-box model predicts failures. The proposed system solves two problems in parallel: (i) anomaly detection and (ii) explanation of the anomaly. For the first problem, we use an unsupervised state-of-the-art autoencoder. For the second problem, we train a rule learning system that learns a mapping from the input features to the autoencoder’s reconstruction error. Both systems run online and in parallel. The autoencoder signals an alarm for the examples with a reconstruction error that exceeds a threshold. The causes of the signal alarm are hard for humans to understand because they result from a non-linear combination of sensor data. The rule that triggers that example describes the relationship between the input features and the autoencoder’s reconstruction error. The rule explains the failure signal by indicating which sensors contribute to the alarm and allowing the identification of the component involved in the failure. The system can present global explanations for the black box model and local explanations for why the black box model predicts a failure. We evaluate the proposed system in a real-world case study of Metro do Porto and provide explanations that illustrate its benefits.

A Meta Algorithm for Interpretable Ensemble Learning: The League of Experts

Background. The importance of explainable artificial intelligence and machine learning (XAI/XML) is increasingly being recognized, aiming to understand how information contributes to decisions, the method’s bias, or sensitivity to data pathologies. Efforts are often directed to post hoc explanations of black box models. These approaches add additional sources for errors without resolving their shortcomings. Less effort is directed into the design of intrinsically interpretable approaches. Methods. We introduce an intrinsically interpretable methodology motivated by ensemble learning: the League of Experts (LoE) model. We establish the theoretical framework first and then deduce a modular meta algorithm. In our description, we focus primarily on classification problems. However, LoE applies equally to regression problems. Specific to classification problems, we employ classical decision trees as classifier ensembles as a particular instance. This choice facilitates the derivation of human-understandable decision rules for the underlying classification problem, which results in a derived rule learning system denoted as RuleLoE. Results. In addition to 12 KEEL classification datasets, we employ two standard datasets from particularly relevant domains—medicine and finance—to illustrate the LoE algorithm. The performance of LoE with respect to its accuracy and rule coverage is comparable to common state-of-the-art classification methods. Moreover, LoE delivers a clearly understandable set of decision rules with adjustable complexity, describing the classification problem. Conclusions. LoE is a reliable method for classification and regression problems with an accuracy that seems to be appropriate for situations in which underlying causalities are in the center of interest rather than just accurate predictions or classifications.

Behavioral and Neural Predictors of Individual Differences in Concept Generalization

Concept learning involves linking related pieces of information to a shared label, such as learning that furry creatures that bark are called “dogs.” People vary in how well they learn concepts and apply them to new situations (generalization). What factors drive these individual differences? In the present study, we tested whether stable aspects of intelligence or transient activations in the brain best predicted concept generalization abilities. To measure aspects of intelligence, subjects underwent an assessment that included measures of working memory, processing speed, perceptual reasoning, and verbal comprehension, which could be combined into an overall IQ. Subjects also completed a concept generalization task while undergoing functional MRI, allowing us to measure activations in brain regions that are part of the explicit rule-learning system (hippocampus, prefrontal cortex) or part of an implicit system that learns without awareness (caudate, posterior visual cortex). To elucidate the shared or dissociable roles of behavioral and neural predictors in concept generalization, we tested the relationship between accuracy in concept generalization and individual differences in measures of intelligence and activation in each brain region of interest. Behaviorally, we found that overall IQ, but not its subcomponents, predicted concept generalization abilities. Neurally, we found that only the activation in the hippocampus predicted concept generalization abilities. Finally, we found that IQ and hippocampal activation each predicted concept generalization independent of each other, indicating that they represent two separate processes that both contribute to generalization success. These results show dissociable contributions of behavioral and neural predictors of concept generalization, suggesting that both stable cognitive abilities and transient brain states influence the ability to learn new concepts.

An ensemble system to predict the spatiotemporal distribution of energy security weaknesses in transmission networks

The security of energy supplies requires the depletion of potential fault events in power transmission networks. To achieve this, sufficient lead time before the happening of a fault event is indispensable for preparing countermeasures. With this inspiration, this paper establishes the fuzzy inference with rare association rule learning system. This ensemble system is designed for the long-term prediction of the spatiotemporal distribution of such energy security weaknesses, that is, to predict when and where these events are more expected to appear. In this system, merely the environmental features rather than the electrical features are needed as inputs. All the selected input features are divided into discrete and continuous features, and are evaluated separately. For the discrete features, the rare association rule learning model is implemented so that the rarely distributed environmental elements are extracted and diagnosed specifically. The risk indices of each element on the overall reliability are worked out as well. For the continuous features, a hierarchical fuzzy inference system along with the rare association rule learning model is deployed to calculate the corresponding risk indices of all the elements. In the hierarchical fuzzy inference system, the probabilistic fuzzy risks are employed instead of the direct fuzzy risks. Then the relative weights of these two sides are optimized. At last, an empirical case based on a practical transmission network is conducted, and the flexibility and the robustness of the proposed system during real applications can be validated consequently.

Semi-Automatic Rule Learning Method Enabling Information Extraction for Ontology Population

Business firms around the world have been generating enormous amounts of domain-related documents. Most of these firms are adapting semantic Web-based techniques into their software systems. Hence, they want to semantically enrich their documents to enable more meaningful querying or processing of the information in the documents. To impart semantics into these documents, ontologies relevant to the business domain should be used. In this context, to populate the domain ontology with the information from the source documents, a method for semi-automatic learning of extraction rules for populating the ontology is presented and implemented in the rule learning system. In addition to the rule learning system, a framework for separating the business logic from application logic and storing the business rules and extraction rules in external user-friendly format is presented in brief. The rule learning system is mainly developed to be a part of the presented framework, but it can be used as a standalone system to learn any decision or association rules too. The framework uses the rule learning system for learning extraction rules. The main idea behind the work presented is to learn extraction rules to be used by an information extraction system (part of the framework) to populate the domain ontology. The extraction rules learned by the rule learning system can be used with any business rules management system (BRMS) with appropriate wrappers to populate the domain ontology.

A semantic approach to a framework for business domain software systems

In enterprise firms, enormous amounts of electronic documents are generated by business analysts and other business domain application users. Applications that use these documents are often driven by business logic that is hard-coded together with application logic. One approach to the separation of business logic from applications is to create and maintain business and information extraction rules in an external, user-friendly format. The drawback of such an externalization is that the business rules, usually, do not have machine interpretable semantics. This situation often leads to misinterpretation of domain analysis documents, which can inhibit the productivity of computer-assisted analytical work and the effectiveness of business solutions. This paper proposes an ontology and rule-based framework for the development of business domain applications, which includes semantic processing of externalized business rules and to some extent externalization of application logic. The creation of external information extraction rules by the business analyst is a cumbersome and time consuming task. In order to overcome this problem, the framework also includes a rule learning system to semi-automate the generation of information extraction rules from source documents with the help of manual annotations. The main idea behind the work presented in this paper is to re-engineer very large enterprise information systems to adapt to Semantic Web computing techniques. The work presented in this paper is inspired by an industrial project.

Bayesian rule learning for biomedical data mining

Disease state prediction from biomarker profiling studies is an important problem because more accurate classification models will potentially lead to the discovery of better, more discriminative markers. Data mining methods are routinely applied to such analyses of biomedical datasets generated from high-throughput 'omic' technologies applied to clinical samples from tissues or bodily fluids. Past work has demonstrated that rule models can be successfully applied to this problem, since they can produce understandable models that facilitate review of discriminative biomarkers by biomedical scientists. While many rule-based methods produce rules that make predictions under uncertainty, they typically do not quantify the uncertainty in the validity of the rule itself. This article describes an approach that uses a Bayesian score to evaluate rule models. We have combined the expressiveness of rules with the mathematical rigor of Bayesian networks (BNs) to develop and evaluate a Bayesian rule learning (BRL) system. This system utilizes a novel variant of the K2 algorithm for building BNs from the training data to provide probabilistic scores for IF-antecedent-THEN-consequent rules using heuristic best-first search. We then apply rule-based inference to evaluate the learned models during 10-fold cross-validation performed two times. The BRL system is evaluated on 24 published 'omic' datasets, and on average it performs on par or better than other readily available rule learning methods. Moreover, BRL produces models that contain on average 70% fewer variables, which means that the biomarker panels for disease prediction contain fewer markers for further verification and validation by bench scientists.

A robot based model of the pre-frontal cortex: planning in a rule learning system

Event Abstract Back to Event A robot based model of the pre-frontal cortex: planning in a rule learning system Armin Duff1* and Paul FMJ Verschure2 1 INI-ETH Zürich, Switzerland 2 SPECS-UPF Barcelona, Spain Humans and animals are able to make near optimal use of their knowledge to achieve their goals. This ability is seen as a hallmark of intelligent behavior. One of the core elements of intelligent behavior is planning. On the neuronal level, planning has been shown to involve the pre-frontal cortex (PFC). Two important properties of the PFC critical for planning are 1) sustained activity through lateral connectivity and 2) synaptic modulation mediated through reward. In the context of the Distributed Adaptive Control (DAC) Architecture [1], we investigate to what extent these two mechanisms combined with a rule learning system is sufficient to support real-time planning in behaving systems. The Distributed Adaptive Control (DAC) architecture comprises three tightly coupled layers; reactive, adaptive and contextual. The reactive and adaptive layers provide the robot with basic behaviors to support simple tasks. The contextual layer coordinates the ongoing behavior in the context of higher level behavioral goals. Recently we extended the contextual layer following the organizational principles of the PFC, accounting for rule learning and rule switching. With this extension we can now incorporate planning capabilities in the contextual layer. The core of the contextual layer is a group of laterally interconnected neurons. The sustained activity of these neurons is driven by perceptual inputs and modulated by lateral coupling and a dopamine like reward mechanism. A competition at the contextual level selects the neurons with the highest activity, i.e. relevance given the current input state and the context, that in turn trigger the Bayes optimal behavioral action [2]. The lateral connections of the contextual layer can be seen as a restricted world model and facilitate planning through propagation of activity. The validity of this acquired world model is assessed in both formal tasks and open-environment foraging. Our experiments show that sustained activity and activity modulation, as found in the PFC, can be used to learn and express sequential rules in real time, and that these can be further extended to support planning. The behavioral results show that the DAC framework is suitable for dynamical environments and constitutes a promising basis towards solving complex real world tasks.

Acceptance of Rules Generated by Machine Learning among Medical Experts

The aim was to evaluate the potential for monotonicity constraints to bias machine learning systems to learn rules that were both accurate and meaningful. Two data sets, taken from problems as diverse as screening for dementia and assessing the risk of mental retardation, were collected and a rule learning system, with and without monotonicity constraints, was run on each. The rules were shown to experts, who were asked how willing they would be to use such rules in practice. The accuracy of the rules was also evaluated. Rules learned with monotonicity constraints were at least as accurate as rules learned without such constraints. Experts were, on average, more willing to use the rules learned with the monotonicity constraints. The analysis of medical databases has the potential of improving patient outcomes and/or lowering the cost of health care delivery. Various techniques, from statistics, pattern recognition, machine learning, and neural networks, have been proposed to "mine" this data by uncovering patterns that may be used to guide decision making. This study suggests cognitive factors make learned models coherent and, therefore, credible to experts. One factor that influences the acceptance of learned models is consistency with existing medical knowledge.

Knowledge discovery by inductive neural networks

A new neural network model for inducing symbolic knowledge from empirical data is presented. This model capitalizes on the fact that the certainty factor-based activation function can improve the network generalization performance from a limited amount of training data. The formal properties of the procedure for extracting symbolic knowledge from such a trained neural network are investigated. In the domain of molecular genetics, a case study demonstrated that the described learning system effectively discovered the prior domain knowledge with some degree of refinement. Also, in cross-validation experiments, the system outperformed C4.5, a commonly used rule learning system.

A Computational Study of Distributed Rule Learning

This report is concerned with a rule learning system called the Distributed Learning System (DLS). Its objective is two-fold: First, as the main contribution, the DLS as a rule-learning technique is described and the resulting computational performance is presented, with definitive computational benefits clearly demonstrated to show the efficacy of using the DLS. Second, the important parameters of the DLS are identified to show the characteristics of the Group Problem Solving (GPS) strategy as implemented in the DLS. On one hand this helps us pinpoint the critical designs of the DLS for effective rule learning; on the other hand this analysis can provide insight into the use of GPS as a more general rule-learning strategy.

Scaling up inductive learning with massive parallelism

Machine learning programs need to scale up to very large data sets for several reasons, including increasing accuracy and discovering infrequent special cases. Current inductive learners perform well with hundreds or thousands of training examples, but in some cases, up to a million or more examples may be necessary to learn important special cases with confidence. These tasks are infeasible for current learning programs running on sequential machines. We discuss the need for very large data sets and prior efforts to scale up machine learning methods. This discussion motivates a strategy that exploits the inherent parallelism present in many learning algorithms. We describe a parallel implementation of one inductive learning program on the CM-2 Connection Machine, show that it scales up to millions of examples, and show that it uncovers special-case rules that sequential learning programs, running on smaller datasets, would miss. The parallel version of the learning program is preferable to the sequential version for example sets larger than about 10K examples. When learning from a public-health database consisting of 3.5 million examples, the parallel rule-learning system uncovered a surprising relationship that has led to considerable follow-up research.

Hypothesis-Driven Constructive Induction in AQ17-HCI: A Method and Experiments

The proposed method for constructive induction searches for concept descriptions in a representation space that is being iteratively improved. In each iteration, the system learns concept description from training examples projected into a newly constructed representation space, using an Aq algorithm-based inductive learning system (AQ15). The learned description is analyzed to determine desirable problem-oriented modifications of the representation space. These modifications include generating new attributes, removing redundant or insignificant ones, and/or agglomerating attribute values into larger units. New attributes are constructed by assigning names to groups of the best-performing characteristic rules for each decision class, and then are used to define the representation space for the next iteration. This iterative process repeats until the created hypotheses satisfy a stopping criterion. In several experiments on learning discrete functions, the developed AQ17-HCI system consistently outperformed, in terms of the prediction accuracy on new examples, all systems that it was compared to, including the AQ15 rule learning system, GREEDY3 and GROVE decision-list learning systems, and REDWOOD and FRINGE decision-tree learning systems. Although the proposed method was developed for the Aq-based rule learning system, it can potentially be adapted to any other inductive learning system. In this sense, it represents a universal new approach to constructive induction.

Computer-aided pipeline operation using genetic algorithms and rule learning. PART II: Rule learning control of a pipeline under normal and abnormal conditions

In this two-paper series, techniques connected with artificial intelligence and genetics are applied to the problem of gas pipeline control. In the first paper, genetic algorithms were applied to two pipeline optimization problems. In this, the second paper, genetic algorithms are used as a basic learning mechanism in a larger rule learning system called a learning classifier system. The learning classifier system is developed and applied to the control of a gas pipeline under normal summer and winter operations as well as abnormal operations during leak events.