Rule-based Explanations Research Articles

Generation of Rule-Based Explanations of CNN Classifiers Using Regional Features

Although Deep Learning networks generally outperform traditional machine learning approaches based on tailored features, they often lack explainability. To address this issue, numerous methods have been proposed, particularly for image-related tasks such as image classification or object segmentation. These methods generate a heatmap that visually explains the classification problem by identifying the most important regions for the classifier. However, these explanations remain purely visual. To overcome this limitation, we introduce a novel CNN explainability method that identifies the most relevant regions in an image and generates a decision tree based on meaningful regional features, providing a rule-based explanation of the classification model. We evaluated the proposed method on a synthetic blob’s dataset and subsequently applied it to two cell image classification datasets with healthy and pathological patterns.

Rule-Based Explanations of Machine Learning Classifiers Using Knowledge Graphs

The use of symbolic knowledge representation and reasoning as a way to resolve the lack of transparency of machine learning classifiers is a research area that has lately gained a lot of traction. In this work, we use knowledge graphs as the underlying framework providing the terminology for representing explanations for the operation of a machine learning classifier escaping the constraints of using the features of raw data as a means to express the explanations, providing a promising solution to the problem of the understandability of explanations. In particular, given a description of the application domain of the classifier in the form of a knowledge graph, we introduce a novel theoretical framework for representing explanations of its operation, in the form of query-based rules expressed in the terminology of the knowledge graph. This allows for explaining opaque black-box classifiers, using terminology and information that is independent of the features of the classifier and its domain of application, leading to more understandable explanations but also allowing the creation of different levels of explanations according to the final end-user.

Towards Transparent Healthcare: Advancing Local Explanation Methods in Explainable Artificial Intelligence.

This paper focuses on the use of local Explainable Artificial Intelligence (XAI) methods, particularly the Local Rule-Based Explanations (LORE) technique, within healthcare and medical settings. It emphasizes the critical role of interpretability and transparency in AI systems for diagnosing diseases, predicting patient outcomes, and creating personalized treatment plans. While acknowledging the complexities and inherent trade-offs between interpretability and model performance, our work underscores the significance of local XAI methods in enhancing decision-making processes in healthcare. By providing granular, case-specific insights, local XAI methods like LORE enhance physicians' and patients' understanding of machine learning models and their outcome. Our paper reviews significant contributions to local XAI in healthcare, highlighting its potential to improve clinical decision making, ensure fairness, and comply with regulatory standards.

An eXplainable Artificial Intelligence Methodology on Big Data Architecture

Although artificial intelligence has become part of everyone’s real life, a trust crisis against such systems is occurring, thus increasing the need to explain black-box predictions, especially in the military, medical, and financial domains. Modern eXplainable Artificial Intelligence (XAI) techniques focus on benchmark datasets, but the cognitive applicability of such solutions under big data settings is still unclear due to memory or computation constraints. In this paper, we extend a model-agnostic XAI methodology, named Cluster-Aided Space Transformation for Local Explanation (CASTLE), to be able to deal with high-volume datasets. CASTLE aims to explain the black-box behavior of predictive models by combining both local (i.e., based on the input sample) and global (i.e., based on the whole scope for action of the model) information. In particular, the local explanation provides a rule-based explanation for the prediction of a target instance as well as the directions to update the likelihood of the predicted class. Our extension leverages modern big data technologies (e.g., Apache Spark) to handle the high volume, variety, and velocity of huge datasets. We have evaluated the framework on five datasets, in terms of temporal efficiency, explanation quality, and model significance. Our results indicate that the proposed approach retains the high-quality explanations associated with CASTLE while efficiently handling large datasets. Importantly, it exhibits a sub-linear, rather than exponential, dependence on dataset size, making it a scalable solution for massive datasets or in any big data scenario.

Exploring the Explainable Aspects and Performance of a Learnable Evolutionary Multiobjective Optimization Method

Multiobjective optimization problems have multiple conflicting objective functions to be optimized simultaneously. The solutions to these problems are known as Pareto optimal solutions, which are mathematically incomparable. Thus, a decision maker must be employed to provide preferences to find the most preferred solution. However, decision makers often lack support in providing preferences and insights in exploring the solutions available. We explore the combination of learnable evolutionary models with interactive indicator-based evolutionary multiobjective optimization to create a learnable evolutionary multiobjective optimization method. Furthermore, we leverage interpretable machine learning to provide decision makers with potential insights about the problem being solved in the form of rule-based explanations. In fact, we show that a learnable evolutionary multiobjective optimization method can offer advantages in the search for solutions to a multiobjective optimization problem. We also provide an open source software framework for other researchers to implement and explore our ideas in their own works. Our work is a step toward establishing a new paradigm in the field on multiobjective optimization: explainable and learnable multiobjective optimization . We take the first steps toward this new research direction and provide other researchers and practitioners with necessary tools and ideas to further contribute to this field.

Global and local interpretability techniques of supervised machine learning black box models for numerical medical data

The most effective machine learning classification techniques, such as artificial neural networks, are not easily interpretable, which limits their usefulness in critical areas, such as medicine, where errors can have severe consequences. Researchers have been working to balance the trade-off between the model performance and interpretability. In this study, seven interpretability techniques (global surrogate, accumulated local effects, local interpretable model-agnostic explanations (LIME), Shapley additive explanations (SHAP), model agnostic post hoc local explanations (MAPLE), local rule-based explanation (LORE), and Contextual Importance and Utility (CIU)) were evaluated to interpret five medical classifiers (multilayer perceptron, support vector machines, random forests, extreme gradient boosting, and naïve bayes) using six model performance metrics and three interpretability technique metrics across six medical numerical datasets. The results confirmed the effectiveness of integrating global and local interpretability techniques, and highlighted the superior performance of global SHAP explainer and local CIU explanations. The quantitative evaluations of explanations emphasised the importance of assessing these interpretability techniques before employing them to interpret black box models.

Understanding Any Time Series Classifier with a Subsequence-based Explainer

The growing availability of time series data has increased the usage of classifiers for this data type. Unfortunately, state-of-the-art time series classifiers are black-box models and, therefore, not usable in critical domains such as healthcare or finance, where explainability can be a crucial requirement. This paper presents a framework to explain the predictions of any black-box classifier for univariate and multivariate time series. The provided explanation is composed of three parts. First, a saliency map highlighting the most important parts of the time series for the classification. Second, an instance-based explanation exemplifies the black-box’s decision by providing a set of prototypical and counterfactual time series. Third, a factual and counterfactual rule-based explanation, revealing the reasons for the classification through logical conditions based on subsequences that must, or must not, be contained in the time series. Experiments and benchmarks show that the proposed method provides faithful, meaningful, stable, and interpretable explanations.

Local Rule-Based Explanations Method Based on Genetic Algorithms with Fitness Sharing

The Local Rule Based Explanations method (LORE) explains decisions of black-box classifiers by using an interpretable model (Decision Tree – DT). The DT is trained with an artificial dataset generated by Genetic Algorithms (GAs). The primary objective of this approach is to replicate the decision boundaries of the black-box model in proximity to the instance under explanation. We show that the artificial examples generated by the GAs in LORE are not necessarily diverse. Consequently, we propose the integration of GAs with fitness sharing in LORE to generate a more diversified subset of artificial examples. The underlying motivation is to ensure that the local decision boundaries of the DT more closely resemble those of the black-box classifier. Experimental results with two classifiers (Multilayer Perceptron and Random Forests), and four classification problems, indicate that LORE with fitness sharing yields more diverse GA populations, consequently leading to improved local explanations. These findings underscore the effectiveness of incorporating fitness sharing into the LORE methodology for enhancing the explainability of black-box classifiers.

Information Lattice Learning

We propose Information Lattice Learning (ILL) as a general framework to learn rules of a signal (e.g., an image or a probability distribution). In our definition, a rule is a coarsened signal used to help us gain one interpretable insight about the original signal. To make full sense of what might govern the signal’s intrinsic structure, we seek multiple disentangled rules arranged in a hierarchy, called a lattice. Compared to representation/rule-learning models optimized for a specific task (e.g., classification), ILL focuses on explainability: it is designed to mimic human experiential learning and discover rules akin to those humans can distill and comprehend. This paper details the math and algorithms of ILL, and illustrates how it addresses the fundamental question “what makes X an X” by creating rule-based explanations designed to help humans understand. Our focus is on explaining X rather than (re)generating it. We present applications in knowledge discovery, using ILL to distill music theory from scores and chemical laws from molecules and further revealing connections between them. We show ILL’s efficacy and interpretability on benchmarks and assessments, as well as a demonstration of ILL-enhanced classifiers achieving human-level digit recognition using only one or a few MNIST training examples (1–10 per class).

Unlocking the Black Box: Explainable Artificial Intelligence (XAI) for Trust and Transparency in AI Systems

Explainable Artificial Intelligence (XAI) has emerged as a critical field in AI research, addressing the lack of transparency and interpretability in complex AI models. This conceptual review explores the significance of XAI in promoting trust and transparency in AI systems. The paper analyzes existing literature on XAI, identifies patterns and gaps, and presents a coherent conceptual framework. Various XAI techniques, such as saliency maps, attention mechanisms, rule-based explanations, and model-agnostic approaches, are discussed to enhance interpretability. The paper highlights the challenges posed by black-box AI models, explores the role of XAI in enhancing trust and transparency, and examines the ethical considerations and responsible deployment of XAI. By promoting transparency and interpretability, this review aims to build trust, encourage accountable AI systems, and contribute to the ongoing discourse on XAI.

On clustering and interpreting with rules by means of mathematical optimization

In this paper, we make Cluster Analysis more interpretable with a new approach that simultaneously allocates individuals to clusters and gives rule-based explanations to each cluster. The traditional homogeneity metric in clustering, namely the sum of the dissimilarities between individuals in the same cluster, is enriched by considering also, for each cluster and its associated explanation, two explainability criteria, namely, the accuracy of the explanation, i.e., how many individuals within the cluster satisfy its explanation, and the distinctiveness of the explanation, i.e., how many individuals outside the cluster satisfy its explanation. Finding the clusters and the explanations optimizing a joint measure of homogeneity, accuracy, and distinctiveness is formulated as a multi-objective Mixed Integer Linear Optimization problem, from which non-dominated solutions are generated. Our approach is tested on real-world datasets.

Computing Rule-Based Explanations by Leveraging Counterfactuals

Sophisticated machine models are increasingly used for high-stakes decisions in everyday life. There is an urgent need to develop effective explanation techniques for such automated decisions. Rule-Based Explanations have been proposed for high-stake decisions like loan applications, because they increase the users' trust in the decision. However, rule-based explanations are very inefficient to compute, and existing systems sacrifice their quality in order to achieve reasonable performance. We propose a novel approach to compute rule-based explanations, by using a different type of explanation, Counterfactual Explanations, for which several efficient systems have already been developed. We prove a Duality Theorem, showing that rule-based and counterfactual-based explanations are dual to each other, then use this observation to develop an efficient algorithm for computing rule-based explanations, which uses the counterfactual-based explanation as an oracle. We conduct extensive experiments showing that our system computes rule-based explanations of higher quality, and with the same or better performance, than two previous systems, MinSetCover and Anchor.

LIMREF: Local Interpretable Model Agnostic Rule-Based Explanations for Forecasting, with an Application to Electricity Smart Meter Data

Accurate electricity demand forecasts play a key role in sustainable power systems. To enable better decision-making especially for demand flexibility of the end-user, it is necessary to provide not only accurate but also understandable and actionable forecasts. To provide accurate forecasts Global Forecasting Models (GFM) that are trained across time series have shown superior results in many demand forecasting competitions and real-world applications recently, compared with univariate forecasting approaches. We aim to fill the gap between the accuracy and the interpretability in global forecasting approaches. In order to explain the global model forecasts, we propose Local Interpretable Model-agnostic Rule-based Explanations for Forecasting (LIMREF), which is a local explainer framework that produces k-optimal impact rules for a particular forecast, considering the global forecasting model as a black-box model, in a model-agnostic way. It provides different types of rules which explain the forecast of the global model and the counterfactual rules, which provide actionable insights for potential changes to obtain different outputs for given instances. We conduct experiments using a large-scale electricity demand dataset with exogenous features such as temperature and calendar effects. Here, we evaluate the quality of the explanations produced by the LIMREF framework in terms of both qualitative and quantitative aspects such as accuracy, fidelity and comprehensibility, and benchmark those against other local explainers.

Explanation-Aware Experience Replay in Rule-Dense Environments

Human environments are often regulated by explicit and complex rulesets. Integrating Reinforcement Learning (RL) agents into such environments motivates the development of learning mechanisms that perform well in rule-dense and exception-ridden environments such as autonomous driving on regulated roads. In this paper, we propose a method for organising experience by means of partitioning the experience buffer into clusters labelled on a per-explanation basis. We present discrete and continuous navigation environments compatible with modular rulesets and 9 learning tasks. For environments with explainable rulesets, we convert rule-based explanations into case-based explanations by allocating state-transitions into clusters labelled with explanations. This allows us to sample experiences in a curricular and task-oriented manner, focusing on the rarity, importance, and meaning of events. We label this concept Explanation-Awareness (XA). We perform XA experience replay (XAER) with intra and inter-cluster prioritisation, and introduce XA-compatible versions of DQN, TD3, and SAC. Performance is consistently superior with XA versions of those algorithms, compared to traditional Prioritised Experience Replay baselines, indicating that explanation engineering can be used in lieu of reward engineering for environments with explainable features.

A Comparative Study of Two Rule-Based Explanation Methods for Diabetic Retinopathy Risk Assessment

Understanding the reasons behind the decisions of complex intelligent systems is crucial in many domains, especially in healthcare. Local explanation models analyse a decision on a single instance, by using the responses of the system to the points in its neighbourhood to build a surrogate model. This work makes a comparative analysis of the local explanations provided by two rule-based explanation methods on RETIPROGRAM, a system based on a fuzzy random forest that analyses the health record of a diabetic person to assess his/her degree of risk of developing diabetic retinopathy. The analysed explanation methods are C-LORE-F (a variant of LORE that builds a decision tree) and DRSA (a method based on rough sets that builds a set of rules). The explored methods gave good results in several metrics, although there is room for improvement in the generation of counterfactual examples.

머신러닝 기반 부도예측모형에서 로컬영역의 도메인 지식 통합 규칙 기반 설명 방법

신용리스크 관리에 해당하는 부도예측모형은 기업에 대한 신용평가라고도 볼 수 있으며 은행을 비롯한 금융기관의 신용평가모형의 기본 지식기반으로 새로운 인공지능 기술을 접목할 수 있는 유망한 분야로 손꼽히고 있다. 고도화된 모형의 실제 응용은 사용자의 수용도가 중요하나 부도예측모형의 경우, 금융전문가 혹은 고객에게 모형의 결과에 대한 설명이 요구되는 분야로 설명력이 없는 모형은 실제로 도입되고 사용자들에게 수용되기에는 어려움이 있다. 결국 모형의 결과에 대한 설명은 모형의 사용자에게 제공되는 것으로 사용자가 납득할 수 있는 설명을 제공하는 것이 모형에 대한 신뢰와 수용을 증진시킬 수 있다. 본 연구에서는 머신러닝 기반 모형에 설명력을 제고하는 방안으로 설명대상 인스턴스에 대하여 로컬영역에서의 설명을 제공하고자 한다. 이를 위해 설명대상의 로컬영역에 유전알고리즘(GA)을 이용하여 가상의 데이터포인트들을 생성한 후, 로컬 대리모델(surrogate model)로 연관규칙 알고리즘을 이용하여 설명대상에 대한 규칙기반 설명(rule-based explanation)을 생성한다. 해석 가능한 로컬 모델의 활용으로 설명을 제공하는 기존의 방법에서 더 나아가 본 연구는 부도예측 모형에 이용된 재무변수의 특성을 반영하여 연관규칙으로 도출된 설명에 도메인 지식을 통합한다. 이를 통해 사용자에게 제공되는 규칙의 현실적 가능성(feasibility)을 확보하고 제공되는 설명의 이해와 수용을 제고하고자 한다. 본 연구에서는 대표적인 블랙박스 모형인 인공신경망 기반 부도예측모형을 기반으로 최신의 규칙기반 설명 방법인 Anchor와 비교하였다. 제안하는 방법은 인공신경망 뿐만 아니라 다른 머신러닝 모형에도 적용 가능한 방법(model-agonistic method)이다.

Experimental Analysis of Trustworthy In-Vehicle Intrusion Detection System Using eXplainable Artificial Intelligence (XAI)

Anomaly-based In-Vehicle Intrusion Detection System (IV-IDS) is one of the protection mechanisms to detect cyber attacks on automotive vehicles. Using artificial intelligence (AI) for anomaly detection to thwart cyber attacks is promising but suffers from generating false alarms and making decisions that are hard to interpret. Consequently, this issue leads to uncertainty and distrust towards such IDS design unless it can explain its behavior, e.g., by using eXplainable AI (XAI). In this paper, we consider the XAI-powered design of such an IV-IDS using CAN bus data from a public dataset, named “Survival”. Novel features are engineered, and a Deep Neural Network (DNN) is trained over the dataset. A visualization-based explanation, “VisExp”, is created to explain the behavior of the AI-based IV-IDS, which is evaluated by experts in a survey, in relation to a rule-based explanation. Our results show that experts’ trust in the AI-based IV-IDS is significantly increased when they are provided with VisExp (more so than the rule-based explanation). These findings confirm the effect, and by extension the need, of explainability in automated systems, and VisExp, being a source of increased explainability, shows promise in helping involved parties gain trust in such systems.

Enhancing Cluster Analysis With Explainable AI and Multidimensional Cluster Prototypes

Explainable Artificial Intelligence (XAI) aims to introduce transparency and intelligibility into the decision-making process of AI systems. Most often, its application concentrates on supervised machine learning problems such as classification and regression. Nevertheless, in the case of unsupervised algorithms like clustering, XAI can also bring satisfactory results. In most cases, such application is based on the transformation of an unsupervised clustering task into a supervised one and providing generalised global explanations or local explanations based on cluster centroids. However, in many cases, the global explanations are too coarse, while the centroid-based local explanations lose information about cluster shape and distribution. In this paper, we present a novel approach called CLAMP (Cluster Analysis with Multidimensional Prototypes) that aids experts in cluster analysis with human-readable rule-based explanations. The developed state-of-the-art explanation mechanism is based on cluster prototypes represented by multidimensional bounding boxes. This allows representing of arbitrary shaped clusters and combines the strengths of local explanations with the generality of global ones.We demonstrate and evaluate the use of our approach in a real-life industrial case study from the domain of steel manufacturing as well as on the benchmark datasets. The explanations generated with CLAMP were more precise than either centroid-based or global ones.

A Quantitative Evaluation of Global, Rule-Based Explanations of Post-Hoc, Model Agnostic Methods

Understanding the inferences of data-driven, machine-learned models can be seen as a process that discloses the relationships between their input and output. These relationships consist and can be represented as a set of inference rules. However, the models usually do not explicit these rules to their end-users who, subsequently, perceive them as black-boxes and might not trust their predictions. Therefore, scholars have proposed several methods for extracting rules from data-driven machine-learned models to explain their logic. However, limited work exists on the evaluation and comparison of these methods. This study proposes a novel comparative approach to evaluate and compare the rulesets produced by five model-agnostic, post-hoc rule extractors by employing eight quantitative metrics. Eventually, the Friedman test was employed to check whether a method consistently performed better than the others, in terms of the selected metrics, and could be considered superior. Findings demonstrate that these metrics do not provide sufficient evidence to identify superior methods over the others. However, when used together, these metrics form a tool, applicable to every rule-extraction method and machine-learned models, that is, suitable to highlight the strengths and weaknesses of the rule-extractors in various applications in an objective and straightforward manner, without any human interventions. Thus, they are capable of successfully modelling distinctively aspects of explainability, providing to researchers and practitioners vital insights on what a model has learned during its training process and how it makes its predictions.

Ranking Rule-Based Automatic Explanations for Machine Learning Predictions on Asthma Hospital Encounters in Patients With Asthma: Retrospective Cohort Study.

BackgroundAsthma hospital encounters impose a heavy burden on the health care system. To improve preventive care and outcomes for patients with asthma, we recently developed a black-box machine learning model to predict whether a patient with asthma will have one or more asthma hospital encounters in the succeeding 12 months. Our model is more accurate than previous models. However, black-box machine learning models do not explain their predictions, which forms a barrier to widespread clinical adoption. To solve this issue, we previously developed a method to automatically provide rule-based explanations for the model’s predictions and to suggest tailored interventions without sacrificing model performance. For an average patient correctly predicted by our model to have future asthma hospital encounters, our explanation method generated over 5000 rule-based explanations, if any. However, the user of the automated explanation function, often a busy clinician, will want to quickly obtain the most useful information for a patient by viewing only the top few explanations. Therefore, a methodology is required to appropriately rank the explanations generated for a patient. However, this is currently an open problem.ObjectiveThe aim of this study is to develop a method to appropriately rank the rule-based explanations that our automated explanation method generates for a patient.MethodsWe developed a ranking method that struck a balance among multiple factors. Through a secondary analysis of 82,888 data instances of adults with asthma from the University of Washington Medicine between 2011 and 2018, we demonstrated our ranking method on the test case of predicting asthma hospital encounters in patients with asthma.ResultsFor each patient predicted to have asthma hospital encounters in the succeeding 12 months, the top few explanations returned by our ranking method typically have high quality and low redundancy. Many top-ranked explanations provide useful insights on the various aspects of the patient’s situation, which cannot be easily obtained by viewing the patient’s data in the current electronic health record system.ConclusionsThe explanation ranking module is an essential component of the automated explanation function, and it addresses the interpretability issue that deters the widespread adoption of machine learning predictive models in clinical practice. In the next few years, we plan to test our explanation ranking method on predictive modeling problems addressing other diseases as well as on data from other health care systems.International Registered Report Identifier (IRRID)RR2-10.2196/5039