Counterfactual Generation Research Articles

Integrating federated learning for improved counterfactual explanations in clinical decision support systems for sepsis therapy

In recent years, we have witnessed both artificial intelligence obtaining remarkable results in clinical decision support systems (CDSSs) and explainable artificial intelligence (XAI) improving the interpretability of these models. In turn, this fosters the adoption by medical personnel and improves trustworthiness of CDSSs. Among others, counterfactual explanations prove to be one such XAI technique particularly suitable for the healthcare domain due to its ease of interpretation, even for less technically proficient staff. However, the generation of high-quality counterfactuals relies on generative models for guidance. Unfortunately, training such models requires a huge amount of data that is beyond the means of ordinary hospitals. In this paper, we therefore propose to use federated learning to allow multiple hospitals to jointly train such generative models while maintaining full data privacy. We demonstrate the superiority of our approach compared to locally generated counterfactuals. Moreover, we prove that generative models for counterfactual generation that are trained using federated learning in a suitable environment perform only marginally worse compared to centrally trained ones while offering the benefit of data privacy preservation. Finally, we integrate our method into a prototypical CDSS for treatment recommendation for sepsis patients, thus providing a proof of concept for real-world application as well as insights and sanity checks from clinical application.

Discord-based counterfactual explanations for time series classification

The opacity inherent in machine learning models presents a significant hindrance to their widespread incorporation into decision-making processes. To address this challenge and foster trust among stakeholders while ensuring decision fairness, the data mining community has been actively advancing the explainable artificial intelligence paradigm. This paper contributes to the evolving field by focusing on counterfactual generation for time series classification models, a domain where research is relatively scarce. We develop, a post-hoc, model agnostic counterfactual explanation algorithm that leverages the Matrix Profile to map time series discords to their nearest neighbors in a target sequence and use this mapping to generate new counterfactual instances. To our knowledge, this is the first effort towards the use of time series discords for counterfactual explanations. We evaluate our algorithm on the University of California Riverside and University of East Anglia archives and compare it to three state-of-the-art univariate and multivariate methods.

Robust anomaly detection via adversarial counterfactual generation

The capability to devise robust outlier and anomaly detection tools is an important research topic in machine learning and data mining. Recent techniques have been focusing on reinforcing detection with sophisticated data generation tools that successfully refine the learning process by generating variants of the data that expand the recognition capabilities of the outlier detector. In this paper, we propose ARN\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\ extrm{ARN}$$\\end{document}, a semi-supervised anomaly detection and generation method based on adversarial counterfactual reconstruction. ARN\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\ extrm{ARN}$$\\end{document} exploits a regularized autoencoder to optimize the reconstruction of variants of normal examples with minimal differences that are recognized as outliers. The combination of regularization and counterfactual reconstruction helps to stabilize the learning process, which results in both realistic outlier generation and substantially extended detection capability. In fact, the counterfactual generation enables a smart exploration of the search space by successfully relating small changes in all the actual samples from the true distribution to high anomaly scores. Experiments on several benchmark datasets show that our model improves the current state of the art by valuable margins because of its ability to model the true boundaries of the data manifold.

Benchmarking Instance-Centric Counterfactual Algorithms for XAI: From White Box to Black Box

This study investigates the impact of machine learning models on the generation of counterfactual explanations by conducting a benchmark evaluation over three different types of models: a decision tree (fully transparent, interpretable, white-box model), a random forest (semi-interpretable, grey-box model), and a neural network (fully opaque, black-box model). We tested the counterfactual generation process using four algorithms (DiCE, WatcherCF, prototype, and GrowingSpheresCF) in the literature in 25 different datasets. Our findings indicate that: (1) Different machine learning models have little impact on the generation of counterfactual explanations; (2) Counterfactual algorithms based uniquely on proximity loss functions are not actionable and will not provide meaningful explanations; (3) One cannot have meaningful evaluation results without guaranteeing plausibility in the counterfactual generation. Algorithms that do not consider plausibility in their internal mechanisms will lead to biased and unreliable conclusions if evaluated with the current state-of-the-art metrics; (4) A counterfactual inspection analysis is strongly recommended to ensure a robust examination of counterfactual explanations and the potential identification of biases.

Overcoming resistance to belief revision and correction of misinformation beliefs: psychophysiological and behavioral effects of a counterfactual mindset

In a series of experiments involving beliefs and misinformation beliefs, we find that individuals who are prompted with a counterfactual mindset are significantly more likely to change their existing beliefs when presented with evidence that contradicts their beliefs. While research finds that beliefs that are considered part of one’s identity are highly resistant to change in the face of evidence that challenges these beliefs, four experiments provide evidence that counterfactual generation causes individuals to adjust beliefs and correct misinformation beliefs in response to contradicting evidence. Indeed, we find that a counterfactual mindset was effective in promoting incorporation of accurate facts and causing individuals to revise misinformation beliefs about COVID vaccination safety for a large sample of individuals who have rejected COVID vaccinations. Finally, the results of the psychophysiological experiment reveal that counterfactual generation alters decision makers’ search strategies, increases their cognitive arousal in response to evidence that challenges their beliefs, and increases their desire to seek out disconfirming evidence. Overall, the four experiments indicate that counterfactual generation can effectively activate mindsets that increase individuals’ willingness to evaluate evidence that contradicts their beliefs and adjust their beliefs in response to evidence.

Ijuice: integer JUstIfied counterfactual explanations

Counterfactual explanations modify the feature values of an instance in order to alter its prediction from an undesired to a desired label. As such, they are highly useful for providing trustworthy interpretations of decision-making in domains where complex and opaque machine learning algorithms are utilized. To guarantee their quality and promote user trust, they need to satisfy the faithfulness desideratum, when supported by the data distribution. We hereby propose a counterfactual generation algorithm for mixed-feature spaces that prioritizes faithfulness through k-justification, a novel counterfactual property introduced in this paper. The proposed algorithm employs a graph representation of the search space and provides counterfactuals by solving an integer program. In addition, the algorithm is classifier-agnostic and is not dependent on the order in which the feature space is explored. In our empirical evaluation, we demonstrate that it guarantees k-justification while showing comparable performance to state-of-the-art methods in feasibility, sparsity, and proximity.

Glacier: guided locally constrained counterfactual explanations for time series classification

In machine learning applications, there is a need to obtain predictive models of high performance and, most importantly, to allow end-users and practitioners to understand and act on their predictions. One way to obtain such understanding is via counterfactuals, that provide sample-based explanations in the form of recommendations on which features need to be modified from a test example so that the classification outcome of a given classifier changes from an undesired outcome to a desired one. This paper focuses on the domain of time series classification, more specifically, on defining counterfactual explanations for univariate time series. We propose Glacier, a model-agnostic method for generating locally-constrained counterfactual explanations for time series classification using gradient search either on the original space or on a latent space that is learned through an auto-encoder. An additional flexibility of our method is the inclusion of constraints on the counterfactual generation process that favour applying changes to particular time series points or segments while discouraging changing others. The main purpose of these constraints is to ensure more reliable counterfactuals, while increasing the efficiency of the counterfactual generation process. Two particular types of constraints are considered, i.e., example-specific constraints and global constraints. We conduct extensive experiments on 40 datasets from the UCR archive, comparing different instantiations of Glacier against three competitors. Our findings suggest that Glacier outperforms the three competitors in terms of two common metrics for counterfactuals, i.e., proximity and compactness. Moreover, Glacier obtains comparable counterfactual validity compared to the best of the three competitors. Finally, when comparing the unconstrained variant of Glacier to the constraint-based variants, we conclude that the inclusion of example-specific and global constraints yields a good performance while demonstrating the trade-off between the different metrics.

Systematic generation and analysis of counterfactuals for compound activity predictions using multi-task models.

Most machine learning (ML) methods produce predictions that are hard or impossible to understand. The black box nature of predictive models obscures potential learning bias and makes it difficult to recognize and trace problems. Moreover, the inability to rationalize model decisions causes reluctance to accept predictions for experimental design. For ML, limited trust in predictions presents a substantial problem and continues to limit its impact in interdisciplinary research, including early-phase drug discovery. As a desirable remedy, approaches from explainable artificial intelligence (XAI) are increasingly applied to shed light on the ML black box and help to rationalize predictions. Among these is the concept of counterfactuals (CFs), which are best understood as test cases with small modifications yielding opposing prediction outcomes (such as different class labels in object classification). For ML applications in medicinal chemistry, for example, compound activity predictions, CFs are particularly intuitive because these hypothetical molecules enable immediate comparisons with actual test compounds that do not require expert ML knowledge and are accessible to practicing chemists. Such comparisons often reveal structural moieties in compounds that determine their predictions and can be further investigated. Herein, we adapt and extend a recently introduced concept for the systematic generation of molecular CFs to multi-task predictions of different classes of protein kinase inhibitors, analyze CFs in detail, rationalize the origins of CF formation in multi-task modeling, and present exemplary explanations of predictions.

Score-Based Counterfactual Generation for Interpretable Medical Image Classification and Lesion Localization.

Deep neural networks (DNNs) have immense potential for precise clinical decision-making in the field of biomedical imaging. However, accessing high-quality data is crucial for ensuring the high-performance of DNNs. Obtaining medical imaging data is often challenging in terms of both quantity and quality. To address these issues, we propose a score-based counterfactual generation (SCG) framework to create counterfactual images from latent space, to compensate for scarcity and imbalance of data. In addition, some uncertainties in external physical factors may introduce unnatural features and further affect the estimation of the true data distribution. Therefore, we integrated a learnable FuzzyBlock into the classifier of the proposed framework to manage these uncertainties. The proposed SCG framework can be applied to both classification and lesion localization tasks. The experimental results revealed a remarkable performance boost in classification tasks, achieving an average performance enhancement of 3-5% compared to previous state-of-the-art (SOTA) methods in interpretable lesion localization.

A Trustworthy Counterfactual Explanation Method with Latent Space Smoothing.

Despite the large-scale adoption of Artificial Intelligence (AI) models in healthcare, there is an urgent need for trustworthy tools to rigorously backtrack the model decisions so that they behave reliably. Counterfactual explanations take a counter-intuitive approach to allow users to explore "what if" scenarios gradually becoming popular in the trustworthy field. However, most previous work on model's counterfactual explanation cannot generate in-distribution attribution credibly, produces adversarial examples, or fails to give a confidence interval for the explanation. Hence, in this paper, we propose a novel approach that generates counterfactuals in locally smooth directed semantic embedding space, and at the same time gives an uncertainty estimate in the counterfactual generation process. Specifically, we identify low-dimensional directed semantic embedding space based on Principal Component Analysis (PCA) applied in differential generative model. Then, we propose latent space smoothing regularization to rectify counterfactual search within in-distribution, such that visually-imperceptible changes are more robust to adversarial perturbations. Moreover, we put forth an uncertainty estimation framework for evaluating counterfactual uncertainty. Extensive experiments on several challenging realistic Chest X-ray and CelebA datasets show that our approach performs consistently well and better than the existing several state-of-the-art baseline approaches.

CLICK: Integrating Causal Inference and Commonsense Knowledge Incorporation for Counterfactual Story Generation

Counterfactual reasoning explores what could have happened if the circumstances were different from what actually occurred. As a crucial subtask, counterfactual story generation integrates counterfactual reasoning into the generative narrative chain, which requires the model to preserve minimal edits and ensure narrative consistency. Previous work prioritizes conflict detection as a first step, and then replaces conflicting content with appropriate words. However, these methods mainly face two challenging issues: (a) the causal relationship between story event sequences is not fully utilized in the conflict detection stage, leading to inaccurate conflict detection, and (b) the absence of proper planning in the content rewriting stage results in a lack of narrative consistency in the generated story ending. In this paper, we propose a novel counterfactual generation framework called CLICK based on causal inference in event sequences and commonsense knowledge incorporation. To address the first issue, we utilize the correlation between adjacent events in the story ending to iteratively calculate the contents from the original ending affected by the condition. The content with the original condition is then effectively prevented from carrying over into the new story ending, thereby avoiding causal conflict with the counterfactual conditions. Considering the second issue, we incorporate structural commonsense knowledge about counterfactual conditions, equipping the framework with comprehensive background information on the potential occurrence of counterfactual conditional events. Through leveraging a rich hierarchical data structure, CLICK gains the ability to establish a more coherent and plausible narrative trajectory for subsequent storytelling. Experimental results show that our model outperforms previous unsupervised state-of-the-art methods and achieves gains of 2.65 in BLEU, 4.42 in ENTScore, and 3.84 in HMean on the TIMETRAVEL dataset.

FS-SCF network: Neural network interpretability based on counterfactual generation and feature selection for fault diagnosis

Interpretability of neural networks aims at the development of models that can give information to the end-user about its inner workings and/or predictions, while keeping the high levels of performance of neural networks. In the context of fault diagnosis, interpretability is necessary for bias detection, model debugging and building trust. This is particularly relevant in the field due to the frequent applications in safety–critical systems, in which an undetected failure can result in highly detrimental outcomes. In this paper, a novel interpretable neural network model is proposed, referred to as Feature Selection and Sparse Counterfactual Generation (FS-SCF) network. It is a multi-task neural network divided into two branches: one for fault diagnosis and feature selection, and one for counterfactual generation. Thus, the model can be interpreted in terms of what are the most relevant features, and the obtained individual predictions are interpreted through counterfactuals. Also, the generated counterfactuals are used to evaluate necessity and sufficiency of each feature. The obtained rankings are then compared to the ranking obtained through the feature selection technique. The proposed approach is tested in two case studies with real data from the industry. Results show that the proposed neural network is able to achieve high levels of performance for fault diagnosis, generate interpretable counterfactuals, and determine the importance of each feature according to a criterion similar to both necessity and sufficiency.

Counterfactual Generation Framework for Few-Shot Learning

Few-shot learning (FSL) that aims to recognize novel classes with few labeled samples is troubled by its data scarcity. Though recent works tackle FSL with data augmentation-based methods, these models fail to maintain the discrimination and diversity of the generated samples due to the distribution shift and intra-class bias caused by the data scarcity, therefore greatly undermining the performance. To this end, we use causal mechanisms, which are constant among independent variables across data distribution, to alleviate such effects. In this sense, we decompose the image information into two independent components: sample-specific and class-agnostic information, and further propose a novel Counterfactual Generation Framework (CGF) to learn the underlying causal mechanisms to synthesize faithful samples for FSL. Specifically, based on the counterfactual inference, we design a class-agnostic feature extractor to capture the sample-specific information, together with a counterfactual generation network to simulate the data generation process from a causal perspective. Moreover, to leverage the power of CGF in counterfactual inference, we further develop a novel classifier that classifies samples based on their distributions of counterfactual generations. Extensive experiments demonstrate the effectiveness of CGF on four FSL benchmarks, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">e.g</i> ., 80.12/86.13% accuracy on 5-way 1/5-shot miniImageNet FSL tasks, significantly improving the performance. Our codes and models are available at https://github.com/eric-hang/CGF.

Mitigating Bias in E-commerce Recommendation Systems: A Causal Inference Approach

Recommendation systems play a crucial role in e-commerce platforms, significantly influencing user experiences and business outcomes. However, these systems often suffer from various biases, leading to suboptimal recommendations and potential unfairness. This paper presents a comprehensive framework for leveraging causal inference techniques to debias recommendation systems in e-commerce contexts. By integrating structural causal models, counterfactual reasoning, and interventional methods, we propose a robust approach to identify and mitigate different types of biases in recommender systems. Our methodology encompasses bias detection, causal model construction, counterfactual generation, and debiased ranking algorithms. The proposed framework aims to improve recommendation fairness and accuracy while maintaining business relevance. This research contributes to the fields of recommender systems and causal inference, providing practitioners with advanced tools for developing more equitable and effective e-commerce platforms.

Explaining Black Box Drug Target Prediction Through Model Agnostic Counterfactual Samples.

Many high-performance DTA deep learning models have been proposed, but they are mostly black-box and thus lack human interpretability. Explainable AI (XAI) can make DTA models more trustworthy, and allows to distill biological knowledge from the models. Counterfactual explanation is one popular approach to explaining the behaviour of a deep neural network, which works by systematically answering the question "How would the model output change if the inputs were changed in this way?". We propose a multi-agent reinforcement learning framework, Multi-Agent Counterfactual Drug-target binding Affinity (MACDA), to generate counterfactual explanations for the drug-protein complex. Our proposed framework provides human-interpretable counterfactual instances while optimizing both the input drug and target for counterfactual generation at the same time. We benchmark the proposed MACDA framework using the Davis and PDBBind dataset and find that our framework produces more parsimonious explanations with no loss in explanation validity, as measured by encoding similarity. We then present a case study involving ABL1 and Nilotinib to demonstrate how MACDA can explain the behaviour of a DTA model in the underlying substructure interaction between inputs in its prediction, revealing mechanisms that align with prior domain knowledge.

Bayesian Spatio-Temporal Prediction and Counterfactual Generation: An Application in Non-Pharmaceutical Interventions in COVID-19.

The spatio-temporal course of an epidemic (such as COVID-19) can be significantly affected by non-pharmaceutical interventions (NPIs) such as full or partial lockdowns. Bayesian Susceptible-Infected-Removed (SIR) models can be applied to the spatio-temporal spread of infectious diseases (STIFs) (such as COVID-19). In causal inference, it is classically of interest to investigate the counterfactuals. In the context of STIF, it is possible to use nowcasting to assess the possible counterfactual realization of disease in an incidence that would have been evidenced with no NPI. Classic lagged dependency spatio-temporal IF models are discussed, and the importance of the ST component in nowcasting is assessed. Real examples of lockdowns for COVID-19 in two US states during 2020 and 2021 are provided. The degeneracy in prediction over longer time periods is highlighted, and the wide confidence intervals characterize the forecasts. For SC, the early and short lockdown contrasted with the longer NJ intervention. The approach here demonstrated marked differences in spatio-temporal disparities across counties with respect to an adherence to counterfactual predictions.

Feature-Weighted Counterfactual-Based Explanation for Bankruptcy Prediction

In recent years, there have been many studies on the application and implementation of machine learning techniques in the financial domain. Implementation of such state-of-the-art models inevitably requires interpretability for users to understand the result and trust. However, as most of the machine learning methods are “black-box,” explainable AI, which aims to provide explanations to users, has become an important research issue. This paper focuses on explanation by counterfactual example for a bankruptcy-prediction model. Counterfactual-based explanation offers an alternative case for users in order for them to have a desired output from the model. This paper proposes a genetic algorithm (GA)-based counterfactual generation algorithm using feature importance whilst taking other key factors into account. Feature importance was derived from a prediction model, and key factors for counterfactuals include closeness to the original dataset and sparsity. The proposed method presented advantages over the nearest contrastive sample and a simple counterfactual generation algorithm in the experiment. Also, it provides relevant and compact explanations to enhance the interpretability of the bankruptcy prediction model.

A Framework and Benchmarking Study for Counterfactual Generating Methods on Tabular Data

Counterfactual explanations are viewed as an effective way to explain machine learning predictions. This interest is reflected by a relatively young literature with already dozens of algorithms aiming to generate such explanations. These algorithms are focused on finding how features can be modified to change the output classification. However, this rather general objective can be achieved in different ways, which brings about the need for a methodology to test and benchmark these algorithms. The contributions of this work are manifold: First, a large benchmarking study of 10 algorithmic approaches on 22 tabular datasets is performed, using nine relevant evaluation metrics; second, the introduction of a novel, first of its kind, framework to test counterfactual generation algorithms; third, a set of objective metrics to evaluate and compare counterfactual results; and, finally, insight from the benchmarking results that indicate which approaches obtain the best performance on what type of dataset. This benchmarking study and framework can help practitioners in determining which technique and building blocks most suit their context, and can help researchers in the design and evaluation of current and future counterfactual generation algorithms. Our findings show that, overall, there’s no single best algorithm to generate counterfactual explanations as the performance highly depends on properties related to the dataset, model, score, and factual point specificities.

Bilateral Asymmetry Guided Counterfactual Generating Network for Mammogram Classification.

Mammogram benign or malignant classification with only image-level labels is challenging due to the absence of lesion annotations. Motivated by the symmetric prior that the lesions on one side of breasts rarely appear in the corresponding areas on the other side, we explore to answer a counterfactual question to identify the lesion areas. This counterfactual question means: given an image with lesions, how would the features have behaved if there were no lesions in the image? To answer this question, we derive a new theoretical result based on the symmetric prior. Specifically, by building a causal model that entails such a prior for bilateral images, we identify to optimize the distances in distribution between i) the counterfactual features and the target side's features in lesion-free areas; and ii) the counterfactual features and the reference side's features in lesion areas. To realize these optimizations for better benign/malignant classification, we propose a counterfactual generative network, which is mainly composed of Generator Adversarial Network and a prediction feedback mechanism, they are optimized jointly and prompt each other. Specifically, the former can further improve the classi?cation performance by generating counterfactual features to calculate lesion areas. On the other hand, the latter helps counterfactual generation by the supervision of classification loss. The utility of our method and the effectiveness of each module in our model can be verified by state-of-the-art performance on INBreast and an in-house dataset and ablation studies.

Detecting Source Contextual Barriers for Understanding Neural Machine Translation

In machine translation evaluation, the traditional wisdom measures model's generalization ability in an average sense, for example by using corpus BLEU. However, the statistics of corpus BLEU cannot provide comprehensive understanding and fine-grained analysis on model's generalization ability. As a remedy, this paper attempts to understand NMT at fine-grained level, by detecting contextual barriers within an unseen input sentence that cause the degradation in model's translation quality. It proposes a principled definition of source contextual barriers as well as its modified version which is tractable in computation and operates at word-level. Based on the modified one, three simple methods are proposed for barrier detection by search-aware risk estimation through counterfactual generation. Extensive analyses are conducted on those detected contextual barrier words on both Zh $\Leftrightarrow$ En NIST benchmarks. Potential usages motivated from barrier words are also discussed.