Counterfactual Explanations in Education: A Systematic Review

In this paper, we propose a methodology designed to deliver actionable insights that help businesses retain customers. While Machine Learning (ML) techniques predict whether a customer is likely to churn, this alone is not enough. Explainable Artificial Intelligence (XAI) methods, such as SHapley Additive Explanations (SHAP) and Local Interpretable Model-agnostic Explanations (LIME), highlight the features influencing the prediction, but businesses need strategies to prevent churn. Counterfactual (CF) explanations bridge this gap by identifying the minimal changes in the business–customer relationship that could shift an outcome from churn to retention, offering steps to enhance customer loyalty and reduce losses to competitors. These explanations might not fully align with business constraints; however, alternative scenarios can be developed to achieve the same objective. Among the six classifiers used to detect churn cases, the Balanced Random Forest classifier was selected for its superior performance, achieving the highest recall score of 0.72. After classification, Diverse Counterfactual Explanations with ML (DiCEML) through Mixed-Integer Linear Programming (MILP) is applied to obtain the required changes in the features, as well as in the range permitted by the business itself. We further apply DiCEML to uncover potential biases within the model, calculating the disparate impact of some features.

Recently, eXplainable AI (XAI) research has focused on the use of counterfactual explanations to address interpretability, algorithmic recourse, and bias in AI system decision-making. The proponents of these algorithms claim they meet users’ requirements for counterfactual explanations. For instance, many claim that the output of their algorithms work as explanations because they prioritise "plausible", "actionable" or "causally important" features in their generated counterfactuals. However, very few of these claims have been tested in controlled psychological studies, and we know very little about which aspects of counterfactual explanations help users to understand AI system decisions. Furthermore, we do not know whether counterfactual explanations are an advance on more traditional causal explanations that have a much longer history in AI (in explaining expert systems and decision trees). Accordingly, we carried out two user studies to (i) test a fundamental distinction in feature-types, between categorical and continuous features, and (ii) compare the relative effectiveness of counterfactual and causal explanations. The studies used a simulated, automated decision-making app that determined safe driving limits after drinking alcohol, based on predicted blood alcohol content, and user responses were measured objectively (users’ predictive accuracy) and subjectively (users’ satisfaction and trust judgments). Study 1 (N=127) showed that users understand explanations referring to categorical features more readily than those referring to continuous features. It also discovered a dissociation between objective and subjective measures: counterfactual explanations elicited higher accuracy of predictions than no-explanation control descriptions but no higher accuracy than causal explanations, yet counterfactual explanations elicited greater satisfaction and trust judgments than causal explanations. Study 2 (N=211) found that users were more accurate for categorically-transformed features compared to continuous ones, and also replicated the results of Study 1. The findings delineate important boundary conditions for current and future counterfactual explanation methods in XAI.

Recently, eXplainable AI (XAI) research has focused on the use of counterfactual explanations to address interpretability, algorithmic recourse, and bias in AI system decision-making. The developers of these algorithms claim they meet user requirements in generating counterfactual explanations with “plausible,” “actionable” or “causally important” features. However, few of these claims have been tested in controlled psychological studies. Hence, we know very little about which aspects of counterfactual explanations really help users understand the decisions of AI systems. Nor do we know whether counterfactual explanations are an advance on more traditional causal explanations that have a longer history in AI (e.g., in expert systems). Accordingly, we carried out three user studies to (1) test a fundamental distinction in feature-types, between categorical and continuous features, and (2) compare the relative effectiveness of counterfactual and causal explanations. The studies used a simulated, automated decision-making app that determined safe driving limits after drinking alcohol, based on predicted blood alcohol content, where users’ responses were measured objectively (using predictive accuracy) and subjectively (using satisfaction and trust judgments). Study 1 ( N \({=}\) 127) showed that users understand explanations referring to categorical features more readily than those referring to continuous features. It also discovered a dissociation between objective and subjective measures: counterfactual explanations elicited higher accuracy than no-explanation controls but elicited no more accuracy than causal explanations, yet counterfactual explanations elicited greater satisfaction and trust than causal explanations. In Study 2 ( N \({=}\) 136) we transformed the continuous features of presented items to be categorical (i.e., binary) and found that these converted features led to highly accurate responding. Study 3 ( N \({=}\) 211) explicitly compared matched items involving either mixed features (i.e., a mix of categorical and continuous features) or categorical features (i.e., categorical and categorically-transformed continuous features), and found that users were more accurate when categorically-transformed features were used instead of continuous ones. It also replicated the dissociation between objective and subjective effects of explanations. The findings delineate important boundary conditions for current and future counterfactual explanation methods in XAI.

Aim: Academic Performance is one of the issues raised in the field of education, which not only affects the academic future of students from various dimensions, but also determines their fate in various fields. Therefore, the current research was conducted with the aim of comparing the effectiveness of an educational package to prevent academic procrastination as well as treatment based on acceptance and commitment on academic motivation and performance of procrastinating students. Methods: The research method was semi-experimental with a pre-test, post-test and follow-up design with a control group. The statistical population included all secondary school students suffering from academic procrastination in city of Isfahan. Among them, 45 people were selected by purposive sampling and randomly divided into two experimental groups (educational package to prevent academic procrastination and treatment based on acceptance and commitment) and a control group (15 people in each group). Research tools included academic procrastination questionnaires (Solomon and Rothblum, 1984), academic motivation (Abdkhodaei et al., 2017) and academic performance (Pham and Taylor, 1999). In order to analyze the data, analysis of variance with repeated measurements was used. Results: The findings showed that the educational package of prevention of academic procrastination and treatment based on acceptance and commitment had a significant effect on the academic motivation and academic performance of procrastinating students (p<0.001) and there is a significant difference between the two interventions, so that the educational package to prevent academic procrastination has been more effective. Conclusion: Those involved in the field of student education are recommended to use the educational package to prevent academic procrastination in order to solve students' academic problems, especially to improve academic motivation and academic performance.

EXplainable Artificial Intelligence ( XAI) methods have gained much momentum lately given their ability to shed the light on the decision function of opaque machine learning models. There are two dominating XAI paradigms: feature attribution and counterfactual explanation methods. While the first family of methods explains why the model made a decision, counterfactual methods aim at answering what-if the input is slightly different and results in another classification decision. Most of the research efforts have focused on answering the why question for time series data modality. In this paper, we aim at answering the what-if question by finding a good balance between a set of desirable counterfactual explanation properties. We propose Shapelet-guided Counterfactual Explanation (SG-CF), a novel optimization-based model that generates interpretable, intuitive post-hoc counterfactual explanations of time series classification models that balance validity, proximity, sparsity, and contiguity. Our experimental results on nine real-world time-series datasets show that our proposed method can generate counterfactual explanations that balance all the desirable counterfactual properties in comparison with other competing baselines.

Counterfactual explanations are increasingly used as an Explainable Artificial Intelligence (XAI) technique to provide stakeholders of complex machine learning algorithms with explanations for data-driven decisions. The popularity of counterfactual explanations resulted in a boom in the algorithms generating them. However, not every algorithm creates uniform explanations for the same instance. Even though in some contexts multiple possible explanations are beneficial, there are circumstances where diversity amongst counterfactual explanations results in a potential disagreement problem among stakeholders. Ethical issues arise when for example, malicious agents use this diversity to fairwash an unfair machine learning model by hiding sensitive features. As legislators worldwide tend to start including the right to explanations for data-driven, high-stakes decisions in their policies, these ethical issues should be understood and addressed. Our literature review on the disagreement problem in XAI reveals that this problem has never been empirically assessed for counterfactual explanations. Therefore, in this work, we conduct a large-scale empirical analysis, on 40 data sets, using 12 explanation-generating methods, for two black-box models, yielding over 192,000 explanations. Our study finds alarmingly high disagreement levels between the methods tested. A malicious user is able to both exclude and include desired features when multiple counterfactual explanations are available. This disagreement seems to be driven mainly by the data set characteristics and the type of counterfactual algorithm. XAI centers on the transparency of algorithmic decision-making, but our analysis advocates for transparency about this self-proclaimed transparency.

Machine learning models are widely used in real-world applications. However, their complexity makes it often challenging to interpret the rationale behind their decisions. Counterfactual explanations (CEs) have emerged as a viable solution for generating comprehensible explanations in eXplainable Artificial Intelligence (XAI). CE provides actionable information to users on how to achieve the desired outcome with minimal modifications to the input. However, current CE algorithms usually operate within the entire feature space when optimising changes to turn over an undesired outcome, overlooking the identification of key contributors to the outcome and disregarding the practicality of the suggested changes. In this study, we introduce a novel methodology, that is named as user feedback-based counterfactual explanation (UFCE), which addresses these limitations and aims to bolster confidence in the provided explanations. UFCE allows for the inclusion of user constraints to determine the smallest modifications in the subset of actionable features while considering feature dependence, and evaluates the practicality of suggested changes using benchmark evaluation metrics. We conducted three experiments with five datasets, demonstrating that UFCE outperforms two well-known CE methods in terms of proximity, sparsity, and feasibility. Reported results indicate that user constraints influence the generation of feasible CEs.

Explainable Artificial Intelligence (XAI) is a set of techniques that allows the understanding of both technical and non-technical aspects of Artificial Intelligence (AI) systems. XAI is crucial to help satisfying the increasingly important demand of trustworthy Artificial Intelligence, characterized by fundamental aspects such as respect of human autonomy, prevention of harm, transparency, accountability, etc. Within XAI techniques, counterfactual explanations aim to provide to end users a set of features (and their corresponding values) that need to be changed in order to achieve a desired outcome. Current approaches rarely take into account the feasibility of actions needed to achieve the proposed explanations, and in particular, they fall short of considering the causal impact of such actions. In this paper, we present Counterfactual Explanations as Interventions in Latent Space (CEILS), a methodology to generate counterfactual explanations capturing by design the underlying causal relations from the data, and at the same time to provide feasible recommendations to reach the proposed profile. Moreover, our methodology has the advantage that it can be set on top of existing counterfactuals generator algorithms, thus minimising the complexity of imposing additional causal constrains. We demonstrate the effectiveness of our approach with a set of different experiments using synthetic and real datasets (including a proprietary dataset of the financial domain).

Many Artificial Intelligence (AI) systems are black boxes, which hinders their deployment. Explainable AI (XAI) approaches which automatically generate counterfactual explanations aim to assist users in scrutinising AI decisions. One property of explanations crucial for their acceptance by users is their coherence. Users perceive counterfactual explanations as coherent if they present a realistic/typical counterfactual scenario that is suitable to explain the factual situation. We design an optimisation-based approach to generate coherent counterfactual explanations applicable to structured data. We demonstrate its applicability and rigorously evaluate its efficacy through functionally grounded and human-grounded evaluation. Results suggest that our approach indeed produces counterfactual explanations that are perceived as coherent by users. More specifically, they are perceived as more realistic, typical, and feasible than state-of-the-art explanations.

Explainable AI (XAI) establishes artificial intelligence (AI) models which combine high-performance capabilities with enhanced transparency for school administrative choices. This systematic literature review aims to study past research about XAI use in education to connect theoretical understanding with real-world practice while advancing ethical AI practices. The current systematic literature review data was gathered from 15 peer-reviewed articles published in renowned databases, including Web of Science, Scopus, Springer, Elsevier, etc. The search was focused on studies determining the application of XAI in education, offering insights from advancements and their implications. The review examines XAI tools alongside their feature sets and operational boundaries and their fundamental needs within educational contexts. The focus of AI and ML researchers on enhancing XAI tools exists, but there are some differences between their targeted audiences and expected results. Interpretable Machine Learning (IML) or XAI produces explanations about prediction outputs while generating customized remedies through tutoring sessions. Adaptive learning systems depend on XAI to develop students’ cognitive abilities for analysis and problem resolution. The intrinsic techniques of XAI in educational data science enable researchers to forecast underrepresented and underperforming student profiles and online learner success rates alongside poor course completion prospect identification for academically struggling students. XAI can help see the learned features and evaluate the bias needed for suspicion about unfair results. XAI can help see the learned features and assess the necessary bias for suspicion about unfair results.

Predictive business process monitoring increasingly leverages sophisticated prediction models. Although sophisticated models achieve consistently higher prediction accuracy than simple models, one major drawback is their lack of interpretability, which limits their adoption in practice. We thus see growing interest in explainable predictive business process monitoring, which aims to increase the interpretability of prediction models. Existing solutions focus on giving factual explanations.While factual explanations can be helpful, humans typically do not ask why a particular prediction was made, but rather why it was made instead of another prediction, i.e., humans are interested in counterfactual explanations. While research in explainable AI produced several promising techniques to generate counterfactual explanations, directly applying them to predictive process monitoring may deliver unrealistic explanations, because they ignore the underlying process constraints. We propose LORELEY, a counterfactual explanation technique for predictive process monitoring, which extends LORE, a recent explainable AI technique. We impose control flow constraints to the explanation generation process to ensure realistic counterfactual explanations. Moreover, we extend LORE to enable explaining multi-class classification models. Experimental results using a real, public dataset indicate that LORELEY can approximate the prediction models with an average fidelity of 97.69\% and generate realistic counterfactual explanations.

With the recent advancements of learning analytics techniques, it is possible to build predictive models of student academic performance at an early stage of a course, using student’s self-regulation learning and affective strategies (SRLAS), and their multiple intelligences (MI). This process can be conducted to determine the most important factors that lead to good academic performance. A quasi-experimental study on 618 undergraduate students was performed to determine student profiles based on these two constructs: MI and SRLAS. After calibrating the students’ profiles, learning analytics techniques were used to study the relationships among the dimensions defined by these constructs and student academic performance using principal component analysis, clustering patterns, and regression and correlation analyses. The results indicate that the logical-mathematical intelligence, intrinsic motivation, and self-regulation have a positive impact on academic performance. In contrast, anxiety and dependence on external motivation have a negative effect on academic performance. A priori knowledge of the characteristics of a student sample and its likely behavior predicted by the models may provide both students and teachers with an early-awareness alert that can help the teachers in designing enhanced proactive and strategic decisions aimed to improve academic performance and reduce dropout rates. From the student side, knowledge about their main academic profile will sharpen their metacognition, which may improve their academic performance.

From local counterfactuals to global feature importance: efficient, robust, and model-agnostic explanations for brain connectivity networks

Counterfactual explanations are a prominent example of post-hoc interpretability methods in the explainable Artificial Intelligence (AI) research domain. Differently from other explanation methods, they offer the possibility to have recourse against unfavourable outcomes computed by machine learning models. However, in this paper we show that retraining machine learning models over time may invalidate the counterfactual explanations of their outcomes. We provide a formal definition of this phenomenon and we introduce a method, namely counterfactual data augmentation, to help improving the robustness of counterfactual explanations over time. We test our method in an empirical study where we simulate different model retraining scenarios. Our results show that counterfactual data augmentation improves the robustness of counterfactual explanations over time, therefore contributing to their use in real-world machine learning applications.

Explainable Artificial Intelligence (XAI) has gained significant attention in recent years due to concerns over the lack of interpretability of Deep Learning models, which hinders their decision-making processes. To address this issue, counterfactual explanations have been proposed to elucidate the reasoning behind a model’s decisions by providing what-if statements as explanations. However, generating counterfactuals traditionally involves solving an optimization problem for each input, making it impractical for real-time feedback. Moreover, counterfactuals must meet specific criteria, including being user-driven, causing minimal changes, and staying within the data distribution. To overcome these challenges, a novel model-agnostic approach called Real-Time Guided Counterfactual Explanations (RTGCEx) is proposed. This approach utilizes autoencoders to generate real-time counterfactual explanations that adhere to these criteria by optimizing a multiobjective loss function. The performance of RTGCEx has been evaluated on two datasets: MNIST and Gearbox, a synthetic time series dataset. The results demonstrate that RTGCEx outperforms traditional methods in terms of speed and efficacy on MNIST, while also effectively identifying and rectifying anomalies in the Gearbox dataset, highlighting its versatility across different scenarios.