Explainable Artificial Intelligence Research Articles

A decision support system for acute lymphoblastic leukemia detection based on explainable artificial intelligence

The detection of acute lymphoblastic leukemia (ALL) via deep learning (DL) has received great interest because of its high accuracy in detecting lymphoblasts without the need for handcrafted feature extraction. However, current DL models, such as convolutional neural networks and vision Transformers, are extremely complex, making them black boxes that perform classification in an obscure way. To compensate for this and increase the explainability of the decisions made by such methods, in this paper, we propose an innovative decision support system for ALL detection that is based on DL and explainable artificial intelligence (XAI). Our approach first introduces causality into the decision with a metric learning approach, enabling a decision to be made by analyzing the most similar images in the database. Second, our method integrates XAI techniques to allow even non-trained personnel to obtain an informed decision by analyzing which regions of the images are most similar and how the samples are organized in the latent space. The results on publicly available ALL databases confirm the validity of our approach in opening the black box while achieving similar or superior accuracy to that of existing approaches.

Development and application of explainable artificial intelligence using machine learning classification for long-term facial nerve function after vestibular schwannoma surgery.

Vestibular schwannomas (VSs) represent the most common cerebellopontine angle tumors, posing a challenge in preserving facial nerve (FN) function during surgery. We employed the Extreme Gradient Boosting machine learning classifier to predict long-term FN outcomes (classified as House-Brackmann grades 1-2 for good outcomes and 3-6 for bad outcomes) after VS surgery. In a retrospective analysis of 256 patients, comprehensive pre-, intra-, and post-operative factors were examined. We applied the machine learning (ML) classifier Extreme Gradient Boosting (XGBoost) for the following binary classification: long-term good and bad FN outcome after VS surgery To enhance the interpretability of our model, we utilized an explainable artificial intelligence approach. Short-term FN function (tau = 0.6) correlated with long-term FN function. The model exhibited an average accuracy of 0.83, a ROC AUC score of 0.91, and Matthew's correlation coefficient score of 0.62. The most influential feature, identified through SHapley Additive exPlanations (SHAP), was short-term FN function. Conversely, large tumor volume and absence of preoperative auditory brainstem responses were associated with unfavorable outcomes. We introduce an effective ML model for classifying long-term FN outcomes following VS surgery. Short-term FN function was identified as the key predictor of long-term function. This model's excellent ability to differentiate bad and good outcomes makes it useful for evaluating patients and providing recommendations regarding FN dysfunction management.

Using PLS-SEM and XAI for causal-predictive services marketing research

Purpose This study aims to redefine approaches to metrics in service marketing by examining the utility of partial least squares – structural equation modeling (PLS-SEM) and eXplainable Artificial Intelligence (XAI) for assessing service quality, with a focus on the airline industry. Design/methodology/approach Using the Airline Passenger Satisfaction data set from Kaggle platform, this study applies PLS-SEM, facilitated by ADANCO software and XAI techniques, specifically using the SHapley Additive exPlanations TreeExplainer model. This study tests several hypotheses to validate the effectiveness of these methodological tools in identifying key determinants of service quality. Findings PLS-SEM analysis categorizes key variables into Delay, Airport Service and In-flight Service, whereas XAI techniques rank these variables based on their impact on service quality. This dual-framework provides businesses a detailed analytical approach customized to specific research needs. Research limitations/implications This study is constrained by the use of a single data set focused on the airline industry, which may limit generalizability. Future research should apply these methodologies across various sectors to enhance a broader applicability. Practical implications The analytical framework offered here equips businesses with the robust tools for a more rigorous and nuanced evaluation of service quality metrics, supporting informed strategic decision-making. Social implications By applying advanced analytics to refine service metrics, businesses can better meet and exceed customer expectations, ultimately elevating the societal standard of service delivery. Originality/value This study contributes to the ongoing discourse on artificial intelligence interpretability in business analytics, presenting an innovative methodological guide for applying PLS-SEM and/or XAI in service marketing research. This approach delivers actionable insights, not only in the airline sector but also across diverse business domains seeking to optimize service quality.

Explaining AI through mechanistic interpretability

Recent work in explainable artificial intelligence (XAI) attempts to render opaque AI systems understandable through a divide-and-conquer strategy. However, this fails to illuminate how trained AI systems work as a whole. Precisely this kind of functional understanding is needed, though, to satisfy important societal desiderata such as safety. To remedy this situation, we argue, AI researchers should seek mechanistic interpretability, viz. apply coordinated discovery strategies familiar from the life sciences to uncover the functional organisation of complex AI systems. Additionally, theorists should accommodate for the unique costs and benefits of such strategies in their portrayals of XAI research.

Predictive Understanding of Stream Salinization in a Developed Watershed Using Machine Learning.

Stream salinization is a global issue, yet few models can provide reliable salinity estimates for unmonitored locations at the time scales required for ecological exposure assessments. Machine learning approaches are presented that use spatially limited high-frequency monitoring and spatially distributed discrete samples to estimate the daily stream-specific conductance across a watershed. We compare the predictive performance of space- and time-unaware Random Forest models and space- and time-aware Recurrent Graph Convolution Neural Network models (KGE: 0.67 and 0.64, respectively) and use explainable artificial intelligence methods to interpret model predictions and understand salinization drivers. These models are applied to the Delaware River Basin, a developed watershed with diverse land uses that experiences anthropogenic salinization from winter deicer applications. These models capture seasonality for the winter first flush of deicers, and the streams with elevated predictions correspond well with indicators of deicer application. This result suggests that these models can be used to identify potential salinity-impaired streams for winter best management practices. Daily salinity predictions are driven primarily by land cover (urbanization) trends that may represent anthropogenic salinization processes and weather at time scales up to three months. Such modeling approaches are likely transferable to other watersheds and can be applied to further understand salinization risks and drivers.

Helping CNAs Generate CVSS Scores Faster and More Confidently Using XAI

The number of cybersecurity vulnerabilities keeps growing every year. Each vulnerability must be reported to the MITRE Corporation and assessed by a Counting Number Authority, which generates a metrics vector that determines its severity score. This process can take up to several weeks, with higher-severity vulnerabilities taking more time. Several authors have successfully used Deep Learning to automate the score generation process and used explainable AI to build trust with the users. However, the explanations that were shown were surface label input saliency on binary classification. This is a limitation, as several metrics are multi-class and there is much more we can achieve with XAI than just visualizing saliency. In this work, we look for actionable actions CNAs can take using XAI. We achieve state-of-the-art results using an interpretable XGBoost model, generate explanations for multi-class labels using SHAP, and use the raw Shapley values to calculate cumulative word importance and generate IF rules that allow a more transparent look at how the model classified vulnerabilities. Finally, we made the code and dataset open-source for reproducibility.

Explainable artificial intelligence on safe balance and its major determinants in stroke patients

This study develops explainable artificial intelligence for predicting safe balance using hospital data, including clinical, neurophysiological, and diffusion tensor imaging properties. Retrospective data from 92 first-time stroke patients from January 2016 to June 2023 was analysed. The dependent variables were independent mobility scores, i.e., Berg Balance Scales with 0 (45 or below) vs. 1 (above 45) measured after three and six months, respectively. Twenty-nine predictors were included. Random forest variable importance was employed for identifying significant predictors of the Berg Balance Scale and testing its associations with the predictors, including Berg Balance Scale after one month and corticospinal tract diffusion tensor imaging properties. Shapley Additive Explanation values were calculated to analyse the directions of these associations. The random forest registered a higher or similar area under the curve compared to logistic regression, i.e., 91% vs. 87% (Berg Balance Scale after three months), 92% vs. 92% (Berg Balance Scale after six months). Based on random forest variable importance values and rankings: (1) Berg Balance Scale after three months has strong associations with Berg Balance Scale after one month, Fugl-Meyer assessment scale, ipsilesional corticospinal tract fractional anisotropy, fractional anisotropy laterality index and age; (2) Berg Balance Scale after six months has strong relationships with Fugl-Meyer assessment scale, Berg Balance Scale after one month, ankle plantar flexion muscle strength, knee extension muscle strength and hip flexion muscle strength. These associations were positive in the SHAP summary plots. Including Berg Balance Scale after one month, Fugl-Meyer assessment scale or ipsilesional corticospinal tract fractional anisotropy in the random forest will increase the probability of Berg Balance Scale after three months being above 45 by 0.11, 0.08, or 0.08. In conclusion, safe balance after stroke strongly correlates with its initial motor function, Fugl-Meyer assessment scale, and ipsilesional corticospinal tract fractional anisotropy. Diffusion tensor imaging information aids in developing explainable artificial intelligence for predicting safe balance after stroke.

Fully Interpretable Deep Learning Model Using IR Thermal Images for Possible Breast Cancer Cases.

Breast cancer remains a global health problem requiring effective diagnostic methods for early detection, in order to achieve the World Health Organization's ultimate goal of breast self-examination. A literature review indicates the urgency of improving diagnostic methods and identifies thermography as a promising, cost-effective, non-invasive, adjunctive, and complementary detection method. This research explores the potential of using machine learning techniques, specifically Bayesian networks combined with convolutional neural networks, to improve possible breast cancer diagnosis at early stages. Explainable artificial intelligence aims to clarify the reasoning behind any output of artificial neural network-based models. The proposed integration adds interpretability of the diagnosis, which is particularly significant for a medical diagnosis. We constructed two diagnostic expert models: Model A and Model B. In this research, Model A, combining thermal images after the explainable artificial intelligence process together with medical records, achieved an accuracy of 84.07%, while model B, which also includes a convolutional neural network prediction, achieved an accuracy of 90.93%. These results demonstrate the potential of explainable artificial intelligence to improve possible breast cancer diagnosis, with very high accuracy.

Spatio-temporal modeling of asthma-prone areas: Exploring the influence of urban climate factors with explainable artificial intelligence (XAI)

Urbanization's impact on climate is increasingly recognized as a significant public health challenge, particularly for respiratory conditions like asthma. Despite progress in understanding asthma, a critical gap remains regarding the interaction between urban environmental factors and asthma-prone areas. This study addresses this gap by applying innovative spatio-temporal modeling techniques with explainable artificial intelligence (XAI). Using data from 872 asthma patients in Tehran, Iran, and 19 factors affecting asthma exacerbations, including climate and air pollution, spatio-temporal modeling was conducted using XGBoost (eXtreme Gradient Boosting) algorithm optimization by the Bat algorithm (BA). Evaluation of asthma-prone area maps using receiver operating characteristic (ROC) curves revealed accuracies of 97.3 % in spring, 97.5 % in summer, 97.8 % in autumn, and 98.4 % in winter. Interpretability analysis of the XGBoost model utilizing the SHAP (Shapley Additive exPlanations) method highlighted rainfall in spring and autumn and temperature in summer and winter as having the most significant impacts on asthma. Particulate matter (PM2.5) in spring, carbon monoxide (CO) in summer, ozone (O3) in autumn, and PM10 in winter exhibited the most substantial effects among air pollution factors. This research enhances understanding of asthma dynamics in urban environments, informing targeted interventions for urban planning strategies to mitigate adverse health consequences of urbanization.

Applied Ethical and Explainable AI in Adversarial Deepfake Detection: From Theory to Real-World Systems

Deepfake technology is advancing by the minute. This gives rise to increased privacy, trust and security risks. Such technology can be used for malicious activities like manipulating public opinion and spreading misinformation using social media. Adversarial machine learning techniques seem to be a strong defense in detecting and flagging deepfake content. But the challenge with practical use of many deepfake detection models is that they operate as black-boxes with little transparency or accountability in their decisions. This paper proposes a framework and guidelines to integrate ethical AI and explainable AI (XAI) - specifically techniques like SHAP and LIME, to make deepfake detection systems more transparent and trustworthy. We will propose guidelines to incorporate techniques which will make deepfake detection systems more accountable and explainable such that they make deepfake detection systems seamlessly deployable in the real world.

Towards counterfactual explanations for ontologies

Debugging and repairing Web Ontology Language (OWL) ontologies has been a key field of research since OWL became a W3C recommendation. One way to understand errors and fix them is through explanations. These explanations are usually extracted from the reasoner and displayed to the ontology authors as is. In the meantime, there has been a recent call in the eXplainable AI (XAI) field to use expert knowledge in the form of knowledge graphs and ontologies. In this paper, a parallel between explanations for machine learning and for ontologies is drawn. This link enables the adaptation of XAI methods to explain ontologies and their entailments. Counterfactual explanations have been identified as a good candidate to solve the explainability problem in machine learning. The CEO (Counterfactual Explanations for Ontologies) method is thus proposed to explain inconsistent ontologies using counterfactual explanations. A preliminary user study is conducted to ensure that using XAI methods for ontologies is relevant and worth pursuing.

Integration of FAERS, DrugBank and SIDER Data for Machine Learning-based Detection of Adverse Drug Reactions

AbstractTraditionally, disproportionality analysis (DPA) methods are employed for signal detection in pharmacovigilance, but these methods utilize only a limited portion of the data available from spontaneous event reports (SERs). This research aims to enhance signal detection by applying machine learning (ML) methods that can leverage additional data. We create a dataset by integrating SER data from the FDA Adverse Event Reporting System (FAERS) with biological and chemical data from DrugBank, and information on known adverse drug reactions (ADRs) from Side Effect Resource (SIDER). The known ADRs from SIDER are used to label the dataset for ML training. Using the AutoML library TPOT, ML models are trained on this dataset. Our findings indicate that ML models, even when trained with the same features as DPA methods, achieve higher recall and precision. Moreover, incorporating additional features related to drugs and events significantly boosts the performance of ML models. Analysis using the explainable AI (XAI) technique SHAP reveals that the drug name, event name, and fifth-level ATC code are the most influential features for model predictions. These ML models offer a promising alternative or supplement to conventional DPA methods for signal detection in pharmacovigilance.

Deep learning to capture leaf shape in plant images: Validation by geometric morphometrics.

Plant leaves play a pivotal role in automated species identification using deep learning (DL). However, achieving reproducible capture of leaf variation remains challenging due to the inherent "black box" problem of DL models. To evaluate the effectiveness of DL in capturing leaf shape, we used geometric morphometrics (GM), an emerging component of eXplainable Artificial Intelligence (XAI) toolkits. We photographed Ranunculus auricomus leaves directly in situ and after herbarization. From these corresponding leaf images, we automatically extracted DL features using a neural network and digitized leaf shapes using GM. The association between the extracted DL features and GM shapes was then evaluated using dimension reduction and covariation models. DL features facilitated the clustering of leaf images by source populations in both in situ and herbarized leaf image datasets, and certain DL features were significantly associated with biological leaf shape variation as inferred by GM. DL features also enabled leaf classification into morpho-phylogenomic groups within the intricate R. auricomus species complex. We demonstrated that simple in situ leaf imaging and DL reproducibly captured leaf shape variation at the population level, while combining this approach with GM provided key insights into the shape information extracted from images by computer vision, a necessary prerequisite for reliable automated plant phenotyping.

Evaluating Explainable Artificial Intelligence (XAI) techniques in chest radiology imaging through a human-centered Lens.

The field of radiology imaging has experienced a remarkable increase in using of deep learning (DL) algorithms to support diagnostic and treatment decisions. This rise has led to the development of Explainable AI (XAI) system to improve the transparency and trust of complex DL methods. However, XAI systems face challenges in gaining acceptance within the healthcare sector, mainly due to technical hurdles in utilizing these systems in practice and the lack of human-centered evaluation/validation. In this study, we focus on visual XAI systems applied to DL-enabled diagnostic system in chest radiography. In particular, we conduct a user study to evaluate two prominent visual XAI techniques from the human perspective. To this end, we created two clinical scenarios for diagnosing pneumonia and COVID-19 using DL techniques applied to chest X-ray and CT scans. The achieved accuracy rates were 90% for pneumonia and 98% for COVID-19. Subsequently, we employed two well-known XAI methods, Grad-CAM (Gradient-weighted Class Activation Mapping) and LIME (Local Interpretable Model-agnostic Explanations), to generate visual explanations elucidating the AI decision-making process. The visual explainability results were shared through a user study, undergoing evaluation by medical professionals in terms of clinical relevance, coherency, and user trust. In general, participants expressed a positive perception of the use of XAI systems in chest radiography. However, there was a noticeable lack of awareness regarding their value and practical aspects. Regarding preferences, Grad-CAM showed superior performance over LIME in terms of coherency and trust, although concerns were raised about its clinical usability. Our findings highlight key user-driven explainability requirements, emphasizing the importance of multi-modal explainability and the necessity to increase awareness of XAI systems among medical practitioners. Inclusive design was also identified as a crucial need to ensure better alignment of these systems with user needs.

Analyzing the dynamics between crude oil spot prices and futures prices by maturity terms: Deep learning approaches to futures-based forecasting

Predicting crude oil prices is critical due to their significant impact on the global economy. While numerous studies have focused on time series forecasting and machine learning methods to predict crude oil spot prices, the use of futures prices segmented by the maturity terms as predictors is comparatively underexplored. This study explored a deep learning approach to investigate using futures prices across different maturities to predict spot prices. Specifically, this paper analyzes the predictive power of one-, two-, three-, six-, and 12-month crude oil futures contracts. This study employs multiple deep learning algorithms, including Multilayer Perceptron (MLP), Recurrent Neural Network (RNN), Long Short-Term Memory (LSTM), Gated Recurrent Unit (GRU), Convolutional Neural Network (CNN), and Temporal Convolutional Neural Network (TCN), to forecast crude oil spot prices. This research investigates the performance of these machine learning models when exploring the relationship between futures and spot prices of crude oil. In addition, this research incorporates extensive hyperparameter tuning to enhance the interpretability of the machine learning models when forecasting spot prices using futures prices, thereby contributing to the field of Explainable Artificial Intelligence (XAI) with an optimal set of hyperparameters. In summary, this research systematically shows results demonstrating predictive power in terms of XAI between spot prices and futures prices with different maturities, and machine learning algorithms.

Cloud spot instance price forecasting multi-headed models tuned using modified PSO

The increasing dependence and demands on cloud infrastructure have brought to light challenges associated with cloud instance pricing. The often unpredictable nature of demand as well as changing costs of supplying a reliable instance can leave companies struggling to appropriately budget to support a healthy cash flow while maintaining operating costs. This work explores the potential of multi-headed recurrent architectures to forecast cloud instance prices based on historical and instance data. Two architectures are explored, long short-term memory (LSTM) and gated recurrent unit (GRU) networks. A modified optimizer is introduced and tested on a publicly available Amazon elastic compute cloud dataset. The GRU model, enhanced by the proposed modified approach, had the most impressive outcomes with an MAE score of 0.000801. Results have undergone meticulous statistical validation with the best-performing models further analyzed using explainable artificial intelligence techniques to provide further insight into model reasoning and information on feature importance.

Mitigating Bias in AI-Driven Recruitment : The Role of Explainable Machine Learning (XAI)

This article explores the critical role of Explainable Artificial Intelligence (XAI) in mitigating bias within AI-driven recruitment processes. As AI becomes increasingly prevalent in hiring practices, concerns about algorithmic bias and fairness have emerged. The article discusses how XAI techniques, such as SHAP and LIME, can be used to detect and interpret potential biases in recruitment algorithms. It examines the implementation of XAI for feature importance analysis, algorithmic bias detection, and disparate impact analysis across different demographic groups. The article addresses the challenges of balancing model complexity with explainability and the limitations of XAI in identifying systemic biases. By implementing XAI strategies, organizations can enhance the fairness and transparency of their hiring practices, ultimately fostering more diverse and equitable workplaces.

Moral reasoning in a digital age: blaming artificial intelligence for incorrect high-risk decisions

AbstractThe increasing involvement of Artificial Intelligence (AI) in moral decision situations raises the possibility of users attributing blame to AI-based systems for negative outcomes. In two experimental studies with a total of $$N = 911$$ participants, we explored the attribution of blame and underlying moral reasoning. Participants had to classify mushrooms in pictures as edible or poisonous with support of an AI-based app. Afterwards, participants read a fictitious scenario in which a misclassification due to an erroneous AI recommendation led to the poisoning of a person. In the first study, increased system transparency through explainable AI techniques reduced blaming of AI. A follow-up study showed that attribution of blame to each actor in the scenario depends on their perceived obligation and capacity to prevent such an event. Thus, blaming AI is indirectly associated with mind attribution and blaming oneself is associated with the capability to recognize a wrong classification. We discuss implications for future research on moral cognition in the context of human–AI interaction.

Explainable AI for Transparent and Trustworthy Tuberculosis Diagnosis: From Mere Pixels to Actionable Insights

Building transparent and trustworthy AI-powered systems for disease diagnosis has become more paramount than ever due to a lack of understanding of black box models. A lack of transparency and explainability in AI-driven models can propagate biases and erode patients' and medical practitioners' trust. To answer this challenge, Explainable AI (XAI) is drastically emerging as a practical solution and approach to tackling ethical concerns in the health sector. The overarching purpose of this paper is to highlight the advancement in XAI for tuberculosis diagnosis (TB) and identify the benefits and challenges associated with improved trust in AI-powered TB diagnosis. We explore the potential of XAI in improving TB diagnosis. We attempt to provide a complete plan to promote XAI. We examine the significant problems associated with using XAI in TB diagnosis. We argue that XAI is critical for reliable TB diagnosis by improving the interpretability of AI decision-making processes and recognising possible biases and mistakes. We evaluate techniques and methods for XAI in TB diagnosis and examine the ethical and societal ramifications. By leveraging explainable AI, we can create a more reliable and trustworthy TB diagnostic framework, ultimately improving patient outcomes and global health. Finally, we provide thorough recommendations for developing and implementing XAI in TB diagnosis using X-ray imaging

Pathological voice detection using optimized deep residual neural network and explainable artificial intelligence

Pathological voice detection using optimized deep residual neural network and explainable artificial intelligence