Explainable AI Models Research Articles

Explainable AI Enabled Infant Mortality Prediction Based on Neonatal Sepsis

Neonatal sepsis is the third most common cause of neonatal mortality and a serious public health problem, especially in developing countries. There have been researches on human sepsis, vaccine response, and immunity. Also, machine learning methodologies were used for predicting infant mortality based on certain features like age, birth weight, gestational weeks, and Appearance, Pulse, Grimace, Activity and Respiration (APGAR) score. Sepsis, which is considered the most determining condition towards infant mortality, has never been considered for mortality prediction. So, we have deployed a deep neural model which is the state of art and performed a comparative analysis of machine learning models to predict the mortality among infants based on the most important features including sepsis. Also, for assessing the prediction reliability of deep neural model which is a black box, Explainable AI models like Dalex and Lime have been deployed. This would help any non-technical personnel like doctors and practitioners to understand and accordingly make decisions.

Explainable Misinformation Detection Across Multiple Social Media Platforms

Web Information Processing (WIP) has enormously impacted modern society since a huge percentage of the population relies on the internet to acquire information. Social Media platforms provide a channel for disseminating information and a breeding ground for spreading misinformation, creating confusion and fear among the population. One of the techniques for the detection of misinformation is machine learning-based models. However, due to the availability of multiple social media platforms, developing and training AI-based models has become a tedious job. Despite multiple efforts to develop machine learning-based methods for identifying misinformation, there has been very limited work on developing an explainable generalized detector capable of robust detection and generating explanations beyond black-box outcomes. Knowing the reasoning behind the outcomes is essential to make the detector trustworthy. Hence employing explainable AI techniques is of utmost importance. In this work, the integration of two machine learning approaches, namely domain adaptation and explainable AI, is proposed to address these two issues of generalized detection and explainability. Firstly the Domain Adversarial Neural Network (DANN) develops a generalized misinformation detector across multiple social media platforms. DANN is employed to generate the classification results for test domains with relevant but unseen data. The DANN-based model, a traditional black-box model, cannot justify and explain its outcome, i.e., the labels for the target domain. Hence a Local Interpretable Model-Agnostic Explanations (LIME) explainable AI model is applied to explain the outcome of the DANN model. To demonstrate these two approaches and their integration for effective explainable generalized detection, COVID-19 misinformation is considered a case study. We experimented with two datasets and compared results with and without DANN implementation. It is observed that using DANN significantly improves the F1 score of classification and increases the accuracy by 5% and AUC by 11%. The results show that the proposed framework performs well in the case of domain shift and can learn domain-invariant features while explaining the target labels with LIME implementation. This can enable trustworthy information processing and extraction to combat misinformation effectively.

Explainable AI and Interpretable Model for Insurance Premium Prediction

Explainable AI and Interpretable Model for Insurance Premium Prediction

Influential factors of intercity patient mobility and its network structure in China

Intercity patient mobility reflects the geographic mismatch between healthcare resources and the population, and has rarely been studied with big data at large spatial scales. In this paper, we investigated the patterns of intercity patient mobility and factors influencing this behavior based on >4 million hospitalization records of patients with chronic kidney disease in China. To provide practical policy recommendations, a role identification framework informed by complex network theory was proposed considering the strength and distribution of connections of cities in mobility networks. Such a mobility network features multiscale community structure with “universal administrative constraints and a few boundary breaches”. We discovered that cross-module visits which accounted for only 20 % of total visits, accounted for >50 % of the total travel distance. The explainable machine learning modeling results revealed that distance has a power-law-like effect on flow volume, and high-quality healthcare resources are an important driving factor. This paper provides not only a methodological reference for patient mobility studies but also valuable insights into public health policies. • More than 4 million multi-year mobility records from a national database were investigated. • The distribution of travel distance of patients differs from the results of general human mobility studies. • The network features multiscale community structure with “universal administrative constraints and a few boundary breaches”. • An identification framework was proposed to understand cities’ roles in the mobility network. • The explainable AI model revealed that distance and high-quality healthcare resources are two critical factors.

Solving transparency in drought forecasting using attention models.

Droughts are one of the most devastating and recurring natural disaster due to a multitude of reasons. Among the different drought studies, drought forecasting is one of the key aspects of effective drought management. The occurrence of droughts is related to a multitude of factors which is a combination of hydro-meteorological and climatic factors. These variables are non-linear in nature, and neural networks have been found to effectively forecast drought. However, classical neural nets often succumb to over-fitting due to various lag components among the variables and therefore, the emergence of new deep learning and explainable models can effectively solve this problem. The present study uses an Attention-based model to forecast meteorological droughts (Standard Precipitation Index) at short-term forecast range (1-3 months) for five sites situated in Eastern Australia. The main aim of the work is to interpret the model outcomes and examine how a deep neural network achieves the forecasting results. The plots show the importance of the variables along with its short-term and long-term dependencies at different lead times. The results indicate the importance of large-scale climatic indices at different sequence dependencies specific to the study site, thus providing an example of the necessity to build a spatio-temporal explainable AI model for drought forecasting. The use of such interpretable models would help the decision-makers and planners to use data-driven models as an effective measure to forecast droughts as they provide transparency and trust while using these models.

Image Embeddings Extracted from CNNs Outperform Other Transfer Learning Approaches in Classification of Chest Radiographs.

To identify the best transfer learning approach for the identification of the most frequent abnormalities on chest radiographs (CXRs), we used embeddings extracted from pretrained convolutional neural networks (CNNs). An explainable AI (XAI) model was applied to interpret black-box model predictions and assess its performance. Seven CNNs were trained on CheXpert. Three transfer learning approaches were thereafter applied to a local dataset. The classification results were ensembled using simple and entropy-weighted averaging. We applied Grad-CAM (an XAI model) to produce a saliency map. Grad-CAM maps were compared to manually extracted regions of interest, and the training time was recorded. The best transfer learning model was that which used image embeddings and random forest with simple averaging, with an average AUC of 0.856. Grad-CAM maps showed that the models focused on specific features of each CXR. CNNs pretrained on a large public dataset of medical images can be exploited as feature extractors for tasks of interest. The extracted image embeddings contain relevant information that can be used to train an additional classifier with satisfactory performance on an independent dataset, demonstrating it to be the optimal transfer learning strategy and overcoming the need for large private datasets, extensive computational resources, and long training times.

Federated Learning of Explainable AI Models in 6G Systems: Towards Secure and Automated Vehicle Networking

This article presents the concept of federated learning (FL) of eXplainable Artificial Intelligence (XAI) models as an enabling technology in advanced 5G towards 6G systems and discusses its applicability to the automated vehicle networking use case. Although the FL of neural networks has been widely investigated exploiting variants of stochastic gradient descent as the optimization method, it has not yet been adequately studied in the context of inherently explainable models. On the one side, XAI permits improving user experience of the offered communication services by helping end users trust (by design) that in-network AI functionality issues appropriate action recommendations. On the other side, FL ensures security and privacy of both vehicular and user data across the whole system. These desiderata are often ignored in existing AI-based solutions for wireless network planning, design and operation. In this perspective, the article provides a detailed description of relevant 6G use cases, with a focus on vehicle-to-everything (V2X) environments: we describe a framework to evaluate the proposed approach involving online training based on real data from live networks. FL of XAI models is expected to bring benefits as a methodology for achieving seamless availability of decentralized, lightweight and communication efficient intelligence. Impacts of the proposed approach (including standardization perspectives) consist in a better trustworthiness of operations, e.g., via explainability of quality of experience (QoE) predictions, along with security and privacy-preserving management of data from sensors, terminals, users and applications.

An Explainable AI Model for Hate Speech Detection on Indonesian Twitter

To avoid citizen disputes, hate speech on social media, such as Twitter, must be automatically detected. The current research in Indonesian Twitter focuses on developing better hate speech detection models. However, there is limited study on the explainability aspects of hate speech detection. The research aims to explain issues that previous researchers have not detailed and attempt to answer the shortcomings of previous researchers. There are 13,169 tweets in the dataset with labels like “hate speech” and “abusive language”. The dataset also provides binary labels on whether hate speech is directed to individual, group, religion, race, physical disability, and gender. In the research, classification is performed by using traditional machine learning models, and the predictions are evaluated using an Explainable AI model, such as Local Interpretable Model-Agnostic Explanations (LIME), to allow users to comprehend why a tweet is regarded as a hateful message. Moreover, models that perform well in classification perceive incorrect words as contributing to hate speech. As a result, such models are unsuitable for deployment in the real world. In the investigation, the combination of XGBoost and logical LIME explanations produces the most logical results. The use of the Explainable AI model highlights the importance of choosing the ideal model while maintaining users’ trust in the deployed model.

Mitigating the Backdoor Attack by Federated Filters for Industrial IoT Applications

The federated learning provides an effective solution to train collaborative models over a large scale of participated Industrial Internet of Things (IIoT) applications with the help of a global server, building an intelligent life. However, the federated learning is vulnerable to the backdoor attack from strong malicious participants. The backdoor attack is inconspicuous and may result in devastating consequences. To resist the attack on IIoT applications, we propose the federated backdoor filter defense that can identify backdoor inputs and restore the data to availability by the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">blur-label-flipping</i> strategy. We build multiple filters with eXplainable AI models on the server and send them to clients randomly, preventing advanced attackers from evading the defense. Our backdoor filters show significant backdoor recognition with the accuracy up to 99%. After the implementation of the blur-label-flipping strategy, victim's local model on suspicious backdoor samples can achieve the accuracy up to 88%.

IHAR—A fog‐driven interpretable human activity recognition system

AbstractHuman activity recognition (HAR) is an active research area that is currently being applied to various healthcare applications such as fall detection, assisted living, etc. These applications make use of the Internet of Things which is widespread across today's world. One of the major challenges in these applications is the need for quick intelligent decisions. The deployment of hierarchical edge‐fog‐cloud computing architecture is the potential solution to address the latency issue. In this article, we proposed an interpretable human activity recognition (IHAR) framework that supports HAR, leveraging the advantages of the fog layer. The proposed framework used a deep learning (DL) model at the cloud infrastructure to classify the activities of humans. The trained DL model is made to run on local fog nodes. However, the DL model being black‐box does not provide any explanations for the output to make them acceptable by the users and physicians. Hence, this article also incorporated an explainable artificial intelligence model at the fog layer, to gain insights into the classified output of the DL model. The effective model‐level insights emphasized the need for explainable HAR. The article also included an analysis of the results of the existing explainable AI models such as LIME and SHAP to understand which model leads to better performance in the domain of HAR. Results show that the SHAP model has a 33% higher success rate in generating explanations when compared to the LIME model.

Accurate auto-labeling of chest X-ray images based on quantitative similarity to an explainable AI model

The inability to accurately, efficiently label large, open-access medical imaging datasets limits the widespread implementation of artificial intelligence models in healthcare. There have been few attempts, however, to automate the annotation of such public databases; one approach, for example, focused on labor-intensive, manual labeling of subsets of these datasets to be used to train new models. In this study, we describe a method for standardized, automated labeling based on similarity to a previously validated, explainable AI (xAI) model-derived-atlas, for which the user can specify a quantitative threshold for a desired level of accuracy (the probability-of-similarity, pSim metric). We show that our xAI model, by calculating the pSim values for each clinical output label based on comparison to its training-set derived reference atlas, can automatically label the external datasets to a user-selected, high level of accuracy, equaling or exceeding that of human experts. We additionally show that, by fine-tuning the original model using the automatically labelled exams for retraining, performance can be preserved or improved, resulting in a highly accurate, more generalized model.

Investigation of a Web-Based Explainable AI Screening for Prolonged Grief Disorder

Losing a loved one through death is known to be one of the most challenging life events. To help the bereaved and their therapists monitor and better understand the factors that contribute to Prolonged Grief Disorder (PGD), we co-designed and studied a web-based explainable AI screening system named “Grief Inquiries Following Tragedy (GIFT).” We used an initial iteration of the system to collect PGD-related data from 611 participants. Using this data, we developed a model that could be used to screen and explain the different factors contributing to PGD. Our results showed that a Random Forest model using Bereavement risk and outcome features performed best in detecting PGD (AUC=0.772), with features such as a negative intepretation of grief and the ability to integrate stressful life events contributing strongly to the model. Afterwards, five grief experts were asked to provide feedback on a mock-up of the results generated by the GIFT model, and discuss the potential value of the explanatory AI model in real-world PGD care. Overall, the grief experts were generally receptive towards using such a tool in a clinical setting and acknowledged the benefit of offering a personalized result to the users based on the explainable AI model. Our results also showed that, in addition to the explainability of the model, the grief experts also preferred a more “empathetic” and “actionable” AI system, especially, when designing for patient end-users.

A Multimodal Approach for the Risk Prediction of Intensive Care and Mortality in Patients with COVID-19.

Background: Although several studies have been launched towards the prediction of risk factors for mortality and admission in the intensive care unit (ICU) in COVID-19, none of them focuses on the development of explainable AI models to define an ICU scoring index using dynamically associated biological markers. Methods: We propose a multimodal approach which combines explainable AI models with dynamic modeling methods to shed light into the clinical features of COVID-19. Dynamic Bayesian networks were used to seek associations among cytokines across four time intervals after hospitalization. Explainable gradient boosting trees were trained to predict the risk for ICU admission and mortality towards the development of an ICU scoring index. Results: Our results highlight LDH, IL-6, IL-8, Cr, number of monocytes, lymphocyte count, TNF as risk predictors for ICU admission and survival along with LDH, age, CRP, Cr, WBC, lymphocyte count for mortality in the ICU, with prediction accuracy 0.79 and 0.81, respectively. These risk factors were combined with dynamically associated biological markers to develop an ICU scoring index with accuracy 0.9. Conclusions: to our knowledge, this is the first multimodal and explainable AI model which quantifies the risk of intensive care with accuracy up to 0.9 across multiple timepoints.

An explainable artificial intelligence approach for decoding the enhancer histone modifications code and identification of novel enhancers in Drosophila

BackgroundEnhancers are non-coding regions of the genome that control the activity of target genes. Recent efforts to identify active enhancers experimentally and in silico have proven effective. While these tools can predict the locations of enhancers with a high degree of accuracy, the mechanisms underpinning the activity of enhancers are often unclear.ResultsUsing machine learning (ML) and a rule-based explainable artificial intelligence (XAI) model, we demonstrate that we can predict the location of known enhancers in Drosophila with a high degree of accuracy. Most importantly, we use the rules of the XAI model to provide insight into the underlying combinatorial histone modifications code of enhancers. In addition, we identified a large set of putative enhancers that display the same epigenetic signature as enhancers identified experimentally. These putative enhancers are enriched in nascent transcription, divergent transcription and have 3D contacts with promoters of transcribed genes. However, they display only intermediary enrichment of mediator and cohesin complexes compared to previously characterised active enhancers. We also found that 10–15% of the predicted enhancers display similar characteristics to super enhancers observed in other species.ConclusionsHere, we applied an explainable AI model to predict enhancers with high accuracy. Most importantly, we identified that different combinations of epigenetic marks characterise different groups of enhancers. Finally, we discovered a large set of putative enhancers which display similar characteristics with previously characterised active enhancers.

SHAP and LIME: An Evaluation of Discriminative Power in Credit Risk.

In credit risk estimation, the most important element is obtaining a probability of default as close as possible to the effective risk. This effort quickly prompted new, powerful algorithms that reach a far higher accuracy, but at the cost of losing intelligibility, such as Gradient Boosting or ensemble methods. These models are usually referred to as “black-boxes”, implying that you know the inputs and the output, but there is little way to understand what is going on under the hood. As a response to that, we have seen several different Explainable AI models flourish in recent years, with the aim of letting the user see why the black-box gave a certain output. In this context, we evaluate two very popular eXplainable AI (XAI) models in their ability to discriminate observations into groups, through the application of both unsupervised and predictive modeling to the weights these XAI models assign to features locally. The evaluation is carried out on real Small and Medium Enterprises data, obtained from official italian repositories, and may form the basis for the employment of such XAI models for post-processing features extraction.

Interpretable and explainable AI (XAI) model for spatial drought prediction

Accurate prediction of any type of natural hazard is a challenging task. Of all the various hazards, drought prediction is challenging as it lacks a universal definition and is getting adverse with climate change impacting drought events both spatially and temporally. The problem becomes more complex as drought occurrence is dependent on a multitude of factors ranging from hydro-meteorological to climatic variables. A paradigm shift happened in this field when it was found that the inclusion of climatic variables in the data-driven prediction model improves the accuracy. However, this understanding has been primarily using statistical metrics used to measure the model accuracy. The present work tries to explore this finding using an explainable artificial intelligence (XAI) model. The explainable deep learning model development and comparative analysis were performed using known understandings drawn from physical-based models. The work also tries to explore how the model achieves specific results at different spatio-temporal intervals, enabling us to understand the local interactions among the predictors for different drought conditions and drought periods. The drought index used in the study is Standard Precipitation Index (SPI) at 12 month scales applied for five different regions in New South Wales, Australia, with the explainable algorithm being SHapley Additive exPlanations (SHAP). The conclusions drawn from SHAP plots depict the importance of climatic variables at a monthly scale and varying ranges of annual scale. We observe that the results obtained from SHAP align with the physical model interpretations, thus suggesting the need to add climatic variables as predictors in the prediction model.

Explainable AI methods in cyber risk management

AbstractArtificial intelligence (AI) methods are becoming widespread, especially when data are not sufficient to build classical statistical models, as is the case for cyber risk management. However, when applied to regulated industries, such as energy, finance, and health, AI methods lack explainability. Authorities aimed at validating machine learning models in regulated fields will not consider black‐box models, unless they are supplemented with further methods that explain why certain predictions have been obtained, and which are the variables that mostly concur to such predictions. Recently, Shapley values have been introduced for this purpose: They are model agnostic, and powerful, but are not normalized and, therefore, cannot become a standardized procedure. In this paper, we provide an explainable AI model that embeds Shapley values with a statistical normalization, based on Lorenz Zonoids, particularly suited for ordinal measurement variables that can be obtained to assess cyber risk.

Which SME is worth an investment? An explainable machine learning approach

Investments in Small and Medium Enterprise (SME) are facilitated by the availability of advanced machine learning (ML) methods, with high computational power and accuracy. However, despite their high accuracy, complex ML models do not provide sufficient explanation and may not be adequate for informed decision-making. In this paper, we propose an explainable AI model that can be used for analyzing SMEs and, in particular, for predicting their expected return, based on their credit risk and expected profitability. Our model is based on Shapley values which allow the predicted values generated by AI models to be interpreted according to the available explanatory variables. To validate our model we have extracted financial performance indicators from the annual balance sheets of 2049 SMEs. As a result of our empirical analysis, we show that the expected return of SMEs can be well predicted and explained by a set of characteristics deduced from their balance sheets. Therefore, their future behavior including risk and return can be predicted.

Explainable AI for Multimodal Credibility Analysis: Case Study of Online Beauty Health (Mis)-Information

“One person’s data or experience is another person’s information” this has become the golden rule of the 21st century which has resulted in a massive reservoir of data and immense amounts of information generation. However, there is no control over the source of this information, accessibility of this information, or the quality of it, which has given rise to the presence of “misinformation.” The research community has reacted by proposing frameworks and difficulties, which are helpful for (different subtasks of) recognizing misinformation. Most of these frameworks, however, fail to consider all the aspects that can contribute to making information “credible”. Furthermore, a valid explanation for each considered feature’s contribution to the model’s decision stands missing in most work. With this in mind, the authors have attempted to produce a system that yields highly accurate decisions, thus effectively separating credible health blogs from their non-credible counterparts while providing valid user-friendly explanations. The study proposes an Explainable AI-assisted Multimodal Credibility Assessment System that examines the credibility of the platform where the blog is hosted, the credibility of the author of the blog and the credibility of the images that contribute to the blog. This novel framework contributes to the existing body of knowledge by assessing the credibility of misleading beauty blogs using multiple crucial modalities which would lead to an insightful information consumption by the users. The proposed pipeline was successfully implemented on multiple carefully curated datasets and correctly identified 274 non credible blogs out of 321 blogs with an accuracy of 97.5%, Precision of 0.973 & F1score of 0.986. Further, the Explainable AI model, with the help of several visualizations displayed the feature contributions for each blog & it’s impact and magnitude in a concise comprehensible format. The framework can be further customized and applied to various domains where presence of misinformation is of high concern such as pharmaceutical drug information, pandemic management, financial advisories, online healthcare services and cyber frauds.

Explainable AI (XAI) Models Applied to Planning in Financial Markets

Regime changes planning in financial markets is well known to be hard to explain and interpret. Can an asset manager ex-plain clearly the intuition of his regime changes prediction on equity market ? To answer this question, we consider a gradi-ent boosting decision trees (GBDT) approach to plan regime changes on S&P 500 from a set of 150 technical, fundamen-tal and macroeconomic features. We report an improved ac-curacy of GBDT over other machine learning (ML) methods on the S&P 500 futures prices. We show that retaining fewer and carefully selected features provides improvements across all ML approaches. Shapley values have recently been intro-duced from game theory to the field of ML. This approach allows a robust identification of the most important variables planning stock market crises, and of a local explanation of the crisis probability at each date, through a consistent features attribution. We apply this methodology to analyse in detail the March 2020 financial meltdown, for which the model of-fered a timely out of sample prediction. This analysis unveils in particular the contrarian predictive role of the tech equity sector before and after the crash.