Explainable AI Models Research Articles

Enhancing transparency in AI-powered customer engagement

This paper addresses the critical challenge of building consumer trust in AI-powered customer engagement by emphasising the necessity for transparency and accountability. Despite the potential of AI to revolutionise business operations and enhance customer experiences, widespread concerns about misinformation and the opacity of AI decision-making processes hinder trust. Surveys highlight a significant lack of awareness among consumers regarding their interactions with AI, alongside apprehensions about bias and fairness in AI algorithms. The paper advocates for the development of explainable AI models that are transparent and understandable to both consumers and organisational leaders, thereby mitigating potential biases and ensuring ethical use. It underscores the importance of organisational commitment to transparency practices beyond mere regulatory compliance, including fostering a culture of accountability, prioritising clear data policies and maintaining active engagement with stakeholders. By adopting a holistic approach to transparency and explainability, businesses can cultivate trust in AI technologies, bridging the gap between technological innovation and consumer acceptance, and paving the way for more ethical and effective AI-powered customer engagements.

A Novel Explainable AI Model for Medical Data Analysis

Abstract This research focuses on the development of an explainable artificial intelligence (Explainable AI or XAI) system aimed at the analysis of medical data. Medical imaging and related datasets present inherent complexities due to their high-dimensional nature and the intricate biological patterns they represent. These complexities necessitate sophisticated computational models to decode and interpret, often leading to the employment of deep neural networks. However, while these models have achieved remarkable accuracy, their ”black-box” nature raises legitimate concerns regarding their interpretability and reliability in the clinical context. To address this challenge, we can consider the following approaches: traditional statistical methods, a singular complex neural network, or an ensemble of simpler neural networks. Traditional statistical methods, though transparent, often lack the nuanced sensitivity required for the intricate patterns within medical images. On the other hand, a singular complex neural network, while powerful, can sometimes be too generalized, making specific interpretations challenging. Hence, our proposed strategy employs a hybrid system, combining multiple neural networks with distinct architectures, each tailored to address specific facets of the medical data interpretation challenges. The key components of this proposed technology include a module for anomaly detection within medical images, a module for categorizing detected anomalies into specific medical conditions and a module for generating user-friendly, clinically-relevant interpretations.

Analyzing and assessing explainable AI models for smart agriculture environments

We analyze a case study in the field of smart agriculture exploiting Explainable AI (XAI) approach, a field of study that aims to provide interpretations and explanations to the behaviour of AI systems. The study regards a multiclass classification problem on the Crop Recommendation dataset. The original task is the prediction of the most adequate crop, according to seven features. In addition to the predictions, two of the most well-known XAI approaches have been used in order to obtain explanations and interpretations of the behaviour of the models: SHAP (SHapley Additive ExPlanations), and LIME (Local Interpretable Model-Agnostic Explanations). Both packages provide easy-to-understand visualizations that allow common users to understand explanations of single predictions even without going into the mathematical details of the algorithms. Within the scientific community criticisms have been raised against these approaches, and recently some papers brought to light some weaknesses. However, the two algorithms are among the most popular in XAI and are still considered points of reference for this field of study.

Explainable AI models for predicting drop coalescence in microfluidics device

In the field of chemical engineering, understanding the dynamics and probability of drop coalescence is not just an academic pursuit, but a critical requirement for advancing process design by applying energy only where it is needed to build necessary interfacial structures, increasing efficiency towards Net Zero manufacture. This research applies machine learning predictive models to unravel the sophisticated relationships embedded in the experimental data on drop coalescence in a microfluidics device. Through the deployment of SHapley Additive exPlanations values, critical features relevant to coalescence processes are consistently identified. Comprehensive feature ablation tests further delineate the robustness and susceptibility of each model. Furthermore, the incorporation of Local Interpretable Model-agnostic Explanations for local interpretability offers an elucidative perspective, clarifying the intricate decision-making mechanisms inherent to each model’s predictions. As a result, this research provides the relative importance of the features for the outcome of drop interactions. It also underscores the pivotal role of model interpretability in reinforcing confidence in machine learning predictions of complex physical phenomena that are central to chemical engineering applications.

Proteome encoded determinants of protein sorting into extracellular vesicles

AbstractExtracellular vesicles (EVs) are membranous structures released by cells into the extracellular space and are thought to be involved in cell‐to‐cell communication. While EVs and their cargo are promising biomarker candidates, sorting mechanisms of proteins to EVs remain unclear. In this study, we ask if it is possible to determine EV association based on the protein sequence. Additionally, we ask what the most important determinants are for EV association. We answer these questions with explainable AI models, using human proteome data from EV databases to train and validate the model. It is essential to correct the datasets for contaminants introduced by coarse EV isolation workflows and for experimental bias caused by mass spectrometry. In this study, we show that it is indeed possible to predict EV association from the protein sequence: a simple sequence‐based model for predicting EV proteins achieved an area under the curve of 0.77 ± 0.01, which increased further to 0.84 ± 0.00 when incorporating curated post‐translational modification (PTM) annotations. Feature analysis shows that EV‐associated proteins are stable, polar, and structured with low isoelectric point compared to non‐EV proteins. PTM annotations emerged as the most important features for correct classification; specifically, palmitoylation is one of the most prevalent EV sorting mechanisms for unique proteins. Palmitoylation and nitrosylation sites are especially prevalent in EV proteins that are determined by very strict isolation protocols, indicating they could potentially serve as quality control criteria for future studies. This computational study offers an effective sequence‐based predictor of EV associated proteins with extensive characterisation of the human EV proteome that can explain for individual proteins which factors contribute to their EV association.

A Comparison of Explainable AI Models on Numeric and Graph-Structured Data

An exponential growth of interest in the healthcare IoT over the past few years has increased the adoption of AI. However, healthcare analytics demands highly accurate and reliable systems that can not afford even a small amount of vulnerability. The black-box nature of AI models raises significant concerns about the computations involved in these predictions, leading to a lack of trust in these models. Explainable Artificial Intelligence (XAI) seeks to resolve this problem by providing a human-understandable explanation for AI decisions, bringing transparency, trust, and fairness to these AI models. This paper evaluates the existing XAI models on numeric and graph-structured healthcare data. The LIME and SHAP models are first used to explain the predictions in the numerical dataset of fetal health prediction. Second, graph-structured data is analyzed using GNN Explainer and PG Explainer. This evaluation suggests that XAI model interpretations benefit healthcare professionals and patients to trust AI predictions.

Genomic insights and advanced machine learning: characterizing autism spectrum disorder biomarkers and genetic interactions

Autism Spectrum Disorder (ASD) is a complex neurodevelopmental condition characterized by altered brain connectivity and function. In this study, we employed advanced bioinformatics and explainable AI to analyze gene expression associated with ASD, using data from five GEO datasets. Among 351 neurotypical controls and 358 individuals with autism, we identified 3,339 Differentially Expressed Genes (DEGs) with an adjusted p-value (≤ 0.05). A subsequent meta-analysis pinpointed 342 DEGs (adjusted p-value ≤ 0.001), including 19 upregulated and 10 down-regulated genes across all datasets. Shared genes, pathogenic single nucleotide polymorphisms (SNPs), chromosomal positions, and their impact on biological pathways were examined. We identified potential biomarkers (HOXB3, NR2F2, MAPK8IP3, PIGT, SEMA4D, and SSH1) through text mining, meriting further investigation. Additionally, ‎we shed light on the roles of RPS4Y1 and KDM5D genes in neurogenesis and neurodevelopment. Our analysis detected 1,286 SNPs linked to ASD-related conditions, of which 14 high-risk SNPs were located on chromosomes 10 and X. We highlighted potential missense SNPs associated with FGFR inhibitors, suggesting that it may serve as a promising biomarker for responsiveness to targeted therapies. Our explainable AI model identified the MID2 gene as a potential ASD biomarker. This research unveils vital genes and potential biomarkers, providing a foundation for novel gene discovery in complex diseases.

Challenges and Experiences in Designing Interpretable KPI-diagnostics for Cloud Applications

Automated root cause analysis of performance problems in modern cloud computing infrastructures is of a high technology value in the self-driving context. Those systems are evolved into large scale and complex solutions which are core for running most of today’s business applications. Hence, cloud management providers realize their mission through a “total” monitoring of data center flows thus enabling a full visibility into the cloud. Appropriate machine learning methods and software products rely on such observation data for real-time identification and remediation of potential sources of performance degradations in cloud operations to minimize their impacts. We describe the existing technology challenges and our experiences while working on designing problem root cause analysis mechanisms which are automatic, application agnostic, and, at the same time, interpretable for human operators to gain their trust. The paper focuses on diagnosis of cloud ecosystems through their Key Performance Indicators (KPI). Those indicators are utilized to build automatically labeled data sets and train explainable AI models for identifying conditions and processes “responsible” for misbehaviors. Our experiments on a large time series data set from a cloud application demonstrate that those approaches are effective in obtaining models that explain unacceptable KPI behaviors and localize sources of issues.

Evaluation of an Explainable Tree-Based AI Model for Thrombophilia Diagnosis and Thrombosis Risk Stratification

Background: Thrombophilia diagnosis can often be a convoluted process involving collection and analysis of clinical data, specialized laboratory testing, and high-level decision-making. This is inherently subjective due to differences in the clinical practice philosophy of each individual practitioner and can vary depending on institutional guidelines and available resources. Patient care and clinical outcomes may be affected as a result, which in turn provides a potential opportunity for optimization of thrombophilia diagnosis using AI. Methods: This retrospective study evaluated the utility and effectiveness of an AI-powered algorithm (XGBoost) programmed to replicate the process of thrombophilia diagnosis. A total of 256 patients were referred by their clinician for thrombophilia evaluation at our ambulatory coagulation clinic between November 2019 and February 2023 and clinical and laboratory data were collected from the electronic medical record. Thrombophilia diagnosis was established (or ruled out) on the basis of the patients' personal and family history of thrombosis as well as according to the results of thrombophilia testing including established acquired and inherited thrombophilia risk factors. The XGBoost, a gradient boosting algorithm for supervised learning, was used to perform a randomized search over a predefined set of tree parameters and to find a well-performing configuration on the data using cross-validation. The dataset contained 12 clinical data parameters and 26 laboratory data parameters. The target variable was two-dimensional, consisting of the thrombophilia probability score and thrombophilia risk factors. Thrombophilia probability scores were calculated based on clinical data from the following criteria: one point assigned each for a) spontaneous thrombotic event; b) mild risk situation; c) recurrent thrombosis; d) atypical thrombosis localization; e) age <50 years at the time of first thrombosis; f) family history of thrombosis and/or first degree relative with an established thrombophilia diagnosis. A probability score of 0 points corresponded to “unlikely thrombophilia”; 1-2 points “thrombophilia cannot be excluded”; 3 points “likely thrombophilia”; and 4+ points “most likely thrombophilia”. Thrombophilia risk factors were categorized by a scale of 0-3 indicating no risk (0), low risk (1), intermediate risk (2), and high risk (3). Results: Six patients were excluded from analysis due to lack of availability of sufficient clinical data. Clinical and laboratory data from the remaining 250 patients were thus included in the XGBoost analysis. The resulting trees and associated feature importance rankings revealed decisive factors for the detection of thrombophilia and suggested modified thresholds (e.g. age at the first occurrence of thrombosis increased from 50 years to 51.5 years) for an objective and standardized diagnostic procedure. The relative contribution of each clinical and laboratory feature to overall performance of the algorithm is shown in the Figure. The dataset was divided into three subsets: 200 train data, 25 validation data, and 25 test data, ensuring a representative distribution across the evaluation process. The model ultimately showed a sensitivity of 100% and a specificity of 100% for the thrombophilia probability score, and a sensitivity of 75% and specificity of 98% for the thrombophilia risk factors (Table). Conclusion: These results highlight the utility of tree-based AI models to support the objective and complex diagnosis of thrombophilia and thrombosis risk stratification. Furthermore, medical professionals can gain helpful insights from this AI-powered decision-making process due to the explainable nature of the model. We expect that the algorithm will show a higher sensitivity and specificity for the thrombophilia risk factors when we repeat the tests with a larger data set, which is currently ongoing.

An eXplainable AI (XAI) model for text-based patent novelty analysis

The technology development cycle continues to accelerate, and novelty analysis is becoming increasingly important for R&D planning as well as in the patent application process. Recently, thanks to significant advances in both text mining and natural language processing, researchers started to look into AI-assisted novelty analysis of technical content including patents. However, existing language models do not take into account the unique characteristics of technical elements in patent documents nor do they provide any explanation of their decisions including which technical elements represent a novelty. Therefore, we developed an eXplainable AI (XAI) model that evaluates novelty, takes into account the claim structure of a patent, and provides an explanation. The proposed framework of an XAI model for patent novelty consists of the following three parts: (1) dataset construction, (2) model architecture, and (3) inference process. The training dataset for patent novelty is constructed using a full-text dataset provided by the European Patent Office (EPO). A self-explainable novelty classification model is proposed and investigated. Using the fitted model, the inference results are then analyzed by extracting patents in the field of vehicle communication networks. The inference process is done by applying the fitted model to patents in the vehicle communication network field and can be expanded to address potential applications. The performance of the proposed model is then verified by comparing it with the results of similar studies. We also discuss practical applications from the perspective of patent examiners and technical planning practitioners. This study involves both an academic contribution that uses a novel approach to technology management via an XAI model and a practical contribution that can be used for patent analysis.

An Explainable AI Model for ICU Admission Prediction of COVID-19 Patients

COVID-19 has overwhelmed hospitals all around the world, because of the very high number of patients requiring intubation. Predicting and anticipating which patients will be intubated can help hospitals better allocate their resources. For this, we created an explainable AI model that receives some medical data for each patient and returns a prediction on whether they will be intubated later on, while also explaining the argumentation process that led to that conclusion. This model can help medical professionals better manage the ICU beds and the patients between hospitals and warn them on the symptoms and signs to look for in patients worsening. To obtain the model, we first trained various ML algorithms on a dataset of COVID ICU admissions. Then, using the most accurate of the models, we obtained the sum of the most important rules, using R’s inTrees framework. With the rules obtained, we produced a program in Gorgias written in Prolog. We added some more rules and changed the ordering to come up with a program that requires some patient vitals and blood results as inputs and then returns the prediction along with its reasoning.

The Impact of AI on Regulatory Compliance and Anti-Money Laundering Efforts in Payment Processing

The paper elaborates on how AI is changing the face of regulatory compliance and anti-money laundering initiatives in the industry for processing payments. This can be done with artificial intelligence tools—natural language processing and machine learning algorithms—applied in areas such as document analysis, anomaly detection, and transaction monitoring. The paper points to increased detection rates, reduced false positives, and improved regulatory reporting associated with the use of AI within the KYC process and customer due diligence procedures more generally. It identifies how AI can transform the financial crime prevention landscape while also acknowledging such challenges as explainability, bias mitigation, and ethical concerns. Finally, some potential future paths in this area, like blockchain integration, federated learning, and the creation of more sophisticated explainable AI models for compliance systems, are discussed.

Towards reliable and explainable AI model for pulmonary nodule diagnosis

Background and ObjectiveLung cancer has the highest mortality rate of all deadly cancers in the world. Early detection is essential to the treatment of lung cancer. However, detection and accurate diagnosis of pulmonary nodules depend heavily on the experience of radiologists and can be a heavy workload for them. Computer-aided diagnosis (CAD) systems have been developed to assist radiologists in nodule detection and diagnosis, greatly easing the workload while increasing diagnosis accuracy. Recent developments in deep learning have greatly improved the performance of CAD systems. However, the lack of model reliability and interpretability remains a major obstacle for its large-scale clinical application. MethodsWe proposed a multi-task explainable deep-learning model for pulmonary nodule diagnosis, named “ExPN-Net”. The proposed ExPN-Net can not only predict lesion malignancy but also identify nodule characteristics. Further, the location of each characteristic can also be visualized for visual interpretability. We first used 3D nnU-Net to obtain the probability maps of lung nodules, which were used as attention for the following classification task. Then, we used an anatomical attention gate (AAG) mechanism to introduce the spatial attention information into the classification network. Finally, the soft activation map (SAM) module is used to generate fine-grained activation maps to indicate the nodule characteristics. To evaluate the performance, we validated ExPN-Net on both public and in-house datasets. 624 and 807 participants were enrolled from the public LIDC and our-inhouse datasets, respectively. The roles of different modules in ExPN-Net are analyzed by ablation experiments. ResultsExPN-Net achieved a test AUC of 0.992 and 0.923 on the LIDC public and in-house datasets, respectively. Moreover, experimental results proved that by incorporating characteristic identification tasks into the multi-task model, the accuracy of the malignancy classification can also be improved. ConclusionsThis work successfully updates the state-of-the-art diagnosis accuracy of pulmonary nodules by incorporating characteristics and anatomical attention mechanism. Furthermore, our multi-task explainable model provides a solution for AI models to better interact with radiologists in a clinical environment. Source code is available at: https://github.com/KumoLiu/explainablePN.

An Explainable AI Approach to Agrotechnical Monitoring and Crop Diseases Prediction in Dnipro Region of Ukraine

Abstract The proliferation of computer-oriented and information digitalisation technologies has become a hallmark across various sectors in today’s rapidly evolving environment. Among these, agriculture emerges as a pivotal sector in need of seamless incorporation of high-performance information technologies to address the pressing needs of national economies worldwide. The aim of the present article is to substantiate scientific and applied approaches to improving the efficiency of computer-oriented agrotechnical monitoring systems by developing an intelligent software component for predicting the probability of occurrence of corn diseases during the full cycle of its cultivation. The object of research is non-stationary processes of intelligent transformation and predictive analytics of soil and climatic data, which are factors of the occurrence and development of diseases in corn. The subject of the research is methods and explainable AI models of intelligent predictive analysis of measurement data on the soil and climatic condition of agricultural enterprises specialised in growing corn. The main scientific and practical effect of the research results is the development of IoT technologies for agrotechnical monitoring through the development of a computer-oriented model based on the ANFIS technique and the synthesis of structural and algorithmic provision for identifying and predicting the probability of occurrence of corn diseases during the full cycle of its cultivation.

1036-P: Developing an Explainable AI Model to Identify Members with Diabetes at High Risk for Worsening Glycemic Control in a Large Accountable Care Network (ACN)

People with diabetes and poor glycemic control (HbA1c ≥9%) have a greater likelihood of diabetes complications, increased healthcare utilization, and higher total cost of care (TCOC). Novel approaches to identify patients at high risk for worsening glycemic control can improve outcomes and contain costs within ACNs. We used XGBoost Decision Trees to build and validate a model predicting worsening glycemic control (HbA1c increase ≥1.5% over 12 months) using Medicare ACN administrative claims and electronic health record data. Eligible members had ≥18 mos. continuous enrollment, were age ≥18 years with Type 1 or 2 diabetes, had ≥2 HbA1c values between 6.5 and 14%, and no disability. Members were split into HbA1c outcome stratified training (80%) and validation (20%) datasets for 10-fold cross-validation. The model utilized 1,490 features including medical history, labs, demographics, SDOH, utilization, and quality of care. Of the 44,007 unique members (mean age 72 years, 51% Female, 61% Non-Hispanic White, mean HbA1c 7.2%, 9% on insulin, HTN 92%, Obesity 71%, CKD 33%, CAD 32%, Heart Failure 15% and COPD 13%), 5.23% had an HbA1c increase ≥1.5% over 12 months. Key features in the final model included duration of diabetes, last HbA1c value, treatment intensification, prescribed insulin or sulfonylureas, weight change, depression, acute renal failure, area deprivation index, and no car access. Model AUC was 0.672 (95% CI ±0.005) in the validation sample. The model was able to identify 23.5% of members with worsening glycemic control in the highest predicted decile of HbA1c progression (sensitivity 23.5% at 10% alert rate). Mean TCOC at 10% alert rate was $43,850 PMPY which was substantially higher than training population at $18,208 PMPY. Our model can identify ACN members with diabetes at high risk for worsening glycemic control that may benefit from education, medication adherence, food insecurity and transportation interventions. Disclosure A.Kermani: None. C.J.Mccall: None. J.A.Gartner: None. D.Decaprio: Employee; ClosedLoop.ai. J.M.Mutz: None. J.S.Fish: None. U.Kollipara: None. J.Acosta: None. M.E.Bowen: Research Support; Boehringer-Ingelheim. J.P.Almandoz: Advisory Panel; Eli Lilly and Company, Novo Nordisk. B.Goldberg: None. A.Rozich: None. M.He: None. S.V.J.Acosta: None.

Explainable machine learning models to analyse maternal health

Maternal health is a significant public health concern for globe and many developing countries. A country like India (with large population), there are considerable disparities in maternal health service utilisation and maternal mortality within and across states. A more than a general healthcare operational policy would suffice, but a precision healthcare strategy would be needed. This article focused on explainable machine learning models that can precisely advise health care intervention policy and medical treatment to an administrative unit rather than a generic policy suggestion for improving maternal health. This study presents an exhaustive list of factors associated with Maternal Mortality Rate (MMR) and a series of explainable AI models. One of models uses CART heuristics to categorise districts (administrative boundaries) into lower and higher MMR classes. Another explainable model, Shapley Additive Explanations (SHAP), used SVM, ANN, boosting, and random forest machine learning models to investigate higher and lower MMR regions. Further, an Explainable Boosting Machine (EBM) also used, and the results are compared for policy suggestions. Some of the ignored features from general social science studies, such as topography and agro-climatic zone characteristics of a particular district, may be crucial in the analysis. Moreover, health infrastructure, insurance, and other factors also influence policymaking. This predictive and explainable work has significant implications for precision healthcare policy design to improve maternal health compared to a broader policy approach.

Earthquake spatial probability and hazard estimation using various explainable AI (XAI) models at the Arabian peninsula

Earthquakes are the most destructive natural hazards because of their adversely severe impacts on urban areas. Earthquakes affect people's lives and properties, thus captivating the extensive attention of seismologists. Carrying out probability and hazard assessment for the prevention, and reduction of mega-events and recovery will be of great significance in affected areas. Given that limited studies have attempted to estimate earthquake Spatial Probability Assessment (SPA) in the Arabian Peninsula, this study aims to evaluate the SPA and Earthquake Hazard Assessment (EHA). This study implements and evaluates various machine learning and explainable-AI (XAI) techniques for the estimation of SPA and EHA in the Arabian Peninsula, explores the contribution and highlights the importance of different factors in the development of AI-based models. A total of twelve factors ranging from seismological to geophysical factors were evaluated. Two machine learning models namely Light Gradient Boosting Machine (LightGBM) and deep Recurrent Neural Networks (RNN) along with three XAI approaches (i.e, Smart predictor, Smart Explainer and Local Interpretable Model-Agnostic Explanation (LIME) model) were investigated. Results of the comparative earthquake SPA estimation demonstrated that the accuracy of 89% and 87% were achieved by LightGBM and RNN models. Moreover, the results of the XAI models show that the Smart Predictor provides better spatial outputs than the other evaluated XAI models. The stable factors identified by Smart Predictor were magnitude variation and earthquake frequency whereas the important factors were magnitude variation, earthquake frequency, depth variation, and seismic gap. Collectively, results of SPA show that, the Gulf of Aden, Red Sea, Iran, and Turkey are falling under a very-high SPA index (0.991–1). Correspondingly, Gulf areas, coastal areas of Saudi Arabia, and areas in the Zagros fault and Anatolian fault zone fall under a very-high hazard zone. This research could support planners, and decision-makers for emergency planning, infrastructure development, and reconstruction projects.

An explainable AI (XAI) model for landslide susceptibility modeling

Landslides are among the most devastating natural hazards, severely impacting human lives and damaging property and infrastructure. Landslide susceptibility maps, which help to identify which regions in a given area are at greater risk of a landslide occurring, are a key tool for effective mitigation. Research in this field has grown immensely, ranging from quantitative to deterministic approaches, with a recent surge in machine learning (ML)-based computational models. The development of ML models, in particular, has undergone a meteoritic rise in the last decade, contributing to the successful development of accurate susceptibility maps. However, despite their success, these models are rarely used by stakeholders owing to their “black box” nature. Hence, it is crucial to explain the results, thus providing greater transparency for the use of such models. To address this gap, the present work introduces the use of an ML-based explainable algorithm, SHapley Additive exPlanations (SHAP), for landslide susceptibility modeling. A convolutional neural network model was used conducted in the CheongJu region in South Korea. A total of 519 landslide locations were examined with 16 landslide-affected variables, of which 70% was used for training and 30% for testing, and the model achieved an accuracy of 89%. Further, the comparison was performed using Support Vector Machine mode, which achieved an accuracy of 84%. The SHAP plots showed variations in feature interactions for both landslide and non-landslide locations, thus providing more clarity as to how the model achieves a specific result. The SHAP dependence plots explained the relationship between altitude and slope, showing a negative relationship with altitude and a positive relationship with slope. This is the first use of an explainable ML model in landslide susceptibility modeling, and we argue that future works should include aspects of explainability to open up the possibility of developing a transferable artificial intelligence model.

Interactive Explainable Deep Learning Model Informs Prostate Cancer Diagnosis at MRI.

Background Clinically significant prostate cancer (PCa) diagnosis at MRI requires accurate and efficient radiologic interpretation. Although artificial intelligence may assist in this task, lack of transparency has limited clinical translation. Purpose To develop an explainable artificial intelligence (XAI) model for clinically significant PCa diagnosis at biparametric MRI using Prostate Imaging Reporting and Data System (PI-RADS) features for classification justification. Materials and Methods This retrospective study included consecutive patients with histopathologic analysis-proven prostatic lesions who underwent biparametric MRI and biopsy between January 2012 and December 2017. After image annotation by two radiologists, a deep learning model was trained to detect the index lesion; classify PCa, clinically significant PCa (Gleason score ≥ 7), and benign lesions (eg, prostatitis); and justify classifications using PI-RADS features. Lesion- and patient-based performance were assessed using fivefold cross validation and areas under the receiver operating characteristic curve. Clinical feasibility was tested in a multireader study and by using the external PROSTATEx data set. Statistical evaluation of the multireader study included Mann-Whitney U and exact Fisher-Yates test. Results Overall, 1224 men (median age, 67 years; IQR, 62-73 years) had 3260 prostatic lesions (372 lesions with Gleason score of 6; 743 lesions with Gleason score of ≥ 7; 2145 benign lesions). XAI reliably detected clinically significant PCa in internal (area under the receiver operating characteristic curve, 0.89) and external test sets (area under the receiver operating characteristic curve, 0.87) with a sensitivity of 93% (95% CI: 87, 98) and an average of one false-positive finding per patient. Accuracy of the visual and textual explanations of XAI classifications was 80% (1080 of 1352), confirmed by experts. XAI-assisted readings improved the confidence (4.1 vs 3.4 on a five-point Likert scale; P = .007) of nonexperts in assessing PI-RADS 3 lesions, reducing reading time by 58 seconds (P = .009). Conclusion The explainable AI model reliably detected and classified clinically significant prostate cancer and improved the confidence and reading time of nonexperts while providing visual and textual explanations using well-established imaging features. © RSNA, 2023 Supplemental material is available for this article. See also the editorial by Chapiro in this issue.

Limited Discriminator GAN using explainable AI model for overfitting problem

Data-driven learning is the most representative deep learning method. Generative adversarial networks (GANs) are designed to generate sufficient data to support such learning. The learning process of GAN models typically trains a generator and discriminator in turn. However, overfitting problems occur when the discriminator depends excessively on the training data. When this problem persists, the image created by the generator shows a similar appearance to the learning image. Images similar to learning images eventually lose the meaning of data augmentation. In this paper, we propose a limited discriminator GAN (LDGAN) model that explains the results of GAN, which is a model that cannot be analyzed externally, such as a black box. The part explained in LDGAN becomes the discriminator model of GAN, and it is possible to check which area of the image is used as the basis for determining fake/real by the discriminator. In the end, a method for limiting the learning of discriminator is proposed based on the described results. Through this, it is possible to avoid the overfitting problem of the discriminator and to generate various images different from the learning image. The LDGAN method allows users to perform meaningful data augmentation with only specific objects except for complex images or backgrounds that require analysis. Compare the LDGAN method with the existing DCGAN and present the extensive simulation results. The extensive simulation result shows that the image generated by the proposed LDGAN including the estimation area is about 10% more.