Layer-wise Relevance Propagation Research Articles

Advancing Attribution-Based Neural Network Explainability through Relative Absolute Magnitude Layer-Wise Relevance Propagation and Multi-Component Evaluation

Recent advancement in deep-neural network performance led to the development of new state-of-the-art approaches in numerous areas. However, the black-box nature of neural networks often prohibits their use in areas where model explainability and model transparency are crucial. Over the years, researchers proposed many algorithms to aid neural network understanding and provide additional information to the human expert. One of the most popular methods being Layer-Wise Relevance Propagation (LRP). This method assigns local relevance based on the pixel-wise decomposition of nonlinear classifiers. With the rise of attribution method research, there has emerged a pressing need to assess and evaluate their performance. Numerous metrics have been proposed, each assessing an individual property of attribution methods such as faithfulness, robustness, or localization. Unfortunately, no single metric is deemed optimal for every case, and researchers often use several metrics to test the quality of the attribution maps. In this work, we address the shortcomings of the current LRP formulations and introduce a novel method for determining the relevance of input neurons through layer-wise relevance propagation. Furthermore, we apply this approach to the recently developed Vision Transformer architecture and evaluate its performance against existing methods on two image classification datasets, namely ImageNet and PascalVOC. Our results clearly demonstrate the advantage of our proposed method. Furthermore, we discuss the insufficiencies of current evaluation metrics for attribution-based explainability and propose a new evaluation metric that combines the notions of faithfulness, robustness, and contrastiveness. We utilize this new metric to evaluate the performance of various attribution-based methods. Our code is available at: https://github.com/davor10105/relative-absolute-magnitude-propagation

An explainable multiscale LSTM model with wavelet transform and layer-wise relevance propagation for daily streamflow forecasting

Developing an accurate and reliable daily streamflow forecasting model is important for facilitating the efficient resource planning and management of hydrological systems. In this study, an explainable multiscale long short-term memory (XM-LSTM) model is proposed for effective daily streamflow forecasting by integrating the à trous wavelet transform (ATWT) for decomposing data, the Boruta algorithm for identifying model inputs, and the layer-wise relevance propagation (LRP) for explaining the prediction results. The proposed XM-LSTM is tested by performing multi-step-ahead forecasting of daily streamflow at four stations in the middle and lower reaches of the Yangtze River basin and compared with the X-LSTM. The X-LSTM is formed by coupling the long short-term memory (LSTM) with the LRP. For comparison, the inputs of these two models are identified by the Boruta selection algorithm. The results show that all models exhibit good ability to forecast daily streamflow, however, the prediction performance decreases as the lead time increases. The XM-LSTM provides a better forecasting performance than the X-LSTM, suggesting the ability of the ATWT to improve the LSTM for daily streamflow forecasting. Moreover, the correlation scores analysis by the LRP shows that the ATWT can extract useful information that influences the daily streamflow from the raw predictors, and the water level has the most significant contribution to streamflow prediction. Accordingly, the XM-LSTM model can be viewed as a potentially useful approach for increasing the accuracy and explainability of streamflow forecasting.

Empowering CAM-Based Methods with Capability to Generate Fine-Grained and High-Faithfulness Explanations

Recently, the explanation of neural network models has garnered considerable research attention. In computer vision, CAM (Class Activation Map)-based methods and LRP (Layer-wise Relevance Propagation) method are two common explanation methods. However, since most CAM-based methods can only generate global weights, they can only generate coarse-grained explanations at a deep layer. LRP and its variants, on the other hand, can generate fine-grained explanations. But the faithfulness of the explanations is too low. To address these challenges, in this paper, we propose FG-CAM (Fine-Grained CAM), which extends CAM-based methods to enable generating fine-grained and high-faithfulness explanations. FG-CAM uses the relationship between two adjacent layers of feature maps with resolution differences to gradually increase the explanation resolution, while finding the contributing pixels and filtering out the pixels that do not contribute. Our method not only solves the shortcoming of CAM-based methods without changing their characteristics, but also generates fine-grained explanations that have higher faithfulness than LRP and its variants. We also present FG-CAM with denoising, which is a variant of FG-CAM and is able to generate less noisy explanations with almost no change in explanation faithfulness. Experimental results show that the performance of FG-CAM is almost unaffected by the explanation resolution. FG-CAM outperforms existing CAM-based methods significantly in both shallow and intermediate layers, and outperforms LRP and its variants significantly in the input layer. Our code is available at https://github.com/dongmo-qcq/FG-CAM.

Towards Diverse Perspective Learning with Selection over Multiple Temporal Poolings

In Time Series Classification (TSC), temporal pooling methods that consider sequential information have been proposed. However, we found that each temporal pooling has a distinct mechanism, and can perform better or worse depending on time series data. We term this fixed pooling mechanism a single perspective of temporal poolings. In this paper, we propose a novel temporal pooling method with diverse perspective learning: Selection over Multiple Temporal Poolings (SoM-TP). SoM-TP dynamically selects the optimal temporal pooling among multiple methods for each data by attention. The dynamic pooling selection is motivated by the ensemble concept of Multiple Choice Learning (MCL), which selects the best among multiple outputs. The pooling selection by SoM-TP's attention enables a non-iterative pooling ensemble within a single classifier. Additionally, we define a perspective loss and Diverse Perspective Learning Network (DPLN). The loss works as a regularizer to reflect all the pooling perspectives from DPLN. Our perspective analysis using Layer-wise Relevance Propagation (LRP) reveals the limitation of a single perspective and ultimately demonstrates diverse perspective learning of SoM-TP. We also show that SoM-TP outperforms CNN models based on other temporal poolings and state-of-the-art models in TSC with extensive UCR/UEA repositories.

Identification of kidney stones in KUB X-ray images using VGG16 empowered with explainable artificial intelligence

A kidney stone is a solid formation that can lead to kidney failure, severe pain, and reduced quality of life from urinary system blockages. While medical experts can interpret kidney-ureter-bladder (KUB) X-ray images, specific images pose challenges for human detection, requiring significant analysis time. Consequently, developing a detection system becomes crucial for accurately classifying KUB X-ray images. This article applies a transfer learning (TL) model with a pre-trained VGG16 empowered with explainable artificial intelligence (XAI) to establish a system that takes KUB X-ray images and accurately categorizes them as kidney stones or normal cases. The findings demonstrate that the model achieves a testing accuracy of 97.41% in identifying kidney stones or normal KUB X-rays in the dataset used. VGG16 model delivers highly accurate predictions but lacks fairness and explainability in their decision-making process. This study incorporates the Layer-Wise Relevance Propagation (LRP) technique, an explainable artificial intelligence (XAI) technique, to enhance the transparency and effectiveness of the model to address this concern. The XAI technique, specifically LRP, increases the model's fairness and transparency, facilitating human comprehension of the predictions. Consequently, XAI can play an important role in assisting doctors with the accurate identification of kidney stones, thereby facilitating the execution of effective treatment strategies.

Stable feature selection utilizing Graph Convolutional Neural Network and Layer-wise Relevance Propagation for biomarker discovery in breast cancer

High-throughput technologies are becoming increasingly important in discovering prognostic biomarkers and in identifying novel drug targets. With Mammaprint, Oncotype DX, and many other prognostic molecular signatures breast cancer is one of the paradigmatic examples of the utility of high-throughput data to deliver prognostic biomarkers, that can be represented in a form of a rather short gene list. Such gene lists can be obtained as a set of features (genes) that are important for the decisions of a Machine Learning (ML) method applied to high-dimensional gene expression data. Several studies have identified predictive gene lists for patient prognosis in breast cancer, but these lists are unstable and have only a few genes in common. Instability of feature selection impedes biological interpretability: genes that are relevant for cancer pathology should be members of any predictive gene list obtained for the same clinical type of patients. Stability and interpretability of selected features can be improved by including information on molecular networks in ML methods. Graph Convolutional Neural Network (GCNN) is a contemporary deep learning approach applicable to gene expression data structured by a prior knowledge molecular network. Layer-wise Relevance Propagation (LRP) and SHapley Additive exPlanations (SHAP) are methods to explain individual decisions of deep learning models. We used both GCNN+LRP and GCNN+SHAP techniques to construct feature sets by aggregating individual explanations. We suggest a methodology to systematically and quantitatively analyze the stability, the impact on the classification performance, and the interpretability of the selected feature sets. We used this methodology to compare GCNN+LRP to GCNN+SHAP and to more classical ML-based feature selection approaches. Utilizing a large breast cancer gene expression dataset we show that, while feature selection with SHAP is useful in applications where selected features have to be impactful for classification performance, among all studied methods GCNN+LRP delivers the most stable (reproducible) and interpretable gene lists.

Interpretable parallel channel encoding convolutional neural network for bearing fault diagnosis

Interpretability plays a crucial role in the application of neural networks for fault diagnosis. Integrating preprocessing methods into neural networks can enhance interpretability while preserving their ‘end-to-end’ characteristics. However, when only redesigning the first layer, subsequent structures still exhibit limited transparency. Additionally, traditional convolution structure is ill-suited for analyzing the readable feature maps derived from vibration signals. To address these challenges, this paper proposes a novel convolution structure for the parameterized signal processing function of the first-layer convolution kernel. This structure incorporates channel mixing for feature augmentation, designs a condensed feature encoder for aggregating and compressing features channel-by-channel, ensures the interpretability of feature map processing, and obtains condensed feature codes to propose smooth activation layer-wise relevance propagation (SA-LRP) method that to perform interpretability analysis. Additionally, cubic feature screening is implemented for diagnostic classification to improve structural fitness. We design experiments using multiple datasets to test various indicators of the structure. The results confirm that connecting our convolution architecture for subsequent analysis outperforms other convolution architectures for the convolution kernel of the first-layer parameterized signal processing function. The interpretability of the model is evaluated through SA-LRP method and validates the interpretability of the model.

Differences in running technique between runners with better and poorer running economy and lower and higher milage: An artificial neural network approach.

Prior studies investigated selected discrete sagittal-plane outcomes (e.g., peak knee flexion) in relation to running economy, hereby discarding the potential relevance of running technique parameters during noninvestigated phases of the gait cycle and in other movement planes. Investigate which components of running technique distinguish groups of runners with better and poorer economy and higher and lower weekly running distance using an artificial neural network (ANN) approach with layer-wise relevance propagation. Forty-one participants (22 males and 19 females) ran at 2.78 m∙s-1 while three-dimensional kinematics and gas exchange data were collected. Two groups were created that differed in running economy or weekly training distance. The three-dimensional kinematic data were used as input to an ANN to predict group allocations. Layer-wise relevance propagation was used to determine the relevance of three-dimensional kinematics for group classification. The ANN classified runners in the correct economy or distance group with accuracies of up to 62% and 71%, respectively. Knee, hip, and ankle flexion were most relevant to both classifications. Runners with poorer running economy showed higher knee flexion during swing, more hip flexion during early stance, and more ankle extension after toe-off. Runners with higher running distance showed less trunk rotation during swing. The ANN accuracy was moderate when predicting whether runners had better, or poorer running economy, or had a higher or lower weekly training distance based on their running technique. The kinematic components that contributed the most to the classification may nevertheless inform future research and training.

Using machine learning to analyze the changes in extreme precipitation in southern China

Convolutional neural network (CNN) is applied to identify the extreme precipitation events (EPEs) in southern China and the physical contributions are quantified for the changes in extreme precipitation in recent decades. The CNN correctly identifies about 96% of the observed EPEs based on the given large-scale atmospheric circulation. The discrimination made by the neural network is revealed by using the layer-wise relevance propagation method. The CNN is inclined to classify the circulation pattern as an extreme precipitation circulation pattern (EPCP), when a deep cyclonic anomaly and intense horizontal pressure gradient appear over southern China at the lower and middle troposphere. The extreme precipitation amount decreases during April and May (AM) and increases in June to August (JJA) after the early-1990s. Result of the quantitative partitioning analysis indicates that the dynamic and thermodynamic effects respectively contribute 297.6% and −234% to the reduction of extreme precipitation in AM. The decline in EPCP frequency dominates the decrease in EPEs. In JJA, the increased extreme precipitation after the early 1990s is attributed to both the interdecadal increase in the EPCP frequency and the increasing trend in the atmospheric moisture. The dynamic and thermodynamic changes play almost equal roles in the increased extreme precipitation in JJA.

Explaining decisions of a light-weight deep neural network for real-time coronary artery disease classification in magnetic resonance imaging

In certain healthcare settings, such as emergency or critical care units, where quick and accurate real-time analysis and decision-making are required, the healthcare system can leverage the power of artificial intelligence (AI) models to support decision-making and prevent complications. This paper investigates the optimization of healthcare AI models based on time complexity, hyper-parameter tuning, and XAI for a classification task. The paper highlights the significance of a lightweight convolutional neural network (CNN) for analysing and classifying Magnetic Resonance Imaging (MRI) in real-time and is compared with CNN-RandomForest (CNN-RF). The role of hyper-parameter is also examined in finding optimal configurations that enhance the model’s performance while efficiently utilizing the limited computational resources. Finally, the benefits of incorporating the XAI technique (e.g. GradCAM and Layer-wise Relevance Propagation) in providing transparency and interpretable explanations of AI model predictions, fostering trust, and error/bias detection are explored. Our inference time on a MacBook laptop for 323 test images of size 100x100 is only 2.6 sec, which is merely 8 milliseconds per image while providing comparable classification accuracy with the ensemble model of CNN-RF classifiers. Using the proposed model, clinicians/cardiologists can achieve accurate and reliable results while ensuring patients’ safety and answering questions imposed by the General Data Protection Regulation (GDPR). The proposed investigative study will advance the understanding and acceptance of AI systems in connected healthcare settings.

Convolutional neural network based data interpretable framework for Alzheimer’s treatment planning

Alzheimer’s disease (AD) is a neurological disorder that predominantly affects the brain. In the coming years, it is expected to spread rapidly, with limited progress in diagnostic techniques. Various machine learning (ML) and artificial intelligence (AI) algorithms have been employed to detect AD using single-modality data. However, recent developments in ML have enabled the application of these methods to multiple data sources and input modalities for AD prediction. In this study, we developed a framework that utilizes multimodal data (tabular data, magnetic resonance imaging (MRI) images, and genetic information) to classify AD. As part of the pre-processing phase, we generated a knowledge graph from the tabular data and MRI images. We employed graph neural networks for knowledge graph creation, and region-based convolutional neural network approach for image-to-knowledge graph generation. Additionally, we integrated various explainable AI (XAI) techniques to interpret and elucidate the prediction outcomes derived from multimodal data. Layer-wise relevance propagation was used to explain the layer-wise outcomes in the MRI images. We also incorporated submodular pick local interpretable model-agnostic explanations to interpret the decision-making process based on the tabular data provided. Genetic expression values play a crucial role in AD analysis. We used a graphical gene tree to identify genes associated with the disease. Moreover, a dashboard was designed to display XAI outcomes, enabling experts and medical professionals to easily comprehend the prediction results.

Transparency in AI Decision Making: A Survey of Explainable AI Methods and Applications

Artificial Intelligence (AI) systems have become pervasive in numerous facets of modern life, wielding considerable influence in critical decision-making realms such as healthcare, finance, criminal justice, and beyond. Yet, the inherent opacity of many AI models presents significant hurdles concerning trust, accountability, and fairness. To address these challenges, Explainable AI (XAI) has emerged as a pivotal area of research, striving to augment the transparency and interpretability of AI systems. This survey paper serves as a comprehensive exploration of the state-of-the-art in XAI methods and their practical applications. We delve into a spectrum of techniques, spanning from model-agnostic approaches to interpretable machine learning models, meticulously scrutinizing their respective strengths, limitations, and real-world implications. The landscape of XAI is rich and varied, with diverse methodologies tailored to address different facets of interpretability. Model-agnostic approaches offer versatility by providing insights into model behavior across various AI architectures. In contrast, interpretable machine learning models prioritize transparency by design, offering inherent understandability at the expense of some predictive performance. Layer-wise Relevance Propagation (LRP) and attention mechanisms delve into the inner workings of neural networks, shedding light on feature importance and decision processes. Additionally, counterfactual explanations open avenues for exploring what-if scenarios, elucidating the causal relationships between input features and model outcomes. In tandem with methodological exploration, this survey scrutinizes the deployment and impact of XAI across multifarious domains. Successful case studies showcase the practical utility of transparent AI in healthcare diagnostics, financial risk assessment, criminal justice systems, and more. By elucidating these use cases, we illuminate the transformative potential of XAI in enhancing decision-making processes while fostering accountability and fairness. Nevertheless, the journey towards fully transparent AI systems is fraught with challenges and opportunities. As we traverse the current landscape of XAI, we identify pressing areas for further research and development. These include refining interpretability metrics, addressing the scalability of XAI techniques to complex models, and navigating the ethical dimensions of transparency in AI decision-making.Through this survey, we endeavor to cultivate a deeper understanding of transparency in AI decision-making, empowering stakeholders to navigate the intricate interplay between accuracy, interpretability, and ethical considerations. By fostering interdisciplinary dialogue and inspiring collaborative innovation, we aspire to catalyze future advancements in Explainable AI, ultimately paving the way towards more accountable and trustworthy AI systems.

Improving deep neural network generalization and robustness to background bias via layer-wise relevance propagation optimization

Features in images’ backgrounds can spuriously correlate with the images’ classes, representing background bias. They can influence the classifier’s decisions, causing shortcut learning (Clever Hans effect). The phenomenon generates deep neural networks (DNNs) that perform well on standard evaluation datasets but generalize poorly to real-world data. Layer-wise Relevance Propagation (LRP) explains DNNs’ decisions. Here, we show that the optimization of LRP heatmaps can minimize the background bias influence on deep classifiers, hindering shortcut learning. By not increasing run-time computational cost, the approach is light and fast. Furthermore, it applies to virtually any classification architecture. After injecting synthetic bias in images’ backgrounds, we compared our approach (dubbed ISNet) to eight state-of-the-art DNNs, quantitatively demonstrating its superior robustness to background bias. Mixed datasets are common for COVID-19 and tuberculosis classification with chest X-rays, fostering background bias. By focusing on the lungs, the ISNet reduced shortcut learning. Thus, its generalization performance on external (out-of-distribution) test databases significantly surpassed all implemented benchmark models.

Interpretable machine learning comprehensive human gait deterioration analysis.

Gait analysis, an expanding research area, employs non-invasive sensors and machine learning techniques for a range of applications. In this study, we investigate the impact of cognitive decline conditions on gait performance, drawing connections between gait deterioration in Parkinson's Disease (PD) and healthy individuals dual tasking. We employ Explainable Artificial Intelligence (XAI) specifically Layer-Wise Relevance Propagation (LRP), in conjunction with Convolutional Neural Networks (CNN) to interpret the intricate patterns in gait dynamics influenced by cognitive loads. We achieved classification accuracies of 98% F1 scores for PD dataset and 95.5% F1 scores for the combined PD dataset. Furthermore, we explore the significance of cognitive load in healthy gait analysis, resulting in robust classification accuracies of 90% ± 10% F1 scores for subject cognitive load verification. Our findings reveal significant alterations in gait parameters under cognitive decline conditions, highlighting the distinctive patterns associated with PD-related gait impairment and those induced by multitasking in healthy subjects. Through advanced XAI techniques (LRP), we decipher the underlying features contributing to gait changes, providing insights into specific aspects affected by cognitive decline. Our study establishes a novel perspective on gait analysis, demonstrating the applicability of XAI in elucidating the shared characteristics of gait disturbances in PD and dual-task scenarios in healthy individuals. The interpretability offered by XAI enhances our ability to discern subtle variations in gait patterns, contributing to a more nuanced comprehension of the factors influencing gait dynamics in PD and dual-task conditions, emphasizing the role of XAI in unraveling the intricacies of gait control.

Sparse Deep Neural Network for Encoding and Decoding the Structural Connectome.

Brain state classification by applying deep learning techniques on neuroimaging data has become a recent topic of research. However, unlike domains where the data is low dimensional or there are large number of available training samples, neuroimaging data is high dimensional and has few training samples. To tackle these issues, we present a sparse feedforward deep neural architecture for encoding and decoding the structural connectome of the human brain. We use a sparsely connected element-wise multiplication as the first hidden layer and a fixed transform layer as the output layer. The number of trainable parameters and the training time is significantly reduced compared to feedforward networks. We demonstrate superior performance of this architecture in encoding the structural connectome implicated in Alzheimer's disease (AD) and Parkinson's disease (PD) from DTI brain scans. For decoding, we propose recursive feature elimination (RFE) algorithm based on DeepLIFT, layer-wise relevance propagation (LRP), and Integrated Gradients (IG) algorithms to remove irrelevant features and thereby identify key biomarkers associated with AD and PD. We show that the proposed architecture reduces 45.1% and 47.1% of the trainable parameters compared to a feedforward DNN with an increase in accuracy by 2.6 % and 3.1% for cognitively normal (CN) vs AD and CN vs PD classification, respectively. We also show that the proposed RFE method leads to a further increase in accuracy by 2.1% and 4% for CN vs AD and CN vs PD classification, while removing approximately 90% to 95% irrelevant features. Furthermore, we argue that the biomarkers (i.e., key brain regions and connections) identified are consistent with previous literature. We show that relevancy score-based methods can yield high discriminative power and are suitable for brain decoding. We also show that the proposed approach led to a reduction in the number of trainable network parameters, an increase in classification accuracy, and a detection of brain connections and regions that were consistent with earlier studies.

Interpretable Sleep Stage Classification Based on Layer-Wise Relevance Propagation

Interpretable Sleep Stage Classification Based on Layer-Wise Relevance Propagation

Probabilistic Convective Initiation Nowcasting Using Himawari-8 AHI with Explainable Deep Learning Models

Abstract Convective initiation (CI) nowcasting is crucial for reducing loss of human life and property caused by severe convective weather. A novel deep learning method based on the U-Net model (named as CIUnet) was developed for forecasting CI during the warm season with eight interest fields of Himawari-8 Advanced Himawari Imager (AHI) and terrain height. The results showed that the CIUnet model produced probability forecasts of CI occurrence location and time with probability of detection (POD) at 93.3% ± 0.3% and false alarm ratio (FAR) at 18.3% ± 0.4% at a lead time of 30 min. Sensitivity and permutation importance experiments on the input fields of the CIUnet model revealed that the differences in brightness temperature for spectral channels were more critical for CI nowcasts than the original infrared channel brightness temperatures. The brightness temperature difference between band 10 (7.3 μm) and band 13 (10.4 μm), which represents the cloud-top height relative to the lower troposphere, is identified as the most important input fields for CI nowcasting. The tri-spectral brightness temperature difference (TTD), which represents cloud-top glaciation, is ranked the second and it significantly reduced the FAR of the CI forecast. Using terrain heights as an extra input feature improved the POD, but slightly overestimated CI over complex terrain. In addition, a layer-wise relevance propagation (LRP) analyses was performed, and confirmed that the CIUnet model can effectively identify the crucial regions and features of the input fields for accurate CI prediction. Therefore, both permutation importance experiments and LPR analyses are useful for improving the CIUnet model and advancing the understanding of CI mechanisms.

Multi-modal deep learning from imaging genomic data for schizophrenia classification.

Schizophrenia (SZ) is a psychiatric condition that adversely affects an individual's cognitive, emotional, and behavioral aspects. The etiology of SZ, although extensively studied, remains unclear, as multiple factors come together to contribute toward its development. There is a consistent body of evidence documenting the presence of structural and functional deviations in the brains of individuals with SZ. Moreover, the hereditary aspect of SZ is supported by the significant involvement of genomics markers. Therefore, the need to investigate SZ from a multi-modal perspective and develop approaches for improved detection arises. Our proposed method employed a deep learning framework combining features from structural magnetic resonance imaging (sMRI), functional magnetic resonance imaging (fMRI), and genetic markers such as single nucleotide polymorphism (SNP). For sMRI, we used a pre-trained DenseNet to extract the morphological features. To identify the most relevant functional connections in fMRI and SNPs linked to SZ, we applied a 1-dimensional convolutional neural network (CNN) followed by layerwise relevance propagation (LRP). Finally, we concatenated these obtained features across modalities and fed them to the extreme gradient boosting (XGBoost) tree-based classifier to classify SZ from healthy control (HC). Experimental evaluation on clinical dataset demonstrated that, compared to the outcomes obtained from each modality individually, our proposed multi-modal approach performed classification of SZ individuals from HC with an improved accuracy of 79.01%. We proposed a deep learning based framework that selects multi-modal (sMRI, fMRI and genetic) features efficiently and fuse them to obtain improved classification scores. Additionally, by using Explainable AI (XAI), we were able to pinpoint and validate significant functional network connections and SNPs that contributed the most toward SZ classification, providing necessary interpretation behind our findings.

Counterfactual Causal-Effect Intervention for Interpretable Medical Visual Question Answering.

Medical Visual Question Answering (VQA-Med) is a challenging task that involves answering clinical questions related to medical images. However, most current VQA-Med methods ignore the causal correlation between specific lesion or abnormality features and answers, while also failing to provide accurate explanations for their decisions. To explore the interpretability of VQA-Med, this paper proposes a novel CCIS-MVQA model for VQA-Med based on a counterfactual causal-effect intervention strategy. This model consists of the modified ResNet for image feature extraction, a GloVe decoder for question feature extraction, a bilinear attention network for vision and language feature fusion, and an interpretability generator for producing the interpretability and prediction results. The proposed CCIS-MVQA introduces a layer-wise relevance propagation method to automatically generate counterfactual samples. Additionally, CCIS-MVQA applies counterfactual causal reasoning throughout the training phase to enhance interpretability and generalization. Extensive experiments on three benchmark datasets show that the proposed CCIS-MVQA model outperforms the state-of-the-art methods. Enough visualization results are produced to analyze the interpretability and performance of CCIS-MVQA.

Development of a Deep Neural Network Forecasting Model for PM2.5 Prediction in the Metropolitan Area and Analysis of Input Factor Importance Using Layer-wise Relevance Propagation

Development of a Deep Neural Network Forecasting Model for PM2.5 Prediction in the Metropolitan Area and Analysis of Input Factor Importance Using Layer-wise Relevance Propagation