Layer-wise Relevance Propagation Research Articles

Explainability of convolutional neural networks for damage diagnosis using transmissibility functions

Structural health monitoring (SHM) is crucial for ensuring the safety and efficiency of engineering systems. Vibration-based signals have been widely employed in numerous studies for damage detection, localization and quantification, thanks to their high sensitivity to structural damages and versatile applicability across various structures. Among these, transmissibility functions (TFs) have emerged as particularly promising because they can be derived from output-only measurements, making them practical for real-world applications. Vibration-based features, including those captured by TFs, provide a wealth of damage-related information, but extracting these features can be challenging with traditional methods and often requires a priori knowledge. To address this challenge, deep learning has gained significant attention for its capacity to handle intricate patterns and extract hidden features. However, deep learning algorithms are often perceived as black-box models due to their complex and heterogeneous layers, which hinder transparency and raise concerns, especially in critical domains. Although numerous studies have highlighted the diagnostic potential of deep learning in SHM with vibration-based data, there has been limited exploration of the rationale behind using deep neural networks (DNNs) for processing TFs. This paper addresses this gap by introducing explainable artificial intelligence (XAI) methods in two distinct TF-based damage diagnostic case studies: a numerical aluminium structural beam and a large-scale experimental steel frame. In the former, damage is simulated as local stiffness reductions, while in the latter, damage is introduced by loosening joint bolts. In both cases, a convolutional neural network (CNN) is used to process raw TF data for damage detection, localization, and quantification in the beam, and for detection and localization in the frame. Subsequently, an XAI method based on the layer-wise relevance propagation (LRP) algorithm is employed to identify the most relevant input features. A pattern analysis is presented to explain recurrent patterns, and a comparison with other XAI methods is proposed for further validation. The results demonstrate that the CNN provides accurate predictions for both case studies, with its focus on damage-sensitive features aligning closely with findings in the existing literature.

Neural network and layer-wise relevance propagation reveal how ice hockey protective equipment restricts players' motion.

Understanding the athlete's movements and the restrictions incurred by protective equipment is crucial for improving the equipment and subsequently, the athlete's performance. The task of equipment improvement is especially challenging in sports including advanced manoeuvres such as ice hockey and requires a holistic approach guiding the researcher's attention toward the right variables. The purposes of this study were (a) to quantify the effects of protective equipment in ice hockey on player's performance and (b) to identify the restrictions incurred by it. Twenty male hockey players performed four different drills with and without protective equipment while their performance was quantified. A neural network accompanied by layer-wise relevance propagation was applied to the 3D kinematic data to identify variables and time points that were most relevant for the neural network to distinguish between the equipment and no equipment condition, and therefore presumable result from restrictions incurred by the protective equipment. The study indicated that wearing the protective equipment, significantly reduced performance. Further, using the 3D kinematics, an artificial neural network could accurately distinguish between the movements performed with and without the equipment. The variables contributing the most to distinguishing between the equipment conditions were related to the upper extremities and movements in the sagittal plane. The presented methodology consisting of artificial neural networks and layer-wise relevance propagation contributed to insights without prior knowledge of how and to which extent joint angles are affected in complex maneuvers in ice hockey in the presence of protective equipment. It was shown that changes to the equipment should support the flexion movements of the knee and hip and should allow players to keep their upper extremities closer to the torso.

Attention Heat Map-Based Black-Box Local Adversarial Attack for Synthetic Aperture Radar Target Recognition

Synthetic aperture radar (SAR) automatic target recognition (ATR) models based on deep neural networks (DNNs) are susceptible to adversarial attacks. In this study, we proposed an SAR black-box local adversarial attack algorithm named attention heat map- based black-box local adversarial attack (AH-BLAA). First, we designed an attention heat map extraction module combined with the layer-wise relevance propagation (LRP) algorithm to obtain the high concerning areas of the SAR-ATR models. Then, to gener- ate SAR adversarial attack examples, we designed a perturbation generator module, introducing the structural dissimilarity (DSSIM) metric in the loss function to limit image distortion and the dif- ferential evolution (DE) algorithm to search for optimal perturba- tions. Experimental results on the MSTAR and FUSAR-Ship datasets showed that compared with existing adversarial attack algorithms, the attack success rate of the AH-BLAA algorithm increased by 0.63% to 33.59% and 1.05% to 17.65%, respectively. Moreover, the low- est perturbation ratios reached 0.23% and 0.13%, respectively.

WB-LRP: Layer-wise relevance propagation with weight-dependent baseline

DeepLift is a special Layer-wise Relevance Propagation (LRP) algorithm that assigns importance to features by evaluating the impact of small perturbations in input features on the output. However, we discover that only perturbations parallel to the plane defined by weights and input features affect the output. To address this issue, we propose a new LRP algorithm that offers more accurate instance-level explanation for neural networks. Firstly, we derive the baselines for various existing LRP algorithms using DeepLift theory and identify that these baselines often disregard model weights, sample features, or both. Secondly, the Weight-dependent Baseline Layer-wise Relevance Propagation (WB-LRP) algorithm is proposed to address this problem, by deriving the norm-invariant rule (m-rule) according to our discovery. Then, the m-rule is extended to the positive norm-invariant rule (m+-rule), focusing specifically on the positive parts of the weights. Finally, we extend the WB-LRP theory to accommodate various baseline settings, creating a framework that integrates nearly all existing LRP algorithms. Experimental results show that the relevance interpretation depends on both model weights and sample features. The proposed WB-LRP algorithm can be designed with different baselines to adjust the proportions of model weights and sample features in the visualization results, thus enabling separate visualization of the relevance of sample features and model weights to the target category.

Enhancing industrial sustainability in complex production systems through energy hotspot identification: A multi-task learning with layer-wise relevance propagation approach

With growing concerns about global warming and environmental degradation, the petrochemical industry has been urged to reduce energy consumption and emissions. However, making operational adjustments in multi-sectional production systems, which involve multiple production stages, is challenging without identifying the areas within the process that are intensive energy consumers. Therefore, this paper proposes a novel hotspot identification methodology in a multi-sectional production system using layer-wise relevance propagation of a multi-task learning model. The proposed model tracks overall energy efficiency as the final output to determine the energy gap, while the activation result identifies areas of production where energy is ineffectively consumed using sectional energy efficiency. The performance of the proposed method is validated using a case study of the vinyl chloride monomer process, which can be divided into five sections. The results show that the proposed methodology effectively captures the relationship between process variables and both tasks (overall and sectional energy efficiency) with an average R2 of 0.9818 and 0.9858 in prediction with unseen data, respectively. The layer-wise relevance propagation result demonstrates the contribution of unit operations to overall energy efficiency values by tracing the regression trajectories. Furthermore, operational adjustment using the overall energy gap as a benchmark condition offers a significant energy consumption reduction of around 15.87 %, reduces operating expenses for utilities by 147,000 USD, and decreases direct carbon dioxide emissions by 4700 tons, helping production to become more energy-efficient and sustainable.

Explainability of deep reinforcement learning algorithms in robotic domains by using Layer-wise Relevance Propagation

A key component to the recent success of reinforcement learning is the introduction of neural networks for representation learning. Doing so allows for solving challenging problems in several domains, one of which is robotics. However, a major criticism of deep reinforcement learning (DRL) algorithms is their lack of explainability and interpretability. This problem is even exacerbated in robotics as they oftentimes cohabitate space with humans, making it imperative to be able to reason about their behavior. In this paper, we propose to analyze the learned representation in a robotic setting by utilizing Graph Networks (GNs). Using the GN and Layer-wise Relevance Propagation (LRP), we represent the observations as an entity-relationship to allow us to interpret the learned policy. We evaluate our approach in two environments in MuJoCo. These two environments were delicately designed to effectively measure the value of knowledge gained by our approach to analyzing learned representations. This approach allows us to analyze not only how different parts of the observation space contribute to the decision-making process but also differentiate between policies and their differences in performance. This difference in performance also allows for reasoning about the agent’s recovery from faults. These insights are key contributions to explainable deep reinforcement learning in robotic settings.

XRL-FlexSBR: Multi-agent reinforcement learning-driven flexible SBR control with explainable performance guarantee under diverse influent conditions

The sequencing batch reactor (SBR) process stands out for its small footprint and operational flexibility. However, the SBR process is highly nonlinear and subject to influent disturbances. In this study, we suggested an explainable multi-agent reinforcement learning (XRL) approach coupled with multi-agent reinforcement learning (MARL) and explainable AI (XAI); then, an XRL-driven flexible SBR control (XRL-FlexSBR) system was developed to conduct multivariate control the SBR process autonomously. Influent big datasets including biochemical oxygen demand (BOD) and total nitrogen (TN) were collected from the wastewater treatment plants (WWTPs) of South Korea. Then, the Gaussian mixture model was utilized to cluster the diverse influent conditions and the SBR mechanistic model was developed. A game abstraction method based on a two-stage attention network (G2ANET), one of MARL algorithms, was employed to manipulate dissolved oxygen (DO) and extra carbon injection (EC) controllers in the SBR process; furthermore, layer-wise relevance propagation (LRP) of explainable AI (XAI) technique was utilized to evaluate a control performance guarantee by G2ANET. The results verified that XRL-FlexSBR can control the DO and EC controllers in the SBR process while reducing the energy consumption by 4.93 % on average and maintaining effluent quality criteria across the diverse influent conditions. Furthermore, XAI explained that the improved control performance of XRL-FlexSBR agents is attributed to their understanding of the mechanism underlying SBR operations without human intervention. Hence the proposed XRL-FlexSBR can flexibly control the SBR to improve sustainability and profitability under varying influent conditions.

Spatio-temporal learning and explaining for dynamic functional connectivity analysis: Application to depression

BackgroundFunctional connectivity has been shown to fluctuate over time. The present study aimed to identifying major depressive disorders (MDD) with dynamic functional connectivity (dFC) from resting-state fMRI data, which would be helpful to produce tools of early depression diagnosis and enhance our understanding of depressive etiology. MethodsThe resting-state fMRI data of 178 subjects were collected, including 89 MDD and 89 healthy controls. We propose a spatio-temporal learning and explaining framework for dFC analysis. A yet effective spatio-temporal model is developed to classifying MDD from healthy controls with dFCs. The model is a stacking neural network model, which learns network structure information by a multi-layer perceptron based spatial encoder, and learns time-varying patterns by a Transformer based temporal encoder. We propose to explain the spatio-temporal model with a two-stage explanation method of importance feature extracting and disorder-relevant pattern exploring. The layer-wise relevance propagation (LRP) method is introduced to extract the most relevant input features in the model, and the attention mechanism with LRP is applied to extract the important time steps of dFCs. The disorder-relevant functional connections, brain regions, and brain states in the model are further explored and identified. ResultsWe achieved the best classification performance in identifying MDD from healthy controls with dFC data. The top important functional connectivity, brain regions, and dynamic states closely related to MDD have been identified. LimitationsThe data preprocessing may affect the classification performance of the model, and this study needs further validation in a larger patient population. ConclusionsThe experimental results demonstrate that the proposed spatio-temporal model could effectively classify MDD, and uncover structural and temporal patterns of dFCs in depression.

Four Transformer-Based Deep Learning Classifiers Embedded with an Attention U-Net-Based Lung Segmenter and Layer-Wise Relevance Propagation-Based Heatmaps for COVID-19 X-ray Scans.

Background: Diagnosing lung diseases accurately is crucial for proper treatment. Convolutional neural networks (CNNs) have advanced medical image processing, but challenges remain in their accurate explainability and reliability. This study combines U-Net with attention and Vision Transformers (ViTs) to enhance lung disease segmentation and classification. We hypothesize that Attention U-Net will enhance segmentation accuracy and that ViTs will improve classification performance. The explainability methodologies will shed light on model decision-making processes, aiding in clinical acceptance. Methodology: A comparative approach was used to evaluate deep learning models for segmenting and classifying lung illnesses using chest X-rays. The Attention U-Net model is used for segmentation, and architectures consisting of four CNNs and four ViTs were investigated for classification. Methods like Gradient-weighted Class Activation Mapping plus plus (Grad-CAM++) and Layer-wise Relevance Propagation (LRP) provide explainability by identifying crucial areas influencing model decisions. Results: The results support the conclusion that ViTs are outstanding in identifying lung disorders. Attention U-Net obtained a Dice Coefficient of 98.54% and a Jaccard Index of 97.12%. ViTs outperformed CNNs in classification tasks by 9.26%, reaching an accuracy of 98.52% with MobileViT. An 8.3% increase in accuracy was seen while moving from raw data classification to segmented image classification. Techniques like Grad-CAM++ and LRP provided insights into the decision-making processes of the models. Conclusions: This study highlights the benefits of integrating Attention U-Net and ViTs for analyzing lung diseases, demonstrating their importance in clinical settings. Emphasizing explainability clarifies deep learning processes, enhancing confidence in AI solutions and perhaps enhancing clinical acceptance for improved healthcare results.

Machine Learning Approaches for Fault Detection and Diagnosis in Electrical Machines: A Comparative Study of Deep Learning and Classical Methods

Fault detection and diagnosis in electrical machines are crucial for ensuring their safe and reliable operation. In recent years, machine learning techniques have emerged as powerful tools for addressing this challenge, offering the potential for more accurate and efficient fault detection and diagnosis compared to traditional methods. Among these techniques, deep learning has gained significant attention due to its ability to automatically learn relevant features from raw data. However, the performance of deep learning models in this domain has not been extensively compared to classical methods. This paper presents a comparative study of deep learning and classical methods for fault detection and diagnosis in electrical machines. The study evaluates the performance of various machine learning algorithms, including deep neural networks, support vector machines, decision trees, and ensemble methods, in detecting and diagnosing faults such as stator winding faults, rotor faults, and bearing faults. The experimental evaluation is conducted using real-world datasets obtained from electrical machines in industrial settings. Performance metrics such as accuracy, precision, recall, and F1-score are used to assess the effectiveness of each approach in detecting and diagnosing faults accurately and efficiently. The results of the study indicate that deep learning approaches, particularly convolutional neural networks (CNNs) and recurrent neural networks (RNNs), outperform classical methods in terms of fault detection and diagnosis accuracy. These deep learning models demonstrate the ability to automatically extract informative features from raw sensor data, enabling them to effectively identify subtle patterns indicative of faults. The study investigates the interpretability of deep learning models compared to classical methods, examining the extent to which the models can provide insights into the underlying causes of faults. While deep learning models typically operate as black boxes, techniques such as layer-wise relevance propagation (LRP) are employed to enhance their interpretability and facilitate the identification of relevant features contributing to fault detection and diagnosis. This comparative study provides valuable insights into the strengths and limitations of deep learning and classical methods for fault detection and diagnosis in electrical machines, offering guidance for practitioners and researchers in selecting appropriate approaches for their specific applications.

Finding the Right XAI Method—A Guide for the Evaluation and Ranking of Explainable AI Methods in Climate Science

Abstract Explainable artificial intelligence (XAI) methods shed light on the predictions of machine learning algorithms. Several different approaches exist and have already been applied in climate science. However, usually missing ground truth explanations complicate their evaluation and comparison, subsequently impeding the choice of the XAI method. Therefore, in this work, we introduce XAI evaluation in the climate context and discuss different desired explanation properties, namely, robustness, faithfulness, randomization, complexity, and localization. To this end, we chose previous work as a case study where the decade of annual-mean temperature maps is predicted. After training both a multilayer perceptron (MLP) and a convolutional neural network (CNN), multiple XAI methods are applied and their skill scores in reference to a random uniform explanation are calculated for each property. Independent of the network, we find that XAI methods such as Integrated Gradients, layerwise relevance propagation, and input times gradients exhibit considerable robustness, faithfulness, and complexity while sacrificing randomization performance. Sensitivity methods, gradient, SmoothGrad, NoiseGrad, and FusionGrad, match the robustness skill but sacrifice faithfulness and complexity for the randomization skill. We find architecture-dependent performance differences regarding robustness, complexity, and localization skills of different XAI methods, highlighting the necessity for research task-specific evaluation. Overall, our work offers an overview of different evaluation properties in the climate science context and shows how to compare and benchmark different explanation methods, assessing their suitability based on strengths and weaknesses, for the specific research problem at hand. By that, we aim to support climate researchers in the selection of a suitable XAI method. Significance Statement Explainable artificial intelligence (XAI) helps to understand the reasoning behind the prediction of a neural network. XAI methods have been applied in climate science to validate networks and provide new insight into physical processes. However, the increasing number of XAI methods can overwhelm practitioners, making it difficult to choose an explanation method. Since XAI methods’ results can vary, uninformed choices might cause misleading conclusions about the network decision. In this work, we introduce XAI evaluation to compare and assess the performance of explanation methods based on five desirable properties. We demonstrate that XAI evaluation reveals the strengths and weaknesses of different XAI methods. Thus, our work provides climate researchers with the tools to compare, analyze, and subsequently choose explanation methods.

Corn leaf disease: insightful diagnosis using VGG16 empowered by explainable AI.

The agricultural sector is pivotal to food security and economic stability worldwide. Corn holds particular significance in the global food industry, especially in developing countries where agriculture is a cornerstone of the economy. However, corn crops are vulnerable to various diseases that can significantly reduce yields. Early detection and precise classification of these diseases are crucial to prevent damage and ensure high crop productivity. This study leverages the VGG16 deep learning (DL) model to classify corn leaves into four categories: healthy, blight, gray spot, and common rust. Despite the efficacy of DL models, they often face challenges related to the explainability of their decision-making processes. To address this, Layer-wise Relevance Propagation (LRP) is employed to enhance the model's transparency by generating intuitive and human-readable heat maps of input images. The proposed VGG16 model, augmented with LRP, outperformed previous state-of-the-art models in classifying corn leaf diseases. Simulation results demonstrated that the model not only achieved high accuracy but also provided interpretable results, highlighting critical regions in the images used for classification. By generating human-readable explanations, this approach ensures greater transparency and reliability in model performance, aiding farmers in improving their crop yields.

Reduction of rejects by combining data from the casting process and automatic X-ray inspection

Automatic inspection of castings with X-rays (radiographic and computed tomography) is widespread for parts that are relevant for safety or have high quality requirements. Examples in the automotive sector are aluminum wheels, chassis parts and new parts within the electric power train. Those parts are automatically inspected, which means that both the image acquisition and the evaluation of the images is done fully automatically. Today, in most industrial implementations, the generated data with a size up to several gigabytes per part is summarized to a simple good or bad decision, according to specification. All other data is dismissed, although this information can be valuable to optimize production processes and thus minimize rejects. This contribution gives an overview about the results of the project Cast Control, which is a collaboration of Fraunhofer Development Center for X-ray Technology EZRT, Fraunhofer Center for Applied Research on Supply Chain Services SCS and industry partner RONAL GROUP. RONAL GROUP is a major aluminum wheel manufacturer, mainly for the OEM market. Within the project we combined serial production data from the low pressure die casting process from a foundry of the RONAL GROUP with the data generated in the automatic X- ray inspection. After collecting a large base of sample data, we were able train a neural network for the prediction of error metrics obtained by X-ray inspection. We apply a combination of layer-wise relevance propagation and dimensionality reduction to find correlations between data of the casting machines (process and sensor) and the characteristics of anomalies detected during X-ray inspection. With this information, it is possible to adjust the casting process in an early stage – even before rejects are produced. This enables the foundry to reduce their rejects rate, which saves costs and energy and results in a better competitiveness.

Analysis of Input Factor Contributions Characteristics by Index in PM<SUB>2.5</SUB> Forecast Using Layer-wise Relevance Propagation and DNN

Analysis of Input Factor Contributions Characteristics by Index in PM<SUB>2.5</SUB> Forecast Using Layer-wise Relevance Propagation and DNN

SNIPPET: A Framework for Subjective Evaluation of Visual Explanations Applied to DeepFake Detection

Explainable Artificial Intelligence (XAI) attempts to help humans understand machine learning decisions better and has been identified as a critical component towards increasing the trustworthiness of complex black-box systems, such as deep neural networks (DNNs). In this paper, we propose a generic and comprehensive framework named SNIPPET and create a user interface for the subjective evaluation of visual explanations, focusing on finding human-friendly explanations. SNIPPET considers human-centered evaluation tasks and incorporates the collection of human annotations. These annotations can serve as valuable feedback to validate the qualitative results obtained from the subjective assessment tasks. Moreover, we consider different user background categories during the evaluation process to ensure diverse perspectives and comprehensive evaluation. We demonstrate SNIPPET on a DeepFake face dataset. Distinguishing real from fake faces is a non-trivial task even for humans, that depends on rather subtle features, making it a challenging use case. Using SNIPPET, we evaluate four popular XAI methods which provide visual explanations: Gradient-weighted Class Activation Mapping (GradCAM), Layer-wise Relevance Propagation (LRP), attention rollout (rollout), and Transformer Attribution (TA). Based on our experimental results, we observe preference variations among different user categories. We find that most people are more favorable to the explanations of rollout. Moreover, when it comes to XAI-assisted understanding, those who have no or lack relevant background knowledge often consider that visual explanations are insufficient to help them understand. We open-source our framework for continued data collection and annotation at https://github.com/XAI-SubjEvaluation/SNIPPET.

Towards understanding the influence of seasons on low-groundwater periods based on explainable machine learning

Abstract. Seasons are known to have a major influence on groundwater recharge and therefore groundwater levels; however, underlying relationships are complex and partly unknown. The goal of this study is to investigate the influence of the seasons on groundwater levels (GWLs), especially during low-water periods. For this purpose, we train artificial neural networks on data from 24 locations spread throughout Germany. We exclusively focus on precipitation and temperature as input data and apply layer-wise relevance propagation to understand the relationships learned by the models to simulate GWLs. We find that the learned relationships are plausible and thus consistent with our understanding of the major physical processes. Our results show that for the investigated locations, the models learn that summer is the key season for periods of low GWLs in fall, with a connection to the preceding winter usually only being subordinate. Specifically, dry summers exhibit a strong influence on low-water periods and generate a water deficit that (preceding) wet winters cannot compensate for. Temperature is thus an important proxy for evapotranspiration in summer and is generally identified as more important than precipitation, albeit only on average. Single precipitation events show by far the largest influences on GWLs, and summer precipitation seems to mainly control the severeness of low-GWL periods in fall, while higher summer temperatures do not systematically cause more severe low-water periods.

Genetic algorithm based deep learning model adaptation for improvising the motor imagery classification

ABSTRACT Deep learning methods have proved a promising performance for electroencephalography-based brain-computer interfaces (EEG-BCI). It is particularly encouraging that a subject-independent model can be trained using a large amount of other subjects’ data. Transfer learning methods such as adaptation or fine-tuning can be used on the pre-trained model to improve the performance. This study examined the influence of fine-tuning on the subject-independent model for EEG-based motor imagery (MI) classification using a genetic algorithm (GA). The proposed method is evaluated on the binary class MI dataset from the Korea University EEG dataset. Results show that the proposed GA-based fine-tuning approach statistically improved the average classification accuracy of the baseline model from 84.46% to 87.29%. More interestingly, our approach shows significant improvement in cases where the performance of the baseline model is poor after fine-tuning using other approaches. Further, layer-wise relevance propagation (LRP) is used to analyze the adapted models to gain a deeper understanding of the neurophysiological explanations underlying the model’s decision.

LRFE: A NOVEL LOCAL RESPONSE FEATURE ELIMINATION PROCESS FOR IDENTIFICATION OF LUNG CANCER CELLS

One of the main causes of cancer-related mortality across the globe is lung cancer. Early-stage lung cancer frequently exhibits no symptoms, which delays diagnosis until the illness has progressed. Before symptoms manifest, screening and early detection techniques can aid in the early diagnosis and treatment of lung cancer. Many olden research papers have implemented image processing and few latest papers have implemented computer vision techniques to detect lung cancer. Particularly when dealing with minor or subtle anomalies, image processing algorithms may not be able to detect lung cancer lesions with sufficient sensitivity and specificity. It is still difficult to increase the detection algorithms' accuracy and dependability, especially when dealing with early-stage lesions or situations where attributes overlap. It takes a lot of processing power, such as high-performance GPUs and enormous memory capacities, to train deep learning models, particularly large-scale convolutional neural networks (CNNs). In this proposed research, the model pre-processes the images using the ostu and sober filter mechanisms because Otsu's approach adjusts to the features of the input image, including noise, contrast, and lighting fluctuations. It is capable of handling images with varying dynamic ranges and intensity distributions without depending on pre-established threshold settings. When it comes to image noise, the Sobel filter is more resilient than other edge detection methods. It produces clearer edge maps and fewer false detections by determining the gradient magnitude, which amplifies edge information while suppressing noise. The features are extracted using the tuned AlexNet pre-trained model, in AlexNet there is a layer known as “Layer-wise Relevance Propagation”. By giving each pixel or feature in the input image a relevance score, the LRP layer offers fine-grained feature attribution. This makes it possible to analyze in great depth which particular elements or areas of the input image are most important for the network to forecast, which helps to clarify the underlying patterns that the network has learned. At last, the extracted features are further reduced using the enhanced feature elimination method. By iteratively selecting subsets of features based on their importance, RFE helps to identify the most relevant features for the classification task. Integrating RFE with SVM classifiers can lead to improved model performance by focusing on the subset of features that are most discriminative and informative for the classification problem.

Constructing personalized characterizations of structural brain aberrations in patients with dementia using explainable artificial intelligence

Deep learning approaches for clinical predictions based on magnetic resonance imaging data have shown great promise as a translational technology for diagnosis and prognosis in neurological disorders, but its clinical impact has been limited. This is partially attributed to the opaqueness of deep learning models, causing insufficient understanding of what underlies their decisions. To overcome this, we trained convolutional neural networks on structural brain scans to differentiate dementia patients from healthy controls, and applied layerwise relevance propagation to procure individual-level explanations of the model predictions. Through extensive validations we demonstrate that deviations recognized by the model corroborate existing knowledge of structural brain aberrations in dementia. By employing the explainable dementia classifier in a longitudinal dataset of patients with mild cognitive impairment, we show that the spatially rich explanations complement the model prediction when forecasting transition to dementia and help characterize the biological manifestation of disease in the individual brain. Overall, our work exemplifies the clinical potential of explainable artificial intelligence in precision medicine.

Advancing Attribution-Based Neural Network Explainability through Relative Absolute Magnitude Layer-Wise Relevance Propagation and Multi-Component Evaluation

Recent advancement in deep-neural network performance led to the development of new state-of-the-art approaches in numerous areas. However, the black-box nature of neural networks often prohibits their use in areas where model explainability and model transparency are crucial. Over the years, researchers proposed many algorithms to aid neural network understanding and provide additional information to the human expert. One of the most popular methods being Layer-Wise Relevance Propagation (LRP). This method assigns local relevance based on the pixel-wise decomposition of nonlinear classifiers. With the rise of attribution method research, there has emerged a pressing need to assess and evaluate their performance. Numerous metrics have been proposed, each assessing an individual property of attribution methods such as faithfulness, robustness, or localization. Unfortunately, no single metric is deemed optimal for every case, and researchers often use several metrics to test the quality of the attribution maps. In this work, we address the shortcomings of the current LRP formulations and introduce a novel method for determining the relevance of input neurons through layer-wise relevance propagation. Furthermore, we apply this approach to the recently developed Vision Transformer architecture and evaluate its performance against existing methods on two image classification datasets, namely ImageNet and PascalVOC. Our results clearly demonstrate the advantage of our proposed method. Furthermore, we discuss the insufficiencies of current evaluation metrics for attribution-based explainability and propose a new evaluation metric that combines the notions of faithfulness, robustness, and contrastiveness. We utilize this new metric to evaluate the performance of various attribution-based methods. Our code is available at: https://github.com/davor10105/relative-absolute-magnitude-propagation