Saliency Map Research Articles

Atrial Fibrillation Type Classification by a Convolutional Neural Network Using Contrast-Enhanced Computed Tomography Images

Catheter ablation therapy, which is a treatment for atrial fibrillation (AF), has a higher recurrence rate as AF duration increases. Compared to paroxysmal AF (PAF), sustained AF is known to cause progressive anatomic remodeling of the left atrium, resulting in enlargement and shape changes. In this study, we used contrast-enhanced computed tomography (CT) to classify atrial fibrillation (AF) into paroxysmal atrial fibrillation (PAF) and long-term persistent atrial fibrillation (LSAF), which have particularly different recurrence rates after catheter ablation. Contrast-enhanced CT images of 30 patients with PAF and 30 patients with LSAF were input into six pretrained convolutional neural networks (CNNs) for the binary classification of PAF and LSAF. In this study, we propose a method that can recognize information regarding the body axis direction of the left atrium by inputting five slices near the left atrium. The classification was visualized by obtaining a saliency map based on score-class activation mapping (CAM). Furthermore, we surveyed cardiologists regarding the classification of AF types, and the results of the CNN classification were compared with the results of physicians’ clinical judgment. The proposed method achieved the highest correct classification rate (81.7%). In particular, models with shallow layers, such as VGGNet and ResNet, are able to capture the overall characteristics of the image and therefore are likely to be suitable for focusing on the left atrium. In many cases, patients with an enlarged left atrium tended to have long-lasting AF, confirming the validity of the proposed method. The results of the saliency map and survey of physicians’ basis for judgment showed that many patients tended to focus on the shape of the left atrium in both classifications, suggesting that this method can classify atrial fibrillation more accurately than physicians, similar to the judgment criteria of physicians.

Temporal Image Sandwiches Enable Link between Functional Data Analysis and Deep Learning for Single-Plant Cotton Senescence

Abstract Senescence is a highly ordered biological process involving resource redistribution away from aging tissues that affects yield and quality in annuals and perennials. Images from 14 unmanned / unoccupied / uncrewed aerial system (UAS, UAV, drone) flights captured the senescence window across two experiments while functional principal component analysis (FPCA) effectively reduced the dimensionality of temporal visual senescence ratings (VSRs) and two vegetation indices: the red chromatic coordinate index (RCC) and transformed normalized difference green and red index (TNDGR). Convolutional neural networks (CNNs) trained on temporally concatenated, or “sandwiched,” UAS images of individual cotton plants (Gossypium hirsutum L.), allowed single-plant analysis (SPA). The first functional principal component scores (FPC1) served as the regression target across six CNN models (M1-M6). Model performance was strongest for FPC1 scores from VSRs (R2 = 0.857 and 0.886 for M1 and M4), strong for TNDGR (R2 = 0.743 and 0.745 for M3 and M6), and strong-to-moderate for RCC (R2 = 0.619 and 0.435 for M2 and M5), with deep learning attention of each model confirmed by activation of plant pixels within saliency maps. Single-plant UAS image analysis across time enabled translatable implementations of high-throughput phenotyping by linking deep learning with functional data analysis (FDA). This has applications for fundamental plant biology, monitoring orchards or other spaced plantings, plant breeding, and genetic research.

A novel interpretative deep neural network with Grad-CAMs heatmap for the early diagnosis of Alzheimers disease

Abstract. In recent years, Alzheimers disease, a disease which targets the patients cognitive abilities and causes dementia, has become increasingly prevalent among the older population. Clinical practice today diagnoses the disease through MRI imaging and cognitive tests based on individual doctors personal experience. This reliance on varying doctoral experiences poses a huge challenge in during its early stages, when the symptoms are subtle and hard for clinicians to discern. As a result, many elderly today are only diagnosed when the disease has already progressed into a later, much obvious stage, significantly reducing their chance of receiving effective treatments. In our work, we applied a novel method in Alzheimers disease early diagnosis. Our unprecedented method uses a state-of-the-art image recognition model, a Vision Transformer (ViT), combined with a saliency map, Grad-CAM, to increase interpretability and alleviate the black box issue in the predicted results. We trained the ViT with ADNI imaging datasets and conducted similar experiments with other popular models for machine learning as baselines for performance comparison. The results show that our model has delivered the best performance out of all traditional methods, with an astonishing 99% accuracy. Compared to an average accuracy of 96% by other commonly used models for data prediction, our method has improved the final prediction accuracy to 3 standard deviation range, thereby accounting for almost all the possible outliers. These breakthroughs, paired with the website we created to increase the models reach, will prove to be a revolutionary contribution to the field of Alzheimers disease diagnosis.

Image enhancement method for paint defects based on polarization fusion technique

The highly reflective nature of paint surfaces poses a challenge for defect detection. In this paper, a polarization fusion based image enhancement algorithm for paint defects is proposed to improve the image quality and complement the masked defect details. Firstly, the linearly polarized photometric (DOLP) image is initially fused with the angle of polarization (AOP) image. The light intensity images and polarization feature images were enhanced using a single scale Retinex algorithm and these images were decomposed and reconstructed using wavelet transform to integrate the low and high frequency subbands based on saliency maps and regional energy features. Experiments were performed on defects at different locations and compared with other classical methods. The results show that the proposed fusion algorithm improves the contrast while providing more complete defect information, clearer texture details and significant advantages over other algorithms.

Explainable Artificial Intelligence (XAI) in Healthcare: Enhancing Transparency and Trust

Artificial Intelligence has now taken a full-fledged role in healthcare and has started driving innovations not only in diagnostics and treatment planning but also in patient monitoring and operational efficiency. This will enable complex medical data analysis, extracting patterns and insights that no human is capable of. However, most of these models are per se opaque-that is, the so-called "black-box" problem-there are still great challenges in areas such as transparency, trust, and ethical applications in a clinical setting. This lack of interpretability can stand in the way of acceptance or integration for AI technologies when issues of understanding and accountability are relevant. Explainable AI solves these problems by making real artificial intelligence decisions understand the decisions made to humans. XAI techniques offer well-understandable and interpretable explanations of the models with minimum degradations in performance. This review article explains in detail the critical role of XAI in healthcare, underpinning how this field can bring more transparency into AI applications. We explain some of the current methods of XAI: model-agnostic techniques like LIME and SHAP, interpretable models relating to decision trees and linear models, and visualization techniques like saliency maps and mechanisms of attention.

Transformative Transparent Hybrid Deep Learning Framework for Accurate Cataract Detection

This paper presents a transformative explainable convolutional neural network (CNN) framework for cataract detection, utilizing a hybrid deep learning model combining Siamese networks with VGG16. By leveraging a learning rate scheduler and Grad-CAM (Gradient-weighted Class Activation Mapping) for explainability, the proposed model not only achieves high accuracy in identifying cataract-infected images but also provides interpretable visual explanations of its predictions. Performance evaluation metrics such as accuracy, precision, recall, and F1 score demonstrate the model’s robustness, with a perfect accuracy of 100%. Grad-CAM visualizations highlight the key image regions—primarily around the iris and pupil—that contribute most to the model’s decision-making, making the system more transparent for clinical use. Additionally, novel statistical analysis methods, including saliency map evaluation metrics like AUC (Area Under the Curve) and the Pointing Game, were employed to quantify the quality of the model’s explanations. These metrics enhance the interpretability of the model and support its practical applicability in medical image analysis. This approach advances the integration of deep learning with explainable AI, offering a robust, accurate, and interpretable solution for cataract detection with the potential for broader adoption in ocular disease diagnosis and medical decision support systems.

Classification and Identification of Frequency-Hopping Signals Based on Jacobi Salient Map for Adversarial Sample Attack Approach.

Frequency-hopping (FH) communication adversarial research is a key area in modern electronic countermeasures. To address the challenge posed by interfering parties that use deep neural networks (DNNs) to classify and identify multiple intercepted FH signals-enabling targeted interference and degrading communication performance-this paper presents a batch feature point targetless adversarial sample generation method based on the Jacobi saliency map (BPNT-JSMA). This method builds on the traditional JSMA to generate feature saliency maps, selects the top 8% of salient feature points in batches for perturbation, and increases the perturbation limit to restrict the extreme values of single-point perturbations. Experimental results in a white-box environment show that, compared with the traditional JSMA method, BPNT-JSMA not only maintains a high attack success rate but also enhances attack efficiency and improves the stealthiness of the adversarial samples.

Chimeric U-Net – Modifying the standard U-Net towards explainability

Healthcare guided by semantic segmentation has the potential to improve our quality of life through early and accurate disease detection. Convolutional Neural Networks, especially the U-Net-based architectures, are currently the state-of-the-art learning-based segmentation methods and have given unprecedented performances. However, their decision-making processes are still an active field of research. In order to reliably utilize such methods in healthcare, explainability of how the segmentation was performed is mandated. To date, explainability is studied and applied heavily in classification tasks. In this work, we propose the Chimeric U-Net, a U-Net architecture with an invertible decoder unit, that inherently brings explainability into semantic segmentation tasks. We find that having the restriction of an invertible decoder does not hinder the performance of the segmentation task. However, the invertible decoder helps to disentangle the class information in the latent space embedding and to construct meaningful saliency maps. Furthermore, we found that with a simple k-Nearest-Neighbours classifier, we could predict the Intersection over Union scores of unseen data, demonstrating that the latent space, constructed by the Chimeric U-Net, encodes an interpretable representation of the segmentation quality. Explainability is an emerging field, and in this work, we propose an alternative approach, that is, rather than building tools for explaining a generic architecture, we propose constraints on the architecture which induce explainability. With this approach, we could peer into the architecture to reveal its class correlations and local contextual dependencies, taking an insightful step towards trustworthy and reliable AI. Code to build and utilize the Chimeric U-Net is made available under:https://github.com/kenrickschulze/Chimeric-UNet---Half-invertible-UNet-in-Pytorch

Incident atrial fibrillation prediction using ECG-based deep learning at a specialized tertiary cardiac care center

Abstract Background Deep learning (DL) applied to electrocardiograms (ECG) is an emerging modality in the prediction of incident atrial fibrillation or atrial flutter (AF). The generalizability of such methods to a tertiary heart center (THC) population has not yet been formally explored. Purpose Evaluate the performance of DL in predicting 5-year incident AF using a resting 12-lead ECG in sinus rhythm acquired at a THC. Methods In a retrospective study, we examined 1.4 million ECGs acquired from over 250,000 adults at a THC between 2004 and 2022. ECGs were excluded if they were performed within 30 days of cardiac surgery, captured a rhythm other than sinus rhythm, or were acquired in patients with pre-existing AF. Incident AF at 5 years was modelled as a binary outcome and was determined on the basis of available outpatient and inpatient clinical databases and ECG diagnoses. Included ECGs were randomly split by distributing patients into training (70%), validation (10%), and test (20%) sets. A ResNet-50 model was trained using the training set. Hyperparameters were optimized using the validation set. The tuned model's performance is reported on the test set. The results at the patient level were derived by averaging the model's probability outputs for ECGs grouped according to both their AF outcome and the patient's identity. Bootstrapping was used to report confidence intervals (CI). Sensitivity and specificity are reported at a classification threshold based on the Matthews correlation coefficient. Saliency maps were used to enhance the model’s explainability. Results A total of 669,782 ECGs (47% of the screened ECGs) were included among 145,323 patients. Mean age was 63±15 years and 62% were male. The 5-year incident AF outcome was observed in 12% of ECGs and 16% of patients. The performance of the tuned model was first evaluated at the ECG level on the test set demonstrating an area under the receiver operating curve (AUC) of 0.75 (95% CI: 0.745-0.753) and an integrated calibration index of 0.009 (95% CI: 0.008-0.011). When testing the model at the patient level to simulate a deployment scenario, the AUC improved to 0.78 (95% CI: 0.768-0.783) with a sensitivity of 50% (95% CI: 49-52) and specificity of 87% (95% CI: 86.5-87.3). Stratified results by sex showed a better AUC in females, 0.81 (95% CI: 0.80-0.82), compared to males, 0.75 (95% CI: 0.74-0.76). Subgroup analyses revealed a trend of improved performance at the patient level with an increasing number of ECGs. Saliency maps highlighted the P-wave area as having the highest influence on the model’s prediction, thereby enhancing the model’s plausibility and explainability. Conclusion A unimodal ECG-based deep learning model showed promising 5-year incident AF prediction performance in a cohort of all-comer patients at a tertiary heart center. Further studies could explore the use of multimodal prediction models that integrate ECGs with other clinical and imaging data.

Artificial intelligence applied to electrocardiographic images for scalable screening of cardiac amyloidosis

Abstract Background Transthyretin amyloid cardiomyopathy (ATTR-CM) remains largely under-recognized, under-diagnosed, and under-treated. We hypothesized that the myocardial remodeling seen in ATTR-CM may be detectable through artificial intelligence (AI) applied to 12-lead electrocardiographic (ECG) images. Purpose To develop an AI-ECG algorithm that can detect ATTR-CM from images of 12-lead electrocardiograms. Methods Across 5 hospitals of a large U.S.-based hospital system, we identified patients with ATTR-CM, defined by the presence of a positive bone scintigraphy scan or pharmacotherapy with an approved transthyretin stabilizer between 2015 and the first half of 2023. The development cohort consisted of 1,011 ECGs from 234 patients (age 79 [IQR:70-85], n=176 (17.4%) women), who were age- and sex-matched in a 10:1 ratio to 10,110 ECGs from 10,110 controls (age 79 [IQR:70-85] years, n=1,800 [17.7%] female). A convolutional neural network (CNN) pre-trained using a bio-contrastive pretext on ECGs before 2015 was fine-tuned for ATTR-CM using 5-fold cross-validation and subsequently tested in an independent set of cases (139 ECGs in 47 patients; age 80 [75-86] years, n=44 (31.7% women)) and matched controls (1390 ECGs and patients) from the second half of 2023. Results The AUROC (area under the receiver operating characteristic curve) of the AI-ECG model for discriminating ATTR-CM in the cross-validation set was 0.90 [95%CI: 0.89-0.91], and in the leave-out, temporally distinct dataset was 0.915 [95%CI: 0.89-0.94] (A), with a sensitivity of 0.85 [95%CI: 0.79-0.91] and specificity 0.80 [95%CI 0.78-0.82]. Representative saliency maps are shown in panel B. In simulations, the positive predictive value ranged from 0.12 [95%CI: 0.10-0.13] at a 3% prevalence level to 0.59 [95%CI: 0.56-0.62] at 25% prevalence. Conclusions We demonstrate that AI applied directly to ECG images represents a promising approach for the screening of ATTR-CM.

Part-Prototype Models in Medical Imaging: Applications and Current Challenges

Recent developments in Artificial Intelligence have increasingly focused on explainability research. The potential of Explainable Artificial Intelligence (XAI) in producing trustworthy computer-aided diagnosis systems and its usage for knowledge discovery are gaining interest in the medical imaging (MI) community to support the diagnostic process and the discovery of image biomarkers. Most of the existing XAI applications in MI are focused on interpreting the predictions made using deep neural networks, typically including attribution techniques with saliency map approaches and other feature visualization methods. However, these are often criticized for providing incorrect and incomplete representations of the black-box models’ behaviour. This highlights the importance of proposing models intentionally designed to be self-explanatory. In particular, part-prototype (PP) models are interpretable-by-design computer vision (CV) models that base their decision process on learning and identifying representative prototypical parts from input images, and they are gaining increasing interest and results in MI applications. However, the medical field has unique characteristics that could benefit from more advanced implementations of these types of architectures. This narrative review summarizes existing PP networks, their application in MI analysis, and current challenges.

Investigating Brain Responses to Transcutaneous Electroacupuncture Stimulation: A Deep Learning Approach

This study investigates the neurophysiological effects of transcutaneous electroacupuncture stimulation (TEAS) on brain activity, using advanced machine learning techniques. This work analyzed the electroencephalograms (EEG) of 48 study participants, in order to analyze the brain’s response to different TEAS frequencies (2.5, 10, 80, and sham at 160 pulses per second (pps)) across 48 participants through pre-stimulation, during-stimulation, and post-stimulation phases. Our approach introduced several novel aspects. EEGNet, a convolutional neural network specifically designed for EEG signal processing, was utilized in this work, achieving over 95% classification accuracy in detecting brain responses to various TEAS frequencies. Additionally, the classification accuracies across the pre-stimulation, during-stimulation, and post-stimulation phases remained consistently high (above 92%), indicating that EEGNet effectively captured the different time-based brain responses across different stimulation phases. Saliency maps were applied to identify the most critical EEG electrodes, potentially reducing the number needed without sacrificing accuracy. A phase-based analysis was conducted to capture time-based brain responses throughout different stimulation phases. The robustness of EEGNet was assessed across demographic and clinical factors, including sex, age, and psychological states. Additionally, the responsiveness of different EEG frequency bands to TEAS was investigated. The results demonstrated that EEGNet excels in classifying EEG signals with high accuracy, underscoring its effectiveness in reliably classifying EEG responses to TEAS and enhancing its applicability in clinical and therapeutic settings. Notably, gamma band activity showed the highest sensitivity to TEAS, suggesting significant effects on higher cognitive functions. Saliency mapping revealed that a subset of electrodes (Fp1, Fp2, Fz, F7, F8, T3, T4) could achieve accurate classification, indicating potential for more efficient EEG setups.

Enhancing Brain Tumor Detection Through Custom Convolutional Neural Networks and Interpretability-Driven Analysis

Brain tumor detection is crucial for effective treatment planning and improved patient outcomes. However, existing methods often face challenges, such as limited interpretability and class imbalance in medical-imaging data. This study presents a novel, custom Convolutional Neural Network (CNN) architecture, specifically designed to address these issues by incorporating interpretability techniques and strategies to mitigate class imbalance. We trained and evaluated four CNN models (proposed CNN, ResNetV2, DenseNet201, and VGG16) using a brain tumor MRI dataset, with oversampling techniques and class weighting employed during training. Our proposed CNN achieved an accuracy of 94.51%, outperforming other models in regard to precision, recall, and F1-Score. Furthermore, interpretability was enhanced through gradient-based attribution methods and saliency maps, providing valuable insights into the model’s decision-making process and fostering collaboration between AI systems and clinicians. This approach contributes a highly accurate and interpretable framework for brain tumor detection, with the potential to significantly enhance diagnostic accuracy and personalized treatment planning in neuro-oncology.

Attenuative Unified U-Net with Conditional PatchMatch Algorithm for Edge Preserving Salient Object Detection

The rise of Deep Neural Networks (DNNs) has significantly boosted salient object detection in computer vision tasks. However, downsampling methods like striding and pooling often introduce bluish artifacts near edges, hindering detection accuracy. To address this, the "Attenuative Unified U-Net with Conditional PatchMatch Algorithm" is proposed. The method employs Gaussian smoothing for noise reduction and depth refinement in preprocessing. In existing methods, the Edge-preserving salient object detection struggles with cluttered backgrounds and connected objects, like a bird on a rock, making it difficult to accurately distinguish edges and salient regions due to shared boundaries. Hence, an Attenuative Unified Backpropagated Entropy UNet is proposed, which integrates an Attention Mechanism that enhance feature map spatial weight assignment, emphasizing features or regions that are judged most relevant for edge and salient region detection. Then, the Cascaded Laplace pyramid is incorporated into the Unified U-Net’s design for multi-scale information processing, capturing contextual details effectively. The Backpropagated Entropy Loss Function is then created to ensure accurate and confident fused image output. The depth map, edge map, and salient area map are combined in the U-Net’s final output layer to create an overall fused image. Next, to addresses depth discontinuity problem a noteworthy novel idea the Conditional PatchMatch Random Field Algorithm (CPMRF), which combines PatchMatch efficiency and CRFs’ contextual modeling. PatchMatch iteratively propagates matches from neighboring patches to find similar patches in the remaining portion of the image and Conditional Random Field (CRF) is the next step, which improves the PatchMatch results by taking into account the connections between the matched patches and their neighboring areas. Hence, the experimental outcomes of the proposed model effectively show the improved detection of edge in salient object detection with better accuracy, precision, recall, MaxF, and F1 score with minimized MAE.

Anatomic Interpretability in Neuroimage Deep Learning: Saliency Approaches for Typical Aging and Traumatic Brain Injury.

The black box nature of deep neural networks (DNNs) makes researchers and clinicians hesitant to rely on their findings. Saliency maps can enhance DNN explainability by suggesting the anatomic localization of relevant brain features. This study compares seven popular attribution-based saliency approaches to assign neuroanatomic interpretability to DNNs that estimate biological brain age (BA) from magnetic resonance imaging (MRI). Cognitively normal (CN) adults ( N = 13,394, 5,900 males; mean age: 65.82 ± 8.89 years) are included for DNN training, testing, validation, and saliency map generation to estimate BA. To study saliency robustness to the presence of anatomic deviations from normality, saliency maps are also generated for adults with mild traumatic brain injury (mTBI, \(\:N\) = 214, 135 males; mean age: 55.3 ± 9.9 years). We assess saliency methods' capacities to capture known anatomic features of brain aging and compare them to a surrogate ground truth whose anatomic saliency is known a priori . Anatomic aging features are identified most reliably by the integrated gradients method, which outperforms all others through its ability to localize relevant anatomic features. Gradient Shapley additive explanations, input × gradient, and masked gradient perform less consistently but still highlight ubiquitous neuroanatomic features of aging (ventricle dilation, hippocampal atrophy, sulcal widening). Saliency methods involving gradient saliency, guided backpropagation, and guided gradient-weight class attribution mapping localize saliency outside the brain, which is undesirable. Our research suggests the relative tradeoffs of saliency methods to interpret DNN findings during BA estimation in typical aging and after mTBI.

Gaze behaviors during free viewing revealed differences in visual salience processing across four major psychiatric disorders: a mega-analysis study of 1012 individuals.

Aberrant salience processing has been proposed as a pathophysiological mechanism underlying psychiatric symptoms in patients with schizophrenia. The gaze trajectories of individuals with schizophrenia have been reported to be abnormal when viewing an image, suggesting anomalous visual salience as one possible pathophysiological mechanism associated with psychiatric diseases. This study was designed to determine whether visual salience is affected in individuals with schizophrenia, and whether this abnormality is unique to patients with schizophrenia. We examined the gaze behaviors of 1012 participants recruited from seven institutes (550 healthy individuals and 238, 41, 50 and 133 individuals with schizophrenia, bipolar disorder, major depressive disorder and autism spectrum disorder, respectively) when they looked at stationary images as they liked, i.e., free-viewing condition. We used an established computational model of salience maps derived from low-level visual features to measure the degree to which the gaze trajectories of individuals were guided by visual salience. The analysis revealed that the saliency at the gaze of individuals with schizophrenia were higher than healthy individuals, suggesting that patients' gazes were guided more by low-level image salience. Among the low-level image features, orientation salience was most affected. Furthermore, a general linear model analysis of the data for the four psychiatric disorders revealed a significant effect of disease. This abnormal salience processing depended on the disease and was strongest in patients with schizophrenia, followed by patients with bipolar disorder, major depressive disorder, and autism spectrum disorder, suggesting a link between abnormalities in salience processing and strength/frequency for psychosis of these disorders.

Prognosis of major bleeding based on residual variables and machine learning for critical patients with upper gastrointestinal bleeding: A multicenter study

BackgroundUpper gastrointestinal bleeding (UGIB) is a significant cause of morbidity and mortality worldwide. This study investigates the use of residual variables and machine learning (ML) models for predicting major bleeding in patients with severe UGIB after their first intensive care unit (ICU) admission. MethodsThe Medical Information Mart for Intensive Care IV and eICU databases were used. Conventional ML and long short-term memory models were constructed using pre-ICU and ICU admission day data to predict the recurrence of major gastrointestinal bleeding. In the models, residual data were utilized by subtracting the normal range from the test result. The models included eight algorithms. Shapley additive explanations and saliency maps were used for feature interpretability. ResultsTwenty-five ML models were developed using data from 2604 patients. The light gradient-boosting machine algorithm model using pre-ICU admission residual data outperformed other models that used test results directly, with an AUC of 0.96. The key factors included aspartate aminotransferase, blood urea nitrogen, albumin, length of ICU admission, and respiratory rate. ConclusionsML models using residuals improved the accuracy and interpretability in predicting major bleeding during ICU admission in patients with UGIB. These interpretable features may facilitate the early identification and management of high-risk patients, thereby improving hemodynamic stability and outcomes.

Anatomic Interpretability in Neuroimage Deep Learning: Saliency Approaches for Typical Aging and Traumatic Brain Injury

The black box nature of deep neural networks (DNNs) makes researchers and clinicians hesitant to rely on their findings. Saliency maps can enhance DNN explainability by suggesting the anatomic localization of relevant brain features. This study compares seven popular attribution-based saliency approaches to assign neuroanatomic interpretability to DNNs that estimate biological brain age (BA) from magnetic resonance imaging (MRI). Cognitively normal (CN) adults (N = 13,394, 5,900 males; mean age: 65.82 ± 8.89 years) are included for DNN training, testing, validation, and saliency map generation to estimate BA. To study saliency robustness to the presence of anatomic deviations from normality, saliency maps are also generated for adults with mild traumatic brain injury (mTBI, N\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$N$$\\end{document} = 214, 135 males; mean age: 55.3 ± 9.9 years). We assess saliency methods’ capacities to capture known anatomic features of brain aging and compare them to a surrogate ground truth whose anatomic saliency is known a priori. Anatomic aging features are identified most reliably by the integrated gradients method, which outperforms all others through its ability to localize relevant anatomic features. Gradient Shapley additive explanations, input × gradient, and masked gradient perform less consistently but still highlight ubiquitous neuroanatomic features of aging (ventricle dilation, hippocampal atrophy, sulcal widening). Saliency methods involving gradient saliency, guided backpropagation, and guided gradient-weight class attribution mapping localize saliency outside the brain, which is undesirable. Our research suggests the relative tradeoffs of saliency methods to interpret DNN findings during BA estimation in typical aging and after mTBI.

Pulp grade monitoring using binocular image through multi-scale feature cross-attention fusion network and saliency map constraint

Pulp grade is an important indicator for performance monitoring in froth flotation process. Previous studies have shown that the pulp grade can be predicted more accurately by using the stereo vision information of froth images. However, due to the low intra-class variation among froth images under the identical working conditions and the similarity in shape and texture among bubbles, it is challenging for current binocular image-based methods for grade prediction. Therefore, to accurately predict the key performance indicators in flotation process, a prediction model based on binocular image fusion is proposed in this paper. First, a calculation method of froth image saliency map is proposed, and the saliency map is used as a priori knowledge to guide the prediction model to learn the characteristics of the region of interest in the image. This measure aims to solve the problem of difficulty in grade prediction caused by low intra-class differences in froth images. Then, a multi-scale feature cross-attention fusion network is introduced, wherein the multi-scale features of the left and right views serve as attention mechanism gating signals to extract common feature from binocular image. After that, a pulp grade prediction model is developed based on Video Transformer Network. The results on actual industrial flotation datasets show that, compared with other pulp grade prediction methods, our proposed approach reduces the mean absolute error and root mean square error by 22.84% and 23.55%, respectively.

Subjective performance assessment protocol for visual explanations-based face verification explainability

The integration of Face Verification (FV) systems into multiple critical moments of daily life has become increasingly prevalent, raising concerns regarding the transparency and reliability of these systems. Consequently, there is a growing need for FV explainability tools to provide insights into the behavior of these systems. FV explainability tools that generate visual explanations, e.g., saliency maps, heatmaps, contour-based visualization maps, and face segmentation maps, show promise in enhancing FV transparency by highlighting the contributions of different face regions to the FV decision-making process. However, evaluating the performance of such explainability tools remains challenging due to the lack of standardized assessment metrics and protocols. In this context, this paper proposes a subjective performance assessment protocol for evaluating the explainability performance of visual explanation-based FV explainability tools through pairwise comparisons of their explanation outputs. The proposed protocol encompasses a set of key specifications designed to efficiently collect the subjects’ preferences and estimate explainability performance scores, facilitating the relative assessment of the explainability tools. This protocol aims to address the current gap in evaluating the effectiveness of visual explanation-based FV explainability tools, providing a structured approach for assessing their performance and comparing with alternative tools. The proposed protocol is exercised and validated through an experiment conducted using two distinct heatmap-based FV explainability tools, notably FV-RISE and CorrRISE, taken as examples of visual explanation-based explainability tools, considering the various types of FV decisions, i.e., True Acceptance (TA), False Acceptance (FA), True Rejection (TR), and False Rejection (FR). A group of subjects with variety in age, gender, and ethnicity was tasked to express their preferences regarding the heatmap-based explanations generated by the two selected explainability tools. The subject preferences were collected and statistically processed to derive quantifiable scores, expressing the relative explainability performance of the assessed tools. The experimental results revealed that both assessed explainability tools exhibit comparable explainability performance for FA, TR, and FR decisions with CorrRISE performing slightly better than FV-RISE for TA decisions.