Discriminative Regions Research Articles

Molecular Tracking of Leishmania major in an Archived Rattus norvegicus Spleen Sample in Iran: A Case Report

Rodents are the primary reservoir hosts for zoonotic cutaneous leishmaniasis (ZCL) caused by Leishmania major. Knowing reservoir hosts is crucial for leishmaniasis surveillance and control programs in endemic areas. In this study, we examined an archived spleen of Rattus norvegicus obtained during a pest control program in 2000 in Tehran, the capital of Iran. The sample was analyzed using polymerase chain reaction (PCR) and sequencing to determine the presence of Trypanosomatidae based on the internal transcribed spacer (ITS) 1 gene. Amplification and sequencing of the discriminative region of the ITS1 gene followed by BLAST analysis showed the highest similarity with L. major isolates. Also, the phylogenetic analysis revealed that our sample was grouped with L. major isolates retrieved from the GenBank database. This finding might support the claim that R. norvegicus acts as a potential reservoir host for L. major. Further studies, including a survey on more rodent samples as well as studying sandflies in the area, might uncover the possible presence of such pathobiological conditions in ZCL transmission in urban and suburban settings.

OPT-STVIT: Video Recognition through Optimized Spatial-Temporal Video Vision Transformers

In this paper, we address the computational chal- lenges associated with video recognition tasks, where video transformers have shown impressive results but come with high computational costs. We introduce Opt-STViT, a token selection framework that dynamically chooses a subset of informative tokens in both temporal and spatial dimensions based on the input video samples. Specifically, we frame token selection as a ranking problem, leveraging a lightweight scorer network to estimate the importance of each token. Only tokens with top scores are retained for downstream processing. In the temporal dimension, we identify and keep the frames most relevant to the action categories, while in the spatial dimension, we pinpoint the most discriminative regions in feature maps without affecting the spatial context used hierarchically in most video transformers. To enable end-to-end training despite the non-differentiable nature of token selection, we employ a perturbed-maximum-based dif- ferentiable Top-K operator. Our extensive experiments, primar- ily conducted on the Kinetics-400 and something-something-V2 datasets using the recently introduced MViT video transformer backbone, demonstrate that our framework achieves similar results while requiring 20 percent less computational resources. We also establish the versatility of our approach across different transformer architectures and video datasets.

Consensus-based iterative meta-pseudo-labeling for deep semi-supervised learning

A known issue that hinders the development of deep learning models is the need for accurate annotation of a large quantity of samples – a time-consuming, labor-intensive, and error-prone task. This limitation is particularly critical in areas where data annotation requires expert knowledge. Semi-supervised learning methods, such as pseudo-labeling, can alleviate the problem by capitalizing on both limited labeled and plentiful unlabeled data; nonetheless, state-of-the-art methods often require pre-trained encoders and validation sets to deliver effective solutions. Herein, we introduce a teacher-student-based iterative meta-pseudo-labeling approach, named consensus Deep Feature Annotation (cons-DeepFA), that enables the training of custom Convolutional Neural Networks (CNNs) from small quantities of labeled samples without reliance on pre-trained encoders and validation sets. cons-DeepFA explores Feature Learning from Image Markers (FLIM) to initialize the filters of a target CNN (student) from minimal data annotation – i.e., user-drawn markers on discriminative regions of a few selected images per class. During each of a few iterations, the latent space of the student's last dense layer is non-linearly projected onto a two-dimensional space for downstream label propagation via an optimum-connectivity-based approach (teacher); afterward, the student is re-trained using pseudo-labeled samples selected by the proposed consensus mechanism, which jointly improves the latent space, its projection, and the student's generalization ability as iterations progress. This strategy was recently introduced with pre-trained encoders by selecting the most confident pseudo-labeled samples to re-train the student. While building on previous methods, cons-DeepFA presents two key contributions. It (i) incorporates FLIM to enable training a custom CNN from scratch with faster convergence, improving its generalization ability, and (ii) introduces a consensus-based procedure over multiple iterations that selects more accurately pseudo-labeled samples for re-training the CNN. Lastly, cons-DeepFA is evaluated on five challenging biological image datasets, demonstrating its effectiveness and competitiveness when compared to seven state-of-the-art methods from four semi-supervised learning paradigms.

Diagnosing Necrotizing Enterocolitis via Fine-Grained Visual Classification.

Necrotizing Enterocolitis (NEC) is a devastating condition affecting prematurely born neonates. Reviewing Abdominal X-rays (AXRs) is a key step in NEC diagnosis, staging and treatment decision-making, but poses significant challenges due to the subtle, difficult-to-identify radiological signs of the disease. In this paper, we propose AIDNEC - AI Diagnosis of NECrotizing enterocolitis, a deep learning method to automatically detect and stratify the severity (surgical or medical) of NEC from no pathology in AXRs. The model is trainable end-to-end and integrates a Detection Transformer and Graph Convolution modules for localizing discriminative areas in AXRs, used to formulate subtle local embeddings. These are then combined with global image features to perform Fine-Grained Visual Classification (FGVC). We evaluate AIDNEC on our GOSH NEC dataset of 1153 images from 334 patients, achieving 79.7% accuracy in classifying NEC against No Pathology. AIDNEC outperforms the backbone by 2.6%, FGVC models by 2.5% and CheXNet by 4.2%, with statistically significant (two-tailed p 0.05) improvements, while providing meaningful discriminative regions to support the classification decision. Additional validation in the publicly available Chest X-ray14 dataset yields comparable performance to state-of-the-art methods, illustrating AIDNEC's robustness in a different X-ray classification task.

Significant feature suppression and cross-feature fusion networks for fine-grained visual classification

The technique of extracting different distinguishing features by locating different part regions to achieve fine-grained visual classification (FGVC) has made significant improvements. Utilizing attention mechanisms for feature extraction has become one of the mainstream methods in computer vision, but these methods have certain limitations. They typically focus on the most discriminative regions and directly combine the features of these parts, neglecting other less prominent yet still discriminative regions. Additionally, these methods may not fully explore the intrinsic connections between higher-order and lower-order features to optimize model classification performance. By considering the potential relationships between different higher-order feature representations in the object image, we can enable the integrated higher-order features to contribute more significantly to the model’s classification decision-making capabilities. To this end, we propose a saliency feature suppression and cross-feature fusion network model (SFSCF-Net) to explore the interaction learning between different higher-order feature representations. These include (1) an object-level image generator (OIG): the intersection of the output feature maps of the last two convolutional blocks of the backbone network is used as an object mask and mapped to the original image for cropping to obtain an object-level image, which can effectively reduce the interference caused by complex backgrounds. (2) A saliency feature suppression module (SFSM): the most distinguishing part of the object image is obtained by a feature extractor, and the part is masked by a two-dimensional suppression method, which improves the accuracy of feature suppression. (3) A cross-feature fusion method (CFM) based on inter-layer interaction: the output feature maps of different network layers are interactively integrated to obtain high-dimensional features, and then the high-dimensional features are channel compressed to obtain the inter-layer interaction feature representation, which enriches the output feature semantic information. The proposed SFSCF-Net can be trained end-to-end and achieves state-of-the-art or competitive results on four FGVC benchmark datasets.

Learning from the few: Fine-grained approach to pediatric wrist pathology recognition on a limited dataset

Wrist pathologies, particularly fractures common among children and adolescents, present a critical diagnostic challenge. While X-ray imaging remains a prevalent diagnostic tool, the increasing misinterpretation rates highlight the need for more accurate analysis, especially considering the lack of specialized training among many surgeons and physicians. Recent advancements in deep convolutional neural networks offer promise in automating pathology detection in trauma X-rays. However, distinguishing subtle variations between pediatric wrist pathologies in X-rays remains challenging. Traditional manual annotation, though effective, is laborious, costly, and requires specialized expertise. In this paper, we address the challenge of pediatric wrist pathology recognition with a fine-grained approach, aimed at automatically identifying discriminative regions in X-rays without manual intervention. We refine our fine-grained architecture through ablation analysis and the integration of LION. Leveraging Grad-CAM, an explainable AI technique, we highlight these regions. Despite using limited data, reflective of real-world medical study constraints, our method consistently outperforms state-of-the-art image recognition models on both augmented and original (challenging) test sets. Our proposed refined architecture achieves an increase in accuracy of 1.06% and 1.25% compared to the baseline method, resulting in accuracies of 86% and 84%, respectively. Moreover, our approach demonstrates the highest fracture sensitivity of 97%, highlighting its potential to enhance wrist pathology recognition.

Accurate, automated classification of radiographic knee osteoarthritis severity using a novel method of deep learning: Plug-in modules

BackgroundFine-grained classification deals with data with a large degree of similarity, such as cat or bird species, and similarly, knee osteoarthritis severity classification [Kellgren–Lawrence (KL) grading] is one such fine-grained classification task. Recently, a plug-in module (PIM) that can be integrated into convolutional neural-network-based or transformer-based networks has been shown to provide strong discriminative regions for fine-grained classification, with results that outperformed the previous deep learning models. PIM utilizes each pixel of an image as an independent feature and can subsequently better classify images with minor differences. It was hypothesized that, as a fine-grained classification task, knee osteoarthritis severity may be classified well using PIMs. The aim of the study was to develop this automated knee osteoarthritis classification model.MethodsA deep learning model that classifies knee osteoarthritis severity of a radiograph was developed utilizing PIMs. A retrospective analysis on prospectively collected data was performed. The model was trained and developed using the Osteoarthritis Initiative dataset and was subsequently tested on an independent dataset, the Multicenter Osteoarthritis Study (test set size: 17,040). The final deep learning model was designed through an ensemble of four different PIMs.ResultsThe accuracy of the model was 84%, 43%, 70%, 81%, and 96% for KL grade 0, 1, 2, 3, and 4, respectively, with an overall accuracy of 75.7%.ConclusionsThe ensemble of PIMs could classify knee osteoarthritis severity using simple radiographs with a fine accuracy. Although improvements will be needed in the future, the model has been proven to have the potential to be clinically useful.

Self-Supervised Medical Image Segmentation Using Deep Reinforced Adaptive Masking.

Self-supervised learning aims to learn transferable representations from unlabeled data for downstream tasks. Inspired by masked language modeling in natural language processing, masked image modeling (MIM) has achieved certain success in the field of computer vision, but its effectiveness in medical images remains unsatisfactory. This is mainly due to the high redundancy and small discriminative regions in medical images compared to natural images. Therefore, this paper proposes an adaptive hard masking (AHM) approach based on deep reinforcement learning to expand the application of MIM in medical images. Unlike predefined random masks, AHM uses an asynchronous advantage actor-critic (A3C) model to predict reconstruction loss for each patch, enabling the model to learn where masking is valuable. By optimizing the non-differentiable sampling process using reinforcement learning, AHM enhances the understanding of key regions, thereby improving downstream task performance. Experimental results on two medical image datasets demonstrate that AHM outperforms state-of-the-art methods. Additional experiments under various settings validate the effectiveness of AHM in constructing masked images.

Spatial-Temporal Co-Attention Learning for Diagnosis of Mental Disorders From Resting-State fMRI Data.

Neuroimaging techniques have been widely adopted to detect the neurological brain structures and functions of the nervous system. As an effective noninvasive neuroimaging technique, functional magnetic resonance imaging (fMRI) has been extensively used in computer-aided diagnosis (CAD) of mental disorders, e.g., autism spectrum disorder (ASD) and attention deficit/hyperactivity disorder (ADHD). In this study, we propose a spatial-temporal co-attention learning (STCAL) model for diagnosing ASD and ADHD from fMRI data. In particular, a guided co-attention (GCA) module is developed to model the intermodal interactions of spatial and temporal signal patterns. A novel sliding cluster attention module is designed to address global feature dependency of self-attention mechanism in fMRI time series. Comprehensive experimental results demonstrate that our STCAL model can achieve competitive accuracies of 73.0 ± 4.5%, 72.0 ± 3.8%, and 72.5 ± 4.2% on the ABIDE I, ABIDE II, and ADHD-200 datasets, respectively. Moreover, the potential for feature pruning based on the co-attention scores is validated by the simulation experiment. The clinical interpretation analysis of STCAL can allow medical professionals to concentrate on the discriminative regions of interest and key time frames from fMRI data.

Accurate identification of individuals with subjective cognitive decline using 3D regional fractal dimensions on structural magnetic resonance imaging

Background and ObjectiveAccurate identification of individuals with subjective cognitive decline (SCD) is crucial for early intervention and prevention of neurodegenerative diseases. Fractal dimensionality (FD) has emerged as a robust and replicable measure, surpassing traditional geometric metrics, in characterizing the intricate fractal geometrical properties of brain structure. Nevertheless, the effectiveness of FD in identifying individuals with SCD remains largely unclear. A 3D regional FD method can be suggested to characterize and quantify the spatial complexity of the precise gray matter, providing insights into cognitive aging and aiding in the automated identification of individuals with SCD. MethodsThis study introduces a novel integer ratio based 3D box-counting fractal analysis (IRBCFA) to quantify regional fractal dimensions (FDs) in structural magnetic resonance imaging (MRI) data. The innovative method overcomes limitations of conventional box-counting techniques by accommodating arbitrary box sizes, thereby enhancing the precision of FD estimation in small, yet neurologically significant, brain regions. ResultsThe application of IRBCFA to two publicly available datasets, OASIS-3 and ADNI, consisting of 520 and 180 subjects, respectively. The method identified discriminative regions of interest (ROIs) predominantly within the limbic system, fronto-parietal region, occipito-temporal region, and basal ganglia-thalamus region. These ROIs exhibited significant correlations with cognitive functions, including executive functioning, memory, social cognition, and sensory perception, suggesting their potential as neuroimaging markers for SCD. The identification model trained on these ROIs demonstrated exceptional performance achieving over 93 % accuracy on the discovery dataset and exceeding 87 % on the independent testing dataset. Furthermore, an exchange experiment between datasets revealed a substantial overlap in discriminative ROIs, highlighting the robustness of our method across diverse populations. ConclusionOur findings indicate that IRBCFA can serve as a valuable tool for quantifying the spatial complexity of gray matter, providing insights into cognitive aging and aiding in the automated identification of individuals with SCD. The demonstrated generalizability and robustness of this method position it as a promising tool for neurodegenerative disease research and offer potential for clinical applications.

Enhanced Attention Guided Teacher–Student Network for Weakly Supervised Object Detection

Weakly Supervised Object Detection (WSOD) has attracted increasing attention due to the convenience and low-cost of acquiring image-level annotations. Most existing WSOD methods follow Multiple Instance Learning (MIL) paradigm to select bounding boxes from proposals based on their classification scores. However, MIL-based WSOD methods often focus on discriminative regions, leading to incomplete and missing instances. To address these issues, we introduce a novel WSOD framework named Enhanced Attention Guided Teacher–Student Network (EATSN). This framework aims to improve detection performance through the guidance of attention map and consistency learning. specifically, we initially train a teacher network using MIL process to generate pseudo ground-truth labels. Subsequently, the weak augmented and strong augmented images are fed into teacher and student models to produce the enhanced attention map. During the training iterations, pseudo labels are utilized to guide the student model, while the teacher model refines its parameters through the Exponential Moving Average(EMA) from the student. Finally we design a proposal selection method that leverages the enhanced attention map and bounding boxes scores to achieve better detection results. Experimental results on benchmark datasets demonstrate that our method achieves comparable performance.

F.1 Oscillatory network markers of subcallosal cingulate deep brain stimulation for depression

Background: Identifying functional biomarkers related to treatment success can aid in optimizing therapy and provide a better understanding of the neural mechanisms of treatment-resistant depression (TRD) and subcallosal cingulate deep brain stimulation (SCC-DBS). Methods: Magnetoencephalography data were obtained from 16 individuals with SCC-DBS for TRD and 25 healthy subjects. We identified region-specific oscillatory modulations that both (i) discriminate individuals with TRD (SCC-DBS OFF) from healthy controls and (ii) discriminate responders from non-responders (SCC-DBS ON). The effects of stimulation intensity and frequency were also explored. Results: Discriminative regions that differentiated responders from non-responders based on modulations of increased alpha (8-12 Hz) and decreased gamma (32-116 Hz) power included nodes of the default mode, central executive, and somatomotor networks, Broca’s area, and lingual gyrus. Furthermore, low stimulation frequency had stronger effects on oscillatory modulation. Conclusions: The identified functional biomarkers implicate modulations of TRD-related activity in brain regions involved in emotional control/processing, motor control, and interactions between speech, vision, and memory – all implicated in depression. These electrophysiological biomarkers have the potential to be used as functional proxies for therapy optimization. Additional stimulation parameter analyses revealed that oscillatory modulations are strengthened by increasing stimulation intensity or reducing frequency, which may benefit SCC-DBS non-responders.

Multi-attentional causal intervention networks for medical image diagnosis

Medical image diagnosis has developed rapidly under the impetus of the deep network. Previous works mainly focus on improving the diagnostic accuracy of models, i.e., first use a backbone network to extract image global features and then feed it into the classifier for diagnosis. However, these methods do not fully explore the transparent and reasonable decision-making process of the final classification results, which is crucial for medical diagnosis. In this paper, we propose a framework called Causal Intervention-based Multi-head Attention network (CaIMA) to enhance the explainability of medical diagnosis from a causal inference perspective, by exploring the inherent causal relationship between multi-region attention and diagnosis results. Specifically, it consists of three key components: (1) The multi-region attention module enables the network to focus on the distinct discriminative lesion regions that hold causal relationships with the predicted outcome. (2) The attention-driven data augmentation module provides accurate localization of discriminative regions and enhances model explainability. (3) The causal intervention module aims to explore the intrinsic causal relationship between the attention map and the predicted outcome, encouraging the network to learn more useful attention maps for medical image diagnosis. Besides, to address the learning difficulty of this network, we further introduce a non-overlapping multiple attentional guidance loss that encourages the learned multiple attention maps to focus on specific lesion regions without overlapping. We compare the proposed CaIMA with state-of-the-art methods on multimedia medical datasets, including three public medical image datasets (Kvasir, ISIC2018, COVID-19) and one private dataset (CLC), and the experimental results substantiate the effectiveness of CaIMA in terms of diagnosis accuracy and explainability.

PCSformer: Pair-wise Cross-scale Sub-prototypes mining with CNN-transformers for weakly supervised semantic segmentation

Generating initial seeds is an important step in weakly supervised semantic segmentation (WSSS). Our approach concentrates on generating and refining initial seeds. The convolutional neural networks (CNNs)–based initial seeds focus only on the most discriminative regions and lack global information about the target. The Vision Transformer (ViT)–based approach can capture long-range feature dependencies due to the unique advantage of the self-attention mechanism. Still, we find that it suffers from distractor object leakage and background leakage problems. Based on these observations, we propose PCSformer, which improves the model’s ability to extract features through a Pair-wise Cross-scale (PC) strategy and solves the problem of distractor object leakage by further extracting potential target features through Sub-Prototypes (SP) mining. In addition, the proposed Conflict Self-Elimination (CSE) module further alleviates the background leakage problem. We validate our approach on the widely adopted Pascal VOC 2012 and MS COCO 2014, and extensive experiments demonstrate our superior performance. Furthermore, our method proves to be competitive for WSSS in medical images and challenging scenarios involving deformable and cluttered scenes. Additionally, we extend the PCSformer to weakly supervised object localization tasks, further highlighting its scalability and versatility. The code is available at https://github.com/ChunmengLiu1/PCSformer.

Graph Representation and Prototype Learning for webly supervised fine-grained image recognition

Webly supervised fine-grained image recognition (FGIR) learns to distinguish sub-ordinate categories based on webly-retrieved data, which can dramatically alleviate the dependency on manually annotated labels. This is quite a challenging task due to the heavy noise labels and the inherent dilemma of small inter-class variance and large intra-class variance. Current webly supervised algorithms learn holistic category prototypes to help correct noisy labels but ignore local features that can distinguish different sub-ordinate categories. In this work, we propose a Graph Representation and Prototype Learning (GRPL) framework to automatically mine discriminative local regions and their interactions with holistic image to learn instance graph representation both category graph prototype to help correct noisy labels and retrieve out-of-distribution (OOD) samples. Specifically, an attention-focused module is designed to extract the discriminative regions and then build a structured graph to correlate them with the holistic image for each instance and an identical graph to model holistic-local correlations for each category. Next, we apply two stacked graph convolution networks to explore holistic-local interaction within each graph and across two graphs to learn graph representation for each instance and graph prototype for each category. Finally, the similarities between the instance-level and prototype-level graph representation are learned to help correct noisy labels and exclude OOD samples. Extensive experiments conducted on several datasets show the proposed approach achieves superior performance compared with current leading algorithms.

A multi-resolution convolutional attention network for efficient diabetic retinopathy classification

Diabetic retinopathy (DR) is a disorder commonly associated with individuals who have diabetes and is caused by excessive glucose levels in the blood, which induce retinal damage. Manual diagnosis of DR by medical experts is time-consuming; hence, a computer-aided diagnosis (CAD) solution for faster identification and improved efficiency is required. Diagnosis of DR is complex due to the prevalence of inter-class differences and small lesions, making it difficult for convolutional neural networks (CNNs) to identify discriminative regions effectively. The most critical challenge is locating complex patterns connected with visual differences and lesion locations. To tackle the aforementioned challenges, this work presents a novel multi-resolution convolutional attention network (MuR-CAN), which improves overall performance by emphasizing discriminative features. MuR-CAN includes a multi-dilation attention block (MDAB), which uses depth-wise convolution layers with various dilation rates to allow the model to prioritize relevant features and capture multi-scale spatial information. This method provides for simultaneously highlighting fine features and subtle visual patterns. Furthermore, the features extracted from the MuR-CAN are utilized to train the support vector machine (SVM) for DR classification. The integration of the SVM classifier with the proposed CNN architecture ensures improved accuracy. The experimental findings demonstrate that the proposed model outperforms the current state-of-the-art methods.

Improving weakly-supervised object localization using adversarial erasing and pseudo label

Weakly-supervised learning approaches have gained significant attention due to their ability to reduce the effort required for human annotations in training neural networks. This paper investigates a framework for weakly-supervised object localization, which aims to train a neural network capable of predicting both the object class and its location using only images and their image-level class labels. The proposed framework consists of a shared feature extractor, a classifier, and a localizer. The localizer predicts pixel-level class probabilities, while the classifier predicts the object class at the image level. Since image-level class labels are insufficient for training the localizer, weakly-supervised object localization methods often encounter challenges in accurately localizing the entire object region. To address this issue, the proposed method incorporates adversarial erasing and pseudo labels to improve localization accuracy. Specifically, novel losses are designed to utilize adversarially erased foreground features and adversarially erased feature maps, reducing dependence on the most discriminative region. Additionally, the proposed method employs pseudo labels to suppress activation values in the background while increasing them in the foreground. The proposed method is applied to two backbone networks (MobileNetV1 and InceptionV3) and is evaluated on three publicly available datasets (ILSVRC-2012, CUB-200-2011, and PASCAL VOC 2012). The experimental results demonstrate that the proposed method outperforms previous state-of-the-art methods across all evaluated metrics.

Classification of Chip-Level Defect Types in Wafer Bin Maps Using Only Wafer-Level Labels

Abstract Defective chips in wafer bin maps (WBMs) form different spatial patterns depending on the root causes of process failures. Therefore, the identification of defect patterns in WBMs can help practitioners identify the root causes. Previous studies have focused on wafer-level classification even though chip-level classification can provide additional information regarding defect locations and defect sizes. Chip-level classification is more challenging than wafer-level classification because existing chip-level classification methods require chip-level labels, which are laborious to collect. We propose a method for chip-level defect classification using only wafer-level labels based on weakly supervised semantic segmentation. We first train a classification network using wafer-level labels and extract class activation maps (CAMs), which are visualizations of the discriminative regions. We then generate chip-level pseudo-labels using the extracted CAMs and use these labels to train a segmentation network, which predicts chip-level defect types. Experimental results verify the effectiveness of the proposed method.

GNDAN: Graph Navigated Dual Attention Network for Zero-Shot Learning.

Zero-shot learning (ZSL) tackles the unseen class recognition problem by transferring semantic knowledge from seen classes to unseen ones. Typically, to guarantee desirable knowledge transfer, a direct embedding is adopted for associating the visual and semantic domains in ZSL. However, most existing ZSL methods focus on learning the embedding from implicit global features or image regions to the semantic space. Thus, they fail to: 1) exploit the appearance relationship priors between various local regions in a single image, which corresponds to the semantic information and 2) learn cooperative global and local features jointly for discriminative feature representations. In this article, we propose the novel graph navigated dual attention network (GNDAN) for ZSL to address these drawbacks. GNDAN employs a region-guided attention network (RAN) and a region-guided graph attention network (RGAT) to jointly learn a discriminative local embedding and incorporate global context for exploiting explicit global embeddings under the guidance of a graph. Specifically, RAN uses soft spatial attention to discover discriminative regions for generating local embeddings. Meanwhile, RGAT employs an attribute-based attention to obtain attribute-based region features, where each attribute focuses on the most relevant image regions. Motivated by the graph neural network (GNN), which is beneficial for structural relationship representations, RGAT further leverages a graph attention network to exploit the relationships between the attribute-based region features for explicit global embedding representations. Based on the self-calibration mechanism, the joint visual embedding learned is matched with the semantic embedding to form the final prediction. Extensive experiments on three benchmark datasets demonstrate that the proposed GNDAN achieves superior performances to the state-of-the-art methods. Our code and trained models are available at https://github.com/shiming-chen/GNDAN.

Progressive Feature Self-Reinforcement for Weakly Supervised Semantic Segmentation

Compared to conventional semantic segmentation with pixel-level supervision, weakly supervised semantic segmentation (WSSS) with image-level labels poses the challenge that it commonly focuses on the most discriminative regions, resulting in a disparity between weakly and fully supervision scenarios. A typical manifestation is the diminished precision on object boundaries, leading to deteriorated accuracy of WSSS. To alleviate this issue, we propose to adaptively partition the image content into certain regions (e.g., confident foreground and background) and uncertain regions (e.g., object boundaries and misclassified categories) for separate processing. For uncertain cues, we propose an adaptive masking strategy and seek to recover the local information with self-distilled knowledge. We further assume that confident regions should be robust enough to preserve the global semantics, and introduce a complementary self-distillation method that constrains semantic consistency between confident regions and an augmented view with the same class labels. Extensive experiments conducted on PASCAL VOC 2012 and MS COCO 2014 demonstrate that our proposed single-stage approach for WSSS not only outperforms state-of-the-art counterparts but also surpasses multi-stage methods that trade complexity for accuracy.