Stanford Dogs Research Articles

Promoting the Shift From Pixel-Level Correlations to Object Semantics Learning by Rethinking Computer Vision Benchmark Data Sets.

In computer vision research, convolutional neural networks (CNNs) have demonstrated remarkable capabilities at extracting patterns from raw pixel data, achieving state-of-the-art recognition accuracy. However, they significantly differ from human visual perception, prioritizing pixel-level correlations and statistical patterns, often overlooking object semantics. To explore this difference, we propose an approach that isolates core visual features crucial for human perception and object recognition: color, texture, and shape. In experiments on three benchmarks-Fruits 360, CIFAR-10, and Fashion MNIST-each visual feature is individually input into a neural network. Results reveal data set-dependent variations in classification accuracy, highlighting that deep learning models tend to learn pixel-level correlations instead of fundamental visual features. To validate this observation, we used various combinations of concatenated visual features as input for a neural network on the CIFAR-10 data set. CNNs excel at learning statistical patterns in images, achieving exceptional performance when training and test data share similar distributions. To substantiate this point, we trained a CNN on CIFAR-10 data set and evaluated its performance on the "dog" class from CIFAR-10 and on an equivalent number of examples from the Stanford Dogs data set. The CNN poor performance on Stanford Dogs images underlines the disparity between deep learning and human visual perception, highlighting the need for models that learn object semantics. Specialized benchmark data sets with controlled variations hold promise for aligning learned representations with human cognition in computer vision research.

Fine-Grained Few-Shot Image Classification Based on Feature Dual Reconstruction

Fine-grained few-shot image classification is a popular research area in deep learning. The main goal is to identify subcategories within a broader category using a limited number of samples. The challenge stems from the high intra-class variability and low inter-class variability of fine-grained images, which often hamper classification performance. To overcome this, we propose a fine-grained few-shot image classification algorithm based on bidirectional feature reconstruction. This algorithm introduces a Mixed Residual Attention Block (MRA Block), combining channel attention and window-based self-attention to capture local details in images. Additionally, the Dual Reconstruction Feature Fusion (DRFF) module is designed to enhance the model’s adaptability to both inter-class and intra-class variations by integrating features of different scales across layers. Cosine similarity networks are employed for similarity measurement, enabling precise predictions. The experiments demonstrate that the proposed method achieves classification accuracies of 96.99%, 98.53%, and 89.78% on the CUB-200-2011, Stanford Cars, and Stanford Dogs datasets, respectively, confirming the method’s efficacy in fine-grained classification tasks.

Adversarially attack feature similarity for fine-grained visual classification

Fine-grained visual classification (FGVC) strives to distinguish images from distinct sub-classes within the same overarching meta-class, which is significant in various practical applications. Existing works mainly employ attention mechanisms to learn discriminative feature representations of objects under weakly supervised learning. In this paper, we argue that this likehood-based attention learning manner often outputs an inadequate feature representation since the available image-level labels fail to provide an explicit supervisory signal for attention learning, especially when the fine-grained images share a small and inconsistent inter-class variance. To alleviate this issue, we consider this challenging task from the perspective of attacking the feature representation between similar sub-classes to maximize the feature discriminativeness via learning adversarial examples, and propose an Adversarial-Aware Fine-Grained Visual Classification Network (A2Net). Specifically, we first propose an adversarial attack module based on projected gradient descent, which appends multiple-scale adversarial perturbations to simulate subclass examples with different similarities. Then, we introduce an adversarial attention generation module that estimates the effect of attention learned on adversarial examples and legitimate examples for the final class prediction through causal inference. The adversarial attention generation module is encouraged to maximize the effect, which provides powerful supervision to capture more attention indicating the discriminative parts. We further propose an adversarial-aware module to learn the feature-level differences between legitimate and adversarial examples, which helps enhance the semantic boundaries of class-specific features for accurate FGVC. The extensive array of experiments conducted serves to underscore the efficacy of the proposed A2Net, outperforming state-of-the-art FGVC methods on CUB-200–2011, FGVC-Aircraft, Stanford Cars, Stanford Dogs, and NABirds benchmarks.

KLSANet: Key local semantic alignment Network for few-shot image classification

Few-shot image classification involves recognizing new classes with a limited number of labeled samples. Current local descriptor-based methods, while leveraging consistent low-level features across visible and invisible classes, face challenges including redundant adjacent information, irrelevant partial representation, and limited interpretability. This paper proposes KLSANet, a few-shot image classification approach based on key local semantic alignment network, which aligns key local semantics for accurate classification. Furthermore, we introduce a key local screening module to mitigate the influence of semantically irrelevant image parts on classification. KLSANet demonstrates superior performance on three benchmark datasets (CUB, Stanford Dogs, Stanford Cars), outperforming state-of-the-art methods in 1-shot and 5-shot settings with average improvements of 3.95% and 2.56% respectively. Visualization experiments demonstrate the interpretability of KLSANet predictions. Code is available at: https://github.com/ZitZhengWang/KLSANet.

Polymorphic Clustering and Approximate Masking Framework for Fine-Grained Insect Image Classification

Insect diversity monitoring is crucial for biological pest control in agriculture and forestry. Modern monitoring of insect species relies heavily on fine-grained image classification models. Fine-grained image classification faces challenges such as small inter-class differences and large intra-class variances, which are even more pronounced in insect scenes where insect species often exhibit significant morphological differences across multiple life stages. To address these challenges, we introduce segmentation and clustering operations into the image classification task and design a novel network model training framework for fine-grained classification of insect images using multi-modality clustering and approximate mask methods, named PCAM-Frame. In the first stage of the framework, we adopt the Polymorphic Clustering Module, and segmentation and clustering operations are employed to distinguish various morphologies of insects at different life stages, allowing the model to differentiate between samples at different life stages during training. The second stage consists of a feature extraction network, called Basenet, which can be any mainstream network that performs well in fine-grained image classification tasks, aiming to provide pre-classification confidence for the next stage. In the third stage, we apply the Approximate Masking Module to mask the common attention regions of the most likely classes and continuously adjust the convergence direction of the model during training using a Deviation Loss function. We apply PCAM-Frame with multiple classification networks as the Basenet in the second stage and conduct extensive experiments on the Insecta dataset of iNaturalist 2017 and IP102 dataset, achieving improvements of 2.2% and 1.4%, respectively. Generalization experiments on other fine-grained image classification datasets such as CUB200-2011 and Stanford Dogs also demonstrate positive effects. These experiments validate the pertinence and effectiveness of our framework PCAM-Frame in fine-grained image classification tasks under complex conditions, particularly in insect scenes.

Feature alignment via mutual mapping for few-shot fine-grained visual classification

Few-shot fine-grained visual classification aims to identify fine-grained concepts with very few samples, which is widely used in many fields, such as the classification of different species of birds in biological research, and the identification of car models in traffic monitoring. Compared with the common few-shot classification task, it encounters difficulties due to significant variations within each class and small gaps between different categories. To address such problems, previous studies primarily project support samples into the space of query samples and employ metric learning to classify query images into their respective categories. However, we observe that such methods are not effective in resolving inter-class variations. To overcome this limitation, we propose a new feature alignment method based on mutual mapping, which simultaneously considers the discriminative features of new samples and classes. Specifically, besides projecting support samples into the space of query samples for reducing intra-class variations, we also project query samples into the space of support samples to increase inter-class variations. Furthermore, a direct position self-reconstruction module is proposed to utilize the location information of objects and obtain more discriminative features. Extensive experiments on four fine-grained benchmarks demonstrate that our approach is competitive when compared with other state-of-the-art methods, in both 1-shot and 5-shot settings. In the case of 5-shot, our method achieved the best performance on all four datasets, with 92.11%, 85.31%, 96.09%, and 94.64% accuracies on CUB-200-2011, Stanford Dogs, Stanford Cars, and Aircaft, respectively.

Keep the Faith: Faithful Explanations in Convolutional Neural Networks for Case-Based Reasoning

Explaining predictions of black-box neural networks is crucial when applied to decision-critical tasks. Thus, attribution maps are commonly used to identify important image regions, despite prior work showing that humans prefer explanations based on similar examples. To this end, ProtoPNet learns a set of class-representative feature vectors (prototypes) for case-based reasoning. During inference, similarities of latent features to prototypes are linearly classified to form predictions and attribution maps are provided to explain the similarity. In this work, we evaluate whether architectures for case-based reasoning fulfill established axioms required for faithful explanations using the example of ProtoPNet. We show that such architectures allow the extraction of faithful explanations. However, we prove that the attribution maps used to explain the similarities violate the axioms. We propose a new procedure to extract explanations for trained ProtoPNets, named ProtoPFaith. Conceptually, these explanations are Shapley values, calculated on the similarity scores of each prototype. They allow to faithfully answer which prototypes are present in an unseen image and quantify each pixel’s contribution to that presence, thereby complying with all axioms. The theoretical violations of ProtoPNet manifest in our experiments on three datasets (CUB-200-2011, Stanford Dogs, RSNA) and five architectures (ConvNet, ResNet, ResNet50, WideResNet50, ResNeXt50). Our experiments show a qualitative difference between the explanations given by ProtoPNet and ProtoPFaith. Additionally, we quantify the explanations with the Area Over the Perturbation Curve, on which ProtoPFaith outperforms ProtoPNet on all experiments by a factor >10^3.

Robust fine‐grained visual recognition with images based on internet of things

AbstractLabeling fine‐grained objects manually is extremely challenging, as it is not only label‐intensive but also requires professional knowledge. Accordingly, robust learning methods for fine‐grained recognition with web images collected from Internet of Things have drawn significant attention. However, training deep fine‐grained models directly using untrusted web images is confronted by two primary obstacles: (1) label noise in web images and (2) domain variance between the online sources and test datasets. To this end, in this study, we mainly focus on addressing these two pivotal problems associated with untrusted web images. To be specific, we introduce an end‐to‐end network that collaboratively addresses these concerns in the process of separating trusted data from untrusted web images. To validate the efficacy of our proposed model, untrusted web images are first collected by utilizing the text category labels found within fine‐grained datasets. Subsequently, we employ the designed deep model to eliminate label noise and ameliorate domain mismatch. And the chosen trusted web data are utilized for model training. Comprehensive experiments and ablation studies validate that our method consistently surpasses other state‐of‐the‐art approaches for fine‐grained recognition tasks in real‐world scenarios, demonstrating a significant improvement margin (2.51% on CUB200‐2011 and 2.92% on Stanford Dogs). The source code and models can be accessed at: https://github.com/Codeczh/FGVC‐IoT.

Fine-grained image classification based on TinyVit object location and graph convolution network

Fine-grained image classification is a branch of image classification. Recently, vision transformer has made excellent progress in the field of image recognition. Its self-attention mechanism can extract very effective image feature information. However, feeding fixed-size image blocks into the network introduces additional noise, which is detrimental to extract discriminative features for fine-grained images. The vision transformer's network model is large, making it difficult to utilize in practice. Moreover, many of today's fine-grained image classification methods focus on mining discriminative features while ignoring the connections within the image. To address these problems, we propose a novel method based on the lightweight TinyVit backbone network. Our approach utilizes the self-attention weight values of TinyVit as a guide to construct an effective object location (OL) module that cuts and enlarges the object area, providing the network with the opportunity to concentrate on the local object. Additionally, we employ the graph convolutional network (GCN) to create a spatial relationship feature learning (SRFL) module that captures spatial context information between image blocks in TinyVit with the help of the transformer's self-attention weights. OL and SRFL collaborate to jointly guide the classification task. The experimental results show that the proposed method achieved competitive performance, with the second-highest classification faccuracy on both the CUB-200–2011 and NABirds datasets. When tested on the Stanford Dogs dataset, our approach outperformed many popular methods. Our code is uploaded on https://github.com/hhhj1999/SRFL_OL.

Pyramid hybrid pooling quantization for efficient fine-grained image retrieval

Deep hashing approaches, including deep quantization and deep binary hashing, have become a common solution to large-scale image retrieval due to their high computation and storage efficiency. Most existing hashing methods cannot produce satisfactory results for fine-grained retrieval, because they usually adopt the outputs of the last CNN layer to generate binary codes. Since deeper layers tend to summarize visual clues, e.g., texture, into abstract semantics, e.g., dogs and cats, the feature produced by the last CNN layer is less effective in capturing subtle but discriminative visual details that mostly exist in shallow layers. To improve fine-grained image hashing, we propose Pyramid Hybrid Pooling Quantization (PHPQ). Specifically, we propose a Pyramid Hybrid Pooling (PHP) module to capture and preserve fine-grained semantic information from multi-level features, which emphasizes the subtle discrimination of different sub-categories. Besides, we propose a learnable quantization module with a partial codebook attention mechanism, which helps to optimize the most relevant codewords and improves the quantization. Comprehensive experiments on two widely-used public benchmarks, i.e., CUB-200-2011 and Stanford Dogs, demonstrate that PHPQ outperforms state-of-the-art methods.

Explicitly learning augmentation invariance for image classification by Consistent Augmentation

Data augmentation is a powerful and widely used technique to improve the generalization of convolutional neural networks. The majority of data augmentation methods emphasize transforming samples to bolster network robustness. However, these methods overlook learning the invariance between the samples augmented by different transformations, impairing the quality of learned representations and generalization of the model. In this paper, we introduce a straightforward yet highly effective method called Consistent Augmentation (CA) to compel networks to learn explicitly the invariance of transformations. Specifically, CA minimizes the bidirectional KL divergence between the predicted distributions of different variants generated from the same samples using different transformations, explicitly enforcing consistency among various semantics-preserving transformations. Implementing CA is intuitive, uncomplicated, and does not introduce additional computational costs during inference. The validation experiments performed on conventional and fine-grained classification tasks demonstrate that CA can be employed across various network architectures and consistently improve their generalization ability. Notably, convolutional neural networks benefit significantly on fine-grained classification tasks, with an improvement of 5.18% and 6.61% compared with other augmentation methods in accuracy observed for the ShuffleNetV2 model on the Stanford Dogs and CUB-200-2011 datasets. Compatibility experiments conducted on the CIFAR-100 dataset reveal that CA improves the accuracy of the Mixup and CutMix methods by 1.25% and 2.11%, respectively. These results underscore the effectiveness of our approach when employed alongside mix-based augmentation methods, further enhancing the capabilities of the model.

Complementary Parts Contrastive Learning for Fine-Grained Weakly Supervised Object Co-Localization

The aim of weakly supervised object co-localization is to locate different objects of the same superclass in a dataset. Recent methods achieve impressive co-localization performance by multiple instance learning and self-supervised learning. However, these methods ignore the common part information shared by fine-grained objects and the influence of the complementary parts on the co-localization of the fine-grained objects. To solve these issues, we propose a complementary parts contrastive learning method for fine-grained weakly supervised object co-localization. The proposed method follows such an assumption that fine-grained object parts with the same/different semantic meaning should have similar/dissimilar feature representations in the feature space. The proposed method tackles two critical issues in this task: <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i</i> ) how to spread the model’s attention and suppress the complex background noise, and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">ii</i> ) how to leverage the cross-category common parts information to mitigate the context co-occurrence problem. To address <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i</i> ), we attempt to integrate local and context cues via three types of attention including self-supervised attention, channel, and spatial attention to spread the model’s attention toward automatically identifying and localizing most discriminative parts of objects in the fine-grained images. To solve <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">ii</i> ), we propose a cross-category object complementarity part contrastive learning module to identify the extracted part regions with different semantic information by pulling the same part features closer and pushing different part features away, which can mitigate the confounding bias caused by the co-occurrence surroundings within specific classes. Extensive qualitative and quantitative evaluations demonstrate the effectiveness of the proposed method on four fine-grained co-localization datasets: CUB-200-2011, Stanford Cars, FGVC-Aircraft, and Stanford Dogs. Code and models are available at https://github.com/Zhao-fan/CPCL.

Improved transfer learning using textural features conflation and dynamically fine-tuned layers.

Transfer learning involves using previously learnt knowledge of a model task in addressing another task. However, this process works well when the tasks are closely related. It is, therefore, important to select data points that are closely relevant to the previous task and fine-tune the suitable pre-trained model's layers for effective transfer. This work utilises the least divergent textural features of the target datasets and pre-trained model's layers, minimising the lost knowledge during the transfer learning process. This study extends previous works on selecting data points with good textural features and dynamically selected layers using divergence measures by combining them into one model pipeline. Five pre-trained models are used: ResNet50, DenseNet169, InceptionV3, VGG16 and MobileNetV2 on nine datasets: CIFAR-10, CIFAR-100, MNIST, Fashion-MNIST, Stanford Dogs, Caltech 256, ISIC 2016, ChestX-ray8 and MIT Indoor Scenes. Experimental results show that data points with lower textural feature divergence and layers with more positive weights give better accuracy than other data points and layers. The data points with lower divergence give an average improvement of 3.54% to 6.75%, while the layers improve by 2.42% to 13.04% for the CIFAR-100 dataset. Combining the two methods gives an extra accuracy improvement of 1.56%. This combined approach shows that data points with lower divergence from the source dataset samples can lead to a better adaptation for the target task. The results also demonstrate that selecting layers with more positive weights reduces instances of trial and error in selecting fine-tuning layers for pre-trained models.

Patch-Level Consistency Regularization in Self-Supervised Transfer Learning for Fine-Grained Image Recognition

Fine-grained image recognition aims to classify fine subcategories belonging to the same parent category, such as vehicle model or bird species classification. This is an inherently challenging task because a classifier must capture subtle interclass differences under large intraclass variances. Most previous approaches are based on supervised learning, which requires a large-scale labeled dataset. However, such large-scale annotated datasets for fine-grained image recognition are difficult to collect because they generally require domain expertise during the labeling process. In this study, we propose a self-supervised transfer learning method based on Vision Transformer (ViT) to learn finer representations without human annotations. Interestingly, it is observed that existing self-supervised learning methods using ViT (e.g., DINO) show poor patch-level semantic consistency, which may be detrimental to learning finer representations. Motivated by this observation, we propose a consistency loss function that encourages patch embeddings of the overlapping area between two augmented views to be similar to each other during self-supervised learning on fine-grained datasets. In addition, we explore effective transfer learning strategies to fully leverage existing self-supervised models trained on large-scale labeled datasets. Contrary to the previous literature, our findings indicate that training only the last block of ViT is effective for self-supervised transfer learning. We demonstrate the effectiveness of our proposed approach through extensive experiments using six fine-grained image classification benchmark datasets, including FGVC Aircraft, CUB-200-2011, Food-101, Oxford 102 Flowers, Stanford Cars, and Stanford Dogs. Under the linear evaluation protocol, our method achieves an average accuracy of 78.5%, outperforming the existing transfer learning method, which yields 77.2%.

Abnormality Detection of Blast Furnace Tuyere Based on Knowledge Distillation and a Vision Transformer

The blast furnace tuyere is a key position in hot metal production and is primarily observed to assess the internal state of the furnace. However, detecting abnormal tuyere conditions has relied heavily on manual judgment, leading to certain limitations. We proposed a tuyere abnormality detection model based on knowledge distillation and a vision transformer (ViT) to address this issue. In this approach, ResNet50 is employed as the Teacher model to distill knowledge into the Student model, ViT. Firstly, we introduced spatial attention modules to enhance the model’s perception and feature-extraction capabilities for different image regions. Furthermore, we simplified the depth of the ViT and improved its self-attention mechanism to alleviate training losses. In addition, we employed the knowledge distillation strategy to achieve model lightweighting and enhance the model’s generalization capability. Finally, we evaluate the model’s performance in tuyere abnormality detection and compare it with other classification methods such as VGG-19, ResNet-101, and ResNet-50. Experimental results showed that our model achieved a classification accuracy of 97.86% on a tuyere image dataset from a company, surpassing the original ViT model by 1.12% and the improved ViT model without knowledge distillation by 0.34%. Meanwhile, the model achieved a competitive classification accuracy of 90.31% and 77.65% on the classical fine-grained image datasets, Stanford Dogs and CUB-200-2011, respectively, comparable to other classification models.

Dual Transformer With Multi-Grained Assembly for Fine-Grained Visual Classification

Fine-grained visual classification requires distinguishing sub-categories within the same super-category, which suffers from small inter-class and large intra-class variances. This paper aims to improve the FGVC task towards better performance, for which we deliver a novel dual Transformer framework (coined Dual-TR) with multi-grained assembly. The Dual-TR is well-designed to encode fine-grained objects by two parallel hierarchies, which is amenable to capturing the subtle yet discriminative cues via the self-attention mechanism in ViT. Specifically, we perform orthogonal multi-grained assembly within the Transformer structure for a more robust representation, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e</i> ., intra-layer and inter-layer assembly. The former aims to explore the informative feature in various self-attention heads within the Transformer layer. The latter pays attention to the token assembly across Transformer layers. Meanwhile, we introduce the constraint of center loss to pull intra-class samples’ compactness and push that of inter-class samples. Extensive experiments show that Dual-TR performs on par with the state-of-the-art methods on four public benchmarks, including CUB-200-2011, NABirds, iNaturalist2017, and Stanford Dogs. The comprehensive ablation studies further demonstrate the effectiveness of architectural design choices.

How to train your pre-trained GAN models

Generative Adversarial Networks (GAN) show excellent performance in various problems of computer vision, computer graphics, and machine learning, but require large amounts of data and huge computational resources. There is also the issue of unstable training. If the generator and discriminator diverge during the training process, the GAN is subsequently difficult to converge. In order to tackle these problems, various transfer learning methods have been introduced; however, mode collapse, which is a form of overfitting, often arises. Moreover, there were limitations in learning the distribution of the training data. In this paper, we provide a comprehensive review of the latest transfer learning methods as a solution to the problem, propose the most effective method of fixing some layers of the generator and discriminator, and discuss future prospects. The model to be used for the experiment is StyleGAN, and the performance evaluation uses Fréchet Inception Distance (FID), coverage, and density. Results of the experiment revealed that the proposed method did not overfit. The model was able to learn the distribution of the training data relatively well compared to the previously proposed methods. Moreover, it outperformed existing methods at the Stanford Cars, Stanford Dogs, Oxford Flower, Caltech-256, CUB-200–2011, and Insect-30 datasets.

Fine-Grained Visual Categorization: A Spatial–Frequency Feature Fusion Perspective

Fine-grained visual categorization is a challenging issue owing to high intra-class and low inter-class variances. Classical approaches rely on pre-trained models or many fine annotations. In this paper, we observe that spatial and frequency information provides distinct image views, and propose a new spatial–frequency feature fusion (SFFF) perspective to handle this challenging issue. Specifically, we design a heterogeneous feature extraction loss function, construct a global and local fusion SFFF network, and propose an importance-sparsity selection strategy. For feature extraction, we focus on the frequency domain feature learning network, extract fine-grained features, and achieve feature complementarity. For feature selection, we propose importance ranking and sparse regularity to constrain spatial–frequency features. For feature fusion, we design a spatial–frequency loss and an inter-layer switching strategy to achieve local-global collaboration. Comparative experiments were performed on popular fine-grained datasets and classic datasets such as CUB200-2011, Stanford Cars, Stanford Dogs, FGVC-Aircraft, and CIFAR100. The effectiveness and outstanding performance of SFFF are confirmed by comparisons with more than 40 state-of-the-art fine-grained categorization methods. Ablation studies and visualizations are provided to facilitate an understanding of our approach.

Hybrid Granularities Transformer for Fine-Grained Image Recognition

Many current approaches for image classification concentrate solely on the most prominent features within an image, but in fine-grained image recognition, even subtle features can play a significant role in model classification. In addition, the large variations in the same class and small differences between different categories that are unique to fine-grained image recognition pose a great challenge for the model to extract discriminative features between different categories. Therefore, we aim to present two lightweight modules to help the network discover more detailed information in this paper. (1) Patches Hidden Integrator (PHI) module randomly selects patches from images and replaces them with patches from other images of the same class. It allows the network to glean diverse discriminative region information and prevent over-reliance on a single feature, which can lead to misclassification. Additionally, it does not increase the training time. (2) Consistency Feature Learning (CFL) aggregates patch tokens from the last layer, mining local feature information and fusing it with the class token for classification. CFL also utilizes inconsistency loss to force the network to learn common features in both tokens, thereby guiding the network to focus on salient regions. We conducted experiments on three datasets, CUB-200-2011, Stanford Dogs, and Oxford 102 Flowers. We achieved experimental results of 91.6%, 92.7%, and 99.5%, respectively, achieving a competitive performance compared to other works.

Loop and distillation: Attention weights fusion transformer for fine‐grained representation

AbstractLearning subtle discriminative feature representation plays a significant role in Fine‐Grained Visual Categorisation (FGVC). The vision transformer (ViT) achieves promising performance in the traditional image classification filed due to its multi‐head self‐attention mechanism. Unfortunately, ViT cannot effectively capture critical feature regions for FGVC due to only focusing on classification token and adopting the strategy of one‐time image input. Besides, the advantage of attention weights fusion is not applied to ViT. To promote the performance of capturing vital regions for FGVC, the authors propose a novel model named RDTrans, which proposes discriminative region with top priority in a recurrent learning way. Specifically, proposed vital regions at each scale will be cropped and amplified as the next input parameters to finally locate the most discriminative region. Furthermore, a distillation learning method is employed to provide better supervision for elevating the generalisation ability. Concurrently, RDTrans can be easily trained end‐to‐end in a weakly‐supervised learning way. Extensive experiments demonstrate that RDTrans yields state‐of‐the‐art performance on four widely used fine‐grained benchmarks, including CUB‐200‐2011, Stanford Cars, Stanford Dogs, and iNat2017.