Fine-grained Classification Task Research Articles

TPTE: Text-Guided Patch Token Exploitation for Unsupervised Fine-Grained Representation Learning

Recent advances in pre-trained vision-language models have successfully boosted the performance of unsupervised image representation in many vision tasks. Most of existing works focus on learning global visual features with Transformers and neglect detailed local cues, leading to suboptimal performance in fine-grained vision tasks. In this article, we propose a text-guided patch token exploitation framework to enhance the discriminative power of unsupervised representation by exploiting more detailed local features. Our text-guided decoder extracts local features with the guidance of texts or learned prompts describing discriminative object parts. We hence introduce a local-global relation distillation loss to promote the joint optimization of local and global features. The proposed method allows to flexibly extract either global or global-local features as the image representation. It significantly outperforms previous methods in fine-grained image retrieval and base-to-new fine-grained classification tasks. For instance, our Recall@1 metric surpasses the recent unsupervised retrieval method STML by 6.0% on the SOP dataset. The code is publicly available at https://github.com/maosnhehe/TPTE .

Accurate, automated classification of radiographic knee osteoarthritis severity using a novel method of deep learning: Plug-in modules

BackgroundFine-grained classification deals with data with a large degree of similarity, such as cat or bird species, and similarly, knee osteoarthritis severity classification [Kellgren–Lawrence (KL) grading] is one such fine-grained classification task. Recently, a plug-in module (PIM) that can be integrated into convolutional neural-network-based or transformer-based networks has been shown to provide strong discriminative regions for fine-grained classification, with results that outperformed the previous deep learning models. PIM utilizes each pixel of an image as an independent feature and can subsequently better classify images with minor differences. It was hypothesized that, as a fine-grained classification task, knee osteoarthritis severity may be classified well using PIMs. The aim of the study was to develop this automated knee osteoarthritis classification model.MethodsA deep learning model that classifies knee osteoarthritis severity of a radiograph was developed utilizing PIMs. A retrospective analysis on prospectively collected data was performed. The model was trained and developed using the Osteoarthritis Initiative dataset and was subsequently tested on an independent dataset, the Multicenter Osteoarthritis Study (test set size: 17,040). The final deep learning model was designed through an ensemble of four different PIMs.ResultsThe accuracy of the model was 84%, 43%, 70%, 81%, and 96% for KL grade 0, 1, 2, 3, and 4, respectively, with an overall accuracy of 75.7%.ConclusionsThe ensemble of PIMs could classify knee osteoarthritis severity using simple radiographs with a fine accuracy. Although improvements will be needed in the future, the model has been proven to have the potential to be clinically useful.

Orchid2024: A cultivar-level dataset and methodology for fine-grained classification of Chinese Cymbidium Orchids

BackgroundChinese Cymbidium orchids, cherished for their deep-rooted cultural significance and significant economic value in China, have spawned a rich tapestry of cultivars. However, these orchid cultivars are facing challenges from insufficient cultivation practices and antiquated techniques, including cultivar misclassification, complex identification, and the proliferation of counterfeit products. Current commercial techniques and academic research primarily emphasize species identification of orchids, rather than delving into that of orchid cultivars within species.ResultsTo bridge this gap, the authors dedicated over a year to collecting a cultivar image dataset for Chinese Cymbidium orchids named Orchid2024. This dataset contains over 150,000 images spanning 1,275 different categories, involving visits to 20 cities across 12 provincial administrative regions in China to gather pertinent data. Subsequently, we introduced various visual parameter-efficient fine-tuning (PEFT) methods to expedite model development, achieving the highest top-1 accuracy of 86.14% and top-5 accuracy of 95.44%.ConclusionExperimental results demonstrate the complexity of the dataset while highlighting the considerable promise of PEFT methods within flower image classification. We believe that our work not only provides a practical tool for orchid researchers, growers and market participants, but also provides a unique and valuable resource for further exploring fine-grained image classification tasks. The dataset and code are available at https://github.com/pengyingshu/Orchid2024.

Pest-ConFormer: A hybrid CNN-Transformer architecture for large-scale multi-class crop pest recognition

Crop pests are acknowledged as the major factors in reducing the yield and quality of agricultural production worldwide. It is an urgent necessity to recognize crop pests accurately to protect the crop in the early stage to reduce the loss for the agricultural economy. Due to the ecological characteristics of the crop pests and the complex background in fields, the crop pests show high inter-class similarity and significant intra-class variation in external morphology appearance, which makes current recognition methods suffer from low classification accuracy and poor generalization ability in complex natural environment recognition tasks. To tackle this problem, a hybrid convolutional neural network and transformer-based model, namely Pest-ConFormer, featured with multi-scale weakly supervised feature selection mechanisms is proposed, which has shown excellent multi-scale discriminative feature extraction in fine-grained visual classification (FGVC) tasks. This method employs a hybrid convolution-transformer encoder architecture pre-training in a self-supervised masked autoencoder manner as a backbone to learn pests’ highly discriminative features across various scales. Next, a dual-path feature aggregation structure with a top-down FPN-like feature pathway and a bottom-up PANet-like feature pathway based on attention mechanisms is designed to learn high-level global context information and low-level local detailed feature representation. Thirdly, a fine-grained classification module using weakly supervised learning is introduced to select the discriminative feature points in different pyramidal levels. Then, these feature points are fed into a graph convolutional network to accomplish classification. Several experiments are conducted on the large-scale multi-class IP102 benchmark dataset, and the proposed method achieves an accuracy of 77.81 % regarding crop pest recognition. The experimental results indicate that our approach outperforms other state-of-the-art methods by nearly 2 percent points, demonstrating that the proposed hybrid architecture with dual-path feature aggregation and fine-grained classification modules can be more effective in the crop pest recognition field than CNN-based methods and can be deployed in the practical natural environment. The source code will be available at https://github.com/mwfang/pestconformer.

DSNet: dual-stream network for fine-grained ship classification in optical remote sensing images

ABSTRACT The fine-grained ship classification (FGSC) task in remote sensing aims to distinguish subordinate ship categories. Due to the high similarity among different subcategories and the significant variations in pose, scale, rotation, and color within the same subcategory in the fine-grained ship datasets, the FGSC task is extremely challenging. To alleviate these issues, a novel approach, named Dual-Stream Network (DSNet), is proposed in this paper for FGSC. DSNet consists of two sub-branches: the Local Residual Branch (LRB) and the Global Representation Branch (GRB). Specifically, the LRB effectively extracts the most discriminative local fine-grained features of the target using Full-scale Skip Connections (FSC) and Stripe Pooling Add (SPA) modules. The GRB captures global contextual information through Swin-Transformer (Swin-T), establishing remote dependencies of the target and extracting global coarse-grained features. Furthermore, a Lightweight Fusion (LF) block is designed in the feature fusion module to combine the feature information from the two branches. This ensures that the discriminative global and local information complement each other without overwhelming one another, thereby effectively improving the performance of the model. Extensive experiments and analyses demonstrate that our proposed method achieves state-of-the-art performance on two mainstream FGSC datasets in remote sensing.

Fine-Grained Few-Shot Image Classification Based on Feature Dual Reconstruction

Fine-grained few-shot image classification is a popular research area in deep learning. The main goal is to identify subcategories within a broader category using a limited number of samples. The challenge stems from the high intra-class variability and low inter-class variability of fine-grained images, which often hamper classification performance. To overcome this, we propose a fine-grained few-shot image classification algorithm based on bidirectional feature reconstruction. This algorithm introduces a Mixed Residual Attention Block (MRA Block), combining channel attention and window-based self-attention to capture local details in images. Additionally, the Dual Reconstruction Feature Fusion (DRFF) module is designed to enhance the model’s adaptability to both inter-class and intra-class variations by integrating features of different scales across layers. Cosine similarity networks are employed for similarity measurement, enabling precise predictions. The experiments demonstrate that the proposed method achieves classification accuracies of 96.99%, 98.53%, and 89.78% on the CUB-200-2011, Stanford Cars, and Stanford Dogs datasets, respectively, confirming the method’s efficacy in fine-grained classification tasks.

WhisPAr: Transferring pre-trained audio models to fine-grained classification via Prompt and Adapter

With the rise of Transformer as the backbone networks, pre-trained models have gradually become a hot spot in deep learning–related fields and play an important role in increasing research and production activities. However, current pre-trained models still have the problems of coarse knowledge granularity and poor zero/few-shot performance and cannot be well applied to fine-grained downstream tasks. In audio processing, considerable structure biases exist between the upstream and downstream tasks, making it difficult to realize cross-task transfer learning. Inspired by the success of Prompt and Adapter techniques in natural language processing (NLP) and computer vision (CV), we propose a cross-task transfer method with good generality to address these problems. We contribute the following: (1) a novel transfer method for audio pre-trained models via deep Prompt and Adapter structures to improve model adaptation to downstream tasks, which is effective and generally applicable and has potential to be used in more powerful audio pre-trained models in the future; (2) a fine-grained classification model implementation based on the abovementioned method WhisPAr, which is experimentally shown to outperform the baseline models in various fine-grained classification tasks (0.73 vs. 0.84, 0.76 vs. 0.81, and 0.47 vs. 0.68); and (3) an in-depth exploration of the mechanisms and laws of Prompt and Adapter. These contributions are expected to drive future research on audio processing and transfer learning, among others.

An Optimization Method for Lightweight Rock Classification Models: Transferred Rich Fine-Grained Knowledge.

Rock image classification represents a challenging fine-grained image classification task characterized by subtle differences among closely related rock categories. Current contrastive learning methods prevalently utilized in fine-grained image classification restrict the model's capacity to discern critical features contrastively from image pairs, and are typically too large for deployment on mobile devices used for in situ rock identification. In this work, we introduce an innovative and compact model generation framework anchored by the design of a Feature Positioning Comparison Network (FPCN). The FPCN facilitates interaction between feature vectors from localized regions within image pairs, capturing both shared and distinctive features. Further, it accommodates the variable scales of objects depicted in images, which correspond to differing quantities of inherent object information, directing the network's attention to additional contextual details based on object size variability. Leveraging knowledge distillation, the architecture is streamlined, with a focus on nuanced information at activation boundaries to master the precise fine-grained decision boundaries, thereby enhancing the small model's accuracy. Empirical evidence demonstrates that our proposed method based on FPCN improves the classification accuracy mobile lightweight models by nearly 2% while maintaining the same time and space consumption.

Remote Sensing Image Harmonization Method for Fine-Grained Ship Classification

Target recognition and fine-grained ship classification in remote sensing face challenges of high inter-class similarity and sample scarcity. A transfer fusion-based ship image harmonization algorithm is proposed to overcome these challenges. This algorithm designs a feature transfer fusion strategy based on the combination of a region-aware instantiation and attention mechanism. Adversarial learning is implemented through an image harmony generator and discriminator module to generate realistic remote sensing ship harmony images. Furthermore, the domain encoder and domain discriminator modules are responsible for extracting feature representations of the foreground and background, and further align the ship foreground with remote sensing ocean background features through feature discrimination. Compared with other advanced image conversion techniques, our algorithm delivers more realistic visuals, improving classification accuracy for six ship types by 3% and twelve types by 2.94%, outperforming Sim2RealNet. Finally, a mixed dataset containing data augmentation and harmonizing samples and real data was proposed for the fine-grained classification task of remote sensing ships. Evaluation experiments were conducted on eight typical fine-grained classification algorithms, and the accuracy of the fine-grained classification for all categories of ships was analyzed. The experimental results show that the mixed dataset proposed in this paper effectively alleviates the long-tail problem in real datasets, and the proposed remote sensing ship data augmentation framework performs better than state-of-the-art data augmentation methods in fine-grained ship classification tasks.

Image Detection Network Based on Enhanced Small Target Recognition Details and Its Application in Fine Granularity

Image detection technology is of paramount importance across various fields. This significance is not only seen in general images with everyday scenes but also holds substantial research value in the field of remote sensing. Remote sensing images involve capturing images from aircraft or satellites. These images typically feature diverse scenes, large image formats, and varying imaging heights, thus leading to numerous small-sized targets in the captured images. Accurately identifying these small targets, which may occupy only a few pixels, is a challenging and active research area. Current methods mainly fall into two categories: enhancing small target features by improving resolution and increasing the number of small targets to bolster training datasets. However, these approaches often fail to address the core distinguishing features of small targets in the original images, thus resulting in suboptimal performance in fine-grained classification tasks. To address this situation, we propose a new network structure DDU (Downsample Difference Upsample), which is based on differential and resolution changing methods in the Neck layer of deep learning networks to enhance the recognition features of small targets, thus further improving the feature richness of recognition and effectively solving the problem of low accuracy in small target object recognition. At the same time, in order to take into account the recognition effect of targets of other sizes in the image, a new attention mechanism called PNOC (protecting the number of channels) is proposed, which integrates small target features and universal object features without losing the number of channels, thereby increasing the accuracy of recognition. And experimental verification was conducted on the PASCAL-VOC dataset. At the same time, it was applied to the testing of the fine-grained MAR20 dataset and found that the performance was better than other classic algorithms. At the same time, because the proposed framework belongs to a one-stage detection method, it has good engineering applicability and scalability, and universality in scientific research applications are good. Through comparative experiments, it was found that our algorithm improved the performance of the mAP by 0.7% compared to the original YOLOv8 algorithm.

Classification of Rainfall Intensity and Cloud Type from Dash Cam Images Using Feature Removal by Masking

Weather Report is an initiative from Weathernews Inc. to obtain sky images and current weather conditions from the users of its weather app. This approach can provide supplementary weather information to radar observations and can potentially improve the accuracy of forecasts However, since the time and location of the contributed images are limited, gathering data from different sources is also necessary. This study proposes a system that automatically submits weather reports using a dash cam with communication capabilities and image recognition technology. This system aims to provide detailed weather information by classifying rainfall intensities and cloud formations from images captured via dash cams. In models for fine-grained image classification tasks, there are very subtle differences between some classes and only a few samples per class. Therefore, they tend to include irrelevant details, such as the background, during training, leading to bias. One solution is to remove useless features from images by masking them using semantic segmentation, and then train each masked dataset using EfficientNet, evaluating the resulting accuracy. In the classification of rainfall intensity, the model utilizing the features of the entire image achieved up to 92.61% accuracy, which is 2.84% higher compared to the model trained specifically on road features. This outcome suggests the significance of considering information from the whole image to determine rainfall intensity. Furthermore, analysis using the Grad-CAM visualization technique revealed that classifiers trained on masked dash cam images particularly focused on car headlights when classifying the rainfall intensity. For cloud type classification, the model focusing solely on the sky region attained an accuracy of 68.61%, which is 3.16% higher than that of the model trained on the entire image. This indicates that concentrating on the features of clouds and the sky enables more accurate classification and that eliminating irrelevant areas reduces misclassifications.

Weight Decay With Tailored Adam on Scale-Invariant Weights for Better Generalization.

Weight decay (WD) is a fundamental and practical regularization technique in improving generalization of current deep learning models. However, it is observed that the WD does not work effectively for an adaptive optimization algorithm (such as Adam), as it works for SGD. Specifically, the solution found by Adam with the WD often generalizes unsatisfactorily. Though efforts have been made to mitigate this issue, the reason for such deficiency is still vague. In this article, we first show that when using the Adam optimizer, the weight norm increases very fast along with the training procedure, which is in contrast to SGD where the weight norm increases relatively slower and tends to converge. The fast increase of weight norm is adverse to WD; in consequence, the Adam optimizer will lose efficacy in finding solution that generalizes well. To resolve this problem, we propose to tailor Adam by introducing a regularization term on the adaptive learning rate, such that it is friendly to WD. Meanwhile, we introduce first moment on the WD to further enhance the regularization effect. We show that the proposed method is able to find solution with small norm and generalizes better than SGD. We test the proposed method on general image classification and fine-grained image classification tasks with different networks. Experimental results on all these cases substantiate the effectiveness of the proposed method in help improving the generalization. Specifically, the proposed method improves the test accuracy of Adam by a large margin and even improves the performance of SGD by 0.84% on CIFAR 10 and 1.03 % on CIFAR 100 with ResNet-50. The code of this article is public available at xxx.

Polymorphic Clustering and Approximate Masking Framework for Fine-Grained Insect Image Classification

Insect diversity monitoring is crucial for biological pest control in agriculture and forestry. Modern monitoring of insect species relies heavily on fine-grained image classification models. Fine-grained image classification faces challenges such as small inter-class differences and large intra-class variances, which are even more pronounced in insect scenes where insect species often exhibit significant morphological differences across multiple life stages. To address these challenges, we introduce segmentation and clustering operations into the image classification task and design a novel network model training framework for fine-grained classification of insect images using multi-modality clustering and approximate mask methods, named PCAM-Frame. In the first stage of the framework, we adopt the Polymorphic Clustering Module, and segmentation and clustering operations are employed to distinguish various morphologies of insects at different life stages, allowing the model to differentiate between samples at different life stages during training. The second stage consists of a feature extraction network, called Basenet, which can be any mainstream network that performs well in fine-grained image classification tasks, aiming to provide pre-classification confidence for the next stage. In the third stage, we apply the Approximate Masking Module to mask the common attention regions of the most likely classes and continuously adjust the convergence direction of the model during training using a Deviation Loss function. We apply PCAM-Frame with multiple classification networks as the Basenet in the second stage and conduct extensive experiments on the Insecta dataset of iNaturalist 2017 and IP102 dataset, achieving improvements of 2.2% and 1.4%, respectively. Generalization experiments on other fine-grained image classification datasets such as CUB200-2011 and Stanford Dogs also demonstrate positive effects. These experiments validate the pertinence and effectiveness of our framework PCAM-Frame in fine-grained image classification tasks under complex conditions, particularly in insect scenes.

A dual contrastive learning-based graph convolutional network with syntax label enhancement for aspect-based sentiment classification

Introduction: Aspect-based sentiment classification is a fine-grained sentiment classification task. State-of-the-art approaches in this field leverage graph neural networks to integrate sentence syntax dependency. However, current methods fail to exploit the data augmentation in encoding and ignore the syntactic relation in sentiment delivery.Methods: In this work, we propose a novel graph neural network-based architecture with dual contrastive learning and syntax label enhancement. Specifically, a contrastive learning-based contextual encoder is designed, integrating sentiment information for semantics learning. Moreover, a weighted label-enhanced syntactic graph neural network is established to use both the syntactic relation and syntax dependency, which optimizes the syntactic weight between words. A syntactic triplet between words is generated. A syntax label-based contrastive learning scheme is developed to map the triplets into a unified feature space for syntactic information learning.Results: Experiments on five publicly available datasets show that our model substantially outperforms the baseline methods.Discussion: As such, the proposed method shows its effectiveness in aspect-based sentiment classification tasks.

Aspect-Based Sentiment Analysis with Explicit Sentiment Augmentations

Aspect-based sentiment analysis (ABSA), a fine-grained sentiment classification task, has received much attention recently. Many works investigate sentiment information through opinion words, such as "good'' and "bad''. However, implicit sentiment data widely exists in the ABSA dataset, whose sentiment polarity is hard to determine due to the lack of distinct opinion words. To deal with implicit sentiment, this paper proposes an ABSA method that integrates explicit sentiment augmentations (ABSA-ESA) to add more sentiment clues. We propose an ABSA-specific explicit sentiment generation method to create such augmentations. Specifically, we post-train T5 by rule-based data and employ three strategies to constrain the sentiment polarity and aspect term of the generated augmentations. We employ Syntax Distance Weighting and Unlikelihood Contrastive Regularization in the training procedure to guide the model to generate the explicit opinion words with the same polarity as the input sentence. Meanwhile, we utilize the Constrained Beam Search to ensure the augmentations are aspect-related. We test ABSA-ESA on two ABSA benchmarks. The results show that ABSA-ESA outperforms the SOTA baselines on implicit and explicit sentiment accuracy.

Cross-Layer and Cross-Sample Feature Optimization Network for Few-Shot Fine-Grained Image Classification

Recently, a number of Few-Shot Fine-Grained Image Classification (FS-FGIC) methods have been proposed, but they primarily focus on better fine-grained feature extraction while overlooking two important issues. The first one is how to extract discriminative features for Fine-Grained Image Classification tasks while reducing trivial and non-generalizable sample level noise introduced in this procedure, to overcome the over-fitting problem under the setting of Few-Shot Learning. The second one is how to achieve satisfying feature matching between limited support and query samples with variable spatial positions and angles. To address these issues, we propose a novel Cross-layer and Cross-sample feature optimization Network for FS-FGIC, C2-Net for short. The proposed method consists of two main modules: Cross-Layer Feature Refinement (CLFR) module and Cross-Sample Feature Adjustment (CSFA) module. The CLFR module further refines the extracted features while integrating outputs from multiple layers to suppress sample-level feature noise interference. Additionally, the CSFA module addresses the feature mismatch between query and support samples through both channel activation and position matching operations. Extensive experiments have been conducted on five fine-grained benchmark datasets, and the results show that the C2-Net outperforms other state-of-the-art methods by a significant margin in most cases. Our code is available at: https://github.com/zenith0923/C2-Net.

Global–local feature learning for fine-grained food classification based on Swin Transformer

Separable object parts, such as the head and tail in a bird, are vital for fine-grained visual classifications. For those objects without separable parts, the classification task relies only on local and global textural image features. Although the Swin Transformer architecture was proposed to efficiently capture both local and global visual features, it still exhibits a bias towards global features. Therefore, our goal is to enhance the local feature learning capability of the Swin Transformer by adding four new modules of the Local Feature Extraction Network (L-FEN), Convolution Patch-Merging (CP), Multi-Path (MP), and Multi-View (MV). The L-FEN enhances the Swin transformer with the improved local feature capture. The CP is a localized and hierarchical adaptation of the Swin’s Patch Merging technique. The MP method integrates features across various Swin stages to accentuate local details. Meanwhile, the MV Swin transformer block supersedes traditional Swin blocks with those incorporating varied receptive fields, ensuring a broader scope of local feature capture. Our enhanced architecture, named Global–Local Swin Transformer (GL-Swin), is applied to solve a fine-grained food classification task. On three major food datasets: ISIA Food-500 UEC Food-256, and Food-101, our GL-Swin achieved accuracies of 66.75%, 85.78%, and 92.93% respectively, consistently outperforming other leading methods.

ACANet: A Fine-grained Image Classification Optimization Method Based on Convolution and Attention Fusion

<p>The key to solve the problem of fine-grained image classification is to find the differentiation regions related to fine-grained features. In this paper, we try to add new network components to the original network and adjust various parameters to try to propose a new fine-grained image classification network. We propose a fine-grained image classification network based on the fusion of asymmetric convolution, convolution and self-attention mechanisms. Firstly, an enhanced module using asymmetric convolution to assist classical convolution proposed to help convolution learn deep features. Secondly, according to the common points of convolution and self-attention mechanism, we invented a fusion module of convolution and self-attention mechanism to improve the learning ability of the network.We integrate these two modules into the residual network and invent a new residual network .Finally, according to the experience, we design a new downsampling layer to adapt to the new component of the attention mechanism and improve the performance of the model. The experiment test on three publicly available datasets, and three methods for comparison. The results show that the new structure can effectively complete the task of fine-grained image classification, and the classification accuracy of different methods and different datasets are significantly improved.</p> <p> </p>

Enhancement of Multiclass Dermoscopic Image Classification with a Fusion Model and Fine-Tuned Deep Neural Networks

Since malignancy of the skin is a frequent and sometimes fatal illness, prompt and precise detection is essential to successful treatment. Dermoscopic imaging is now emerging as a helpful tool for dermatologists in aiding with the identification of various types of skin lesions and offering timely treatments. Due to varied image conditions, data unbalancing, and fine-grained pattern classification task are is challenging in skin cancer detection. Dermoscopic images vary greatly in quality, illumination, and orientation, making it challenging for models to efficiently generalize across varied image conditions. To obtain accurate and reliable skin cancer detection, such challenges require the utilization of sophisticated deep learning techniques and rigorous model development. In multiple classes skin malignancy classification, convolutional neural networks outperform dermatology professionals, according to recent research. In the proposed system, firstly a preprocessing technique has been developed that includes hair and noise removal, data augmentation, and image resizing with model parameters. Then training has been performed using transfer learning on the fusion of DenseNet121 and ResNet50V2 model on the HAM10000 dataset. The proposed technique's performance is assessed using measures such as sensitivity, f-1 score, specificity, and accuracy. For multi-class skin lesion classification, the suggested technique achieves a precision of 97.88%, F-Score of 98%, sensitivity of 99%, accuracy of 97.08%, and Mathew's correlation coefficient (MCC) of 96.4%, it executes superior to cutting-edge techniques.

Fine-Grained Lesion Classification Framework for Early Auxiliary Diagnosis.

The deep neural networks are envisaged for the early disease diagnosis from medical images. However, in the early stage of the disease, the medical images of patients and healthy people have only subtle visual differences. Distinguishing the medical images for early diagnosis belongs to the Fine-Grained Visual Classification (FGVC) task. Many recent works are based on a standard FGVC learning paradigm: locate the discriminative regions first and then classify by fusing the information of these regions. However, it is still not enough for medical images. Because the shape and size of the lesions are variable, and the relationship between lesions and the background is complex. In order to solve these problems, we propose a fine-grained lesion classification framework for early auxiliary diagnosis. We first locate and extract multiple lesions with different sizes and shapes from the original image and then fuse the feature of lesion and background based on attention mechanism. As shown by experiment results in two real-world clinical data sets, our model can locate accurately and perform better.