Discriminative Local Features Research Articles

Transformer-Based Weakly Supervised Learning for Whole Slide Lung Cancer Image Classification.

Image analysis can play an important role in supporting histopathological diagnoses of lung cancer, with deep learning methods already achieving remarkable results. However, due to the large scale of whole-slide images (WSIs), creating manual pixel-wise annotations from expert pathologists is expensive and time-consuming. In addition, the heterogeneity of tumors and similarities in the morphological phenotype of tumor subtypes have caused inter-observer variability in annotations, which limits optimal performance. Effective use of weak labels could potentially alleviate these issues. In this paper, we propose a two-stage transformer-based weakly supervised learning framework called Simple Shuffle-Remix Vision Transformer (SSRViT). Firstly, we introduce a Shuffle-Remix Vision Transformer (SRViT) to retrieve discriminative local tokens and extract effective representative features. Then, the token features are selected and aggregated to generate sparse representations of WSIs, which are fed into a simple transformer-based classifier (SViT) for slide-level prediction. Experimental results demonstrate that the performance of our proposed SSRViT is significantly improved compared with other state-of-the-art methods in discriminating between adenocarcinoma, pulmonary sclerosing pneumocytoma and normal lung tissue (accuracy of 96.9% and AUC of 99.6%).

Maize Leaf Disease Recognition Based on Improved Convolutional Neural Network ShuffleNetV2.

The occurrence of maize diseases is frequent but challenging to manage. Traditional identification methods have low accuracy and complex model structures with numerous parameters, making them difficult to implement on mobile devices. To address these challenges, this paper proposes a corn leaf disease recognition model SNMPF based on convolutional neural network ShuffleNetV2. In the down-sampling module of the ShuffleNet model, the max pooling layer replaces the deep convolutional layer to perform down-sampling. This improvement helps to extract key features from images, reduce the overfitting of the model, and improve the model's generalization ability. In addition, to enhance the model's ability to express features in complex backgrounds, the Sim AM attention mechanism was introduced. This mechanism enables the model to adaptively adjust focus and pay more attention to local discriminative features. The results on a maize disease image dataset demonstrate that the SNMPF model achieves a recognition accuracy of 98.40%, representing a 4.1 percentage point improvement over the original model, while its size is only 1.56 MB. Compared with existing convolutional neural network models such as EfficientNet, MobileViT, EfficientNetV2, RegNet, and DenseNet, this model offers higher accuracy and a more compact size. As a result, it can automatically detect and classify maize leaf diseases under natural field conditions, boasting high-precision recognition capabilities. Its accurate identification results provide scientific guidance for preventing corn leaf disease and promote the development of precision agriculture.

Multi-granularity tooth analysis via YOLO-based object detection models for effective tooth detection and classification

Accurate detection and classification of teeth is the first step in dental disease diagnosis. However, the same class of tooth exhibits significant variations in surface appearance. Moreover, the complex geometrical structure poses challenges in learning discriminative features among the different tooth classes. Due to these complex features, tooth classification is one of the challenging research domains in deep learning. To address the aforementioned issues, the presented study proposes discriminative local feature extraction at different granular levels using YOLO models. However, this necessitates a granular intra-oral image dataset. To facilitate this requirement, a dataset at three granular levels (two, four, and seven teeth classes) is developed. YOLOv5, YOLOv6, and YOLOv7 models were trained using 2,790 images. The results indicate superior performance of YOLOv6 for two-class classification problems. The model generated a mean average precision (mAP) value of 94%. However, as the granularity level is increased, the performance of YOLO models decreases. For, four and seven-class classification problems, the highest mAP value of 87% and 79% was achieved by YOLOv5 respectively. The results indicate that different levels of granularity play an important role in tooth detection and classification. The YOLO’s performance gradually decreased as the granularity decreased especially at the finest granular level.

TagCLIP: A Local-to-Global Framework to Enhance Open-Vocabulary Multi-Label Classification of CLIP without Training

Contrastive Language-Image Pre-training (CLIP) has demonstrated impressive capabilities in open-vocabulary classification. The class token in the image encoder is trained to capture the global features to distinguish different text descriptions supervised by contrastive loss, making it highly effective for single-label classification. However, it shows poor performance on multi-label datasets because the global feature tends to be dominated by the most prominent class and the contrastive nature of softmax operation aggravates it. In this study, we observe that the multi-label classification results heavily rely on discriminative local features but are overlooked by CLIP. As a result, we dissect the preservation of patch-wise spatial information in CLIP and proposed a local-to-global framework to obtain image tags. It comprises three steps: (1) patch-level classification to obtain coarse scores; (2) dual-masking attention refinement (DMAR) module to refine the coarse scores; (3) class-wise reidentification (CWR) module to remedy predictions from a global perspective. This framework is solely based on frozen CLIP and significantly enhances its multi-label classification performance on various benchmarks without dataset-specific training. Besides, to comprehensively assess the quality and practicality of generated tags, we extend their application to the downstream task, i.e., weakly supervised semantic segmentation (WSSS) with generated tags as image-level pseudo labels. Experiments demonstrate that this classify-then-segment paradigm dramatically outperforms other annotation-free segmentation methods and validates the effectiveness of generated tags. Our code is available at https://github.com/linyq2117/TagCLIP.

Semantics-enhanced discriminative descriptor learning for LiDAR-based place recognition

LiDAR-based place recognition (LPR) aims to localize autonomous vehicles and mobile robots relative to pre-built maps or retrieve previously visited places. However, the complexity of real-world scenes and changes in viewpoint are significant challenges for place recognition. As high-level information, semantics makes it easier to distinguish geometrically similar scene situations. Unlike most existing methods that rely solely on a single type of information (geometric or semantic) to construct scene descriptors, we consider the complementary nature of the semantic and geometric information and propose a semantics-enhanced discriminative feature learning method for LPR. Specifically, we first develop a Multi-layer Fusion Feature Extraction Network (MFFEN) based on the transformer encoder to hierarchically fuse local geometric and semantic information and utilize contextual information for extracting discriminative local features. To obtain semantic information, we introduce the dynamic graph convolution network to extract local semantic features with local relations. In addition, to weaken the interference of redundancy and dynamic objects in the scene, we design a semantics-guided local attention network (SLAN) to focus on salient local features that are helpful for recognizing scenes, thereby enhancing the descriptive ability of the global descriptor. Extensive experiments on public datasets KITTI and KITTI-360 demonstrate that the proposed method performs better than recent LiDAR-based methods on the 3D place recognition task. For instance, it achieves the mean F1max score of 96.9% on the KITTI dataset, surpassing the strongest prior model by 2.7%.

Deeply Unsupervised Patch Re-Identification for Pre-Training Object Detectors.

Unsupervised pre-training aims at learning transferable features that are beneficial for downstream tasks. However, most state-of-the-art unsupervised methods concentrate on learning global representations for image-level classification tasks instead of discriminative local region representations, which limits their transferability to region-level downstream tasks, such as object detection. To improve the transferability of pre-trained features to object detection, we present Deeply Unsupervised Patch Re-ID (DUPR), a simple yet effective method for unsupervised visual representation learning. The patch Re-ID task treats individual patch as a pseudo-identity and contrastively learns its correspondence in two views, enabling us to obtain discriminative local features for object detection. Then the proposed patch Re-ID is performed in a deeply unsupervised manner, appealing to object detection, which usually requires multi-level feature maps. Extensive experiments demonstrate that DUPR outperforms state-of-the-art unsupervised pre-trainings and even the ImageNet supervised pre-training on various downstream tasks related to object detection.

Focus and imagine: Occlusion suppression and repairing transformer for occluded person re-identification

Occlusion scenarios pose a great challenge to person re-identification (ReID) task because various occlusions may weaken the discriminative features and introduce interference. Recently, Transformer-based networks, which can aggregate features of all the image patches to construct global features adaptively, have shown advantages in occluded person ReID. Existing methods mainly adopted Transformer as a feature extractor and enhanced local features from the output of the Transformer encoder. However, during the processing of self-attention blocks, disturbing features from occlusions may be diffused into all the tokens, making it difficult to construct effective local features. Therefore, we consider predicting the occlusion situation of images before feature extraction and guiding the Transformer encoder to focus on visible regions, suppressing interference from occlusion. Furthermore, we propose to imagine the partial target under occlusion and reconstruct pseudo-holistic features for more robust retrieval. To this end, the Occlusion Suppression and Repairing Transformer (OSRTrans) is proposed. First, we use a self-supervised occlusion predictor to predict occlusion scores of image patches. Then the Occlusion Suppression Encoder (OSE), guided by occlusion predictions, suppresses the interference from occlusion regions and constructs a global feature. Finally, inspired by contrastive learning, the Feature Repairing Head (FRH) is proposed to reconstruct pseudo-holistic features. Our method enhances model’s ability of extracting discriminative local features, and achieve the state-of-the-art performance on occluded person ReID benchmarks, e.g., Rank-1 of 72.9% on Occluded-DukeMTMC.

Dual-stream feature fusion network for person re-identification

Person re-identification (Re-ID) has made significant progress in recent years. However, it still faces numerous challenges in real scenarios. Although researchers have proposed various solutions, the issue of similar clothing colors remains an obstacle in improving the performance of person re-identification. To solve this issue, we propose a dual-stream feature fusion network (DSFF-Net) to extract discriminative features from pedestrian images in two color spaces. Specifically, a dual-stream network is designed to extract RGB global features, grayscale global features, and local features of pedestrian images to increase the richness of pedestrian representations. A channel attention module is designed to direct the network to focus on the salient features of pedestrians. An embedding mixed pooling is designed, which integrates the outputs of global average pooling (GAP) and global max pooling (GMP) to obtain more discriminative global features. Besides, it can also remove redundant information and increase the discrimination of pedestrian representations. A fine-grained local feature embedding fusion operation is designed to obtain more discriminative local features by embedding and fusing fine-grained local features of RGB and grayscale pedestrian images. Since the final pedestrian representation fuses both global features and fine-grained discriminative features in RGB and grayscale spaces, DSFF-Net increases the discriminative capability and richness of pedestrian representations. Moreover, we conduct extensive experiments on three datasets, Market-1501 DukeMTMC-Reid, and CUHK03, and our method achieves the Rank-1/mAP of 95.9%/89.1%, 89.0%/79.2%, and 81.2%/78.7%, respectively. Experimental results show that the performance of DSFF-Net is better than those of most of the state-of-the-art person Re-ID methods.

TransDD: A transformer-based dual-path decoder for improving the performance of thoracic diseases classification using chest X-ray

Manually and accurately detecting thoracic diseases from CXR images is a time-consuming task that requires experienced radiologists. Therefore, automated thoracic diseases classification has great significance. However, most existing methods solely leverage the feature maps extracted from CXR images to classify thoracic diseases, without effectively connecting the correlation between the local discriminative lesion features and their corresponding labels. To address this issue, we innovatively introduce a learnable label embedding as queries to detect and match class-related features from the feature maps, and then processed by a novel Transformer-based dual-path decoder (TransDD) to facilitate interaction. The proposed TransDD is comprised of three key components: spatial reduction attention (SRA), dual-path attention (DPA), and feature enhancement module (FEM). SRA is employed in simplifying the complexity of self-attention, while DPA is specifically designed to connect the explicit correlation between the features and labels. Moreover, FEM is used to boost the expressiveness of local features. Subsequently, the classification attention block is utilized to balance two classification scores based on the feature output and label output, respectively. The proposed TransDD-PVT attained SOTA performance on the ChestX-ray14 dataset, achieving a mean area under the receiver operating characteristic (AUC) of 83.1% across all 14 classes. Also, our method achieves 94.31% accuracy and 93.31% sensitivity on three-class classifications. Extensive experiments conducted on several datasets demonstrate the powerful ability of our TransDD to improve the performance of thoracic diseases classification. It can serve as a plug-and-play structure to improve the classification performance of both CNNs and recent Transformer-based backbones.

End-to-End Convolutional Network and Spectral-Spatial Transformer Architecture for Hyperspectral Image Classification

Although convolutional neural networks (CNNs) have proven successful for hyperspectral image classification (HSIC), it is difficult to characterize the global dependencies between HSI pixels at long-distance ranges and spectral bands due to their limited receptive domain. The transformer can compensate well for this shortcoming, but it suffers from a lack of image-specific inductive biases (i.e., localization and translation equivariance) and contextual position information compared with CNNs. To overcome the aforementioned challenges, we introduce a simply structured, end-to-end convolutional network and spectral–spatial transformer (CNSST) architecture for HSIC. Our CNSST architecture consists of two essential components: a simple 3D-CNN-based hierarchical feature fusion network and a spectral–spatial transformer that introduces inductive bias information. The former employs a 3D-CNN-based hierarchical feature fusion structure to establish the correlation between spectral and spatial (SAS) information while capturing richer inductive bias and more discriminative local spectral-spatial hierarchical feature information, while the latter aims to establish the global dependency among HSI pixels while enhancing the acquisition of local information by introducing inductive bias information. Specifically, the spectral and inductive bias information is incorporated into the transformer’s multi-head self-attention mechanism (MHSA), thus making the attention spectrally aware and location-aware. Furthermore, a Lion optimizer is exploited to boost the classification performance of our newly developed CNSST. Substantial experiments conducted on three publicly accessible hyperspectral datasets unequivocally showcase that our proposed CNSST outperforms other state-of-the-art approaches.

Manifold embedded global and local discriminative features selection for single-shot multi-categories clothing recognition and retrieval

PurposeSingle-shot multi-category clothing recognition and retrieval play a crucial role in online searching and offline settlement scenarios. Existing clothing recognition methods based on RGBD clothing images often suffer from high-dimensional feature representations, leading to compromised performance and efficiency.Design/methodology/approachTo address this issue, this paper proposes a novel method called Manifold Embedded Discriminative Feature Selection (MEDFS) to select global and local features, thereby reducing the dimensionality of the feature representation and improving performance. Specifically, by combining three global features and three local features, a low-dimensional embedding is constructed to capture the correlations between features and categories. The MEDFS method designs an optimization framework utilizing manifold mapping and sparse regularization to achieve feature selection. The optimization objective is solved using an alternating iterative strategy, ensuring convergence.FindingsEmpirical studies conducted on a publicly available RGBD clothing image dataset demonstrate that the proposed MEDFS method achieves highly competitive clothing classification performance while maintaining efficiency in clothing recognition and retrieval.Originality/valueThis paper introduces a novel approach for multi-category clothing recognition and retrieval, incorporating the selection of global and local features. The proposed method holds potential for practical applications in real-world clothing scenarios.

Locally-Enriched Cross-Reconstruction for Few-Shot Fine-Grained Image Classification

Few-shot fine-grained image classification has attracted considerable attention in recent years for its realistic setting to imitate how humans conduct recognition tasks. Metric-based few-shot classifiers have achieved high accuracies. However, their metric function usually requires two arguments of vectors, while transforming or reshaping three-dimensional feature maps to vectors can result in loss of spatial information. Image reconstruction is thus involved to retain more appearance details: the test images are reconstructed by different classes and then classified to the one with the smallest reconstruction error. However, discriminative local information, vital to distinguish sub-categories in fine-grained images with high similarities, is not well elaborated when only the base features from a usual embedding module are adopted for reconstruction. Hence, we propose the novel local content-enriched cross-reconstruction network (LCCRN) for few-shot fine-grained classification. In LCCRN, we design two new modules: the local content-enriched module (LCEM) to learn the discriminative local features, and the cross-reconstruction module (CRM) to fully engage the local features with the appearance details obtained from a separate embedding module. The classification score is calculated based on the weighted sum of reconstruction errors of the cross-reconstruction tasks, with weights learnt from the training process. Extensive experiments on four fine-grained datasets showcase the superior classification performance of LCCRN compared with the state-of-the-art few-shot classification methods. Codes are available at: https://github.com/lutsong/LCCRN.

Unsupervised Local Discrimination for Medical Images.

Contrastive learning, which aims to capture general representation from unlabeled images to initialize the medical analysis models, has been proven effective in alleviating the high demand for expensive annotations. Current methods mainly focus on instance-wise comparisons to learn the global discriminative features, however, pretermitting the local details to distinguish tiny anatomical structures, lesions, and tissues. To address this challenge, in this paper, we propose a general unsupervised representation learning framework, named local discrimination (LD), to learn local discriminative features for medical images by closely embedding semantically similar pixels and identifying regions of similar structures across different images. Specifically, this model is equipped with an embedding module for pixel-wise embedding and a clustering module for generating segmentation. And these two modules are unified by optimizing our novel region discrimination loss function in a mutually beneficial mechanism, which enables our model to reflect structure information as well as measure pixel-wise and region-wise similarity. Furthermore, based on LD, we propose a center-sensitive one-shot landmark localization algorithm and a shape-guided cross-modality segmentation model to foster the generalizability of our model. When transferred to downstream tasks, the learned representation by our method shows a better generalization, outperforming representation from 18 state-of-the-art (SOTA) methods and winning 9 out of all 12 downstream tasks. Especially for the challenging lesion segmentation tasks, the proposed method achieves significantly better performance. The source codes are publicly available at https://github.com/HuaiChen-1994/LDLearning.

Person Re-Identification Using Local Relation-Aware Graph Convolutional Network.

Local feature extractions have been verified to be effective for person re-identification (re-ID) in recent literature. However, existing methods usually rely on extracting local features from single part of a pedestrian while neglecting the relationship of local features among different pedestrian images. As a result, local features contain limited information from one pedestrian image, and cannot benefit from other pedestrian images. In this paper, we propose a novel approach named Local Relation-Aware Graph Convolutional Network (LRGCN) to learn the relationship of local features among different pedestrian images. In order to completely describe the relationship of local features among different pedestrian images, we propose overlap graph and similarity graph. The overlap graph formulates the edge weight as the overlap node number in the node's neighborhoods so as to learn robust local features, and the similarity graph defines the edge weight as the similarity between the nodes to learn discriminative local features. To propagate the information for different kinds of nodes effectively, we propose the Structural Graph Convolution (SGConv) operation. Different from traditional graph convolution operations where all nodes share the same parameter matrix, SGConv learns different parameter matrices for the node itself and its neighbor nodes to improve the expressive power. We conduct comprehensive experiments to verify our method on four large-scale person re-ID databases, and the overall results show LRGCN exceeds the state-of-the-art methods.

Vehicle re-identification based on grouping aggregation attention and cross-part interaction

Vehicle re-identification (Re-ID) plays an important role in intelligent transportation systems. To solve the key problem of small inter-class difference, we propose a vehicle Re-ID network based on grouping aggregation attention and cross-part interaction (GACP), which uses a channel context local interaction attention block (CCLI) and a cross-part interaction module (CPIM) to extract global and local discriminative features, respectively. The CCLI constructs the context information of each channel and performs local interaction to realize the communication of local and long-range information, thus effectively infer the weights of channels. In addition, the CCLI further reduces the number of parameters and improves the attention effect through a grouping aggregation module and an attention enhancement constraint. The CPIM enhances the correlation between vehicle parts obtained by rigid segmentation on feature maps to mine robust local information. Extensive experiments on two popular datasets VeRi776 and VehicleID demonstrate the effectiveness of the proposed method.

Transformer-based descriptors with fine-grained region supervisions for visual place recognition

Visual place recognition is a fundamental component in autonomous systems and robotics, which is easily limited in the real world with different viewpoints and changes in appearance. Existing approaches to tackle this problem mainly rely on dominant CNN-based architectures, which are difficult to model global correlation. Most recently, there has emerged little work that focuses on the effectiveness of the Transformer in modeling long-range dependencies, but this strategy ignores local interactions thus fails to localize the really important regions. To address the above issue, this paper proposes an effective Transformer-based architecture that takes full advantages of the strengths of Transformer related to global context modeling and local specific region capturing. We first design a dual-level Transformer descriptor encoder to successively perform self-attention within local windows and global extent of the CNN feature map to obtain multi-scale spatial context, which combines local interaction and global information. Specifically, multi-layer classification tokens from the Transformer encoder are integrated to form the global image representation. Moreover, a Transformer-guided geometric verification module is introduced to leverage the strengths of the hierarchical Transformer’s inherent self-attention mechanism for fusing multi-level attention, which is employed to filter the output token to obtain key patches and associated attention weights for achieving spatial matching. Finally, we propose a descriptor refinement strategy that employs fine-grained region-level supervisions to further enhance the capability of the network to learn local discriminative features, which effectively alleviates the confusion caused from weak image-level labels. Extensive experiments on benchmark datasets show that our approach outperforms state-of-the-art methods with promising performance.

Registration‐enhanced multiple instance learning for cervical cancer whole slide image classification

AbstractHistopathological diagnosis is the golden standard for cancer diagnosis and grading. The past decade has witnessed great successes of vision‐based deep learning in computer‐aided cancer diagnosis. However, fully‐supervised methods require collecting large amounts of annotated data, which is a major challenge. In this research, we propose RMIL, a novel registration‐enhanced multiple instance learning pipeline, for cervical cancer biopsy grading of whole slide images as well as patch‐level regions. It requires only slide‐level annotations and a small amount of patch‐level annotations for training. To enhance it, an efficient registration method is implemented to generate more labeled patches with discriminative local features as data augmentations. In addition, attention mechanisms and self‐supervised learning are introduced to enhance feature extraction. Experiments showed that its classification accuracy is increased by 11.11%, compared to the vanilla multiple instance learning methods. The method has great potential for transferring to other scenarios.

LET-Net: locally enhanced transformer network for medical image segmentation

AbstractMedical image segmentation has attracted increasing attention due to its practical clinical requirements. However, the prevalence of small targets still poses great challenges for accurate segmentation. In this paper, we propose a novel locally enhanced transformer network (LET-Net) that combines the strengths of transformer and convolution to address this issue. LET-Net utilizes a pyramid vision transformer as its encoder and is further equipped with two novel modules to learn more powerful feature representation. Specifically, we design a feature-aligned local enhancement module, which encourages discriminative local feature learning on the condition of adjacent-level feature alignment. Moreover, to effectively recover high-resolution spatial information, we apply a newly designed progressive local-induced decoder. This decoder contains three cascaded local reconstruction and refinement modules that dynamically guide the upsampling of high-level features by their adaptive reconstruction kernels and further enhance feature representation through a split-attention mechanism. Additionally, to address the severe pixel imbalance for small targets, we design a mutual information loss that maximizes task-relevant information while eliminating task-irrelevant noises. Experimental results demonstrate that our LET-Net provides more effective support for small target segmentation and achieves state-of-the-art performance in polyp and breast lesion segmentation tasks.

Fine-Grained Image Retrieval via Object Localization

In this paper, a network consisting of an object localization module and a discriminative feature extraction module is designed for fine-grained image retrieval (FGIR). In order to reduce the interference of complex backgrounds, the object localization module is introduced into the network before feature extraction. By selecting the convolutional feature descriptors, the main object is separated from the background, and thus, most of the interference is filtered out. Further, in order to improve the overall performance of the network, a discriminative filter bank is introduced into the network as the local feature detector. Hence, the local discriminative features can be extracted directly from the original feature map. The experimental results based on the CUB-200-2011 and Cars-196 datasets demonstrate that the proposed method can improve the performance of FGIR.

Facial expression recognition network with slow convolution and zero-parameter attention mechanism

Facial expression recognition (FER) has been successfully applied to the fields of automatic driving, classroom education, and many other industries. However, a lot of existing problems induced by different lighting, posture, and skin color make many difficulties in extracting effective features, resulting in low accuracy of FER. to address these issues, a novel slow evolution and zero parameter attention mechanism (SC-ZAM) is proposed to reliably extract discriminative feature from facial images for FER. In the proposed SC-ZAM, Slow convolution module is used to reduce the speed of image dimensionality reduction, extract considerably reliable facial features, and enhances the representation of these features to extract more effective global information of facial expressions. Meanwhile, zero-parameter attention modules are applied to enhance the focus on the local discriminative feature areas. The improved attention module focuses on highly distinguishable facial features and restrains the weight of non-facial areas to locate more detailed local facial areas in the global information extracted from the network. Experimental results demonstrate that our SC-ZAM method is robust against different FER tasks on our infant dataset and several public datasets, and is superior to other state-of-the-art methods.