Person Re-identification Datasets Research Articles

Person re-identification transformer with patch attention and pruning

Person re-identification (Re-ID), which is widely used in surveillance and tracking systems, aims to search individuals as they move between different camera views by maintaining identity across various camera views. In the realm of person re-identification (Re-ID), recent advancements have introduced convolutional neural networks (CNNs) and vision transformers (ViTs) as promising solutions. While CNN-based methods excel in local feature extraction, ViTs have emerged as effective alternatives to CNN-based person Re-ID, offering the ability to capture long-range dependencies through multi-head self-attention without relying on convolution and downsampling. However, it still faces challenges such as changes in illumination, viewpoint, pose, low resolutions, and partial occlusions. To address the limitations of widely used person Re-ID datasets and improve the generalization, we present a novel person Re-ID method that enhances global and local information interactions using self-attention modules within a ViT network. It leverages dynamic pruning to extract and prioritize essential image patches effectively. The designed patch selection and pruning for person Re-ID model resulted in a robust feature extractor even in scenarios with partial occlusion, background clutter, and illumination variations. Empirical validation demonstrates its superior performance compared to previous approaches and its adaptability across various domains.

A New Symmetry-Based Transformer for Text Spotting in Person and Vehicle Re-Identification Images

Text spotting in person and vehicle re-identification images is complex due to the presence of multiple views of the same person and vehicle. Most existing models focus on text spotting in natural scene images, our work focuses on spotting in person and vehicle re-identification images. The rationale behind this work is that the person and the vehicles share symmetry properties and the bib number in the torso and license plate number in the vehicle are text. The method divides the input image into patches, and it explores vision transformation for encoding the patches into linear patches. The linearly embedded patches are fed to the feature similarity index step, which involves phase congruency and gradient magnitude to detect symmetric patches. The transformer is proposed to encode and capture textual information from the symmetry patches for text detection and recognition. The decoder receives the attention features from the encoder and fetches a multi-task head with the information about the detected and recognized text. The experiments on person and vehicle image benchmark, viz. (Person) Re-ID, RBNR, UFPR-ALPR and RodoSol datasets show significant improvement in performance when compared to other text spotting models. The effectiveness of the proposed model is validated by testing on the benchmark datasets, namely, ICDAR 2015, Total-Text and CTW1500 of natural scene images. Furthermore, cross-data validation shows the proposed method is independent of domains.

Enhancing person re-identification via Uncertainty Feature Fusion Method and Auto-weighted Measure Combination

Person re-identification (Re-ID) is a challenging task that involves identifying the same person across different camera views in surveillance systems. Current methods usually rely on features from single-camera views, which can be limiting when dealing with multiple cameras and challenges such as changing viewpoints and occlusions. In this paper, a new approach is introduced that enhances the capability of ReID models through the Uncertain Feature Fusion Method (UFFM) and Auto-weighted Measure Combination (AMC). UFFM generates multi-view features using features extracted independently from multiple images to mitigate view bias. However, relying only on similarity based on multi-view features is limited because these features ignore the details represented in single-view features. Therefore, we propose the AMC method to generate a more robust similarity measure by combining various measures. Our method significantly improves Rank@1 (Rank-1 accuracy) and Mean Average Precision (mAP) when evaluated on person re-identification datasets. Combined with the BoT Baseline on challenging datasets, we achieve impressive results, with a 7.9% improvement in Rank@1 and a 12.1% improvement in mAP on the MSMT17 dataset. On the Occluded-DukeMTMC dataset, our method increases Rank@1 by 22.0% and mAP by 18.4%.

LRMM: Low rank multi-scale multi-modal fusion for person re-identification based on RGB-NI-TI

Person Re-identification is a crucial task in video surveillance, aiming to match person images from non-overlapping camera views. Recent methods introduce the Near-Infrared (NI) modality to alleviate the limitations of traditional single visible light modality under low-light conditions, while they overlook the importance of modality-related information. To incorporate more additional complementary information to assist traditional person re-identification tasks, in this paper, a novel RGB-NI-TI multi-modal person re-identification approach is proposed. First, we design a multi-scale multi-modal interaction module to facilitate cross-modal information fusion across multiple scales. Secondly, we propose a low-rank multi-modal fusion module that leverages the feature and weight parallel decomposition and then employs low-rank modality-specific factors for multimodal fusion. It aims to make the model more efficient in fusing multiple modal features while reducing complexity. Finally, we propose a multiple modalities prototype loss to supervise the network jointly with the cross-entropy loss, enforcing the network to learn modality-specific information by improving the intra-class cross-modality similarity and expanding the inter-class difference. The experimental results on benchmark multi-modal Re-ID datasets (RGBNT201, RGBNT100, MSVR310) and constructed person Re-ID datasets (multimodal version Market1501, PRW) validate the effectiveness of the proposed approach compared with the state-of-the-art methods.

Highly Efficient Active Learning With Tracklet-Aware Co-Cooperative Annotators for Person Re-Identification.

Supervised person re-identification (ReID) has attracted widespread attentions in the computer vision community due to its great potential in real-world applications. However, the demand of human annotation heavily limits the application as it is costly to annotate identical pedestrians appearing from different cameras. Thus, how to reduce the annotation cost while preserving the performance remains challenging and has been studied extensively. In this article, we propose a tracklet-aware co-cooperative annotators' framework to reduce the demand of human annotation. Specifically, we partition the training samples into different clusters and associate adjacent images in each cluster to produce the robust tracklet which decreases the annotation requirements significantly. Besides, to further reduce the cost, we introduce a powerful teacher model in our framework to implement the active learning strategy and select the most informative tracklets for human annotator, the teacher model itself, in our setting, also acts as an annotator to label the relatively certain tracklets. Thus, our final model could be well-trained with both confident pseudo-labels and human-given annotations. Extensive experiments on three popular person ReID datasets demonstrate that our approach could achieve competitive performance compared with state-of-the-art methods in both active learning and unsupervised learning (USL) settings.

A Person Re-Identification Method for Fusing Convolutional Attention and Transformer Architecture

Rapid and accurate localization of target person via person re-identification is one of the important methods in intelligent security systems,but faces great challenges due to complex environmental changes. In order to build a model applicable to various complex scenes, deep learning-based person re-identification models have become a key research direction for researchers. The existing convolutional neural network-based or transformer-based deep learning models are still have certain shortcomings in the extraction of local details and are not able to cope well with realistic complex scenes. To address this problem, a method fusing convolutional neural network and transformer (FCAT) is proposed to enhance the transformer's attention to local detail information. This method mainly improves the transformer's ability to extract local detail features indirectly by embedding convolutional space attention and channel attention respectively to enhance the attention to important regions and important channel features in the image. Comparative ablation experiments on three publicly available person re-identification datasets demonstrate that FCAT achieves comparable results to existing methods on non-occluded datasets, and significantly improves performance on occluded datasets. The proposed model is also has a lightweight effect, and the inference speed is improved without increasing the amount of outer computation and model parameters.

FRCE: Transformer-based feature reconstruction and cross-enhancement for occluded person re-identification

Occluded person re-identification (ReID) aims to capture the same person from different camera views with partial occlusion. In real-world scenarios, any object can become a random occluder, which is still an unsettled challenge for person ReID. Previous methods used auxiliary models to locate or reconstruct the occluded parts for ReID, but these methods are affected by external models or unable to model well, leaving the difficulty unresolved. Currently, Transformer has been widely applied to occluded person ReID tasks, but due to the lack of attention to local features, these methods still have limitations. Therefore, we propose a Feature Reconstruction and Cross-Enhancement (FRCE) network. Firstly, a Cross-level Feature Reconstruction Distiller (CFRD) is designed to improve the feature richness in the recognition scope and generate higher quality identity feature mapping by reconstructing the effective recognition region of the target. Secondly, we introduce a Multi-Dimensional Local Refinement (MDLR) module to deeply excavate local details in spatial and channel dimensions, seek subtle target cues and similarity relationships at local angles, and achieve more efficient ReID. Extensive experiments on five challenging occluded and holistic person ReID datasets demonstrate the effectiveness of our approach.

Inter-camera Identity Discrimination for Unsupervised Person Re-identification

Unsupervised person re-identification (Re-ID) has garnered significant attention because of its data-friendly nature, as it does not require labeled data. Existing approaches primarily address this challenge by employing feature-clustering techniques to generate pseudo-labels. In addition, camera-proxy-based methods have emerged because of their impressive ability to cluster sample identities. However, these methods often blur the distinctions between individuals within inter-camera views, which is crucial for effective person re-ID. To address this issue, this study introduces an inter-camera-identity-difference-based contrastive learning framework for unsupervised person Re-ID. The proposed framework comprises two key components: (1) a different sample cross-view close-range penalty module and (2) the same sample cross-view long-range constraint module. The former aims at penalizing excessive similarity among different subjects across inter-camera views, whereas the latter mitigates the challenge of excessive dissimilarity among the same subject across camera views. To validate the performance of our method, we conducted extensive experiments on three existing person Re-ID datasets (Market-1501, MSMT17, and PersonX). The results demonstrate the effectiveness of the proposed method, which shows a promising performance. The code is available at https://github.com/hooldylan/IIDCL .

Parameter-Efficient Person Re-Identification in the 3D Space.

People live in a 3D world. However, existing works on person re-identification (re-id) mostly consider the semantic representation learning in a 2D space, intrinsically limiting the understanding of people. In this work, we address this limitation by exploring the prior knowledge of the 3D body structure. Specifically, we project 2D images to a 3D space and introduce a novel parameter-efficient omni-scale graph network (OG-Net) to learn the pedestrian representation directly from 3D point clouds. OG-Net effectively exploits the local information provided by sparse 3D points and takes advantage of the structure and appearance information in a coherent manner. With the help of 3D geometry information, we can learn a new type of deep re-id feature free from noisy variants, such as scale and viewpoint. To our knowledge, we are among the first attempts to conduct person re-id in the 3D space. We demonstrate through extensive experiments that the proposed method: (1) eases the matching difficulty in the traditional 2D space; 2) exploits the complementary information of 2D appearance and 3D structure; 3) achieves competitive results with limited parameters on four large-scale person re-id datasets; and 4) has good scalability to unseen datasets. Our code, models, and generated 3D human data are publicly available at https://github.com/layumi/person-reid-3d.

Situational diversity in video person re-identification: introducing MSA-BUPT dataset

Thanks to the success of deep learning over the past few years, the video person re-identification (ReID) algorithms have achieved high accuracy on multiple public benchmark datasets. However, the available video person ReID datasets cover a limited range of real-world scenarios, and they have several obvious limitations: limited camera viewing angles, tiny variations of the shooting scene, and even errors in manual labels. These disadvantages prevent video person ReID from being widely used in real-life scenarios. In this work, a new high-quality multi-situation video person ReID dataset, named MSA-BUPT, is built to promote the video person ReID task in large-scale urban surveillance. Specifically, MSA-BUPT contains 684 identities, 2,665 trajectories, and nearly 250,000 frames from 200-h videos across various complex scenarios. Person attribute annotations and unannotated video data are also provided for other research perspectives, such as cross-modality ReID, cross-domain ReID, and so on. Furthermore, two plug-and-play components are used to improve retrieval capabilities: a new scenario-based data augmentation method is proposed to alleviate the person misalignment problem; a re-ranking strategy based on person attribute is applied to make secondary adjustments to the content to the results of the model. The extensive experimental results show that the above methods improve the performance of some representative state-of-the-art models on the new dataset.

Identity-Guided Collaborative Learning for Cloth-Changing Person Reidentification.

Cloth-changing person reidentification (ReID) is a newly emerging research topic aimed at addressing the issues of large feature variations due to cloth-changing and pedestrian view/pose changes. Although significant progress has been achieved by introducing extra information (e.g., human contour sketching information, human body keypoints, and 3D human information), cloth-changing person ReID remains challenging because pedestrian appearance representations can change at any time. Moreover, human semantic information and pedestrian identity information are not fully explored. To solve these issues, we propose a novel identity-guided collaborative learning scheme (IGCL) for cloth-changing person ReID, where the human semantic is effectively utilized and the identity is unchangeable to guide collaborative learning. First, we design a novel clothing attention degradation stream to reasonably reduce the interference caused by clothing information where clothing attention and mid-level collaborative learning are employed. Second, we propose a human semantic attention and body jigsaw stream to highlight the human semantic information and simulate different poses of the same identity. In this way, the extraction features not only focus on human semantic information that is unrelated to the background but are also suitable for pedestrian pose variations. Moreover, a pedestrian identity enhancement stream is proposed to enhance the identity importance and extract more favorable identity robust features. Most importantly, all these streams are jointly explored in an end-to-end unified framework, and the identity is utilized to guide the optimization. Extensive experiments on six public clothing person ReID datasets (LaST, LTCC, PRCC, NKUP, Celeb-reID-light, and VC-Clothes) demonstrate the superiority of the IGCL method. It outperforms existing methods on multiple datasets, and the extracted features have stronger representation and discrimination ability and are weakly correlated with clothing.

Mask-guided discriminative feature network for occluded person re-identification

In recent years, although research on person re-identification (ReID) has made significant progress, occluded person ReID remains a major challenge. In real-world scenes, persons are often occluded by various obstacles such as vehicles, umbrellas, and other persons. This leads to the noisy interference and the loss of visual information, which result in poor ReID performance of occluded persons. To address this issue, we propose an end-to-end Mask-guided Discriminative Feature Network (MDFNet). First, MDFNet adopts a dual-branch architecture with a shared encoder as the Feature Extraction module for paired images. Each pair of images consist of one image from the training set and its corresponding occluded image generated through an occlusion augmentation strategy. Second, MDFNet utilizes a Mask-guided Discriminative Feature Enhancement and Fusion (MDFEF) module to achieve the fusion and enhancement of global and local features for high-quality person representations. MDFEF module effectively suppresses the interference of occlusion, enriches the representation capacity of person features, and enables the model to focus more on the important features in non-occluded regions. Furthermore, MDFNet introduces a sparse pairwise loss for enabling the model to dynamically adapt to intra-class variations and reduce the negative impact of complex occlusions. The experimental results on four challenging person ReID datasets demonstrate the effectiveness of the proposed method.

Joint training with local soft attention and dual cross-neighbor label smoothing for unsupervised person re-identification

AbstractExisting unsupervised person re-identification approaches fail to fully capture the fine-grained features of local regions, which can result in people with similar appearances and different identities being assigned the same label after clustering. The identity-independent information contained in different local regions leads to different levels of local noise. To address these challenges, joint training with local soft attention and dual cross-neighbor label smoothing (DCLS) is proposed in this study. First, the joint training is divided into global and local parts, whereby a soft attention mechanism is proposed for the local branch to accurately capture the subtle differences in local regions, which improves the ability of the re-identification model in identifying a person’s local significant features. Second, DCLS is designed to progressively mitigate label noise in different local regions. The DCLS uses global and local similarity metrics to semantically align the global and local regions of the person and further determines the proximity association between local regions through the cross information of neighboring regions, thereby achieving label smoothing of the global and local regions throughout the training process. In extensive experiments, the proposed method outperformed existing methods under unsupervised settings on several standard person re-identification datasets.

MHDNet: A Multi-Scale Hybrid Deep Learning Model for Person Re-Identification

The primary objective of person re-identification is to identify individuals from surveillance videos across various scenarios. Conventional pedestrian recognition models typically employ convolutional neural network (CNN) and vision transformer (ViT) networks to extract features, and while CNNs are adept at extracting local features through convolution operations, capturing global information can be challenging, especially when dealing with high-resolution images. In contrast, ViT rely on cascaded self-attention modules to capture long-range feature dependencies, sacrificing local feature details. In light of these limitations, this paper presents the MHDNet, a hybrid network structure for pedestrian recognition that combines convolutional operations and self-attention mechanisms to enhance representation learning. The MHDNet is built around the Feature Fusion Module (FFM), which harmonizes global and local features at different resolutions. With a parallel structure, the MHDNet model maximizes the preservation of local features and global representations. Experiments on two person re-identification datasets demonstrate the superiority of the MHDNet over other state-of-the-art methods.

Doubly stochastic subdomain mining with sample reweighting for unsupervised domain adaptive person re-identification

Clustering-based unsupervised domain adaptive person re-identification methods have achieved remarkable progress. However, existing works are easy to fall into local minimum traps due to the optimization of two variables, feature representation and pseudo labels. Besides, the model can also be hurt by the inevitable false assignment of pseudo labels. In order to solve these problems, we propose the Doubly Stochastic Subdomain Mining (DSSM) to prevent the nonconvex optimization from falling into local minima in this paper. And we also design a novel reweighting algorithm based on the similarity correlation coefficient between samples which is referred to as Maximal Heterogeneous Similarity (MHS), it can reduce the adverse effect caused by noisy labels. Extensive experiments on two popular person re-identification datasets demonstrate that our method outperforms other state-of-the-art works. The source code is available at https://github.com/Tchunansheng/DSSM.

Adapt only once: Fast unsupervised person re-identification via relevance-aware guidance

Unsupervised domain adaptive person re-identification (UDA person reID) defines a task where labels in target domains are totally unknown while source domains are fully labeled. Assigning reliable labels quickly is a critical issue for UDA person reID that could be applied in the real-world scenarios. Recent studies focus on obtaining pseudo labels by clustering algorithms and then training the reID model with these labels. However, the main limitation of these methods is the high time complexity, which is caused by the calculation of all pair-wise similarities and multiple iterations in the clustering algorithm to obtain reliable results. When the data is very large or the feature dimensions are very high, the memory and time cost requirements of the clustering algorithm can increase rapidly.In this paper, we provide a fast unsupervised domain adaptive person reID framework (FUReID), which calculates the relevance between unlabeled samples only once to adapt to the new scenarios without any iterations in the stage of label generation. Especially, instead of pursuing accurate labels, FUReID considers constructing a lightweight paradigm to generate coarse labels and then refine these labels during the training stage. Therefore, FUReID designs a prototype-guided labeling method that only relies on calculating the relevance between the prototype vectors and the samples, and assigning coarse labels with noise. Then, to alleviate the issue of noise, FUReID designs a label-flexible training network with an adaptive selection strategy to refine those coarse labels progressively. For several widely-used person reID datasets, our method achieves 81.7%, 26.2%, and 87.7% in mAP on Market1501, MSMT17 and PersonX, respectively. Code is available at https://github.com/AILab90/FUReID.

Pedestrian Re-Identification Algorithm Based on Attention Pooling Saliency Region Detection and Matching

The recognition accuracy of the pedestrian re-identification algorithm is affected by factors such as the scale, posture, occlusion level, and appearance of a pedestrian in an image. These factors bring great challenges to the further development of deep learning theory in the field of person re-identification. On the one hand, existing pedestrian re-identification model cannot accurately extract attention information containing contextual information and discriminative pedestrian features. On the other hand, images of the same pedestrian taken with different cameras have the problem of different degrees of occlusion. Therefore, we introduce an attention pooling mechanism, which allows the model to automatically focus on discriminative regions of pedestrian images. The model can then learn attention maps by automatically focusing on visually salient pedestrian regions. It can effectively address the influence of factors on the accuracy of pedestrian re-identification. Furthermore, we propose a pedestrian re-identification method based on saliency region detection and matching. The method first extracts multiple local saliency regions from pedestrian images using the pedestrian joint point detection method, then marks and extracts the positions of the abovementioned saliency regions, and finally performs matching and sorting. It can correct the impact caused by occlusion on the accuracy of pedestrian re-identification. We conduct tests and experiments on three public person re-identification datasets, and the results show that our method not only achieves the best recognition accuracy on the aforementioned datasets but also has better robustness.

Unsupervised Joint Contrastive Learning for Aerial Person Re-Identification and Remote Sensing Image Classification

Unsupervised person re-identification (Re-ID) aims to match the query image of a person with images in the gallery without the use of supervision labels. Most existing methods usually generate pseudo-labels through clustering algorithms for contrastive learning, which inevitably results in noisy labels assigned to samples. In addition, methods that only apply contrastive learning at the clustering level fail to fully consider instance-level relationships between instances. Motivated by this, we propose a joint contrastive learning (JCL) framework for unsupervised person Re-ID. Our proposed method involves creating two memory banks to store features of cluster centroids and instances and applies cluster and instance-level contrastive learning, respectively, to jointly optimize the neural networks. The cluster-level contrastive loss is used to promote feature compactness within the same cluster and reinforce identity similarity. The instance-level contrastive loss is used to distinguish easily confused samples. In addition, we use a WaveBlock attention module (WAM), which can continuously wave feature map blocks and introduce attention mechanisms to produce more robust feature representations of a person without considerable information loss. Furthermore, we enhance the quality of our clustering by leveraging camera label information to eliminate clusters containing single camera captures. Extensive experimental results on two widely used person Re-ID datasets verify the effectiveness of our JCL method. Meanwhile, we also used two remote sensing datasets to demonstrate the generalizability of our method.

Multi-Scale Image- and Feature-Level Alignment for Cross-Resolution Person Re-Identification

Cross-Resolution Person Re-Identification (re-ID) aims to match images with disparate resolutions arising from variations in camera hardware and shooting distances. Most conventional works utilize Super-Resolution (SR) models to recover Low Resolution (LR) images to High Resolution (HR) images. However, because the SR models cannot completely compensate for the missing information in the LR images, there is still a large gap between the HR image recovered from the LR images and the real HR images. To tackle this challenge, we propose a novel Multi-Scale Image- and Feature-Level Alignment (MSIFLA) framework to align the images on multiple resolution scales at both the image and feature level. Specifically, (i) we design a Cascaded Multi-Scale Resolution Reconstruction (CMSR2) module, which is composed of three cascaded Image Reconstruction (IR) networks, and can continuously reconstruct multiple variables of different resolution scales from low to high for each image, regardless of image resolution. The reconstructed images with specific resolution scales are of similar distribution; therefore, the images are aligned on multiple resolution scales at the image level. (ii) We propose a Multi-Resolution Representation Learning (MR2L) module which consists of three-person re-ID networks to encourage the IR models to preserve the ID-discriminative information during training separately. Each re-ID network focuses on mining discriminative information from a specific scale without the disturbance from various resolutions. By matching the extracted features on three resolution scales, the images with different resolutions are also aligned at the feature-level. We conduct extensive experiments on multiple public cross-resolution person re-ID datasets to demonstrate the superiority of the proposed method. In addition, the generalization of MSIFLA in handling cross-resolution retrieval tasks is verified on the UAV vehicle dataset.

UnA-Mix: Rethinking Image Mixtures for Unsupervised Person Re-Identification

With the development of ultra-long-range visual sensors, the application of unsupervised person re-identification algorithms to them has become increasingly important. However, these algorithms inevitably generate noisy pseudo-labels, which seriously hinder the performance of tasks over a large range. Mixup, a data enhancement technique, has been validated in supervised learning for its generalization to noisy labels. Based on this observation, to our knowledge, this study is the first to explore the impact of the mixup technique on unsupervised person re-identification, which is a downstream task of contrastive learning, in detail. Specifically, mixup was applied in different locations (at the pixel level and feature level) in an unsupervised person re-identification framework to explore its influences on task performance. In addition, based on the richness of the information contained in the person samples to be mixed, we propose an uncertainty-aware mixup (UnA-Mix) method, which reduces the over-learning of simple person samples and avoids the information damage that occurs when information-rich person samples are mixed. The experimental results on three benchmark person re-identification datasets demonstrated the applicability of the proposed method, especially on the MSMT17, where it outperformed state-of-the-art methods by 5.2% and 4.8% in terms of the mAP and rank-1, respectively.