Person Re-identification Task Research Articles

Situational diversity in video person re-identification: introducing MSA-BUPT dataset

AbstractThanks to the success of deep learning over the past few years, the video person re-identification (ReID) algorithms have achieved high accuracy on multiple public benchmark datasets. However, the available video person ReID datasets cover a limited range of real-world scenarios, and they have several obvious limitations: limited camera viewing angles, tiny variations of the shooting scene, and even errors in manual labels. These disadvantages prevent video person ReID from being widely used in real-life scenarios. In this work, a new high-quality multi-situation video person ReID dataset, named MSA-BUPT, is built to promote the video person ReID task in large-scale urban surveillance. Specifically, MSA-BUPT contains 684 identities, 2,665 trajectories, and nearly 250,000 frames from 200-h videos across various complex scenarios. Person attribute annotations and unannotated video data are also provided for other research perspectives, such as cross-modality ReID, cross-domain ReID, and so on. Furthermore, two plug-and-play components are used to improve retrieval capabilities: a new scenario-based data augmentation method is proposed to alleviate the person misalignment problem; a re-ranking strategy based on person attribute is applied to make secondary adjustments to the content to the results of the model. The extensive experimental results show that the above methods improve the performance of some representative state-of-the-art models on the new dataset.

IGIE-net: Cross-modality person re-identification via intermediate modality image generation and discriminative information enhancement

Given an RGB image (or an IR image) of a pedestrian, the task of cross-modality person re-identification aims to retrieve images of a specific pedestrian from an IR (or RGB) gallery. However, the huge modality discrepancy between RGB and IR images significantly affects the performance of cross-modality person re-identification. For the purpose of reducing the impact of modality discrepancy and extracting more discriminative pedestrian features, a new cross-modality person re-identification network is proposed by us, which is based on intermediate modality image generation and discriminative information enhancement (IGIE-Net). Specifically, we design an intermediate modality image generation module (IMIGM) to effectively weaken the impact of modality discrepancy, which first extracts the modality-specific and modality-shared information contained in RGB and IR images separately, and then generates intermediate RGB and intermediate IR images by adaptively fusing original RGB and original IR images with their corresponding modality-specific and modality-shared information separately. In addition, we also design a discriminative information enhancement module (DIEM) to obtain more discriminative pedestrian representations by enhancing the discriminative information contained in the deep features of pedestrians. Extensive experiments on the publicly available SYSU-MM01 and RegDB datasets indicate that the performance of IGIE-Net reaches the current advanced level.

Instance-level Adversarial Source-free Domain Adaptive Person Re-identification

Domain adaption (DA) for person re-identification (ReID) has attained considerable progress by transferring knowledge from a source domain with labels to a target domain without labels. Nonetheless, most of the existing methods require access to source data, which raises privacy concerns. Source-free DA has recently emerged as a response to these privacy challenges, yet its direct application to open-set pedestrian re-identification tasks is hindered by the reliance on a shared category space in existing methods. Current source-free DA approaches for person ReID still encounter several obstacles, particularly the divergence-agnostic problem and the notable domain divergence due to the absent source data. In this article, we introduce an Instance-level Adversarial Mutual Teaching (IAMT) framework, which utilizes adversarial views to tackle the challenges mentioned above. Technically, we first elaborately develop a variance-based division (VBD) module to segregate the target data into instance-level subsets based on their similarity and dissimilarity to the source using the source-trained model, implicitly tackling the divergence-agnostic problem. To mitigate domain divergence, we additionally introduce a dynamic adversarial alignment (DAA) strategy, aiming to enhance the consistence of feature distribution across domains by employing adversarial instances from the target data to confuse the discriminators. Experiments reveal the superiority of the IAMT over state-of-the-art methods for DA person ReID tasks, while preserving the privacy of the source data.

Multi-Prompts Learning with Cross-Modal Alignment for Attribute-Based Person Re-identification

The fine-grained attribute descriptions can significantly supplement the valuable semantic information for person image, which is vital to the success of person re-identification (ReID) task. However, current ReID algorithms typically failed to effectively leverage the rich contextual information available, primarily due to their reliance on simplistic and coarse utilization of image attributes. Recent advances in artificial intelligence generated content have made it possible to automatically generate plentiful fine-grained attribute descriptions and make full use of them. Thereby, this paper explores the potential of using the generated multiple person attributes as prompts in ReID tasks with off-the-shelf (large) models for more accurate retrieval results. To this end, we present a new framework called Multi-Prompts ReID (MP-ReID), based on prompt learning and language models, to fully dip fine attributes to assist ReID task. Specifically, MP-ReID first learns to hallucinate diverse, informative, and promptable sentences for describing the query images. This procedure includes (i) explicit prompts of which attributes a person has and furthermore (ii) implicit learnable prompts for adjusting/conditioning the criteria used towards this person identity matching. Explicit prompts are obtained by ensembling generation models, such as ChatGPT and VQA models. Moreover, an alignment module is designed to fuse multi-prompts (i.e., explicit and implicit ones) progressively and mitigate the cross-modal gap. Extensive experiments on the existing attribute-involved ReID datasets, namely, Market1501 and DukeMTMC-reID, demonstrate the effectiveness and rationality of the proposed MP-ReID solution.

Enhancing long-term person re-identification using global, local body part, and head streams

This work addresses the task of long-term person re-identification. Typically, person re-identification assumes that people do not change their clothes, which limits its applications to short-term scenarios. To overcome this limitation, we investigate long-term person re-identification, which considers both clothes-changing and clothes-consistent scenarios. In this paper, we propose a novel framework that effectively learns and utilizes both global and local information. The proposed framework consists of three streams: global, local body part, and head streams. The global and head streams encode identity-relevant information from an entire image and a cropped image of the head region, respectively. Both streams encode the most distinct, less distinct, and average features using the combinations of adversarial erasing, max pooling, and average pooling. The local body part stream extracts identity-related information for each body part, allowing it to be compared with the same body part from another image. Since body part annotations are not available in re-identification datasets, pseudo-labels are generated using clustering. These labels are then utilized to train a body part segmentation head in the local body part stream. The proposed framework is trained by backpropagating the weighted summation of the identity classification loss, the pair-based loss, and the pseudo body part segmentation loss. To demonstrate the effectiveness of the proposed method, we conducted experiments on three publicly available datasets (Celeb-reID, PRCC, and VC-Clothes). The experimental results demonstrate that the proposed method outperforms the previous state-of-the-art method.

Focus and imagine: Occlusion suppression and repairing transformer for occluded person re-identification

Occlusion scenarios pose a great challenge to person re-identification (ReID) task because various occlusions may weaken the discriminative features and introduce interference. Recently, Transformer-based networks, which can aggregate features of all the image patches to construct global features adaptively, have shown advantages in occluded person ReID. Existing methods mainly adopted Transformer as a feature extractor and enhanced local features from the output of the Transformer encoder. However, during the processing of self-attention blocks, disturbing features from occlusions may be diffused into all the tokens, making it difficult to construct effective local features. Therefore, we consider predicting the occlusion situation of images before feature extraction and guiding the Transformer encoder to focus on visible regions, suppressing interference from occlusion. Furthermore, we propose to imagine the partial target under occlusion and reconstruct pseudo-holistic features for more robust retrieval. To this end, the Occlusion Suppression and Repairing Transformer (OSRTrans) is proposed. First, we use a self-supervised occlusion predictor to predict occlusion scores of image patches. Then the Occlusion Suppression Encoder (OSE), guided by occlusion predictions, suppresses the interference from occlusion regions and constructs a global feature. Finally, inspired by contrastive learning, the Feature Repairing Head (FRH) is proposed to reconstruct pseudo-holistic features. Our method enhances model’s ability of extracting discriminative local features, and achieve the state-of-the-art performance on occluded person ReID benchmarks, e.g., Rank-1 of 72.9% on Occluded-DukeMTMC.

Multiple-local feature and attention fused person re-identification method

Person re-identification (ReID) is widely used in intelligent security, monitoring, criminal investigation and other fields. Aiming at the problems of local occlusion, scale misalignment and attitude change of pedestrian images in actual scenes, we propose a Multi-local Feature and Attention fused network (MFA) used for person re-identification task. Firstly, Channel Point Affinity Attention module (CPAA) is embedded in the backbone network to enhance the ability of the network for extracting local details. The feature map output from the backbone network is horizontally segmented into four local feature maps, and further four branch networks are concatenated to the feature map of the backbone network. The four local feature maps are used to guide the four branch networks to pay more attention on different areas of pedestrians through Global Local Aligned loss (GLA) function. Finally, the pedestrian feature vector containing multi-local features is obtained. The mAP of the network on Market-1501, DukeMTMC-reID,CUHK03 and MSMT17 datasets were 88.6%, 81.4%, 79.5% and 64.7%, and the Rank-1 was 95.8%, 90.1%, 81.2% and 84.1% respectively. In addition, the model also obtained 73.2% and 68.1% of Rank-1 on partial dataset Patial-REID and Patial-iLIDS, respectively. Recently, The MFA model parameter is 28.3M and the inference efficiency is approximately 32 fps to an image with a resulation of 256 × 128. Compared with other ReID methods, our proposed methods achieved a competitive performance for ReID task. The code was available at github:git@github.com:ISCLab-Bistu/MFA.git.

Enhanced visible–infrared person re-identification based on cross-attention multiscale residual vision transformer

Visible–infrared (VI) person re-identification (Re-ID) is a critical identification task that involves retrieving and matching images of an individual using both infrared and visible imaging modalities. To improve the performance, researchers have developed methods to obtain implicit feature information; however, this degrades with fewer discriminative features. To address this issue, we propose a weighted fused cross-attention multi-scale residual vision transformer (WF-CAMReViT) approach to re-identify the appropriate person from visible–infrared modality images by integrating the cross-attention multi-scale residual vision transformer architecture with Opposition-based Dove Swarm Optimization (ODSO). The proposed framework aims to bridge the domain gap between the visible and infrared modalities and significantly improve the re-identification performance. RGB (visible) and infrared (IR) images of persons are gathered from standard datasets, subjected to a cross-attention multi-scale residual vision transformer network to collect features, and then fuse using minimal weight. We also propose Opposition-based DSO to find the minimal weight. The weighted fused features are then subjected to the final decoder layer of CAMReViT to perceive the characteristics of each modality. In this study, model-aware enhancement (MAE) loss is develop to improve the modality information capacity of modality-shared features. Then, the experimental results on the SYSU-MM01 and RegDB datasets are compared with state-of-the-art transformer-based visible–infrared person Re-ID tasks to verify the efficacy of the proposed model.

Anchor Association Learning for Unsupervised Video Person Re-Identification.

Video-based person re-identification (re-id) has attracted a significant attention in recent years due to the increasing demand of video surveillance. However, existing methods are usually based on the supervised learning, which requires vast labeled identities across cameras and is not suitable for real scenes. Although some unsupervised approaches have been proposed for video re-id, their performance is far from satisfactory. In this article, we propose an unsupervised anchor association learning (UAAL) framework to address the video-based person re-id task, in which the feature representation of each sampled tracklet is regarded as an anchor. Specifically, we first propose an intracamera anchor association learning (IAAL) term that learns the discriminative anchor by utilizing the affiliation relations between an image and the anchors in each camera. Then, the exponential moving average (EMA) strategy is employed to update the anchor and the updated anchors are stored into an anchor memory module. On top of that, a cross-camera anchor association learning (CAAL) term is introduced to mine potential positive anchor pairs across cameras by presenting a cyclic ranking anchor alignment and threshold filtering method. Extensive experiments conducted on two public datasets show the superiority of the proposed method; for example, our method achieves 73.2% for rank-1 accuracy and 60.1% for mean average precision (mAP) score, respectively, on MARS, similarly 89.7% and 87.0% on DukeMTMC-VideoReID.

Address the Unseen Relationships: Attribute Correlations in Text Attribute Person Search.

Text attribute person search aims to identify the particular pedestrian by textual attribute information. Compared to person re-identification tasks which requires imagery samples as its query, text attribute person search is more useful under the circumstance where only witness is available. Most existing text attribute person search methods focus on improving the matching correlation and alignments by learning better representations of person-attribute instance pairs, with few consideration of the latent correlations between attributes. In this work, we propose a graph convolutional network (GCN) and pseudo-label-based text attribute person search method. Concretely, the model directly constructs the attribute correlations by label co-occurrence probability, in which the nodes are represented by attribute embedding and edges are by the filtered correlation matrix of attribute labels. In order to obtain better representations, we combine the cross-attention module (CAM) and the GCN. Furthermore, to address the unseen attribute relationships, we update the edge information through the instances through testing set with high predicted probability thus to better adapt the attribute distribution. Extensive experiments illustrate that our model outperforms the existing state-of-the-art methods on publicly available person search benchmarks: Market-1501 and PETA.

A Generative-Based Image Fusion Strategy for Visible-Infrared Person Re-Identification

Cross-modality person re-identification task is a challenging task aiming to recognize images of the same identity between different modalities. To alleviate the cross-modality discrepancies between images, existing approaches mainly guide models to mine modality invariant features. Although those approaches are effective, they lose the modality-specific features that include important information beneficial to VI-ReID. Therefore, some approaches are using generative adversarial networks to compensate for modality information. However, the quality of images generated by these methods is usually poor, and most of them focus only on the learning of modality-sharable features. To solve these problems, this paper proposes a generative-based cross-modality image fusion strategy (GC-IFS), which can generate high-quality cross-modality paired images and fuse the information of the two modalities. Firstly, considering the importance of the identity discriminative information of the generated image, we propose a contrastive-learning image generation (CLIG) network to generate cross-modality paired images. Meanwhile, to fully integrate and utilize the information of the two modalities and eliminate the influence of cross-modality discrepancies, we design a part-based dual multi-modality feature fusion (P-DMFF) module to extract the unified feature representation. Extensive experiments on SYSU-MM01 and RegDB datasets demonstrate that our strategy outperforms the state-of-the-art methods for the VI-ReID task.

Source-free Style-diversity Adversarial Domain Adaptation with Privacy-preservation for person re-identification

Unsupervised domain adaptation (UDA) techniques for person re-identification (ReID) have been extensively studied to facilitate the transfer of knowledge from labeled source domains to unlabeled target domains. However, the need to access the source data raises privacy concerns in real-world scenarios. To overcome this limitations, source-free domain adaptation (SFDA) was introduced, enabling adaptation without requiring access to the source data, relying instead on a well-trained source model. Nevertheless, existing SFDA methods assume a shared label space and overlook the significance of domain-style discrepancies in person ReID, limiting their applicability to source-free domain adaptive person ReID. In this paper, we present a novel approach called Source-free Style-diversity Adversarial Domain Adaptation with Privacy-preservation (S2ADAP) for person ReID to address these challenges. Our approach effectively handles inter-domain pedestrian appearance style differences using GAN-based domain-style diversity augmentation and intra-domain individual style misalignment through adversarial mutual teaching learning, avoiding access to data from the source domain. We leverage a pre-trained model as a person appearance style encoder to enhance source-similar style diversity in the target domain and achieve intra-domain individual style alignment by introducing the domain style discriminator to promote the discriminability of person semantic features for domain adaptation. The experimental results on publicly available person ReID datasets affirm the efficacy of our approach, offering a promising and privacy-preserving solution for person ReID tasks.

A Memorizing and Generalizing Framework for Lifelong Person Re-Identification.

In this paper, we introduce a challenging yet practical setting for person re-identification (ReID) task, named lifelong person re-identification (LReID), which aims to continuously train a ReID model across multiple domains and the trained model is required to generalize well on both seen and unseen domains. It is therefore critical to learn a ReID model that can learn a generalized representation without forgetting knowledge of seen domains. In this paper, we propose a new MEmorizing and GEneralizing framework (MEGE) for LReID, which can jointly prevent the model from forgetting and improve its generalization ability. Specifically, our MEGE is composed of two novel modules, i.e., Adaptive Knowledge Accumulation (AKA) and differentiable Ranking Consistency Distillation (RCD). Taking inspiration from the cognitive processes in the human brain, we endow AKA with two special capacities, knowledge representation and knowledge operation by graph convolution networks. AKA can effectively mitigate catastrophic forgetting on seen domains while improving the generalization ability to unseen domains. By considering the ranking factor that is specifically important in ReID, RCD is designed to distill the ranking knowledge in a differentiable manner, which can further prevent the catastrophic forgetting. To supporting the study of LReID, we build a new and large-scale benchmark with two practical evaluation protocols that consider the metrics of non-forgetting and generalization. Experiments demonstrate that 1) our MEGE framework can effectively improve the performance on seen and unseen domains under the domain-incremental learning constraint, and that 2) the proposed MEGE outperforms state-of-the-art competitors by large margins.

MHSA-Net: Multihead Self-Attention Network for Occluded Person Re-Identification.

This article presents a novel person reidentification model, named multihead self-attention network (MHSA-Net), to prune unimportant information and capture key local information from person images. MHSA-Net contains two main novel components: multihead self-attention branch (MHSAB) and attention competition mechanism (ACM). The MHSAB adaptively captures key local person information and then produces effective diversity embeddings of an image for the person matching. The ACM further helps filter out attention noise and nonkey information. Through extensive ablation studies, we verified that the MHSAB and ACM both contribute to the performance improvement of the MHSA-Net. Our MHSA-Net achieves competitive performance in the standard and occluded person Re-ID tasks.

Dynamic gradient reactivation for backward compatible person re-identification

We study the backward compatible problem for person re-identification (Re-ID), which aims to constrain the features of an updated new model to be comparable with the existing features from the old model in galleries. Most of the existing works adopt distillation-based methods, which focus on pushing new features to imitate the distribution of the old ones. However, the distillation-based methods are intrinsically sub-optimal since it forces the new feature space to imitate the inferior old feature space. To address this issue, we propose the Ranking-based Backward Compatible Learning (RBCL), which directly optimizes the ranking metric between new features and old features. Different from previous methods, RBCL only pushes the new features to find best-ranking positions in the old feature space instead of strictly alignment, and is in line with the ultimate goal of backward retrieval. However, the sharp sigmoid function used to make the ranking metric differentiable also incurs the gradient vanish issue, therefore stems the ranking refinement during the later period of training. To address this issue, we propose the Dynamic Gradient Reactivation (DGR), which can reactivate the suppressed gradients by adding dynamically computed constants during the forward step. To further help target the best-ranking positions, we include the Neighbor Context Agents (NCAs) to approximate the entire old feature space during training. Unlike previous works that mainly test on the in-domain settings, we make the early attempt to introduce the cross-domain settings (including both supervised and unsupervised) for the backward compatible person Re-ID task, which are more challenging yet meaningful. The experimental results on all five settings show that the proposed RBCL outperforms previous state-of-the-art methods by large margins.

Graph Sampling-Based Multi-Stream Enhancement Network for Visible-Infrared Person Re-Identification.

With the increasing demand for person re-identification (Re-ID) tasks, the need for all-day retrieval has become an inevitable trend. Nevertheless, single-modal Re-ID is no longer sufficient to meet this requirement, making Multi-Modal Data crucial in Re-ID. Consequently, a Visible-Infrared Person Re-Identification (VI Re-ID) task is proposed, which aims to match pairs of person images from the visible and infrared modalities. The significant modality discrepancy between the modalities poses a major challenge. Existing VI Re-ID methods focus on cross-modal feature learning and modal transformation to alleviate the discrepancy but overlook the impact of person contour information. Contours exhibit modality invariance, which is vital for learning effective identity representations and cross-modal matching. In addition, due to the low intra-modal diversity in the visible modality, it is difficult to distinguish the boundaries between some hard samples. To address these issues, we propose the Graph Sampling-based Multi-stream Enhancement Network (GSMEN). Firstly, the Contour Expansion Module (CEM) incorporates the contour information of a person into the original samples, further reducing the modality discrepancy and leading to improved matching stability between image pairs of different modalities. Additionally, to better distinguish cross-modal hard sample pairs during the training process, an innovative Cross-modality Graph Sampler (CGS) is designed for sample selection before training. The CGS calculates the feature distance between samples from different modalities and groups similar samples into the same batch during the training process, effectively exploring the boundary relationships between hard classes in the cross-modal setting. Some experiments conducted on the SYSU-MM01 and RegDB datasets demonstrate the superiority of our proposed method. Specifically, in the VIS→IR task, the experimental results on the RegDB dataset achieve 93.69% for Rank-1 and 92.56% for mAP.

Self-similarity guided probabilistic embedding matching based on transformer for occluded person re-identification

In real scenes, the occluded situation caused by different factors is the key to the person re-identification (Re-ID) problem. Many methods retrieve the corresponding images in the gallery by employing feature matching. However, the occluded region is still not well addressed, and the noise generated inevitably increases the difficulty of Re-ID. Therefore, we propose a Self-Similarity guided Probabilistic Embedding Matching (SSPEM) method to solve the problem of occluded person Re-ID. Specifically, we first design a Feature Similarity Enhancement (FSE) module to perform self-similarity calculation based on the extracted features of the same pedestrians by Vision Transformer (ViT) and complete feature enhancement by suppressing irrelevant information. Then, we design a probabilistic embedding matching (PEM) module to represent the features as probability distributions in a common embedding space, which is able to learn more feature structures. The uncertainty for the occluded region can maximize the metric computation to obtain features with better matching. Extensive experiments on five challenging datasets for occluded and holistic person Re-ID tasks are conducted, and the results show the effectiveness of our proposed SSPEM method.

Intermediary-Guided Bidirectional Spatial–Temporal Aggregation Network for Video-Based Visible-Infrared Person Re-Identification

This work focuses on the task of Video-based Visible-Infrared Person Re-Identification, a promising technique for achieving 24-hour surveillance systems. Two main issues in this field are modality discrepancy mitigating and spatial-temporal information mining. In this work, we propose a novel method, named Intermediary-guided Bidirectional spatial-temporal Aggregation Network (IBAN), to address both issues at once. Specifically, IBAN is designed to learn modality-irrelevant features by leveraging the anaglyph data of pedestrian images to serve as the intermediary. Furthermore, a bidirectional spatial-temporal aggregation module is introduced to exploit the spatial-temporal information of video data, while mitigating the impact of noisy image frames. Finally, we design an Easy-sample-based loss to guide the final embedding space and further improve the model’s generalization performance. Extensive experiments on Video-based Visible-Infrared benchmarks show that IBAN achieves promising results and outperforms the state-of-the-art ReID methods by a large margin, improving the rank-1/mAP by 1.29%/3.46% at the Infrared to Visible situation, and by 5.04%/3.27% at the Visible to Infrared situation. The source code of the proposed method will be released at https://github.com/lhf12278/IBAN.

Approaches to Improve the Quality of Person Re-Identification for Practical Use.

The idea of the person re-identification (Re-ID) task is to find the person depicted in the query image among other images obtained from different cameras. Algorithms solving this task have important practical applications, such as illegal action prevention and searching for missing persons through a smart city's video surveillance. In most of the papers devoted to the problem under consideration, the authors propose complex algorithms to achieve a better quality of person Re-ID. Some of these methods cannot be used in practice due to technical limitations. In this paper, we propose several approaches that can be used in almost all popular modern re-identification algorithms to improve the quality of the problem being solved and do not practically increase the computational complexity of algorithms. In real-world data, bad images can be fed into the input of the Re-ID algorithm; therefore, the new Filter Module is proposed in this paper, designed to pre-filter input data before feeding the data to the main re-identification algorithm. The Filter Module improves the quality of the baseline by 2.6% according to the Rank1 metric and 3.4% according to the mAP metric on the Market-1501 dataset. Furthermore, in this paper, a fully automated data collection strategy from surveillance cameras for self-supervised pre-training is proposed in order to increase the generality of neural networks on real-world data. The use of self-supervised pre-training on the data collected using the proposed strategy improves the quality of cross-domain upper-body Re-ID on the DukeMTMC-reID dataset by 1.0% according to the Rank1 and mAP metrics.

An efficient multi-scale channel attention network for person re-identification

At present, occlusion and similar appearance pose serious challenges to the task of person re-identification. In this work, we propose an efficient multi-scale channel attention network (EMCA) to learn robust and more discriminative features to solve these problems. Specifically, we designed a novel cross-channel attention module (CCAM) in EMCA and placed it after different layers in the backbone. The CCAM includes local cross-channel interaction (LCI) and channel weight integration (CWI). LCI focuses on both the maximum pooling features and the average pooling features to generate channel weights through convolutional layers, respectively. CWI combines the two channel weights to generate richer and more discriminant channel weights. Experiments on four popular person Re-ID datasets (Market-1501, DukeMTMC-ReID, CUHK-03 (detected) and MSMT17) show that the performance of our EMCA is consistently significantly superior to the existing state-of-the-art methods.