Person Re-identification Methods Research Articles

Multi-modal person re-identification based on transformer relational regularization

For robust multi-modal person re-identification (re-ID) models, it is crucial to effectively utilize the complementary information and constraint relationships among different modalities. However, current multi-modal methods often overlook the correlation between modalities at the feature fusion stage. To address this issue, we propose a novel multimodal person re-ID method called Transformer Relation Regularization (TRR). Firstly, we introduce an adaptive collaborative matching module that facilitates the exchange of useful information by mining feature correspondences between modalities. This module allows for the integration of complementary information, enhancing the re-ID performance. Secondly, we propose an enhanced embedded module that corrects general information using discriminative information within each modality. By leveraging this approach, we improve the model’s stability in challenging multi-modal environments. Lastly, we propose an adaptive triple loss to enhance sample utilization efficiency and mitigate the problem of inconsistent representation among multimodal samples. This loss function optimizes the model’s ability to distinguish between different individuals, leading to improved re-ID accuracy. Experimental results on several challenging visible-infrared person re-ID benchmark datasets demonstrate that our proposed TRR method achieves optimal performance. Additionally, extensive ablation studies validate the effective contribution of each component to the overall model. In summary, our proposed TRR method effectively leverages complementary information, addresses the correlation between modalities, and improves the re-ID performance in multi-modal scenarios. The results obtained from various benchmark datasets and the comprehensive analysis support the efficacy of our approach.

Asymmetric double networks mutual teaching for unsupervised person Re-identification

Unsupervised person re-identification (Re-ID) has always been challenging in computer vision. It has received much attention from researchers because it does not require any labeled information and can be freely deployed to new scenarios. Most unsupervised person Re-ID research studies produce and optimize pseudo-labels by iterative clustering algorithms on a single network. However, these methods are easily affected by noisy labels and feature variations caused by camera shifts, which will limit the optimization of pseudo-labels. In this paper, we propose an Asymmetric Double Networks Mutual Teaching (ADNMT) architecture that uses two asymmetric networks to generate pseudo-labels for each other by clustering, and the pseudo-labels are updated and optimized by alternate training. Specifically, ADNMT contains two asymmetric networks. One network is a multiple granularity network, which extracts pedestrian features of multiple granularity that correspond to numerous classifiers, and the other network is a conventional backbone network, which extracts pedestrian features that correspond to a classifier. Furthermore, because the camera style changes seriously affect the generalization ability of the proposed model, this paper designs Similarity Compensation of Inter-Camera (SCIC) and Similarity Suppression of Intra-Camera (SSIC) according to the camera ID of the pedestrian images to optimize the similarity measure. Extensive experiments on multiple Re-ID benchmark datasets show that our proposed method achieves superior performance compared with the state‐of‐the‐art unsupervised person re-identification methods.

Unsupervised multi-source domain adaptation for person re-identification via sample weighting

The aim of unsupervised domain adaptation (UDA) in person re-identification (re-ID) is to develop a model that can identify the same individual across different cameras in the target domain, using labeled data from the source domain and unlabeled data from the target domain. However, existing UDA person re-ID methods typically assume a single source domain and a single target domain, and seldom consider the scenario of multiple source domains and a single target domain. In the latter scenario, differences in sample size between domains can lead to biased training of the model. To address this, we propose an unsupervised multi-source domain adaptation person re-ID method via sample weighting. Our approach utilizes multiple source domains to leverage valuable label information and balances the inter-domain sample imbalance through sample weighting. We also employ an adversarial learning method to align the domains. The experimental results, conducted on four datasets, demonstrate the effectiveness of our proposed method.

A Person Re-Identification Method Based on Multi-Branch Feature Fusion

Due to the lack of a specific design for scenarios such as scale change, illumination difference, and occlusion, current person re-identification methods are difficult to put into practice. A Multi-Branch Feature Fusion Network (MFFNet) is proposed, and Shallow Feature Extraction (SFF) and Multi-scale Feature Fusion (MFF) are utilized to obtain robust global feature representations while leveraging the Hybrid Attention Module (HAM) and Anti-erasure Federated Block Network (AFBN) to solve the problems of scale change, illumination difference and occlusion in scenes. Finally, multiple loss functions are used to efficiently converge the model parameters and enhance the information interaction between the branches. The experimental results show that our method achieves significant improvements over Market-1501, DukeMTMC-reID, and MSMT17. Especially on the MSMT17 dataset, which is close to real-world scenarios, MFFNet improves by 1.3 and 1.8% on Rank-1 and mAP, respectively.

Person re-identification method with Mahalanobis TRM triplet on multi-branch network

Person re-identification method with Mahalanobis TRM triplet on multi-branch network

Deep Learning Based Occluded Person Re-Identification: A Survey

Occluded person re-identification (Re-ID) focuses on addressing the occlusion problem when retrieving the person of interest across non-overlapping cameras. With the increasing demand for intelligent video surveillance and the application of person Re-ID technology, the real-world occlusion problem draws considerable interest from researchers. Although a large number of occluded person Re-ID methods have been proposed, there are few surveys that focus on occlusion. To fill this gap and help boost future research, this article provides a systematic survey of occluded person Re-ID. In this work, we review recent deep learning based occluded person Re-ID research. First, we summarize the main issues caused by occlusion as four groups: position misalignment, scale misalignment, noisy information, and missing information. Second, we categorize existing methods into six solution groups: matching, image transformation, multi-scale features, attention mechanism, auxiliary information, and contextual recovery. We also discuss the characteristics of each approach, as well as the issues they address. Furthermore, we present the performance comparison of recent occluded person Re-ID methods on four public datasets: Partial-ReID, Partial-iLIDS, Occluded-ReID, and Occluded-DukeMTMC. We conclude the study with thoughts on promising future research directions.

Face and body-shape integration model for cloth-changing person re-identification

Among the existing deep learning-based person re-identification (ReID) methods, human parsing based on semantic segmentation is the most promising solution for ReID because such models can learn to identify fine-grained details of different body parts or apparel of a target semantically. However, intra-class variations such as illumination changes, multi-pose angles, and cloth-changing (CC) across different non-overlapping camera viewpoints present a crucial challenge for this approach. Among these challenges, a person CC is the most distinctive problem for ReID models, which often fail to associate the target in new cloth against the learned feature semantics of the previous cloth worn in a different timeline. In this paper, we propose a face and body-shape integration (FBI) network as a tactical solution to address the long-term person CC-ReID problem. The FBI comprises hierarchically stacked parsing and edge prediction (PEP) CNN blocks that generate fine-grained human-parsing output at the initial stage. We then aligned the PEP to our proposed model agnostic plug-in feature overlay module (FOM) to mask cloth-relevant body attributes except the facial features pooled from the input sample. Thus, our human parsing PEP and FOM modules are attuned to discriminatively learn cloth-irrelevant features of the target pedestrian(s) to optimize the effectiveness of person ReID in solitary or minimally crowded areas. In our extensive person CC-ReID experiments, our FBI model achieves 83.4/61.8 in R1 and 91.7/65.8 in mAP evaluation results on the PRCC and LTCC datasets, respectively; thereby significantly out-competing several previous state-of-the-art ReID methods, and validating the effectiveness of the FBI.

Weakly Supervised Cross-Domain Person Re-Identification Algorithm Based on Small Sample Learning

This paper proposes a weakly supervised cross-domain person re-identification (Re-ID) method based on small sample data. In order to reduce the cost of data collection and annotation, the model design focuses on extracting and abstracting the information contained in the data under limited conditions. In this paper, we focus on the problems of strong data dependence, weak cross-domain capability and low accuracy in Re-ID in weakly supervised scenarios. Our contributions are as follows: first, we implement a joint training framework with the help of small sample learning and cross-domain migration for Re-ID. Second, with the help of residual compensation and fusion attention module, the RCFA module is designed, and the model framework is built on this basis to improve the cross-domain ability of the model. Third, to solve the problem of low accuracy caused by insufficient data coverage of small samples, a fusion of shallow features and deep features is designed to enable the model to weighted fusion of shallow detail information and deep semantic information. Finally, by selecting different camera images in Market1501 dataset and DukeMTMC-reID dataset as small samples, respectively, and introducing another dataset data for joint training, we demonstrate the feasibility of this joint training framework, which can perform weakly supervised cross-domain Re-ID based on small sample data.

Dual Clustering Co-Teaching With Consistent Sample Mining for Unsupervised Person Re-Identification

In unsupervised person Re-ID, peer-teaching strategy leveraging two networks to facilitate training has been proven to be an effective method to deal with the pseudo label noise. However, training two networks with a set of noisy pseudo labels reduces the complementarity of the two networks and results in label noise accumulation. To handle this issue, this paper proposes a novel Dual Clustering Co-teaching (DCCT) approach. DCCT mainly exploits the features extracted by two networks to generate two sets of pseudo labels separately by clustering with different parameters. Each network is trained with the pseudo labels generated by its peer network, which can increase the complementarity of the two networks to reduce the impact of noises. Furthermore, we propose dual clustering with dynamic parameters (DCDP) to make the network adaptive and robust to dynamically changing clustering parameters. Moreover, Consistent Sample Mining (CSM) is proposed to find the samples with unchanged pseudo labels during training for potential noisy sample removal. Extensive experiments demonstrate the effectiveness of the proposed method, which outperforms the state-of-the-art unsupervised person Re-ID methods by a considerable margin and surpasses most methods utilizing camera information.

An adaptive self-correction joint training framework for person re-identification with noisy labels

Current person re-identification (ReID) methods heavily rely on well-annotated training data, and their performance suffers from significant degradation in the presence of noisy labels that are ubiquitous in real-life scenes. The reason is that noisy labels not only affect the prediction results of the classifier, but also impede feature refinement, making it difficult to distinguish between different person features. To address these issues, we propose an Adaptive Self-correction Classification (ASC) loss and an Adaptive Margin Self-correction Triplet (AMSTri) loss. Specifically, ASC loss helps the network to produce better predictions by balancing annotations and prediction labels, and pays more attention to the minority samples with the help of a focusing factor. On the other hand, the AMSTri loss introduces an adaptive margin that varies with sample features to accommodate complex data variations, and utilizes predicted labels to generate reliable triples for feature refinement. We then present an end-to-end adaptive self-correction joint training framework incorporating ASC loss and AMSTri loss to achieve a robust ReID model. Our comprehensive experiments demonstrate that the proposed framework outperforms most existing counterparts.

Multi-Scale Semantic and Detail Extraction Network for Lightweight Person Re-Identification

Exploring multi-level information to obtain fine-grained features is the key factor to improve the performance of person re-identification (Re-ID). However, existing models for person Re-ID only focus on learning the high-level semantic information while neglecting the low-level detail information. To alleviate this issue, we propose a lightweight person Re-ID method termed Multi-Scale Semantic and Detail Extraction Network (MSDENet) to obtain robustness and discriminative feature representation for the Re-ID task. Specifically, we design a Series Channel-Spatial Attention (SCSA) and embed it into the lightweight backbone network to focus on the key parts of pedestrian images. Meanwhile, we propose a Multi-Scale Semantic and Detail Extraction (MSDE) method to extract multi-scale features of semantic information and detail information, which can effectively capture the feature diversity of pedestrian images. Furthermore, we design a Feature Enhancement Fusion (FEF), which enhances and fuses the fine-grained features of semantic extraction and detail extraction branches to better obtain the discriminative feature representation. Extensive experiments conducted on popular datasets Market1501, MSMT17, and CUHK03 demonstrate that the proposed MSDENet has competitive performance compared with the state-of-the-art methods.

Self-supervised consistency based on joint learning for unsupervised person re-identification

Recently, unsupervised domain-adaptive person Re-identification methods have been extensively studied thanks to not requiring annotations, and have achieved excellent performance. Most of the existing methods aim to train the Re-ID model for learning a discriminative feature representation. However, they usually only consider training the model to learn a global feature of a pedestrian image, but neglecting the local feature, which restricts the further improvement of model performance. To address this problem, two local branches are added to the networks, aiming to allow the model to focus on local feature containing identity information. Furthermore, we propose a self-supervised consistency constraint to further improve the robustness of the model. Specifically, the self-supervised consistency constraint uses the basic data augmentation operations without other auxiliary networks, which can improve the performance of the model effectively. Then, a learnable memory matrix is designed to store the mapping vectors that maps person features into probability distributions. Finally, extensive experiments are conducted on multiple commonly used person Re-ID datasets to verify the effectiveness of the proposed generative adversarial networks fusing global and local features. Experimental results reveal that our method achieves comparable results to the state-of-the-art methods.

MSIF: multi-spectrum image fusion method for cross-modality person re-identification

MSIF: multi-spectrum image fusion method for cross-modality person re-identification

A double transformer residual super-resolution network for cross-resolution person re-identification

Cross-resolution person re-identification is a challenging problem in the field of person re-identification. In order to solve the problem of resolution mismatch, many studies introduce super-resolution into person re-identification tasks. In this work, we propose a cross-resolution person re-identification method based on double transformer residual super-resolution network (DTRSR), which mainly includes super-resolution module and person re-identification module. In the super-resolution module, we propose the double transformer network as our attention module. First of all, we divide the features extracted by the residual network. Then calculate the similarity between each local feature and the global feature after average pooling and maximum pooling respectively, which makes our module quickly capture the hidden weight information in the spatial domain. In the person re-identification module, we propose an effective fusion method based on key point features (KPFF). The key point extraction model can not only solve the problem that local features can not be accurately aligned, but also remove the interference of background noise. In order to fully mine the relationship between the features of each key point, we calculate the two-way correlation between each key point feature and other features, and then superimpose the two-way correlation with the feature itself to get the superposition feature which contains global and local information. The effectiveness of this method is proved by extensive experiments. Compared with the most advanced methods, the test results in the three datasets show that our method improves rank-1 by 1.1%, 3.5% and 1.7%; and rank-5 by 1.3%, 1.7% and 0.3%; and rank-10 by 0.1%, 0.4% and 0.1%, respectively.

Prompting and Tuning: A Two-Stage Unsupervised Domain Adaptive Person Re-identification Method on Vision Transformer Backbone

This paper explores the Vision Transformer (ViT) backbone for Unsupervised Domain Adaptive (UDA) person Re-Identification (Re-ID). While some recent studies have validated ViT for supervised Re-ID, no study has yet to use ViT for UDA Re-ID. We observe that the ViT structure provides a unique advantage for UDA Re-ID, i.e., it has a prompt (the learnable class token) at its bottom layer, that can be used to efficiently condition the deep model for the underlying domain. To utilize this advantage, we propose a novel two-stage UDA pipeline named Prompting And Tuning (PAT) which consists of a prompt learning stage and a subsequent fine-tuning stage. In the first stage, PAT roughly adapts the model from source to target domain by learning the prompts for two domains, while in the second stage, PAT fine-tunes the entire backbone for further adaption to increase the accuracy. Although these two stages both adopt the pseudo labels for training, we show that they have different data preferences. With these two preferences, prompt learning and fine-tuning integrated well with each other and jointly facilitated a competitive PAT method for UDA Re-ID.

Transformer-based global–local feature learning model for occluded person re-identification

Most recent occluded person re-identification (re-ID) methods usually learn global features directly from pedestrian images, or use additional pose estimation and semantic analysis model to learn local features, while ignoring the relationship between global and local features, thus incorrectly retrieving different pedestrians with similar attributes as the same pedestrian. Moreover, learning local features using auxiliary models brings additional computational cost. In this work, we propose a Transformer-based dual-branch feature learning model for occluded person re-ID. Firstly, we propose a global–local feature interaction module to learn the relationship between global and local features, thus enhancing the richness of information in pedestrian features. Secondly, we randomly erase local areas in the input image to simulate the real occlusion situation, thereby improving the model’s adaptability to the occlusion scene. Finally, a spilt group module is introduced to explore the local distinguishing features of pedestrian. Numerous experiments validate the effectiveness of our proposed method.

Robust feature mining transformer for occluded person re-identification

The person Re-identification (ReID) technology in real scenes is still a challenging task, especially when the target pedestrian is occluded. Existing occluded person ReID methods distinguish visible regions through pose information or semantic information. These methods focus on the body information in the visible region, which can perform well for ReID. However, since these approaches require the help of auxiliary networks, they are not easy to obtain efficient models. In addition, under the pose guidance, these networks focus only on the visible body parts and ignore other regional features. Therefore, we propose a Robust Feature Mining Transformer (RFMT) method to tackle the above problems of occluded person re-identification. Specifically, when dealing with occlusion, the proposed Residual Transformer (ResT) layer allows for a more efficient spread of image features through cross-layer connections, thereby reducing the loss of pedestrian information. Furthermore, obtaining complete pedestrian features becomes a big challenge in occluded scenes. Our proposed Global Attention (GAtt) enhances feature extraction by accurately focusing on the most representative part of images. And then, we design a hyperparametric centroid loss (Lctr) to catch stable features. Meanwhile, this loss can reduce the image search time during training and retrieval. Many experiments have demonstrated that our RFMT performs better than the state-of-the-art ReID methods on public occluded and holistic person re-identification datasets.

Cross-Modality Person Re-Identification Algorithm Based on Two-Branch Network

Person re-identification is the technique of identifying the same person in different camera shots, known as ReID for short. Most existing models focus on single-modality person re-identification involving only visible images. However, the visible modality is not suitable for low-light environments or at night, when crime is frequent. In contrast, infrared images can reflect the nighttime environment, and most surveillance systems are equipped with dual-mode cameras that can automatically switch between visible and infrared modalities based on light conditions. In contrast to visible-light cameras, infrared (IR) cameras can still capture enough information from the scene in those dark environments. Therefore, the problem of visible-infrared cross-modality person re-identification (VI-ReID) is proposed. To improve the identification rate of cross-modality person re-identification, a cross-modality person re-identification method based on a two-branch network is proposed. Firstly, we use infrared image colorization technology to convert infrared images into color images to reduce the differences between modalities and propose a visible-infrared cross-modality person re-identification algorithm based on Two-Branch Network with Double Constraints (VI-TBNDC), which consists of two main components: a two-branch network for feature extraction and a double-constrained identity loss for feature learning. The two-branch network extracts the features of both data sets separately, and the double-constrained identity loss ensures that the learned feature representations are discriminative enough to distinguish different people from two different patterns. The effectiveness of the proposed method is verified by extensive experimental analysis, and the method achieves good recognition accuracy on the visible-infrared image person re-identification standard dataset SYSU-MM01.

Few-shot person re-identification based on Feature Set Augmentation and Metric Fusion

Person re-identification identifies pedestrians by analyzing image information from surveillance videos. However, it faces challenges like occlusion, changing lighting, and costly annotation. Thus, it is often performed in a few-shot environment with limited images. In response to the problem of insufficient available pedestrian images in person re-identification, a few-shot person re-identification method based on feature set augmentation and metric fusion is proposed. In this work, firstly, a feature augmentation method is introduced into the feature embedding module. This method introduces multi-head self-attention in different feature extraction layers and spatial attention in feature fusion of different feature extraction layers, which can extract more diverse and discriminative pedestrian features. Secondly, a dual metric method combining Euclidean and cosine distance is proposed in the metric module to comprehensively measure the absolute spatial distance and directional difference of pedestrian features. In this way, the reliability of pedestrian similarity measurement is improved. Then, pedestrian feature similarity scores are obtained separately using the dual metric and relation metric methods. Finally, the combined metric score is obtained by weighted fusion, and the combined metric score is used to construct the joint loss to realize the overall optimization and training of the network. Experimental results on three small datasets, Market-mini, Duke-mini, and MSMT17-mini, show that the proposed method significantly improves recognition performance compared to other few-shot learning algorithms. Specifically, in scenarios 5-way 1-shot and 5-way 5-shot, the average recognition accuracies are 92.54% and 96.99%, 87.93% and 96.08%, and 71.68% and 84.51%, respectively.

Cross-Domain Person Re-identification Combining Feature Concatenation and Attention

To improve the insufficient generalization and poor cross-domain capability of the existing direct cross-dataset person re-identification methods, a cross-domain person re-identification method combining feature concatenation and attention (FCANet) is proposed. The deep features of the network are concatenated to complement the feature information and obtain discriminatively feature, and the position attention module is introduced to enhance the data feature representation capability of the cross-domain task, using the joint training network of label smooth cross-entropy loss and triplet loss, model training in the source domain, and directly deploy to the target domain for testing. To verify the performance of the proposed method, it was experimented on three public datasets of Market1501, DukeMTMC-reID and MSMT17, which mAP and Rank1can reach 51.4% and 62.7% on Market1501. The results show that the proposed method has good performance in improving the generalization of cross-domain tasks, and the recognition accuracy outperforms the domain generalization algorithms of comparison.