Person Re-identification Models Research Articles

A Multi-Level Relation-Aware Transformer model for occluded person re-identification

Occluded person re-identification (Re-ID) is a challenging task, as pedestrians are often obstructed by various occlusions, such as non-pedestrian objects or non-target pedestrians. Previous methods have heavily relied on auxiliary models to obtain information in unoccluded regions, such as human pose estimation. However, these auxiliary models fall short in accounting for pedestrian occlusions, thereby leading to potential misrepresentations. In addition, some previous works learned feature representations from single images, ignoring the potential relations among samples. To address these issues, this paper introduces a Multi-Level Relation-Aware Transformer (MLRAT) model for occluded person Re-ID. This model mainly encompasses two novel modules: Patch-Level Relation-Aware (PLRA) and Sample-Level Relation-Aware (SLRA). PLRA learns fine-grained local features by modeling the structural relations between key patches, bypassing the dependency on auxiliary models. It adopts a model-free method to select key patches that have high semantic correlation with the final pedestrian representation. In particular, to alleviate the interference of occlusion, PLRA captures the structural relations among key patches via a two-layer Graph Convolution Network (GCN), effectively guiding the local feature fusion and learning. SLRA is designed to facilitate the model to learn discriminative features by modeling the relations among samples. Specifically, to mitigate noisy relations of irrelevant samples, we present a Relation-Aware Transformer (RAT) block to capture the relations among neighbors. Furthermore, to bridge the gap between training and testing phases, a self-distillation method is employed to transfer the sample-level relations captured by SLRA to the backbone. Extensive experiments are conducted on four occluded datasets, two partial datasets and two holistic datasets. The results show that the proposed MLRAT model significantly outperforms existing baselines on four occluded datasets, while maintains top performance on two partial datasets and two holistic datasets.

Patch-based tendency camera multi-constraint learning for unsupervised person re-identification

Unsupervised person re-identification (ReID) is a task that aims to retrieve pedestrians across different cameras from unlabeled data. Existing methods rely on clustering to generate pseudo-labels, but they are inevitably noisy. Although pseudo-label refinement approaches have been presented, the essentiality of patch contours is ignored. The tendency analysis of retrieval between global and patch features has not been well investigated. In this paper, we propose a Patch-based Tendency Camera Multi-Constraint Learning (PTCML) model for unsupervised person ReID. First, to explore the tendentious retrieval of global and patch features, we design Ranking Tendency Similarity (RTS) score by gauging the distribution discrepancy of distance changes. Second, based on RTS score, we propose a Tendency-based Mutual Complementation (TMC) loss to improve the quality of global and patch pseudo-labels. Third, to resist camera variations, we propose an Adaptive Camera Multi-Constraint (ACM) loss to optimize recognition results with camera distribution constraint and instance constraint simultaneously. Finally, numerous experiments on Market-1501 and MSMT17 demonstrate that our method can significantly surpass the state-of-the-art performance.

Progressive spatial–temporal transfer model for unsupervised person re-identification

Progressive spatial–temporal transfer model for unsupervised person re-identification

Cross-modality person re-identification based on intermediate modal generation

In the context of cross-modal person re-identification, researchers often employ methods that utilize visible modality information to generate both an ‘X’ modality and a grayscale modality, enhancing the accuracy of person re-identification models. A lightweight network causes the ‘X’ modality through self-supervised learning of labels from visible images. In contrast, the grayscale modality is obtained through simple linear accumulation of the three RGB color channels from visual images. It can be observed that both the ‘X’ modality and grayscale modality are derived from visible images, which fails to establish a connection between the visible and infrared modalities. Therefore, this paper proposes an intermediate modality generation module to produce intermediate modality representations dynamically. By combining information from the visible, infrared, and intermediate modalities, the model is encouraged to capture modality-invariant features with cross-modal consistency. This enables person of the same identity to exhibit similar feature representations across different modalities, thereby mitigating the impact of distribution differences between visible and infrared modalities. Additionally, to facilitate the learning of appropriate intermediate modality representations, a distribution migration strategy is introduced. It guides the intermediate modality to maintain the correct distance from the visible and infrared modalities by optimizing the weights of the loss functions, preventing inadequate feature learning caused by an excessive focus on specific modality information. Furthermore, a mixed augmentation approach is proposed to alleviate disparities among multiple modalities further. By randomly cropping and combining regions of visible (infrared) modality images with infrared (visible) modality images, the generalization ability of the model in heterogeneous modalities is enhanced. Extensive comparative experiments are conducted on the SYSU-MM01 and RegDB datasets, yielding mAP values of 57.2% and 85.82%, respectively. The superior mAP performance on the RegDB dataset compared to most existing methods validates the effectiveness of the proposed approach.

Spatial-Temporal Federated Learning for Lifelong Person Re-identification on Distributed Edges

Data drift is a thorny challenge when deploying person re-identification (ReID) models into real-world devices, where the data distribution is significantly different from that of the training environment and keeps changing. To tackle this issue, we propose a federated spatial-temporal incremental learning approach, named FedSTIL, which leverages both lifelong learning and federated learning to continuously optimize models deployed on many distributed edge clients. Unlike previous efforts, FedSTIL aims to mine spatial-temporal correlations among the knowledge learnt from different edge clients. Specifically, the edge clients first periodically extract general representations of drifted data to optimize their local models. Then, the learnt knowledge from edge clients will be aggregated by centralized parameter server, where the knowledge will be selectively and attentively distilled from spatial- and temporal-dimension with carefully designed mechanisms. Finally, the distilled informative spatial-temporal knowledge will be sent back to correlated edge clients to further improve the recognition accuracy of each edge client with a lifelong learning method. Extensive experiments on a mixture of five real-world datasets demonstrate that our method outperforms others by nearly 4% in Rank-1 accuracy, while reducing communication cost by 62%. All implementation codes are publicly available on https://github.com/MSNLAB/Federated-Lifelong-Person-ReID.

Multiple integration model for single-source domain generalizable person re-identification

Domain generalizable (DG) person re-identification (re-ID) aims to train a model on labeled source domains which can perform well on invisible target domains. Because of the distribution shifts between different domains, it is a challenging task. Existing methods address this challenge by using multiple source domains to train a model which requires more data, manual labor, and computation. In contrast, we pay attention to the single-source DG re-ID task, that is, only one source domain data is accessible for training. However, due to the limited availability of training data, this task is more difficult. In this paper, a novel MulTiple Integration (MTI) model is introduced for single-source DG person re-ID. By integrating multiple reliable perturbations, the generalization performance can be improved. Specifically, MTI model contains two types of integration modules, one is shallow-level compensation (SLC) and the other is deep-level integration (DLI). For SLC, according to the idea of continual learning, the shallow-level information of the ImageNet pre-trained ResNet-50 branch is introduced and fused with the shallow-level information of our backbone network. In this way, massive information in ImageNet can be used to prevent the disastrous forgetting of the pre-trained information, and information compensation can be provided for backbone network. Additionally, we propose a hybrid integrated normalization layer to fuse information and improve the model’s generalization performance. For DLI, a wave transformer block is introduced in the deep layer of the backbone, which can integrate the information of a batch images and contain reliable disturbance, so that the robustness of the model can be promoted. Extensive experimental results demonstrate the superiority of our model.

Rethinking Person Re-Identification via Semantic-based Pretraining

Pretraining is a dominant paradigm in computer vision. Generally, supervised ImageNet pretraining is commonly used to initialize the backbones of person re-identification (Re-ID) models. However, recent works show a surprising result that CNN-based pretraining on ImageNet has limited impacts on Re-ID system due to the large domain gap between ImageNet and person Re-ID data. To seek an alternative to traditional pretraining, here we investigate semantic-based pretraining as another method to utilize additional textual data against ImageNet pretraining. Specifically, we manually construct a diversified FineGPR-C caption dataset for the first time on person Re-ID events. Based on it, a pure semantic-based pretraining approach named VTBR is proposed to adopt dense captions to learn visual representations with fewer images. We train convolutional neural networks from scratch on the captions of FineGPR-C dataset, and then transfer them to downstream Re-ID tasks. Comprehensive experiments conducted on benchmark datasets show that our VTBR can achieve competitive performance compared with ImageNet pretraining—despite using up to 1.4× fewer images, revealing its potential in Re-ID pretraining. Our source code is also publicly available at https://github.com/JeremyXSC/VTBR .

Meta separation–fusion for generalizable person re-identification

Domain generalization (DG) person re-identification (Re-ID) with a good generalization ability on unseen domains has received widespread attention. Existing DG person Re-ID methods usually adopt some multi-source domain training strategies to improve the generalization ability of their models. However, the style of unseen domains usually changes with application scenarios, which requires DG person Re-ID models to be able to adapt to variable real-world application scenarios. In view of this, we propose a meta separation–fusion network (MSF-Net) for DG Re-ID based on a multi-source domain training strategy and a meta-learning framework. Specifically, we first design a feature separation module to separate the identity-related and background features of samples. Then, we design a feature fusion module to diversify the features of meta-test samples. Through the feature separation and fusion, we can obtain meta-test sample features that contain multiple image styles. More importantly, the feature separation and fusion cooperate with each other to achieve a meta separation–fusion strategy, by which our model can learn more image styles using a limited number of samples in source domains and thus has a stronger generalization ability on unseen domains. Extensive experimental results show that our MSF-Net has reliable performance.

Appearance-Pose Joint Coordinates Information Collaboration Model for clothes-changing person re-identification

Clothes-changing person re-identification (CC-ReID) is of great importance in the field of intelligent surveillance as it can be used to help police officers find criminals who want to evade pursuit by changing clothes. Currently, the best-performing algorithm overly emphasizes discriminative information in appearance, which limits the generalization capability of CC-ReID. In this paper, we innovatively propose an Appearance-Pose Joint Coordinate Information Collaboration Model (AP-ICM), which further enhances the generalization capability of the CC-ReID method. FIRST, the pose joint coordinates, a novel auxiliary information that encodes the body shape, is introduced. Compared to the existing auxiliary information ( i.e. contour sketches, human masks, heatmaps) used to encode body shape, it avoids the interference of changes of clothing shape and the interference from areas unrelated to the joints, thus the model can learn discriminative information from the body shape more efficiently. Experiments demonstrate that the pose joint coordinates is more effective than these existing auxiliary information in assisting the model to learn discriminative information from the body shape. THEN, a novel Second-order Structural Information is designed to help the model learn the discriminative information from the pose joint coordinates more easily, thus improving the generalization capability. Experiment demonstrate that the Second-order Structural Information is effective in enhancing generalization capacity. NEXT, a novel Semantic Representation endows the model with the ability to understand semantic differences between joints, so that the model can learn features more accurately during training, which in turn makes it easier to acquire strong generalization capability. Experiment demonstrate that the Semantic Representation is effective in enhancing generalization capacity. FINALLY, by adopting a Information Collaboration Module, we successfully integrate discriminative information extracted from the pose joint coordinates into the best-performing algorithm. Experiments demonstrate that our AP-ICM achieves Rank-1 of 68.50% and 82.90% on two generic CC-ReID datasets, PRCC and VC-Clothes, respectively, outperforming the best-performing algorithm by 2.30% and 2.90%, respectively.

Multi-modal person re-identification based on transformer relational regularization

For robust multi-modal person re-identification (re-ID) models, it is crucial to effectively utilize the complementary information and constraint relationships among different modalities. However, current multi-modal methods often overlook the correlation between modalities at the feature fusion stage. To address this issue, we propose a novel multimodal person re-ID method called Transformer Relation Regularization (TRR). Firstly, we introduce an adaptive collaborative matching module that facilitates the exchange of useful information by mining feature correspondences between modalities. This module allows for the integration of complementary information, enhancing the re-ID performance. Secondly, we propose an enhanced embedded module that corrects general information using discriminative information within each modality. By leveraging this approach, we improve the model’s stability in challenging multi-modal environments. Lastly, we propose an adaptive triple loss to enhance sample utilization efficiency and mitigate the problem of inconsistent representation among multimodal samples. This loss function optimizes the model’s ability to distinguish between different individuals, leading to improved re-ID accuracy. Experimental results on several challenging visible-infrared person re-ID benchmark datasets demonstrate that our proposed TRR method achieves optimal performance. Additionally, extensive ablation studies validate the effective contribution of each component to the overall model. In summary, our proposed TRR method effectively leverages complementary information, addresses the correlation between modalities, and improves the re-ID performance in multi-modal scenarios. The results obtained from various benchmark datasets and the comprehensive analysis support the efficacy of our approach.

MHSA-Net: Multihead Self-Attention Network for Occluded Person Re-Identification.

This article presents a novel person reidentification model, named multihead self-attention network (MHSA-Net), to prune unimportant information and capture key local information from person images. MHSA-Net contains two main novel components: multihead self-attention branch (MHSAB) and attention competition mechanism (ACM). The MHSAB adaptively captures key local person information and then produces effective diversity embeddings of an image for the person matching. The ACM further helps filter out attention noise and nonkey information. Through extensive ablation studies, we verified that the MHSAB and ACM both contribute to the performance improvement of the MHSA-Net. Our MHSA-Net achieves competitive performance in the standard and occluded person Re-ID tasks.

Face and body-shape integration model for cloth-changing person re-identification

Among the existing deep learning-based person re-identification (ReID) methods, human parsing based on semantic segmentation is the most promising solution for ReID because such models can learn to identify fine-grained details of different body parts or apparel of a target semantically. However, intra-class variations such as illumination changes, multi-pose angles, and cloth-changing (CC) across different non-overlapping camera viewpoints present a crucial challenge for this approach. Among these challenges, a person CC is the most distinctive problem for ReID models, which often fail to associate the target in new cloth against the learned feature semantics of the previous cloth worn in a different timeline. In this paper, we propose a face and body-shape integration (FBI) network as a tactical solution to address the long-term person CC-ReID problem. The FBI comprises hierarchically stacked parsing and edge prediction (PEP) CNN blocks that generate fine-grained human-parsing output at the initial stage. We then aligned the PEP to our proposed model agnostic plug-in feature overlay module (FOM) to mask cloth-relevant body attributes except the facial features pooled from the input sample. Thus, our human parsing PEP and FOM modules are attuned to discriminatively learn cloth-irrelevant features of the target pedestrian(s) to optimize the effectiveness of person ReID in solitary or minimally crowded areas. In our extensive person CC-ReID experiments, our FBI model achieves 83.4/61.8 in R1 and 91.7/65.8 in mAP evaluation results on the PRCC and LTCC datasets, respectively; thereby significantly out-competing several previous state-of-the-art ReID methods, and validating the effectiveness of the FBI.

Stochastic Gradient Perturbation: An Implicit Regularizer for Person Re-Identification

Generalization of the person re-identification (ReID) model plays an important role in practical application, and we discuss a simple yet effective regularizer to improve it inspired by Adversarial Training (AT). AT has been indicated as an advanced regularizer due to its adversarial mechanism, ability to mine hard samples, and nature of data augmentation. However, serving as an augmentation-based regularizer, AT shows low diversity of the perturbation, excessive computational cost, and the optimization dilemma between adversarial robustness and accuracy for ReID task, and is thus suboptimal. To tackle these limitations and get a more effective regularizer for ReID, we rethink the nature of AT and unveil that the adversarial data augmentation is essentially reflected by gradients. Based on this, a novel implicit regularizer, named Stochastic Gradient Perturbation (SGP), is proposed, which naturally brings three merits: 1) Better diversity of the perturbation due to the proposed non-directional stochastic perturbations rather than directional adversarial perturbations. 2) Lower computational cost due to the proposed implicit gradient augmentation rather than explicitly additional data. 3) The optimization dilemma of the adversarial robustness and generalization is naturally overcome since SGP contains the adversarial gradient perturbation. Further, we put forward a perspective that the generalization and adversarial robustness may have an inter unity. Experiments on the baseline and SOTA models demonstrate powerful performances of the plugged-played SGP, and both generalization and adversarial robustness can be guaranteed.

Transformer-based global–local feature learning model for occluded person re-identification

Most recent occluded person re-identification (re-ID) methods usually learn global features directly from pedestrian images, or use additional pose estimation and semantic analysis model to learn local features, while ignoring the relationship between global and local features, thus incorrectly retrieving different pedestrians with similar attributes as the same pedestrian. Moreover, learning local features using auxiliary models brings additional computational cost. In this work, we propose a Transformer-based dual-branch feature learning model for occluded person re-ID. Firstly, we propose a global–local feature interaction module to learn the relationship between global and local features, thus enhancing the richness of information in pedestrian features. Secondly, we randomly erase local areas in the input image to simulate the real occlusion situation, thereby improving the model’s adaptability to the occlusion scene. Finally, a spilt group module is introduced to explore the local distinguishing features of pedestrian. Numerous experiments validate the effectiveness of our proposed method.

Learning consistent region features for lifelong person re-identification

The lifelong person re-identification (LRe-ID) model retrieves a person across multiple cameras in continuous data streams and learns new coming datasets incrementally. However, there are two well-known challenges: catastrophic forgetting and generalization loss, which arise due to parameter updates and domain shifts. While there has been encouraging progress in balancing these challenges, few existing methods have addressed them from a unified feature perspective. Inspired by the Complementary Learning Systems theory, an effective framework is proposed to share consistent features and extract discriminative features for each sample. Specifically, this framework consists of Property Region Features, Feature Adaption and Feature Perspicacity. Property Region Features are the parametric representation of consistent region features, Feature Adaption and Feature Perspicacity are responsible for diversity features generation and discriminative features extraction, respectively. Moreover, a cascade knowledge distillation structure is introduced to guarantee Property Feature consistency, and correspondingly, a weighted distillation loss function is designed to prevent generalization loss on current domains caused by overlapping historical knowledge. Extensive experiments conducted on twelve Re-ID datasets, including both rehearsal and no-rehearsal settings, clearly validate the superiority of our method over state-of-the-art competitors, with significantly improved performance. The source code will be released on https://github.com/whisperH/ConRFL/.

Structural redundancy reduction based efficient training for lightweight person re-identification

Deploying person re-identification (Re-ID) models in edge devices is of great importance in the field of intelligent surveillance. It can not only relieve the computational pressure on cloud data centers, but also effectively protects data privacy (e.g., pedestrian identity). However, current Re-ID methods mostly rely on the large-scale backbone with huge parameters, making them impossible to deploy in edge devices. To solve this problem, light-weight Re-ID backbones came into being. The current proposed light-weight Re-ID backbones are all multi-branch parallel structures based on multi-scale ideas. Nonetheless, an excessive number of parallel branches tend to extract redundant information, which takes up too much storage resources and easily leads to overfitting of the model structure. These problems limit the performance of the current models. In this paper, we innovatively propose a blocks-by-blocks combine-and-expand (BB-CE) training method. First we construct a base architecture using the proposed single-branch multi-scale (SMS) module. Then, by combining and expanding the blocks in the base architecture stage by stage during training, BB-CE training method can reduce the redundancy of the structure, utilize the multi-scale information more efficiently, and thus improve the matching accuracy of re-identification. Extensive experimental results on four generic person Re-ID datasets demonstrate that the proposed method outperforms the state-of-the-art algorithms.

COVID-19 contact tracking based on person reidentification and geospatial data

Efficient contact tracing is a crucial step in preventing the spread of COVID-19. However, the current methods rely heavily on manual investigation and truthful reporting by high-risk individuals. Mobile applications and Bluetooth-based contact tracing methods have also been adopted, but privacy concerns and reliance on personal data have limited their effectiveness. To address these challenges, in this paper, a geospatial big data method that combines person reidentification and geospatial information for contact tracing is proposed. The proposed real-time person reidentification model can identify individuals across multiple surveillance cameras, and the surveillance data is fused with geographic information and mapped onto a 3D geospatial model to track movement trajectories. After real-world verification, the proposed method achieves a first accuracy rate of 91.56%, a first-five accuracy rate of 97.70%, and a mean average precision of 78.03% with an inference speed of 13 ms per image. Importantly, the proposed method does not rely on personal information, mobile phones, or wearable devices, avoiding the limitations of existing contact tracing schemes and providing significant implications for public health in the post-COVID-19 era.

POSNet: a hybrid deep learning model for efficient person re-identification

POSNet: a hybrid deep learning model for efficient person re-identification

Learning transformer-based attention region with multiple scales for occluded person re-identification

Occluded person re-identification(Re-ID), with the aim of matching occluded person pairs under cross-camera, remains challenging due to incomplete information and spatial misalignment. The state-of-the-art (SOTA) methods usually include a two-stage architecture based on the existing pose estimation models or the attention mechanism to generate human masks to extract features, which complicate the model and introduce additional biases. To address this issue, we propose a novel end-to-end transformer-based occluded person Re-ID model. Specifically, our model contains two crucial components: (1) the features of global and non-occluded person regions are extracted by two independent Transformer-based feature extraction networks respectively; (2) the distribution of common non-occluded human regions is learnt via a multiheaded self-attention mechanism, and then the Minimized Character-box Proposal (MCP) is utilized to generate accurate shared non-occluded crops. In our model, non-occluded human regions are not annotated and only weakly-supervision of ID labels with multiheaded self-attention are employed to jointly learn the distribution. Further, the human feature contains multi-scale information which is extracted from our dual-branch architecture. Extensive experiment results on four benchmarks of person Re-ID for two tasks (occluded, partial) demonstrate the effectiveness of our proposed framework which achieves the SOTA or the comparable performance on all benchmarks.

Knowledge-Preserving continual person re-identification using Graph Attention Network

Person re-identification (ReID), considered as a sub-problem of image retrieval, is critical for intelligent security. The general practice is to train a deep model on images from a particular scenario (also known as a domain) and perform retrieval tests on images from the same domain. Thus, the model has to be retrained to ensure good performance on unseen domains. Unfortunately, retraining will introduce the so called catastrophic forgetting problem existing in deep learning models. To address this problem, we propose a Continual person re-identification model via a Knowledge-Preserving (CKP) mechanism. The proposed model is able to accumulate knowledge from continuously changing scenarios. The knowledge is updated via a graph attention network from the human cognitive-inspired perspective as the scenario changes. The accumulated knowledge is used to guide the learning process of the proposed model on image samples from new-coming domains. We finally evaluate and compare CKP with fine-tuning, continual learning in image classification and person re-identification, and joint training. Experiments on representative benchmark datasets (Market1501, DukeMTMC, CUHK03, CUHK-SYSU, and MSMT17, which arrive in different orders) demonstrate the advantages of the proposed model in preventing forgetting, and experiments on other benchmark datasets (GRID, SenseReID, CUHK01, CUHK02, VIPER, iLIDS, and PRID, which are not available during training) demonstrate the generalization ability of the proposed model. The CKP outperforms the best comparative model by 0.58% and 0.65% on seen domains (datasets available during training), and by 0.95% and 1.02% on never seen domains (datasets not available during training) in terms of mAP and Rank1, respectively. Arrival order of the training datasets, guidance of accumulated knowledge for learning new knowledge and parameter settings are also discussed.