Person Re-identification Models Research Articles

A Person Re-Identification Method for Fusing Convolutional Attention and Transformer Architecture

Rapid and accurate localization of target person via person re-identification is one of the important methods in intelligent security systems,but faces great challenges due to complex environmental changes. In order to build a model applicable to various complex scenes, deep learning-based person re-identification models have become a key research direction for researchers. The existing convolutional neural network-based or transformer-based deep learning models are still have certain shortcomings in the extraction of local details and are not able to cope well with realistic complex scenes. To address this problem, a method fusing convolutional neural network and transformer (FCAT) is proposed to enhance the transformer's attention to local detail information. This method mainly improves the transformer's ability to extract local detail features indirectly by embedding convolutional space attention and channel attention respectively to enhance the attention to important regions and important channel features in the image. Comparative ablation experiments on three publicly available person re-identification datasets demonstrate that FCAT achieves comparable results to existing methods on non-occluded datasets, and significantly improves performance on occluded datasets. The proposed model is also has a lightweight effect, and the inference speed is improved without increasing the amount of outer computation and model parameters.

In-Depth Analysis of GAF-Net: Comparative Fusion Approaches in Video-Based Person Re-Identification

This study provides an in-depth analysis of GAF-Net, a novel model for video-based person re-identification (Re-ID) that matches individuals across different video sequences. GAF-Net combines appearance-based features with gait-based features derived from skeletal data, offering a new approach that diverges from traditional silhouette-based methods. We thoroughly examine each module of GAF-Net and explore various fusion methods at the both score and feature levels, extending beyond initial simple concatenation. Comprehensive evaluations on the iLIDS-VID and MARS datasets demonstrate GAF-Net’s effectiveness across scenarios. GAF-Net achieves state-of-the-art 93.2% rank-1 accuracy on iLIDS-VID’s long sequences, while MARS results (86.09% mAP, 89.78% rank-1) reveal challenges with shorter, variable sequences in complex real-world settings. We demonstrate that integrating skeleton-based gait features consistently improves Re-ID performance, particularly with long, more informative sequences. This research provides crucial insights into multi-modal feature integration in Re-ID tasks, laying a foundation for the advancement of multi-modal biometric systems for diverse computer vision applications.

GCReID: Generalized continual person re-identification via meta learning and knowledge accumulation

Person re-identification (ReID) has made good progress in stationary domains. The ReID model must be retrained to adapt to new scenarios (domains) as they emerge unexpectedly, which leads to catastrophic forgetting. Continual learning trains the model in the order of domain emergence to alleviate catastrophic forgetting. However, generalization ability of the model is still limited due to the distribution difference between training and testing domains. To address the above problem, we propose the generalized continual person re-Identification (GCReID) model to continuously train an anti-forgetting and generalizable model. We endeavor to increase the diversity of samples by prior to simulate unseen domains. Meta-train and meta-test are adopted to enhance generalization of the model. Universal knowledge extracted from all seen domains and the simulated domains is stored in a set of feature embeddings. The knowledge is continually updated and applied to guide meta-train and meta-test via a graph attention network. Extensive experiments on 12 benchmark datasets and comparisons with 6 representative models demonstrate the effectiveness of the proposed model GCReID in enhancing generalization performance on unseen domains and alleviating catastrophic forgetting of seen domains. The code will be available at https://github.com/DFLAG-NEU/GCReID if our work is accepted.

PSF-C-Net: A Counterfactual Deep Learning Model for Person Re-Identification Based on Random Cropping Patch and Shuffling Filling

In the task of person re-identification (re-ID), capturing the long-range dependency of instances is crucial for accurate identification. The existing methods excel at extracting local features but often overlook the global information of instance images. To address this limitation, we propose a convolution-based counterfactual learning framework, called PSF-C-Net, to focus on global information rather than local detailed features. PSF-C-Net adopts a parameter-sharing dual-path structure to perform counterfactual operations in the prediction space. It takes both the actual instance image and a counterfactual instance image that disrupts the contextual relationship as the input. The counterfactual framework enables the interpretable modeling of global features without introducing additional parameters. Additionally, we propose a novel method for generating counterfactual instance images, which effectively constructs an explicit counterfactual space, to reliably implement counterfactual strategies. We have conducted extensive experiments to evaluate the performance of PSF-C-Net on the Market-1501 and Duke-MTMC-reID datasets. The results demonstrate that PSF-C-Net achieves state-of-the-art performance.

A Multi-Level Relation-Aware Transformer model for occluded person re-identification

Occluded person re-identification (Re-ID) is a challenging task, as pedestrians are often obstructed by various occlusions, such as non-pedestrian objects or non-target pedestrians. Previous methods have heavily relied on auxiliary models to obtain information in unoccluded regions, such as human pose estimation. However, these auxiliary models fall short in accounting for pedestrian occlusions, thereby leading to potential misrepresentations. In addition, some previous works learned feature representations from single images, ignoring the potential relations among samples. To address these issues, this paper introduces a Multi-Level Relation-Aware Transformer (MLRAT) model for occluded person Re-ID. This model mainly encompasses two novel modules: Patch-Level Relation-Aware (PLRA) and Sample-Level Relation-Aware (SLRA). PLRA learns fine-grained local features by modeling the structural relations between key patches, bypassing the dependency on auxiliary models. It adopts a model-free method to select key patches that have high semantic correlation with the final pedestrian representation. In particular, to alleviate the interference of occlusion, PLRA captures the structural relations among key patches via a two-layer Graph Convolution Network (GCN), effectively guiding the local feature fusion and learning. SLRA is designed to facilitate the model to learn discriminative features by modeling the relations among samples. Specifically, to mitigate noisy relations of irrelevant samples, we present a Relation-Aware Transformer (RAT) block to capture the relations among neighbors. Furthermore, to bridge the gap between training and testing phases, a self-distillation method is employed to transfer the sample-level relations captured by SLRA to the backbone. Extensive experiments are conducted on four occluded datasets, two partial datasets and two holistic datasets. The results show that the proposed MLRAT model significantly outperforms existing baselines on four occluded datasets, while maintains top performance on two partial datasets and two holistic datasets.

Patch-based tendency camera multi-constraint learning for unsupervised person re-identification

Unsupervised person re-identification (ReID) is a task that aims to retrieve pedestrians across different cameras from unlabeled data. Existing methods rely on clustering to generate pseudo-labels, but they are inevitably noisy. Although pseudo-label refinement approaches have been presented, the essentiality of patch contours is ignored. The tendency analysis of retrieval between global and patch features has not been well investigated. In this paper, we propose a Patch-based Tendency Camera Multi-Constraint Learning (PTCML) model for unsupervised person ReID. First, to explore the tendentious retrieval of global and patch features, we design a Ranking Tendency Similarity (RTS) score by gauging the distribution discrepancy of distance changes. Second, based on RTS score, we propose a Tendency-based Mutual Complementation (TMC) loss to improve the quality of global and patch pseudo-labels. Third, to resist camera variations, we propose an Adaptive Camera Multi-Constraint (ACM) loss to optimize recognition results with camera distribution constraint and instance constraint simultaneously. Finally, numerous experiments on Market-1501 and MSMT17 demonstrate that our method can significantly surpass the state-of-the-art performance.

Progressive spatial–temporal transfer model for unsupervised person re-identification

Progressive spatial–temporal transfer model for unsupervised person re-identification

Cross-modality person re-identification based on intermediate modal generation

In the context of cross-modal person re-identification, researchers often employ methods that utilize visible modality information to generate both an ‘X’ modality and a grayscale modality, enhancing the accuracy of person re-identification models. A lightweight network causes the ‘X’ modality through self-supervised learning of labels from visible images. In contrast, the grayscale modality is obtained through simple linear accumulation of the three RGB color channels from visual images. It can be observed that both the ‘X’ modality and grayscale modality are derived from visible images, which fails to establish a connection between the visible and infrared modalities. Therefore, this paper proposes an intermediate modality generation module to produce intermediate modality representations dynamically. By combining information from the visible, infrared, and intermediate modalities, the model is encouraged to capture modality-invariant features with cross-modal consistency. This enables person of the same identity to exhibit similar feature representations across different modalities, thereby mitigating the impact of distribution differences between visible and infrared modalities. Additionally, to facilitate the learning of appropriate intermediate modality representations, a distribution migration strategy is introduced. It guides the intermediate modality to maintain the correct distance from the visible and infrared modalities by optimizing the weights of the loss functions, preventing inadequate feature learning caused by an excessive focus on specific modality information. Furthermore, a mixed augmentation approach is proposed to alleviate disparities among multiple modalities further. By randomly cropping and combining regions of visible (infrared) modality images with infrared (visible) modality images, the generalization ability of the model in heterogeneous modalities is enhanced. Extensive comparative experiments are conducted on the SYSU-MM01 and RegDB datasets, yielding mAP values of 57.2% and 85.82%, respectively. The superior mAP performance on the RegDB dataset compared to most existing methods validates the effectiveness of the proposed approach.

Spatial-Temporal Federated Learning for Lifelong Person Re-identification on Distributed Edges

Data drift is a thorny challenge when deploying person re-identification (ReID) models into real-world devices, where the data distribution is significantly different from that of the training environment and keeps changing. To tackle this issue, we propose a federated spatial-temporal incremental learning approach, named FedSTIL, which leverages both lifelong learning and federated learning to continuously optimize models deployed on many distributed edge clients. Unlike previous efforts, FedSTIL aims to mine spatial-temporal correlations among the knowledge learnt from different edge clients. Specifically, the edge clients first periodically extract general representations of drifted data to optimize their local models. Then, the learnt knowledge from edge clients will be aggregated by centralized parameter server, where the knowledge will be selectively and attentively distilled from spatial- and temporal-dimension with carefully designed mechanisms. Finally, the distilled informative spatial-temporal knowledge will be sent back to correlated edge clients to further improve the recognition accuracy of each edge client with a lifelong learning method. Extensive experiments on a mixture of five real-world datasets demonstrate that our method outperforms others by nearly 4% in Rank-1 accuracy, while reducing communication cost by 62%. All implementation codes are publicly available on https://github.com/MSNLAB/Federated-Lifelong-Person-ReID.

Towards universal person re-identification： survey on the applications of large-scale self-supervised pre-train model for person re-identification

Towards universal person re-identification： survey on the applications of large-scale self-supervised pre-train model for person re-identification

Anti-Forgetting Adaptation for Unsupervised Person Re-Identification.

Regular unsupervised domain adaptive person re-identification (ReID) focuses on adapting a model from a source domain to a fixed target domain. However, an adapted ReID model can hardly retain previously-acquired knowledge and generalize to unseen data. In this paper, we propose a Dual-level Joint Adaptation and Anti-forgetting (DJAA) framework, which incrementally adapts a model to new domains without forgetting source domain and each adapted target domain. We explore the possibility of using prototype and instance-level consistency to mitigate the forgetting during the adaptation. Specifically, we store a small number of representative image samples and corresponding cluster prototypes in a memory buffer, which is updated at each adaptation step. With the buffered images and prototypes, we regularize the image-to-image similarity and image-to-prototype similarity to rehearse old knowledge. After the multi-step adaptation, the model is tested on all seen domains and several unseen domains to validate the generalization ability of our method. Extensive experiments demonstrate that our proposed method significantly improves the anti-forgetting, generalization and backward-compatible ability of an unsupervised person ReID model.

Multiple integration model for single-source domain generalizable person re-identification

Domain generalizable (DG) person re-identification (re-ID) aims to train a model on labeled source domains which can perform well on invisible target domains. Because of the distribution shifts between different domains, it is a challenging task. Existing methods address this challenge by using multiple source domains to train a model which requires more data, manual labor, and computation. In contrast, we pay attention to the single-source DG re-ID task, that is, only one source domain data is accessible for training. However, due to the limited availability of training data, this task is more difficult. In this paper, a novel MulTiple Integration (MTI) model is introduced for single-source DG person re-ID. By integrating multiple reliable perturbations, the generalization performance can be improved. Specifically, MTI model contains two types of integration modules, one is shallow-level compensation (SLC) and the other is deep-level integration (DLI). For SLC, according to the idea of continual learning, the shallow-level information of the ImageNet pre-trained ResNet-50 branch is introduced and fused with the shallow-level information of our backbone network. In this way, massive information in ImageNet can be used to prevent the disastrous forgetting of the pre-trained information, and information compensation can be provided for backbone network. Additionally, we propose a hybrid integrated normalization layer to fuse information and improve the model’s generalization performance. For DLI, a wave transformer block is introduced in the deep layer of the backbone, which can integrate the information of a batch images and contain reliable disturbance, so that the robustness of the model can be promoted. Extensive experimental results demonstrate the superiority of our model.

Rethinking Person Re-Identification via Semantic-based Pretraining

Pretraining is a dominant paradigm in computer vision. Generally, supervised ImageNet pretraining is commonly used to initialize the backbones of person re-identification (Re-ID) models. However, recent works show a surprising result that CNN-based pretraining on ImageNet has limited impacts on Re-ID system due to the large domain gap between ImageNet and person Re-ID data. To seek an alternative to traditional pretraining, here we investigate semantic-based pretraining as another method to utilize additional textual data against ImageNet pretraining. Specifically, we manually construct a diversified FineGPR-C caption dataset for the first time on person Re-ID events. Based on it, a pure semantic-based pretraining approach named VTBR is proposed to adopt dense captions to learn visual representations with fewer images. We train convolutional neural networks from scratch on the captions of FineGPR-C dataset, and then transfer them to downstream Re-ID tasks. Comprehensive experiments conducted on benchmark datasets show that our VTBR can achieve competitive performance compared with ImageNet pretraining—despite using up to 1.4× fewer images, revealing its potential in Re-ID pretraining. Our source code is also publicly available at https://github.com/JeremyXSC/VTBR .

Meta separation–fusion for generalizable person re-identification

Domain generalization (DG) person re-identification (Re-ID) with a good generalization ability on unseen domains has received widespread attention. Existing DG person Re-ID methods usually adopt some multi-source domain training strategies to improve the generalization ability of their models. However, the style of unseen domains usually changes with application scenarios, which requires DG person Re-ID models to be able to adapt to variable real-world application scenarios. In view of this, we propose a meta separation–fusion network (MSF-Net) for DG Re-ID based on a multi-source domain training strategy and a meta-learning framework. Specifically, we first design a feature separation module to separate the identity-related and background features of samples. Then, we design a feature fusion module to diversify the features of meta-test samples. Through the feature separation and fusion, we can obtain meta-test sample features that contain multiple image styles. More importantly, the feature separation and fusion cooperate with each other to achieve a meta separation–fusion strategy, by which our model can learn more image styles using a limited number of samples in source domains and thus has a stronger generalization ability on unseen domains. Extensive experimental results show that our MSF-Net has reliable performance.

Appearance-Pose Joint Coordinates Information Collaboration Model for clothes-changing person re-identification

Clothes-changing person re-identification (CC-ReID) is of great importance in the field of intelligent surveillance as it can be used to help police officers find criminals who want to evade pursuit by changing clothes. Currently, the best-performing algorithm overly emphasizes discriminative information in appearance, which limits the generalization capability of CC-ReID. In this paper, we innovatively propose an Appearance-Pose Joint Coordinate Information Collaboration Model (AP-ICM), which further enhances the generalization capability of the CC-ReID method. FIRST, the pose joint coordinates, a novel auxiliary information that encodes the body shape, is introduced. Compared to the existing auxiliary information ( i.e. contour sketches, human masks, heatmaps) used to encode body shape, it avoids the interference of changes of clothing shape and the interference from areas unrelated to the joints, thus the model can learn discriminative information from the body shape more efficiently. Experiments demonstrate that the pose joint coordinates is more effective than these existing auxiliary information in assisting the model to learn discriminative information from the body shape. THEN, a novel Second-order Structural Information is designed to help the model learn the discriminative information from the pose joint coordinates more easily, thus improving the generalization capability. Experiment demonstrate that the Second-order Structural Information is effective in enhancing generalization capacity. NEXT, a novel Semantic Representation endows the model with the ability to understand semantic differences between joints, so that the model can learn features more accurately during training, which in turn makes it easier to acquire strong generalization capability. Experiment demonstrate that the Semantic Representation is effective in enhancing generalization capacity. FINALLY, by adopting a Information Collaboration Module, we successfully integrate discriminative information extracted from the pose joint coordinates into the best-performing algorithm. Experiments demonstrate that our AP-ICM achieves Rank-1 of 68.50% and 82.90% on two generic CC-ReID datasets, PRCC and VC-Clothes, respectively, outperforming the best-performing algorithm by 2.30% and 2.90%, respectively.

Multi-modal person re-identification based on transformer relational regularization

For robust multi-modal person re-identification (re-ID) models, it is crucial to effectively utilize the complementary information and constraint relationships among different modalities. However, current multi-modal methods often overlook the correlation between modalities at the feature fusion stage. To address this issue, we propose a novel multimodal person re-ID method called Transformer Relation Regularization (TRR). Firstly, we introduce an adaptive collaborative matching module that facilitates the exchange of useful information by mining feature correspondences between modalities. This module allows for the integration of complementary information, enhancing the re-ID performance. Secondly, we propose an enhanced embedded module that corrects general information using discriminative information within each modality. By leveraging this approach, we improve the model’s stability in challenging multi-modal environments. Lastly, we propose an adaptive triple loss to enhance sample utilization efficiency and mitigate the problem of inconsistent representation among multimodal samples. This loss function optimizes the model’s ability to distinguish between different individuals, leading to improved re-ID accuracy. Experimental results on several challenging visible-infrared person re-ID benchmark datasets demonstrate that our proposed TRR method achieves optimal performance. Additionally, extensive ablation studies validate the effective contribution of each component to the overall model. In summary, our proposed TRR method effectively leverages complementary information, addresses the correlation between modalities, and improves the re-ID performance in multi-modal scenarios. The results obtained from various benchmark datasets and the comprehensive analysis support the efficacy of our approach.

MHSA-Net: Multihead Self-Attention Network for Occluded Person Re-Identification.

This article presents a novel person reidentification model, named multihead self-attention network (MHSA-Net), to prune unimportant information and capture key local information from person images. MHSA-Net contains two main novel components: multihead self-attention branch (MHSAB) and attention competition mechanism (ACM). The MHSAB adaptively captures key local person information and then produces effective diversity embeddings of an image for the person matching. The ACM further helps filter out attention noise and nonkey information. Through extensive ablation studies, we verified that the MHSAB and ACM both contribute to the performance improvement of the MHSA-Net. Our MHSA-Net achieves competitive performance in the standard and occluded person Re-ID tasks.

Face and body-shape integration model for cloth-changing person re-identification

Among the existing deep learning-based person re-identification (ReID) methods, human parsing based on semantic segmentation is the most promising solution for ReID because such models can learn to identify fine-grained details of different body parts or apparel of a target semantically. However, intra-class variations such as illumination changes, multi-pose angles, and cloth-changing (CC) across different non-overlapping camera viewpoints present a crucial challenge for this approach. Among these challenges, a person CC is the most distinctive problem for ReID models, which often fail to associate the target in new cloth against the learned feature semantics of the previous cloth worn in a different timeline. In this paper, we propose a face and body-shape integration (FBI) network as a tactical solution to address the long-term person CC-ReID problem. The FBI comprises hierarchically stacked parsing and edge prediction (PEP) CNN blocks that generate fine-grained human-parsing output at the initial stage. We then aligned the PEP to our proposed model agnostic plug-in feature overlay module (FOM) to mask cloth-relevant body attributes except the facial features pooled from the input sample. Thus, our human parsing PEP and FOM modules are attuned to discriminatively learn cloth-irrelevant features of the target pedestrian(s) to optimize the effectiveness of person ReID in solitary or minimally crowded areas. In our extensive person CC-ReID experiments, our FBI model achieves 83.4/61.8 in R1 and 91.7/65.8 in mAP evaluation results on the PRCC and LTCC datasets, respectively; thereby significantly out-competing several previous state-of-the-art ReID methods, and validating the effectiveness of the FBI.

Stochastic Gradient Perturbation: An Implicit Regularizer for Person Re-Identification

Generalization of the person re-identification (ReID) model plays an important role in practical application, and we discuss a simple yet effective regularizer to improve it inspired by Adversarial Training (AT). AT has been indicated as an advanced regularizer due to its adversarial mechanism, ability to mine hard samples, and nature of data augmentation. However, serving as an augmentation-based regularizer, AT shows low diversity of the perturbation, excessive computational cost, and the optimization dilemma between adversarial robustness and accuracy for ReID task, and is thus suboptimal. To tackle these limitations and get a more effective regularizer for ReID, we rethink the nature of AT and unveil that the adversarial data augmentation is essentially reflected by gradients. Based on this, a novel implicit regularizer, named Stochastic Gradient Perturbation (SGP), is proposed, which naturally brings three merits: 1) Better diversity of the perturbation due to the proposed non-directional stochastic perturbations rather than directional adversarial perturbations. 2) Lower computational cost due to the proposed implicit gradient augmentation rather than explicitly additional data. 3) The optimization dilemma of the adversarial robustness and generalization is naturally overcome since SGP contains the adversarial gradient perturbation. Further, we put forward a perspective that the generalization and adversarial robustness may have an inter unity. Experiments on the baseline and SOTA models demonstrate powerful performances of the plugged-played SGP, and both generalization and adversarial robustness can be guaranteed.

Transformer-based global–local feature learning model for occluded person re-identification

Most recent occluded person re-identification (re-ID) methods usually learn global features directly from pedestrian images, or use additional pose estimation and semantic analysis model to learn local features, while ignoring the relationship between global and local features, thus incorrectly retrieving different pedestrians with similar attributes as the same pedestrian. Moreover, learning local features using auxiliary models brings additional computational cost. In this work, we propose a Transformer-based dual-branch feature learning model for occluded person re-ID. Firstly, we propose a global–local feature interaction module to learn the relationship between global and local features, thus enhancing the richness of information in pedestrian features. Secondly, we randomly erase local areas in the input image to simulate the real occlusion situation, thereby improving the model’s adaptability to the occlusion scene. Finally, a spilt group module is introduced to explore the local distinguishing features of pedestrian. Numerous experiments validate the effectiveness of our proposed method.