Person Re-identification Methods Research Articles

Mix-Modality Person Re-Identification: A New and Practical Paradigm

Current visible-infrared cross-modality person re-identification research has only focused on exploring the bi-modality mutual retrieval paradigm, and we propose a new and more practical mix-modality retrieval paradigm. Existing V isible- I nfrared person re-identification (VI-ReID) methods have achieved some results in the bi-modality mutual retrieval paradigm by learning the correspondence between visible and infrared modalities. However, significant performance degradation occurs due to the modality confusion problem when these methods are applied to the new mix-modality paradigm. Therefore, this paper proposes a M ix- M odality person re-identification (MM-ReID) task, explores the influence of modality mixing ratio on performance, and constructs mix-modality test sets for existing datasets according to the new mix-modality testing paradigm. To solve the modality confusion problem in MM-ReID, we propose a C ross- I dentity D iscrimination H armonization L oss (CIDHL) adjusting the distribution of samples in the hyperspherical feature space, pulling the centers of samples with the same identity closer, and pushing away the centers of samples with different identities while aggregating samples with the same modality and the same identity. Furthermore, we propose a M odality B ridge S imilarity O ptimization S trategy (MBSOS) to optimize the cross-modality similarity between the query and queried samples with the help of the similar bridge sample in the gallery. Extensive experiments demonstrate that compared to the original performance of existing cross-modality methods on MM-ReID, the addition of our CIDHL and MBSOS demonstrates a general improvement.

Image–Text Person Re-Identification with Transformer-Based Modal Fusion

Existing person re-identification methods utilizing CLIP (Contrastive Language-Image Pre-training) mostly suffer from coarse-grained alignment issues. This is primarily due to the original design intention of the CLIP model, which aims at broad and global alignment between images and texts to support a wide range of image–text matching tasks. However, in the specific domain of person re-identification, local features and fine-grained information are equally important in addition to global features. This paper proposes an innovative modal fusion approach, aiming to precisely locate the most prominent pedestrian information in images by combining visual features extracted by the ResNet-50 model with text representations generated by a text encoder. This method leverages the cross-attention mechanism of the Transformer Decoder to enable text features to dynamically guide visual features, enhancing the ability to identify and locate the target pedestrian. Experiments conducted on four public datasets, namely MSMT17, Market1501, DukeMTMC, and Occluded-Duke, demonstrate that our method outperforms the baseline network by 5.4%, 2.7%, 2.6%, and 9.2% in mAP, and by 4.3%, 1.7%, 2.7%, and 11.8% in Rank-1, respectively. This method exhibits excellent performance and provides new research insights for the task of person re-identification.

Progressive de-preference task-specific processing for generalizable person re-identification

Recently, domain generalization (DG) person re-identification (ReID) has attracted attention. Existing DG person ReID methods train on mixed datasets containing all source domains. However, these mixed datasets have huge inter-domain differences because of varying data distributions across different source domains. Such differences hinder models from learning domain-invariant representations, affecting generalization on unseen domains. To address this issue, we propose a progressive de-preference task-specific processing network (PDTP-Net) for DG person ReID. Initially, we design a progressive de-preference domain segmentation strategy to mitigate inter-domain differences by dividing multiple source domains into different phases, each comprising several training tasks. We then design a global and task-specific processing module that enhances extraction of domain-invariant features by integrating statistical information from other source domains. Finally, we design a multi-granularity attention module and a group-aware batch normalization strategy to ensure the features are more discriminative and better suited for person ReID tasks. The proposed model is validated using three DG person ReID experimental protocols: Protocol-1, Protocol-2, and leave-one-out experiments. On Protocol-1, the model improves mean average precision (mAP) and Rank-1 accuracy on all datasets by an average of 0.7% and 0.3%, respectively. On Protocol-2, the model improves mAP and Rank-1 accuracy on all datasets by an average of 2.525% and 2.725%, respectively. On the leave-one-out experiments, the model improves mAP and Rank-1 accuracy on all tasks by an average of 0.65% and 0.18%, respectively. The results on several popular datasets suggest that the model achieves state-of-the-art performance in DG person ReID.

Person re-identification transformer with patch attention and pruning

Person re-identification (Re-ID), which is widely used in surveillance and tracking systems, aims to search individuals as they move between different camera views by maintaining identity across various camera views. In the realm of person re-identification (Re-ID), recent advancements have introduced convolutional neural networks (CNNs) and vision transformers (ViTs) as promising solutions. While CNN-based methods excel in local feature extraction, ViTs have emerged as effective alternatives to CNN-based person Re-ID, offering the ability to capture long-range dependencies through multi-head self-attention without relying on convolution and downsampling. However, it still faces challenges such as changes in illumination, viewpoint, pose, low resolutions, and partial occlusions. To address the limitations of widely used person Re-ID datasets and improve the generalization, we present a novel person Re-ID method that enhances global and local information interactions using self-attention modules within a ViT network. It leverages dynamic pruning to extract and prioritize essential image patches effectively. The designed patch selection and pruning for person Re-ID model resulted in a robust feature extractor even in scenarios with partial occlusion, background clutter, and illumination variations. Empirical validation demonstrates its superior performance compared to previous approaches and its adaptability across various domains.

Multiple-local feature and attention fused person re-identification method

Person re-identification (ReID) is widely used in intelligent security, monitoring, criminal investigation and other fields. Aiming at the problems of local occlusion, scale misalignment and attitude change of pedestrian images in actual scenes, we propose a Multi-local Feature and Attention fused network (MFA) used for person re-identification task. Firstly, Channel Point Affinity Attention module (CPAA) is embedded in the backbone network to enhance the ability of the network for extracting local details. The feature map output from the backbone network is horizontally segmented into four local feature maps, and further four branch networks are concatenated to the feature map of the backbone network. The four local feature maps are used to guide the four branch networks to pay more attention on different areas of pedestrians through Global Local Aligned loss (GLA) function. Finally, the pedestrian feature vector containing multi-local features is obtained. The mAP of the network on Market-1501, DukeMTMC-reID,CUHK03 and MSMT17 datasets were 88.6%, 81.4%, 79.5% and 64.7%, and the Rank-1 was 95.8%, 90.1%, 81.2% and 84.1% respectively. In addition, the model also obtained 73.2% and 68.1% of Rank-1 on partial dataset Patial-REID and Patial-iLIDS, respectively. Recently, The MFA model parameter is 28.3M and the inference efficiency is approximately 32 fps to an image with a resulation of 256 × 128. Compared with other ReID methods, our proposed methods achieved a competitive performance for ReID task. The code was available at github:git@github.com:ISCLab-Bistu/MFA.git.

Diversity-Representativeness Replay and Knowledge Alignment for Lifelong Vehicle Re-identification

Lifelong vehicle re-identification (LVReID) aims to match a target vehicle across multiple cameras, considering non-stationary and continuous data streams, which fits the needs of the practical application better than traditional vehicle re-identification. Nonetheless, this area has received relatively little attention. Recently, methods for lifelong person re-identification (LPReID) have been emerging, with replay-based methods achieving the best results by storing a small number of instances from previous tasks for retraining, thus effectively reducing catastrophic forgetting. However, these methods cannot be directly applied to LVReID because they fail to simultaneously consider the diversity and representativeness of replayed data, resulting in biases between the subset stored in the memory buffer and the original data. They randomly sample classes, which may not adequately represent the distribution of the original data. Additionally, these methods fail to consider the rich variation in instances of the same vehicle class due to factors such as vehicle orientation and lighting conditions. Therefore, preserving more informative classes and instances for replay helps maintain information from previous tasks and may mitigate the model’s forgetting of old knowledge. In view of this, we propose a novel Diversity-Representativeness Dual-Stage Sampling Replay (DDSR) strategy for LVReID that constructs an effective memory buffer through two stages, i.e. , Cluster-Centric Class Selection and Diverse Instance Mining. Specifically, we first perform class-level sampling based on density in the clustered class-centered feature space and then further mine the diverse, high-quality instances within the selected classes. In addition, we introduce Maximum Mean Discrepancy loss to align the feature distribution between replay data and the new arrivals and apply L2 regularisation in the parameter space to facilitate knowledge transfer, thus enhancing the model’s generalization ability to new tasks. Extensive experiments demonstrate effective improvements of our method compared to current state-of-the-art lifelong ReID methods on the VeRi-776, VehicleID, and VERI-Wild datasets.

TLCSFI: A Pose-Guided Person Re-Identification Method with Two-Level Channel–Spatial Feature Integration

Person re-identification methods currently encounter challenges in feature learning, primarily due to difficulties in expressing the correlation between local features and integrating global and local features effectively. To address these issues, a pose-guided person re-identification method with Two-Level Channel–Spatial Feature Integration (TLCSFI) is proposed. In TLCSFI, a two-level integration mechanism is implemented. At the first level, TLCSFI integrates the spatial information from local features to generate fine-grained spatial features. At the second level, the fine-grained spatial feature and the coarse-grained channel feature are integrated together to complete channel–spatial feature integration. In the method, a Pose-based Spatial Feature Integration (PSFI) module is introduced to generate the pose union feature, which calculates intra-body affinity to guide the integration of spatial information among local pose feature maps. Then, a Channel and Spatial Union Feature Integration (CSUFI) module is proposed to efficiently integrate the channel information of the global feature and the spatial information of the pose union feature. Two individual networks are designed to extract channel and spatial information, respectively, in CSUFI, which are then weighted and integrated. Experiments are conducted on three publicly available datasets to evaluate TLCSFI, and the experimental results demonstrate its competitive performance.

A Person Re-Identification Method for Fusing Convolutional Attention and Transformer Architecture

Rapid and accurate localization of target person via person re-identification is one of the important methods in intelligent security systems,but faces great challenges due to complex environmental changes. In order to build a model applicable to various complex scenes, deep learning-based person re-identification models have become a key research direction for researchers. The existing convolutional neural network-based or transformer-based deep learning models are still have certain shortcomings in the extraction of local details and are not able to cope well with realistic complex scenes. To address this problem, a method fusing convolutional neural network and transformer (FCAT) is proposed to enhance the transformer's attention to local detail information. This method mainly improves the transformer's ability to extract local detail features indirectly by embedding convolutional space attention and channel attention respectively to enhance the attention to important regions and important channel features in the image. Comparative ablation experiments on three publicly available person re-identification datasets demonstrate that FCAT achieves comparable results to existing methods on non-occluded datasets, and significantly improves performance on occluded datasets. The proposed model is also has a lightweight effect, and the inference speed is improved without increasing the amount of outer computation and model parameters.

Adaptive Style Normalization and Similarity Grouping for Unsupervised Cross-Domain Person Re-Identification

Abstract In unsupervised cross-domain person re-identification, feature extraction often includes identity-irrelevant information, reducing model generalization. Additionally, ignoring potential similarities within the target domain leads to noisy pseudo-labels. To address these issues, this paper introduces a method for cross-domain person re-identification that combines adaptive style normalization with similarity grouping. First, adaptive style normalization is applied on the backbone network to mitigate loss of information caused by instance normalization, thus improving the generalization ability and identity recognition accuracy of the model. Second, similarity grouping is employed to uncover potential similarities in target domain, overcoming the problem of loud pseudo-labels and further improve model performance. Using this method, mAP accuracy increased by 2.27% on the Market-1501 dataset and by 4.26% on the DukeMTMC-ReID dataset, enhancing its application in the real world.

A review on video person re-identification based on deep learning

Person Re-Identification (ReID) is an essential technology for matching a person across non-overlapping cameras. It has attracted increasing attention in recent years due to its wide range of applications in various real-world scenarios such as security surveillance and criminal investigation. Different from other person ReID tasks, video-based ReID uses a video clip as the retrieval input, which can provide more promising ReID performance because that the video has rich information on appearance, motion cues and pose variations on temporal pipeline. Over the last few years, many deep learning-based video person ReID have been proposed to address various challenges, such as illumination variation, complex background, occlusion, etc. To provide a more comprehensive and readable review on existing video-based person ReID methods, we propose a novel taxonomy method that observes existing methods from four perspectives: data, algorithms, computing power, and applications. Specifically, we first introduce some popular datasets and evaluation criterion used for video-based person ReID. Next, from limited data and little annotation view, we introduce data augmentation and unsupervised learning ReID. From algorithm view, we focus on reviewing supervised methods including spatial feature learning, temporal feature learning and spatio-temporal feature learning, and further discuss and conduct a systematic comparison among these approaches. From complex open-world application view, we mainly summarized domain adaption and multimodal ReID. From insufficient GPU computing power view, we mainly discuss modality-agnostic unified large-scale ReID and their lightweighting. Finally, we provide a discussion of open problems and potential research directions for the community.

A visible-infrared person re-identification method based on meta-graph isomerization aggregation module

Due to different imaging principles of visible-infrared cameras, there are modal differences between similar person images. For visible-infrared person re-identification (VI-ReID), existing works focus on extracting and aligning cross-modal global descriptors in the shared feature space, while ignoring local variations and graph-structural correlations in cross-modal image pairs. In order to bridge the modal differences with graph-structure between key-parts, a Meta-Graph Isomerization Aggregation Module (MIAM) is proposed, which includes Meta-Graph Node Isomerization Module (MNI) and Dual Aggregation Module (DA). In order to completely describe discriminative in-graph local features, MNI establishes meta-secondary cyclic isomorphism relations of in-graph local features by using a multi-branch embedding generation mechanism. Then, the local features not only contain the limited information of the fixed regions, but also benefit from the neighboring regions. Meanwhile, the secondary node generation process considers similar and different nodes of pedestrian graph structure to reduce the interference of identity differences. In addition, the Dual Aggregation (DA) module combines spatial self-attention and channel self-attention to establish the interdependence of modal heterogeneous graph structure pairs and achieve inter-modal feature aggregation. To match heterogeneous graph structures, a Node Center Joint Mining Loss (NCJM Loss) is proposed to constrain the distance between the node centers of heterogeneous graphs. The experiments performed on the SYSU-MM01, RegDB and LLCM public datasets demonstrate that the proposed method performs excellently in VI-ReID.

Cross-Modality Person Re-Identification Method with Joint-Modality Generation and Feature Enhancement.

In order to minimize the disparity between visible and infrared modalities and enhance pedestrian feature representation, a cross-modality person re-identification method is proposed, which integrates modality generation and feature enhancement. Specifically, a lightweight network is used for dimension reduction and augmentation of visible images, and intermediate modalities are generated to bridge the gap between visible images and infrared images. The Convolutional Block Attention Module is embedded into the ResNet50 backbone network to selectively emphasize key features sequentially from both channel and spatial dimensions. Additionally, the Gradient Centralization algorithm is introduced into the Stochastic Gradient Descent optimizer to accelerate convergence speed and improve generalization capability of the network model. Experimental results on SYSU-MM01 and RegDB datasets demonstrate that our improved network model achieves significant performance gains, with an increase in Rank-1 accuracy of 7.12% and 6.34%, as well as an improvement in mAP of 4.00% and 6.05%, respectively.

Unsupervised multi-source domain adaptation for person re-identification via sample weighting

The aim of unsupervised domain adaptation (UDA) in person re-identification (re-ID) is to develop a model that can identify the same individual across different cameras in the target domain, using labeled data from the source domain and unlabeled data from the target domain. However, existing UDA person re-ID methods typically assume a single source domain and a single target domain, and seldom consider the scenario of multiple source domains and a single target domain. In the latter scenario, differences in sample size between domains can lead to biased training of the model. To address this, we propose an unsupervised multi-source domain adaptation person re-ID method via sample weighting. Our approach utilizes multiple source domains to leverage valuable label information and balances the inter-domain sample imbalance through sample weighting. We also employ an adversarial learning method to align the domains. The experimental results, conducted on four datasets, demonstrate the effectiveness of our proposed method.

A unified deep semantic expansion framework for domain-generalized person re-identification

Supervised Person Re-identification (Person ReID) methods have achieved excellent performance when training and testing within one camera network. However, they usually suffer from considerable performance degradation when applied to different camera systems. In recent years, many Domain Adaptation Person ReID methods have been proposed, achieving impressive performance without requiring labeled data from the target domain. However, these approaches still need the unlabeled data of the target domain during the training process, making them impractical in many real-world scenarios. Our work focuses on the more practical Domain Generalized Person Re-identification (DG-ReID) problem. Given one or more source domains, it aims to learn a generalized model that can be applied to unseen target domains. One promising research direction in DG-ReID is the use of implicit deep semantic feature expansion, and our previous method, Domain Embedding Expansion (DEX), is one such example that achieves powerful results in DG-ReID. However, in this work we show that DEX and other similar implicit deep semantic feature expansion methods, due to limitations in their proposed loss function, fail to reach their full potential on large evaluation benchmarks as they have a tendency to saturate too early. Leveraging on this analysis, we propose Unified Deep Semantic Expansion, our novel framework that unifies implicit and explicit semantic feature expansion techniques in a single framework to mitigate this early over-fitting and achieve a new state-of-the-art (SOTA) in all DG-ReID benchmarks. Further, we apply our method on more general image retrieval tasks, also surpassing the current SOTA in all of these benchmarks by wide margins.

Cross-modal semantic aligning and neighbor-aware completing for robust text–image person retrieval

Most existing text–image person re-identification (TIReID) methods are performed in an ideal environment where both image and text instances are fully intact and identity annotated. However, in real-world open environments, these ideal assumptions often cannot be satisfied. In this study, we are the first to explore how to enhance the robustness of the TIReID model to better adapt to open environments, focusing on two aspects: 1) unlabeled multi-modal data, and 2) incomplete multi-modal data. Here, unlabeled multi-modal data faces two key issues: intra-class variations and cross-modal aligning problems, while incomplete data leads to data semantic inconsistency problems and severe performance degradation. In order to tackle the above issues, we propose a novel Cross-modal Semantic Aligning and Neighbor-aware Completing (CANC) method for robust text–image person re-identification (RTIReID). Specifically, to tackle the intra-class variation problem, we propose intra-view prototype contrastive matching for image and text modalities, which helps the model learn intra-modal discriminative representations while increasing inter-class distance and suppressing intra-class variation. Furthermore, to resolve the cross-modal alignment problem, we propose the cross-view instance projection matching to establish the relationship between different modal representations. Additionally, regarding how to effectively handle incomplete samples, we introduce nearest neighbor consistent completion, which can restore high-quality completion features. We adopt CLIP-ViT and CLIP-Xformer as image and text encoders respectively, and Rank-k metrics (k=1,5,10) as primary evaluation metrics. Extensive experiments on several public datasets demonstrate that our method can effectively address TIReID in open environments, consistently surpassing several SOTA TIReID methods.

Model-aware privacy-preserving with start trigger method for person re-identification

Person Re-identification (ReID) could search for the same pedestrian from non-overlapping cameras, which completes the pedestrian location and search purpose. However, the process contains much sensitive pedestrian information and raises serious privacy problems. Conventional methods mainly remove identity-related features from pedestrian images to alleviate the privacy issue. Unfortunately, these strategies cause pedestrian information loss and poor data utility. In the paper, we propose a novel Model-Aware Privacy-Preserving with Start Trigger (MPST) method, which not only prevents personal identity for third parties but also achieves accurate pedestrian location for authorized organizations. The core idea is that authorized organizations obtain the start trigger to activate the ReID model that has the ability to search for target pedestrians, while third parties (i.e., hackers) cannot employ the ReID model to complete the pedestrian matching task without the start trigger. To this end, we develop a universal adversarial algorithm to learn an ingenious start trigger for the person ReID system. Afterwards, we further design a model-aware training strategy to facilitate our deep model to perceive issued instructions by synthetically utilizing the start trigger and original pedestrian samples. As a result, we successfully install an activate button to change the ReID model state for deciding whether the deep model has the ability to search for pedestrians or not. Abundant experiments demonstrate that the proposed MPST is effective for pedestrian identity anonymization. Our study achieves superior performance for authorized organizations and completes the privacy protection goal.

Enhancing Person Re-Identification through Attention-Driven Global Features and Angular Loss Optimization.

To address challenges related to the inadequate representation and inaccurate discrimination of pedestrian attributes, we propose a novel method for person re-identification, which leverages global feature learning and classification optimization. Specifically, this approach integrates a Normalization-based Channel Attention Module into the fundamental ResNet50 backbone, utilizing a scaling factor to prioritize and enhance key pedestrian feature information. Furthermore, dynamic activation functions are employed to adaptively modulate the parameters of ReLU based on the input convolutional feature maps, thereby bolstering the nonlinear expression capabilities of the network model. By incorporating Arcface loss into the cross-entropy loss, the supervised model is trained to learn pedestrian features that exhibit significant inter-class variance while maintaining tight intra-class coherence. The evaluation of the enhanced model on two popular datasets, Market1501 and DukeMTMC-ReID, reveals improvements in Rank-1 accuracy by 1.28% and 1.4%, respectively, along with corresponding gains in the mean average precision (mAP) of 1.93% and 1.84%. These findings indicate that the proposed model is capable of extracting more robust pedestrian features, enhancing feature discriminability, and ultimately achieving superior recognition accuracy.

Weakly Supervised Tracklet Association Learning With Video Labels for Person Re-Identification.

Supervised person re-identification (re-id) methods require expensive manual labeling costs. Although unsupervised re-id methods can reduce the requirement of the labeled datasets, the performance of these methods is lower than the supervised alternatives. Recently, some weakly supervised learning-based person re-id methods have been proposed, which is a balance between supervised and unsupervised learning. Nevertheless, most of these models require another auxiliary fully supervised datasets or ignore the interference of noisy tracklets. To address this problem, in this work, we formulate a weakly supervised tracklet association learning (WS-TAL) model only leveraging the video labels. Specifically, we first propose an intra-bag tracklet discrimination learning (ITDL) term. It can capture the associations between person identities and images by assigning pseudo labels to each person image in a bag. And then, the discriminative feature for each person is learned by utilizing the obtained associations after filtering the noisy tracklets. Based on that, a cross-bag tracklet association learning (CTAL) term is presented to explore the potential tracklet associations between bags by mining reliable positive tracklet pairs and hard negative pairs. Finally, these two complementary terms are jointly optimized to train our re-id model. Extensive experiments on the weakly labeled datasets demonstrate that WS-TAL achieves 88.1% and 90.3% rank-1 accuracy on the MARS and DukeMTMC-VideoReID datasets respectively. The performance of our model surpasses the state-of-the-art weakly supervised models by a large margin, even outperforms some fully supervised re-id models.

ReFID: Reciprocal Frequency-aware Generalizable Person Re-identification via Decomposition and Filtering

Domain generalization of person re-identification aims to conduct testing across domains that have not been previously encountered, without utilizing target domain data during the training stage. As the number of source domains increases, the relationships between training samples become more complex. This can lead to domain-invariant features that include certain instance-level spurious correlations, which can impact the model’s ability to generalize further. To overcome this limitation, the Reciprocal Frequency-aware Generalizable Person Re-identification method is proposed in this article, which aims to utilize spectral feature correlation learning to transmit frequency component information and generate more discriminative hybrid features. A module called Bilateral Frequency Component-guided Attention is developed to help the network understand high-level semantic and texture information from various frequency features. Furthermore, to reduce the impact of noise from the frequency domain, this article proposes an innovative module called Fourier Noise Masquerade Filtering. This module enhances the portability of frequency domain components while simultaneously suppressing elements that do not contribute to generalization. Extensive experimental results on various datasets demonstrate that our method is effective and superior to the state-of-the-art methods.

Instance-aware diversity feature generation for unsupervised person re-identification

Unsupervised person re-identification (Re-ID) methods have made significant progress by exploiting contrastive learning from unlabeled data. However, the previous approaches including cluster-level or instance-level contrast loss, did not fully explore inherent commonality of each identified individual from unlabeled samples, where the divergence of individual cluster and convergence of different clusters leads to a set of noisy pseudo labels which may result in label noise accumulation. To address this issue, we propose an instance-aware diversity feature generation (IDFG) framework, which can form a stable clustering feature space by exhuming diverse counterparts of given exemplars to update memory dictionary of each cluster, so as to reduce the effect of noisy labels. Specifically, we combines instance segmentation and masked auto-encoder to generate foreground-invariant diversity counterparts of given exemplars to reduce inter-class convergence caused by background similarity between different identification instances. Further, we introduce an instance-aware diversity feature mining module, which gradually creates more reliable clusters to generate more robust pseudo labels by exploiting the compactness and independence of clustering to update the memory dictionary. Extensive experiments demonstrate that the proposed IDFG framework achieves impressive performances of 85.6%, 73.7%, and 31.0% mAP on Market1501, DukeMTMC-reID and MSMT17, respectively.