Person Re-identification Framework Research Articles

UnA-Mix: Rethinking Image Mixtures for Unsupervised Person Re-Identification

With the development of ultra-long-range visual sensors, the application of unsupervised person re-identification algorithms to them has become increasingly important. However, these algorithms inevitably generate noisy pseudo-labels, which seriously hinder the performance of tasks over a large range. Mixup, a data enhancement technique, has been validated in supervised learning for its generalization to noisy labels. Based on this observation, to our knowledge, this study is the first to explore the impact of the mixup technique on unsupervised person re-identification, which is a downstream task of contrastive learning, in detail. Specifically, mixup was applied in different locations (at the pixel level and feature level) in an unsupervised person re-identification framework to explore its influences on task performance. In addition, based on the richness of the information contained in the person samples to be mixed, we propose an uncertainty-aware mixup (UnA-Mix) method, which reduces the over-learning of simple person samples and avoids the information damage that occurs when information-rich person samples are mixed. The experimental results on three benchmark person re-identification datasets demonstrated the applicability of the proposed method, especially on the MSMT17, where it outperformed state-of-the-art methods by 5.2% and 4.8% in terms of the mAP and rank-1, respectively.

A Three-Stage Framework for Video-Based Visible-Infrared Person Re-Identification

A Three-Stage Framework for Video-Based Visible-Infrared Person Re-Identification

A Versatile Framework for Multi-Scene Person Re-Identification.

Person Re-identification (ReID) has been extensively developed for a decade in order to learn the association of images of the same person across non-overlapping camera views. To overcome significant variations between images across camera views, mountains of variants of ReID models were developed for solving a number of challenges, such as resolution change, clothing change, occlusion, modality change, and so on. Despite the impressive performance of many ReID variants, these variants typically function distinctly and cannot be applied to other challenges. To our best knowledge, there is no versatile ReID model that can handle various ReID challenges at the same time. This work contributes to the first attempt at learning a versatile ReID model to solve such a problem. Our main idea is to form a two-stage prompt-based twin modeling framework called VersReID. Our VersReID firstly leverages the scene label to train a ReID Bank that contains abundant knowledge for handling various scenes, where several groups of scene-specific prompts are used to encode different scene-specific knowledge. In the second stage, we distill a V-Branch model with versatile prompts from the ReID Bank for adaptively solving the ReID of different scenes, eliminating the demand for scene labels during the inference stage. To facilitate training VersReID, we further introduce the multi-scene properties into self-supervised learning of ReID via a multi-scene prioris data augmentation (MPDA) strategy. Through extensive experiments, we demonstrate the success of learning an effective and versatile ReID model for handling ReID tasks under multi-scene conditions without manual assignment of scene labels in the inference stage, including general, low-resolution, clothing change, occlusion, and cross-modality scenes. Codes and models will be made publicly available.

A Memorizing and Generalizing Framework for Lifelong Person Re-Identification.

In this paper, we introduce a challenging yet practical setting for person re-identification (ReID) task, named lifelong person re-identification (LReID), which aims to continuously train a ReID model across multiple domains and the trained model is required to generalize well on both seen and unseen domains. It is therefore critical to learn a ReID model that can learn a generalized representation without forgetting knowledge of seen domains. In this paper, we propose a new MEmorizing and GEneralizing framework (MEGE) for LReID, which can jointly prevent the model from forgetting and improve its generalization ability. Specifically, our MEGE is composed of two novel modules, i.e., Adaptive Knowledge Accumulation (AKA) and differentiable Ranking Consistency Distillation (RCD). Taking inspiration from the cognitive processes in the human brain, we endow AKA with two special capacities, knowledge representation and knowledge operation by graph convolution networks. AKA can effectively mitigate catastrophic forgetting on seen domains while improving the generalization ability to unseen domains. By considering the ranking factor that is specifically important in ReID, RCD is designed to distill the ranking knowledge in a differentiable manner, which can further prevent the catastrophic forgetting. To supporting the study of LReID, we build a new and large-scale benchmark with two practical evaluation protocols that consider the metrics of non-forgetting and generalization. Experiments demonstrate that 1) our MEGE framework can effectively improve the performance on seen and unseen domains under the domain-incremental learning constraint, and that 2) the proposed MEGE outperforms state-of-the-art competitors by large margins.

Efficient Person Reidentification for IoT-Assisted Cyber–Physical Systems

The main objective of this study is to propose a cyber-physical system (CPS) based person re-identification framework for smart surveillance. The Internet of Things (IoT) based interconnected vision sensors in smart cities are considered essential elements of a CPS, and contribute significantly to urban security. However, the re-identification of targeted persons using emerging edge AI techniques still faces certain challenges. To improve efficiency at the edge and overcome the traditional sensing of video cameras, we employed an AI-based person re-identification framework for CPS that is functional in IoT environments. In addition, we present dual attention dilated network (DADNet), which integrates an energy-efficient convolutional neural network (CNN) with a self-attention module to substantially improve the person matching probability. Further, we applied dual feature fusion to intelligently integrate discriminative and robust features using early and late fusion strategies that allow DADNet to significantly consider the foreground and marginally utilize the background information. Furthermore, we impose diversity orthogonality regularization over several CNN layers, which boosts the performance of DADNet, resulting in an appropriate usage over IoT networks. A comprehensive set of ablation studies, comparison with other state-of-the-art approaches, and a time complexity analysis confirm the strength of our DADNet for re-identification tasks in AI-enabled IoT settings that are well-suited for a CPS.

A plug-and-play noise-label correction framework for unsupervised domain adaptation person re-identification

A plug-and-play noise-label correction framework for unsupervised domain adaptation person re-identification

An adaptive self-correction joint training framework for person re-identification with noisy labels

Current person re-identification (ReID) methods heavily rely on well-annotated training data, and their performance suffers from significant degradation in the presence of noisy labels that are ubiquitous in real-life scenes. The reason is that noisy labels not only affect the prediction results of the classifier, but also impede feature refinement, making it difficult to distinguish between different person features. To address these issues, we propose an Adaptive Self-correction Classification (ASC) loss and an Adaptive Margin Self-correction Triplet (AMSTri) loss. Specifically, ASC loss helps the network to produce better predictions by balancing annotations and prediction labels, and pays more attention to the minority samples with the help of a focusing factor. On the other hand, the AMSTri loss introduces an adaptive margin that varies with sample features to accommodate complex data variations, and utilizes predicted labels to generate reliable triples for feature refinement. We then present an end-to-end adaptive self-correction joint training framework incorporating ASC loss and AMSTri loss to achieve a robust ReID model. Our comprehensive experiments demonstrate that the proposed framework outperforms most existing counterparts.

Multi-source adaptive meta-learning framework for domain generalization person re-identification

Multi-source adaptive meta-learning framework for domain generalization person re-identification

Pedestrian Re-ID based on feature consistency and contrast enhancement

For robust person re-identification(Re-ID), the key is effectively learning the features of body parts and their long-distance dependence. ResNet and Transformer are respectively good at learning local dependence and long-distance dependence between region features due to their respective special structures. In order to fully integrate the advantages of the two models, we propose a novel person Re-ID framework that effectively incorporates pixel-level region features, posture-level relation features and the long-distance dependence of region features. Specifically, we design a Semantic Correction Module (SCM) that corrects pixel-level region features and posture-level relation features in a masked manner to generate discriminative fine-grained features with high pose semantics. Considering the semantic inconsistency between relation features and region features, we propose a Contrastive Association Module (CAM) to interactively enhances the long-distance correlation and local saliency of features in a self-attention way. Finally, to improve the robustness of local and global features, we construct a CAM layer to enhance the representation of features based on their potential relationships. Extensive experiment results on general and occlusion datasets demonstrate that our approach performs favorably against the state-of-the-art methods, e.g. 96% Rank-1 on Market-1501.

A 6G-Enabled Lightweight Framework for Person Re-Identification on Distributed Edges

In the upcoming 6G era, edge artificial intelligence (AI), as a key technology, will be able to deliver AI processes anytime and anywhere by the deploying of AI models on edge devices. As a hot issue in public safety, person re-identification (Re-ID) also needs its models to be urgently deployed on edge devices to realize real-time and accurate recognition. However, due to complex scenarios and other practical reasons, the performance of the re-identification model is poor in practice. This is especially the case in public places, where most people have similar characteristics, and there are environmental differences, as well other such characteristics that cause problems for identification, and which make it difficult to search for suspicious persons. Therefore, a novel end-to-end suspicious person re-identification framework deployed on edge devices that focuses on real public scenarios is proposed in this paper. In our framework, the video data are cut images and are input into the You only look once (YOLOv5) detector to obtain the pedestrian position information. An omni-scale network (OSNet) is applied through which to conduct the pedestrian attribute recognition and re-identification. Broad learning systems (BLSs) and cycle-consistent adversarial networks (CycleGAN) are used to remove the noise data and unify the style of some of the data obtained under different shooting environments, thus improving the re-identification model performance. In addition, a real-world dataset of the railway station and actual problem requirements are provided as our experimental targets. The HUAWEI Atlas 500 was used as the edge equipment for the testing phase. The experimental results indicate that our framework is effective and lightweight, can be deployed on edge devices, and it can be applied for suspicious person re-identification in public places.

Dual Pseudo Label Refinement for Unsupervised Domain Adaptive Person Re-Identification

Although clustering-based unsupervised domain adaptive person re-identification has achieved promising progress, negative influence of noisy pseudo labels is still unsolved. Training model with noisy labeled data will mislead feature representation and ultimately affect the improvement of model performance. To tackle the above problem, we propose a dual pseudo label refinement framework for unsupervised domain adaptive person re-identification. It has two pseudo label refinement modules, one learns cross consistency of corresponding features between two collaborative networks and the other explores mutual consistency between global and local feature spaces. By working complementarily and jointly, the two modules optimize the quality of training datatset. Specifically, a mean teacher-student framework with multiple branches is designed to extract global and local features. Cross consistency of corresponding features between mean teacher and student networks is learned to select candidate samples for training. Meanwhile, mutual consistency of global and local features in mean teacher network is also explored to select other candidates. Finally, the intersection of two selected candidate datasets is used to re-train the framework. Being trained with selected reliable samples iteratively, the framework becomes more and more effective and robust. Extensive results on benchmark datasets confirm that our dual pseudo label refinement brings performance improvement and outperforms the state-of-the-art clustering-based unsupervised domain adaptive re-identification approaches.

Learning Domain Invariant Representations for Generalizable Person Re-Identification.

Generalizable person Re-Identification (ReID) aims to learn ready-to-use cross-domain representations for direct cross-data evaluation, which has attracted growing attention in the recent computer vision (CV) community. In this work, we construct a structural causal model (SCM) among identity labels, identity-specific factors (clothing/shoes color etc.), and domain-specific factors (background, viewpoints etc.). According to the causal analysis, we propose a novel Domain Invariant Representation Learning for generalizable person Re-Identification (DIR-ReID) framework. Specifically, we propose to disentangle the identity-specific and domain-specific factors into two independent feature spaces, based on which an effective backdoor adjustment approximate implementation is proposed for serving as a causal intervention towards the SCM. Extensive experiments have been conducted, showing that DIR-ReID outperforms state-of-the-art (SOTA) methods on large-scale domain generalization (DG) ReID benchmarks.

Pose-driven attention-guided image generation for person re-Identification

Person re-identification (re-ID) concerns the matching of subject images across different camera views in a multi camera surveillance system. One of the major challenges in person re-ID is pose variations across the camera network, which significantly affects the appearance of a person. Existing development data lack adequate pose variations to carry out effective training of person re-ID systems. To solve this issue, in this paper we propose an end-to-end pose-driven attention-guided generative adversarial network, to generate multiple poses of a person. We propose to attentively learn and transfer the subject pose through an attention mechanism. A semantic-consistency loss is proposed to preserve the semantic information of the person during pose transfer. To ensure fine image details are realistic after pose translation, an appearance discriminator is used while a pose discriminator is used to ensure the pose of the transferred images will exactly be the same as the target pose. We show that by incorporating the proposed approach in a person re-identification framework, realistic pose transferred images and state-of-the-art re-identification results can be achieved.

Specialized re-ranking: A novel retrieval-verification framework for cloth changing person re-identification

Cloth changing person re-identification(Re-ID) can work under more complicated scenarios with higher security than normal Re-ID and biometric techniques and is therefore extremely valuable in applications. Meanwhile, the wide range of appearance flexibility results in more similar-looking, confusing images, which is the weakness of the widely used retrieval methods. In this work, we shed light on how to handle these similar images. Specifically, we propose a novel retrieval-verification framework. Given an image, the retrieval module will search for a shot list of similar images quickly. Our proposed verification network will then compare the probe image with these candidate images by contrasting local details for their similarity scores. An innovative ranking strategy is also introduced to achieve a good balance between retrieval and verification results. Comprehensive experiments are conducted to show the effectiveness of our framework and its capability in improving the state-of-the-art methods remarkably on both synthetic and realistic datasets.

Dual Prototype Contrastive learning with Fourier Generalization for Domain Adaptive Person Re-identification

Unsupervised domain adaptive (UDA) person re-identification (ReID) focuses on improving the model’s generalization capability from one labeled source domain to the unlabeled target domain. Recently, contrastive learning based on pseudo label assignment has attracted much attention and dominated the field. However, existing methods usually consider bridging the domain gap at the feature level, and they only pull each query instance to get close to its cluster centroid which is stored or computed through a memory bank. The relationship of inter-instance within cluster, especially with the hard instances, is ignored. To this end, we propose a Dual Prototype Contrastive learning with Fourier Generalization (DPCFG) framework for domain adaptive Person Re-identification. First, we introduce the Fourier Generalization (FG) strategy at image level to bridge the domain gap. Concretely, the FG strategy is implemented by replacing the amplitude component of each source domain image with a randomly selected target domain image. Then, the Dual Prototype Contrastive learning (DPC) strategy is further developed to fully exploit the hard positive instances within each cluster. DPC optimizes two contrastive losses by forcing each query to be close to two prototypes: the cluster centroid prototype and the hard positive prototype. The cluster centroid prototype ensures the basic classification accuracy, and the hard positive prototype further improves the classification accuracy by dynamically depicting a certain class boundary for each cluster as the model iterates. Experimental results on the real-world datasets, Market, DukeMTMC-reID, and MSMT17, and synthetic dataset PersonX, demonstrate that DPCFG is effective and achieves state-of-the-art UDA person ReID performance.

Person re-identification in indoor videos by information fusion using Graph Convolutional Networks

Information contained in visual appearance and gait features can play an important role in designing computer vision assisted person re-identification (ReID) systems. Fusion of appearance and gait features has not yet been tested in this domain of research despite its great potential to solve some of the intriguing challenges faced by the research community due to viewpoint variations, illumination change, varying recording setups, etc. This paper proposes a new deep learning framework for person re-identification in videos. The framework referred to as Fused Graph Network (FGN-ReID) uses an information fusion strategy to deal with the aforementioned variations. We have used visual appearance and gait features independently and fused these features for re-identification of persons in close shot indoor video recordings. A feature similarity-based score-level fusion strategy has been adopted to fuse spatio-temporal and gait features using Graph Convolutional Networks (GCN). The ReID problem has been mapped to an inherent graph searching problem. In our proposed framework, appearance and gait features represent the nodes. And the relations between spatio-temporal segments denote the edges. Experiments using CASIA-B dataset reveal that the proposed method is more effective as compared to existing methods. We have improved the rank-1 accuracy by a margin of 8%–12% as compared to the baseline algorithms. The mean average precision (mAP) has improved significantly. The method has several applications including visual surveillance, biometric authentication, etc.

Camera domain adaptation based on cross-patch transformers for person re-identification

• We propose a transformer-based framework for cross-camera person re-identification. • A novel module is proposed to obtain effective patch tokens from input images. • The proposed framework outperforms the state-of-the-art methods on three datasets. As an essential task applied to video surveillance, person re-identification (Re-ID) suffers from variations across different cameras. In this paper we propose an effective transformer-based Re-ID framework for learning the identity-discriminative and camera-invariant feature representations. In contrast to the recent direction of using generative models to augment training data and enhance the invariance to input variations, we show that explicitly designing a novel adversarial loss from the perspective of feature representation learning helps to penalize the distribution discrepancy across multiple camera domains effectively. Recently, the pure transformer model has gained much attention due to its strong representation capabilities. We employ a pure transformer encoder to extract a global feature vector for the patch tokens of each person image. Notably, a novel cross-patch encoder is introduced to obtain structural information between image patches. Extensive experiments on three challenging datasets demonstrate the effectiveness and superiority of the proposed learning framework.

Short range correlation transformer for occluded person re-identification

Occluded person re-identification is one of the challenging areas of computer vision, which faces problems such as inefficient feature representation and low recognition accuracy. Recently, vision transformer is introduced into the field of re-identification and achieved state-of-the-art results by constructing global feature relationships between patch sequences. However, vision transformer is not good at capturing short-range correlations of patch sequence and exploiting spatial correlation in patch sequence, which leads to a decrease in the accuracy and robustness of the network in the face of occluded person re-identification. Therefore, to address the above problems, we design a partial feature transformer-based occluded person re-identification framework named PFT. The proposed PFT utilizes three modules to enhance the efficiency of vision transformer. (1) Patch full dimension enhancement module. We design a learnable tensor with the same size as patch sequences, which is full-dimensional and deeply embedded in patch sequences to enrich the diversity of training samples. (2) Fusion and reconstruction module. We extract the less important part of obtained patch sequences, and fuse them with original patch sequence to reconstruct the original patch sequences. (3) Spatial Slicing Module. We slice and group patch sequences from spatial direction, which can effectively improve the short-range correlation of patch sequences. Experimental results over occluded and holistic re-identification datasets demonstrate that the proposed PFT network achieves superior performance consistently and outperforms the state-of-the-art methods.

Unbiased feature enhancement framework for cross-modality person re-identification

Unbiased feature enhancement framework for cross-modality person re-identification

Graph-Based Local Feature Adaptation for Cross-Domain Person Re-Identification

The performance of cross-domain person re-identification has been greatly improved in recent years. However, there are still two problems in existing cross-domain person re-identification methods. First, most of them conduct domain adaptation on features that contain background noise. Second, they ignore the correlation between different features, including intra-domain and inter-domain. To overcome these problems, we propose a novel Graph-based Local Feature Adaptation (GLFA) framework for cross-domain person re-identification, which promotes domain adaptation by correlating intra-domain and inter-domain semantic local features with graph convolutional network. Specifically, in the feature extraction stage, we utilize a parsing model to extract semantic local features for both source and target domain samples, so as to avoid the feature shift caused by background discrepancy. In the feature adaptation stage, we apply two stacked Graph Convolutional Networks (GCN) to propagate semantic local feature information within each domain and across different domains respectively. It promotes the transfer of specific knowledge from the source domain to the target domain. Furthermore, to ensure the features from different domains updated by GCN are well aligned, a local feature distribution alignment loss is introduced on the top of GCN. The combination of GCN and alignment loss enables our framework diminish the feature shift caused by other factors effectively. Extensive experiments on Market-1501, DukeMTMC-reID and MSMT17 datasets demonstrate that our method outperforms the state-of-the-art methods evidently.