Unsupervised Person Re-identification Research Articles

Camera-aware progressive learning for unsupervised person re-identification

Camera-aware progressive learning for unsupervised person re-identification

Unsupervised person re-identification via multi-domain joint learning

Deep learning techniques have achieved impressive progress in the task of person re-identification. However, how to generalize a learned model from the source domain to the target domain remains a long-standing challenge. Inspired by the fact that the enrichment of data diversity and the utilization of miscellaneous semantic features can lead to better generalization ability, we design a model that integrates a novel data augmentation method with a multi-label assignment strategy to achieve semantic features decoupling in the source domain. The pre-trained model is employed to extract several kinds of semantic features from the target dataset, and each kind of semantic features is regarded as a specific domain. We then cluster features of each domain and exploit the connection between different clustering results to perform self-distillation for generating more reliable pseudo labels. Finally, the obtained pseudo labels are used to fine-tune the pre-trained model to achieve model transfer from the source domain to the target one. Extensive experiments demonstrate that our approach outperforms some state-of-the-art methods by a clear margin and even surpass some supervised methods. Source code is available at: https://www.github.com/flychen321/MDJL.

Diversity feature constraint based on heterogeneous data for unsupervised person re-identification

Person re-identification (ReID) based on heterogeneous data aims to search for the same pedestrian from different modalities. The existing unsupervised heterogeneous ReID method overly relies on pseudo labels and ignores the inter-image feature relationship. In the paper, we propose a novel Diversity Feature Constraint (DFC) method to simultaneously consider the clustering-level and instance-level feature relationship for unsupervised heterogeneous ReID. On the one hand, we employ the clustering algorithm to produce pseudo labels for heterogeneous images. Then, the clustering-level constraint is designed to optimize the model. On the other hand, considering that the clustering algorithm may generate some noise, we propose the complementary intra-modality instance-level constraint to correlate any two intra-modality images. Meanwhile, for eliminating the inter-modality discrepancy, the inter-modality instance-level constraint is developed to decrease the large inter-modality gap. We construct the potential feature relationship between heterogeneous images to constrain the feature distribution. By experiments, we prove that over-reliance on pseudo labels generates limited performance. Exploring inter-image potential relationships is an important way to solve the unsupervised problem. Extensive results demonstrate that DFC achieves superior performance that outperforms other methods by a large margin, improving 15.23% and 9.37% at rank-1 and mAP indexes compared with the clustering method on SYSU-MM01.

Reliability modeling and contrastive learning for unsupervised person re-identification

Unsupervised person re-identification (ReID) aims to learn discriminative identity features in scenarios without a ground-truth. Fully unsupervised person ReID methods usually iterate between pseudo-labels prediction and representation learning and have achieved promising performance. However, these methods are usually hampered by noisy labels. To address this issue, we propose a reliability modeling and contrastive learning (RMCL) method, which aims to reduce the impact of noisy labels and enhance the robustness of the model to hard samples. On the basis of existing work, first, we define the concept of probabilistic stability and design a stability estimation scheme to improve pseudo-labels reliability modeling. Second, we explore the reliability–informativity function to redefine the weights of samples, which can be easily introduced to existing optimization methods. Finally, we expand the range of hard samples and design an identity hard contrastive loss, which is used to increase the robustness of the model to hard samples. Experiments on three large-scale person ReID datasets (Market-1501, DukeMTMC-reID and MSMT17) validate the effectiveness of RMCL, which surpasses state-of-the-art fully unsupervised methods and cross-domain methods. Furthermore, we explore the superiority of RMCL in a new “three2one” scenario.

Recent progress in person re-ID

行人再识别通过大时空范围内跨摄像机目标行人图像的检索与匹配，可实现人脸等生物特征失效情况下的行人关联，已成为智能视频监控系统的关键环节和支撑技术，并在智慧公安、智慧城市等国民经济建设中发挥了重要作用。近年行人再识别技术吸引了越来越多的关注，并取得了快速发展与进步。本文在对行人再识别技术进行简介的基础上，面向行人再识别的技术发展和落地应用需求与挑战，总结分析遮挡行人再识别、无监督行人再识别、虚拟数据生成、域泛化行人再识别、换装行人再识别、跨模态行人再识别和行人搜索等热点方向的前沿进展，归纳其发展现状和存在问题，最后对行人再识别技术的发展趋势进行展望。希望通过总结和分析，能够为研究人员开展行人再识别相关研究、推动行人再识别技术进步提供参考。;Person re-Identification(person re-ID) technique aims to solve the problem of association and matching of target person images across multiple cameras within a camera network of a surveillance system, especially in the case of face, iris and other biometrics recognition failure under non-cooperative application scenarios, and has become one of the key component and supporting technique for intelligent video surveillance systems and applications in intelligent public security and smart cities. Recently, person re-ID has attracted more and more attention from both academia and industry, and has made rapid development and progress. To meet the requirement of its technical challenges and application needs of person re-ID in practical scenarios, this paper will first give a brief introduction to the development history, commonly used datasets and evaluation metrics. Then, the recent progress in hot research topics of person re-ID is extensively reviewed and analyzed, which includes:occluded person re-ID, unsupervised person re-ID, virtual data generation, domain generalization, clothchanging person re-ID, cross-modal person re-ID and person search. First, to address the problem of impact of possible occlusions on the performance of person re-ID, recent progress in occluded person re-ID is first reviewed, in which the popular datasets for occluded person re-ID are briefly introduced, and the two major categories of occluded person re-ID models are then further reviewed. Second, facing the challenges of low-efficiency and high-cost data annotation and great impact of training data on the performance of person re-ID, unsupervised person re-ID and virtual data generation for person re-ID emerges as two hot topics in person re-ID. The paper elaborates the recent advances of unsupervised person reID, which can be classified into three major categories:pseudo label generation-based models, domain transfer-based models, and other related models, which take into consideration of the extra information like time-stamps, camera labels besides person image. Third, the-state-of-the-art works on virtual data generation for person re-ID are reviewed, with detailed introduction and performance comparisons of major virtual datasets. Fourth, recent researches on domain generalization person re-ID will be reviewed, which are classified into five categories:batch/instance normalization models, domain invariant feature learning models, deep-learning-based explicit image matching models, models based on mixture of experts and meta-learning-based models. Fifth, since most current person re-ID models largely depend on the color appearance of persons' clothes, clothes-changing person re-ID becomes a challenging setting, in which person images can exhibit large intra-class variation and small inter-class variation. Typical cloth-changing person re-ID datasets are introduced and the recent researches will then be reviewed, in which models in the first category explicitly introduces extra cloth-appearance-independent features like contour and face while the second try to decouple the cloth features and person ID features. Sixth, to compensate the drawbacks of conventional person re-ID of visible light/RGB images in natural complex scenes like poor lighting conditions in the night, the-state-of-the-art of cross-modal person re-ID, which aims to resolve the problem through other visible RGB images-excluded heterogeneous data, are reviewed, with a brief introduction of commonly used cross-modal person re-ID datasets first and then four sub-categories models according to the different modalities employed, including:RGB-infrared image person re-ID, RGB image-text person re-ID, RGB image-sketch person re-ID, and RGB-depth image person re-ID, respectively. Seventh, since existing person re-ID benchmarks and methods mainly focus on matching cropped person images between queries and candidates and is different from practical scenarios where the bounding box annotations of persons are often unavailable, person search, which jointly considers person detection and person re-ID in a single framework, becomes a new hot research topic. The typical datasets and recent progress on person search are reviewed. Finally, the existing challenges and development trend of person re-ID techniques are discussed. It is hoped that the summary and analysis can provide reference for relevant researchers to carry out research on person re-ID and promote the progress of person re-ID techniques and applications.

Learning to Purification for Unsupervised Person Re-Identification.

Unsupervised person re-identification is a challenging and promising task in computer vision. Nowadays unsupervised person re-identification methods have achieved great progress by training with pseudo labels. However, how to purify feature and label noise is less explicitly studied in the unsupervised manner. To purify the feature, we take into account two types of additional features from different local views to enrich the feature representation. The proposed multi-view features are carefully integrated into our cluster contrast learning to leverage more discriminative cues that the global feature easily ignored and biased. To purify the label noise, we propose to take advantage of the knowledge of teacher model in an offline scheme. Specifically, we first train a teacher model from noisy pseudo labels, and then use the teacher model to guide the learning of our student model. In our setting, the student model could converge fast with the supervision of the teacher model thus reduce the interference of noisy labels as the teacher model greatly suffered. After carefully handling the noise and bias in the feature learning, our purification modules are proven to be very effective for unsupervised person re-identification. Extensive experiments on two popular person re-identification datasets demonstrate the superiority of our method. Especially, our approach achieves a state-of-the-art accuracy 85.8% @mAP and 94.5% @Rank-1 on the challenging Market-1501 benchmark with ResNet-50 under the fully unsupervised setting. Code has been available at: https://github.com/tengxiao14/Purification_ReID.

Hybrid Contrastive Learning for Unsupervised Person Re-Identification

Unsupervised person re-identification (Re-ID) aims to learn discriminative features without human-annotated labels. Recently, contrastive learning provides a new prospect for unsupervised person Re-ID, and existing methods mainly constrain the feature similarity among easy sample pairs. However, the feature similarity among hard sample pairs is neglected, which causes suboptimal performance in unsupervised person Re-ID. In the paper, we propose a novel Hybrid Contrastive Model (HCM) to perform the identity-level contrastive learning and the image-level contrastive learning for unsupervised person Re-ID, which adequately explores the feature similarity among hard sample pairs. Specifically, for the identity-level contrastive learning, an identity-based memory is constructed to store pedestrian features. Accordingly, we define the dynamic contrast loss to identify identity information with dynamic factor for distinguishing hard/easy samples. As for the image-level contrastive learning, an image-based memory is established to store each image feature. We design the sample constraint loss to explore the similarity relationship between hard positive and negative sample pairs. Furthermore, we optimize the two contrastive learning processes in one unified framework to make use of their own advantages as so to constrain the feature distribution for extracting potential information. Extensive experiments prove that the proposed HCM distinctly outperforms the state-of-the-art methods.

Camera Invariant Feature Learning for Unsupervised Person Re-Identification

Fully unsupervised person re-identification (ReID) methods aim to learn discriminative features without using labeled ReID data. Because these methods are easily affected by camera discrepancies, similar studies have typically designed optimization methods to enable the model to learn camera-invariant features. However, they often ignore the impact of camera discrepancies on clustering results. Specifically, camera discrepancies will reduce the intra-class camera diversity and promote the generation of noise labels. To solve the above problems, we propose a unified unsupervised learning framework: camera invariant feature learning (CIFL) framework. First, we designed a novel DBSCAN-NN algorithm in the CIFL framework that improves the intra-class camera diversity by forcibly merging samples from different cameras. Then, we designed feature ensemble clustering that improves the accuracy of the pseudo-labels by clustering feature ensembles. In addition, we designed an optimization method for camera discrepancies: stochastic pulled loss. With the stochastic pulled loss, the ReID model is forced to learn camera-invariant features. We verified the effectiveness and generalization of CIFL on four ReID datasets (Market-1501, DukeMTMC-reID, MSMT17 and CUHK03-NP). The experimental results show that CIFL not only outperforms the existing fully unsupervised methods but also is superior to the unsupervised domain adaptation methods.

Dual-level contrastive learning for unsupervised person re-identification

Unsupervised person re-identification (re-ID) has drawn increasing attention in the community because it is more data-friendly when applied in the real world. Most existing works leverage instance discrimination learning to guide the model to learn image features for person matching. However, the instance discrimination may cause unexpected repulsion among similar samples, which makes the unsupervised feature learning unstable. To address this problem, we propose a dual-level contrastive learning (DLCL) framework to mine both the intra-instance and inter-instance similarities. The DLCL framework consists of two tasks: instance-instance contrastive learning (IICL) and instance-community contrastive learning (ICCL). The IICL aims to mine the intra-instance similarity via pulling an original sample and its augmented versions closer and pushing different instances away. The ICCL is proposed to capture the inter-instance similarity by attracting similar instances to the same sample community, which can reduce the unexpected repulsion brought by instance discrimination. The combination of IICL and ICCL can enable the model to learn more robust and discriminative image features. Extensive experimental results on Market-1501 and DukeMTMC-reID indicate the effectiveness of our method for unsupervised person re-ID.

Pseudo-Label Noise Prevention, Suppression and Softening for Unsupervised Person Re-Identification

Unsupervised person re-identification (ReID), including fully unsupervised ReID and unsupervised domain adaptive ReID, remains a challenge for the fields of biometrics and computer vision due to its difficulty in learning with unlabeled target domain data. Existing state-of-the-art methods, most of which generate pseudo-labels via unsupervised clustering for model optimization, are inevitably hampered by the under-explored problem of pseudo-label noise. Motivated by this, we propose a novel joint framework termed pseudo-label Noise Prevention, Suppression, and Softening (NPSS) for unsupervised person re-identification. Instead of refining generated label noise after clustering as many existing methods do, we start solving this issue from the source of pseudo-label noise by proposing a new Dynamic Camera-Adaptive Clustering (DCAC), which dynamically involves camera information to prevent noise caused by cross-camera variance, thus improving their quality during clustering. Moreover, we propose an Online Domain Union (ODU) mechanism for the classification model learning on the target domain via involving source domain data with their ground-truth labels, which effectively suppresses the indelible noisy pseudo-labels. Furthermore, we present the Self-Consistency Constraint (SCC) to soften the label noise in a single model with reduced computation and network parameter cost, which achieves intra-sample knowledge ensembling with our global-local SCC and cross-sample knowledge ensembling with our inter-instance SCC. Experiments demonstrate the effectiveness of our method as it surpasses state-of-the-art methods by a large margin on Market-1501, DukeMTMC-ReID, and MSMT17 benchmarks. The code is available at https://github.com/hjwang-824/NPSS.

Rethinking Sampling Strategies for Unsupervised Person Re-Identification.

Unsupervised person re-identification (re-ID) remains a challenging task. While extensive research has focused on the framework design and loss function, this paper shows that sampling strategy plays an equally important role. We analyze the reasons for the performance differences between various sampling strategies under the same framework and loss function. We suggest that deteriorated over-fitting is an important factor causing poor performance, and enhancing statistical stability can rectify this problem. Inspired by that, a simple yet effective approach is proposed, termed group sampling, which gathers samples from the same class into groups. The model is thereby trained using normalized group samples, which helps alleviate the negative impact of individual samples. Group sampling updates the pipeline of pseudo-label generation by guaranteeing that samples are more efficiently classified into the correct classes. It regulates the representation learning process, enhancing statistical stability for feature representation in a progressive fashion. Extensive experiments on Market-1501, DukeMTMC-reID and MSMT17 show that group sampling achieves performance comparable to state-of-the-art methods and outperforms the current techniques under purely camera-agnostic settings. Code has been available at https://github.com/ucas-vg/GroupSampling.

A Real-Time Memory Updating Strategy for Unsupervised Person Re-Identification

Recently, clustering-based methods have been the dominant solution for unsupervised person re-identification (ReID). Memory-based contrastive learning is widely used for its effectiveness in unsupervised representation learning. However, we find that the inaccurate cluster proxies and the momentum updating strategy do harm to the contrastive learning system. In this paper, we propose a real-time memory updating strategy (RTMem) to update the cluster centroid with a randomly sampled instance feature in the current mini-batch without momentum. Compared to the method that calculates the mean feature vectors as the cluster centroid and updating it with momentum, RTMem enables the features to be up-to-date for each cluster. Based on RTMem, we propose two contrastive losses, i.e., sample-to-instance and sample-to-cluster, to align the relationships between samples to each cluster and to all outliers not belonging to any other clusters. On the one hand, sample-to-instance loss explores the sample relationships of the whole dataset to enhance the capability of density-based clustering algorithm, which relies on similarity measurement for the instance-level images. On the other hand, with pseudo-labels generated by the density-based clustering algorithm, sample-to-cluster loss enforces the sample to be close to its cluster proxy while being far from other proxies. With the simple RTMem contrastive learning strategy, the performance of the corresponding baseline is improved by 9.3% on Market-1501 dataset. Our method consistently outperforms state-of-the-art unsupervised learning person ReID methods on three benchmark datasets. Code is made available at:https://github.com/PRIS-CV/RTMem.

Progressive unsupervised video person re-identification with accumulative motion and tracklet spatial–temporal correlation

Person re-identification (re-id) methods based on supervised learning require a large number of manually labeled samples for training, resulting in poor scalability in actual re-id applications. Existing unsupervised video person re-id methods typically focus on extracting appearance features from pedestrian videos, ignoring motion information and the fact that people usually move in groups, i.e., pedestrian spatio–temporal co-occurrence patterns. The key factor for unsupervised video person re-id is to effectively exploit both spatio–temporal clues from video sequences and cross-camera tracklet association. In this work, we propose a progressive deep learning method for unsupervised person re-id via tracklet association with spatio–temporal correlation (TASTC). Specifically, we first divide uniformly each tracklet into multiple temporally localized slices according to a time pyramid structure. Then, an initial re-id model is trained based on a two-stream convolutional architecture, and the accumulative motion context information of temporally localized slices of the tracklets per camera is learned. Finally, combining accumulative motion and tracklet spatial–temporal correlation, we associate tracklets across cameras and update the re-id model. The above steps are iterated to optimize the re-id model progressively. Experiment results demonstrate that the proposed method significantly outperforms the current state-of-the-art unsupervised video person re-identification methods on three video-based benchmark datasets, ILIDS-VID, MARS, and DukeMTMC-VideoReID.

Unsupervised person re-identification by patch-based nearest neighbor mining

Unsupervised person re-identification (re-ID) is still challenging due to the difficulty in learning discriminative features without labeling information. Some existing methods use the instance specificity learning to learn image features without direct supervision signals. However, this kind of instance specificity learning treats each sample as an individual instance and can not mine the inter-sample relationships, which limits the ability to explore the category boundaries. To address the unsupervised person re-ID, this paper proposes a patch-based nearest neighbor mining (PNNM) method. The proposed PNNM utilizes a patch-based feature extraction model to capture patch features from person images. Then the patch discrimination mining (PDM) is applied to supervise the model to learn a discriminative patch feature space. Meanwhile, the nearest neighbor mining (NNM) is introduced to guide the model to mine the image-level discrimination from unlabeled person images. The NNM aims to explore nearest neighbor sets with high category consistency on the whole training dataset. A two-stage training process is built with the PDM and NNM to enable the model to capture both patch-level and image-level visual clues. Extensive experiments are conducted to show the effectiveness of the proposed PNNM method in the unsupervised person re-ID task.

Multi-granularity Pseudo-label Collaboration for unsupervised person re-identification

Unsupervised person re-identification (ReID) aims at training a model through using fully unlabeled training images. Most successful approaches combine clustering-based pseudo-label prediction with ReID model learning and perform the two steps in an alternating fashion. However, there are some person images, which have common appearance but different identities or have same identity but large intra-variations, and they are prone to be assigned wrong pseudo-labels. Existing methods typically infer person image pseudo-labels in a single pseudo-label space, which is insufficient due to losing the accurate labels of these hard images. This limits model’s power to cope with varying changes. To address this problem, we propose a Multi-granularity Pseudo-label Collaboration (MPC) method for unsupervised person ReID. Firstly, a multi-granularity pseudo-label prediction (MPP) method is proposed to predict the pseudo-labels of person images. MPP generates multiple pseudo-label spaces with each one inferring the person image pseudo-labels under a different granularity at cluster size. Secondly, the multiple pseudo-label spaces are applied to learn a mixture of experts (MoE) model. Each expert model is supervised by the pseudo-labels in a specific pseudo-label space, enabling MoE model to handle different variations. The output of these expert models are fused critically by a bilateral attention aggregation (BAA) to generate aggregation image features. Meanwhile, a label mix-up method is designed for refining pseudo-labels. It fuses labels across different pseudo-label spaces based on label co-occurrence to promote aggregation feature learning. We extensively evaluate the proposed MPC method under both image-based and video-based unsupervised person ReID settings. MPC significantly improves the current baseline method by a margin of 5.4% and 3.7% in mAP, and 3.4% and 3.9% in Rank1 accuracy on the image-based datasets Market-1501 and DukeMTMC-ReID, respectively. In particular, MPC method achieves state-of-the-art performance on video-based datasets with mAP of 71.4% and 87.3% on MARS and DukeMTMC-VideoReID, respectively.

Global Relation-Aware Contrast Learning for Unsupervised Person Re-Identification

The goal of unsupervised person re-identification (Re-ID) is to use unlabeled person images to learn discriminative features. In recent years, many approaches have adopted clustered pseudo labels to construct proxies for contrastive learning, and have thereby achieved great success. However, existing methods of this kind only utilize local structures within IDs to design their proxies while ignoring the relations between samples of different IDs, which limits the improvement for inter-ID discriminative ability. To resolve this issue, we propose a Global Relation-Aware Contrast Learning (GRACL) method for the task of unsupervised Re-ID. Our method first sets up two proxies for each cluster to capture the inter- and intra-ID relations respectively, which enables us to both effectively increase inter-ID variances and reduce the intra-ID discrepancies. Specifically, the samples that are most different from those in different clusters are selected as inter-ID relation-aware proxies, while those that are least similar to samples from the same clusters are employed as intra-ID relation-aware proxies. With the aid of these proxies, we design both inter- and intra-ID relation-aware contrastive learning modules to facilitate model learning. By pulling each sample close to the positive proxy, we can obtain identity-invariant discriminative features. Experiments on five widely-used Re-ID datasets prove that our GRACL model outperforms current state-of-the-art approaches to a remarkable extent.

Unsupervised person re-identification based on high-quality pseudo labels

Unsupervised person re-identification based on high-quality pseudo labels

Population-Based Evolutionary Gaming for Unsupervised Person Re-identification

Population-Based Evolutionary Gaming for Unsupervised Person Re-identification

Unsupervised Person Re-Identification via Multi-Label Classification

The challenge of unsupervised person re-identification (ReID) lies in learning discriminative features without true labels. This paper formulates unsupervised person ReID as a multi-label classification task to progressively seek true labels. Our method starts by assigning each person image with a single-class label, then evolves to multi-label classification by leveraging the updated ReID model for label prediction. The label prediction comprises similarity computation and cycle consistency to ensure the quality of predicted labels. To boost the ReID model training efficiency in multi-label classification, we further propose the memory-based multi-label classification loss (MMCL). MMCL works with memory-based non-parametric classifier and integrates multi-label classification and single-label classification in a unified framework. Our label prediction and MMCL work iteratively and substantially boost the ReID performance. Experiments on several large-scale person ReID datasets demonstrate the superiority of our method in unsupervised person ReID. Our method also allows to use labeled person images in other domains. Under this transfer learning setting, our method also achieves state-of-the-art performance.

Graph correlation-refined centroids for unsupervised person re-identification

Graph correlation-refined centroids for unsupervised person re-identification