Person Re-identification Methods Research Articles

A Self-Adaptive Gallery Construction Method for Open-World Person Re-Identification.

Person re-identification, or simply re-id, is the task of identifying again a person who has been seen in the past by a perception system. Multiple robotic applications, such as tracking or navigate-and-seek, use re-identification systems to perform their tasks. To solve the re-id problem, a common practice consists in using a gallery with relevant information about the people already observed. The construction of this gallery is a costly process, typically performed offline and only once because of the problems associated with labeling and storing new data as they arrive in the system. The resulting galleries from this process are static and do not acquire new knowledge from the scene, which is a limitation of the current re-id systems to work for open-world applications. Different from previous work, we overcome this limitation by presenting an unsupervised approach to automatically identify new people and incrementally build a gallery for open-world re-id that adapts prior knowledge with new information on a continuous basis. Our approach performs a comparison between the current person models and new unlabeled data to dynamically expand the gallery with new identities. We process the incoming information to maintain a small representative model of each person by exploiting concepts of information theory. The uncertainty and diversity of the new samples are analyzed to define which ones should be incorporated into the gallery. Experimental evaluation in challenging benchmarks includes an ablation study of the proposed framework, the assessment of different data selection algorithms that demonstrate the benefits of our approach, and a comparative analysis of the obtained results with other unsupervised and semi-supervised re-id methods.

Multiple camera styles learning for unsupervised person re-identification

Most clustering-based unsupervised person re-identification methods usually first assign pseudo-labels to pedestrian images using a clustering algorithm and then train a network using the labeled pedestrian images in each iteration. However, clustering algorithms may produce some clusters that only contain pedestrian images from fewer cameras, thus resulting in these clusters having fewer camera styles. Training a network using the pedestrian images from these clusters may degrade the capability of the network to retrieve pedestrians across cameras. Moreover, unsupervised learning methods usually utilize only contrastive loss that is based on self-supervised learning. However, lack of metric loss with mandatory guidance from labels may lead the network to fail to learn sufficiently the discriminative information of pedestrians. To address these issues, we propose a multiple camera styles learning (MCSL) method for unsupervised person re-identification. Specifically, we design a novel cluster filtration module (CFM) to ensure the clustering algorithm to keep the focus on pedestrian identities rather than camera styles by filtering out the clusters with fewer camera styles. In addition, we also design a multi-granularity feature metric (MGFM) loss to introduce metric learning with labels into unsupervised learning. The effectiveness of the proposed MCSL is demonstrated by the experimental results (Rank-1 accuracy) on Market1501 (93.7%), DukeMTMC(86.0%) and MSMT17 (71.12%), significantly reducing the gap between unsupervised and supervised performance on person re-identification.

Discriminative feature mining with relation regularization for person re-identification

The appearance attribute and pose are two important and complementary features, so integrating them can effectively alleviate the impact of misalignment and occlusion on re-identification. In this paper, we deeply investigate the inner relation between attribute features and the spatial semantic relation between key-point region features of the pose in a person image and propose a person re-identification method based on discriminative feature mining with relation regularization. Firstly, an attribute relation detector based on nonlinear graph convolution is built on mining the inner correlation between attribute features of a person, providing relational attribute features for more effectively distinguishing persons with a similar appearance. Then, we construct a hierarchical pose pyramid to model the multi-grained semantic features of key-point regions of the pose and propose intra-graph and cross-graph node relation information propagation structures to infer the spatial semantic relation between node features within-graph and between-graph. This module is robust to complex pose changes and can suppress noise background redundancy caused by inaccurate key point detection and occlusion. Finally, a refined feature model is proposed to effectively fuse the global appearance feature with the relational attribute and multi-grained pose features, thus providing a more discriminative fusion feature for person re-identification. Many experiments on three large-scale datasets verify the effectiveness and state-of-the-art performance of the proposed method.

Body Correlation Network for Person Re-Identifications

<p>Visual information accounts for approximately 80 to 85 percent of the information available daily in modern cities. As such, person re-identification, an instance-level image retrieval task, has become an important research topic in computer vision, machine learning, and other fields in recent years. Traditional person re-identification methods based on convolutional neural networks only extract the global feature information of people. Thus, when external factors such as changes in occlusion and illumination disturb people, the recognition performance of these methods substantially decreases. We therefore develop body correlation network (BC-Net), which takes full advantage of images of body parts and the correlations between them. Specifically, BC-Net uses body part feature information and correlation feature information as nodes and edges, respectively, and then uses graph convolutions to learn the overall topology of people. To improve use of crucial feature information, we also design a unique method of propagation between nodes and edges. We conduct extensive comparative experiments on the Market-1501 and DukeMTMC-reID datasets, and the results demonstrate that BC-Net outperforms other state-of-the-art techniques.</p> <p> </p>

Reliability modeling and contrastive learning for unsupervised person re-identification

Unsupervised person re-identification (ReID) aims to learn discriminative identity features in scenarios without a ground-truth. Fully unsupervised person ReID methods usually iterate between pseudo-labels prediction and representation learning and have achieved promising performance. However, these methods are usually hampered by noisy labels. To address this issue, we propose a reliability modeling and contrastive learning (RMCL) method, which aims to reduce the impact of noisy labels and enhance the robustness of the model to hard samples. On the basis of existing work, first, we define the concept of probabilistic stability and design a stability estimation scheme to improve pseudo-labels reliability modeling. Second, we explore the reliability–informativity function to redefine the weights of samples, which can be easily introduced to existing optimization methods. Finally, we expand the range of hard samples and design an identity hard contrastive loss, which is used to increase the robustness of the model to hard samples. Experiments on three large-scale person ReID datasets (Market-1501, DukeMTMC-reID and MSMT17) validate the effectiveness of RMCL, which surpasses state-of-the-art fully unsupervised methods and cross-domain methods. Furthermore, we explore the superiority of RMCL in a new “three2one” scenario.

Complementary Attention-Driven Contrastive Learning With Hard-Sample Exploring for Unsupervised Domain Adaptive Person Re-ID

Unsupervised domain adaptive (UDA) methods for person re-identification (Re-ID) aim to transfer the knowledge of the labeled source domain to the unlabeled target domain without further annotations, which is challenging due to the drift of label distribution and the missing of target domain labels. Improving the clustering accuracy of pseudo-labels can help the model fit the target domain. However, the errors of pseudo-label noise will be accumulated during training, which is harmful to the model performance. Moreover, the hard samples can lead to a large gap between intra-class features and a small gap between inter-class features. To address these problems, this paper proposes a complementary attention-driven contrastive learning with hard-sample exploring (CACHE) algorithm. In CACHE, on one hand, the complementary attention module is used to improve the discriminability of the features. The obtained discriminative features can reduce noisy pseudo-labels and improve the clustering accuracy of pseudo labels; On the other hand, we explore the hard samples based on the instance relationship and cluster relationship for contrastive learning. This way can make the cluster more compact. Extensive experiments on three large-scale person re-identification benchmarks demonstrate the effectiveness of the proposed method, which significantly outperforms state-of-the-art methods in terms of mAP and CMC.

Camera Invariant Feature Learning for Unsupervised Person Re-Identification

Fully unsupervised person re-identification (ReID) methods aim to learn discriminative features without using labeled ReID data. Because these methods are easily affected by camera discrepancies, similar studies have typically designed optimization methods to enable the model to learn camera-invariant features. However, they often ignore the impact of camera discrepancies on clustering results. Specifically, camera discrepancies will reduce the intra-class camera diversity and promote the generation of noise labels. To solve the above problems, we propose a unified unsupervised learning framework: camera invariant feature learning (CIFL) framework. First, we designed a novel DBSCAN-NN algorithm in the CIFL framework that improves the intra-class camera diversity by forcibly merging samples from different cameras. Then, we designed feature ensemble clustering that improves the accuracy of the pseudo-labels by clustering feature ensembles. In addition, we designed an optimization method for camera discrepancies: stochastic pulled loss. With the stochastic pulled loss, the ReID model is forced to learn camera-invariant features. We verified the effectiveness and generalization of CIFL on four ReID datasets (Market-1501, DukeMTMC-reID, MSMT17 and CUHK03-NP). The experimental results show that CIFL not only outperforms the existing fully unsupervised methods but also is superior to the unsupervised domain adaptation methods.

Frequency Information Disentanglement Network for Video-Based Person Re-Identification.

Recently, most video-based person re-identification (Re-ID) methods adopt complex model or multi-scaled information to explore more discriminative spatio-temporal clues, thus achieving better retrieval accuracy. However, we witness that these approaches involve significant higher computation costs but only improve limited performances. Therefore, the overarching goal at this stage is to solve video Re-ID on the trade-off between accuracy and efficiency, thereby boosting the application in real scenarios. Frequency transform provides advantages of simplified representation, identification of hidden information and noise filtering in signal processing. Motivated by this, we treat the complex spatio-temporal feature as signal and convert it to frequency domain. By directly analyzing frequency clues, complex feature extraction procedures can be avoided. Specifically, this paper proposes a novel paradigm by categorizing video features into low/high and spatial/temporal frequency information. Then, with the help of 3D DCT, we theoretically establish the transform equivalence relationship between spatio-temporal domain and frequency domain. Finally, this paper proposes a simple and intuitive Frequency Information Disentanglement Network (FIDN) for video Re-ID. By extracting and applying both low and high frequency spatio-temporal features from a disentangling way, FIDN achieves comprehensive and discriminative video representation. Extensive experiments indicate that FIDN reaches the state-of-the-arts with only one convolution layer addition against baseline.

Improving Person Re-Identification With Multi-Cue Similarity Embedding and Propagation

Most existing person re-identification (Re-ID) methods rely on the visual appearance of the human body. However, face cues are rarely explored in the Re-ID community despite the face that it is an important biometric identifier for human beings. In this work, we propose a Similarity Ensemble Framework (SEF) that uses multi-cue similarity embedding and propagation to effectively fuse body and face information for person re-identification. Specifically, for each query, we first perform standard pedestrian retrieval using body and face cues, respectively, to obtain some candidate results with high confidence. Next, the body and face similarities are combined and embedded into a shared space as node features, and two graphs with the same nodes and different edges with respect to body and face affinities are constructed. Then, the similarity features are propagated in both body and face graphs using graph convolution to capture the relationship among the candidates using different cues. Lastly, the refined features are used to compute the final similarities with the query. The proposed method not only combines the similarities of body and face, but also takes into account the relationship among all the other candidate samples under different cues. Extensive experiments demonstrate that the use of face cues effectively improves the performance of person Re-ID even if the performance obtained by the face alone is much lower than that of the body, suggesting that our approach is able to capture valuable information beyond body from weaker face cues in person Re-ID scenarios.

Weakly Supervised Distribution Discrepancy Minimization Learning With State Information for Person Re-Identification

Weakly supervised person re-identification (Re-ID) is appealing to handle real-world tasks by using state information that is available without manual annotation. At present, most methods perform unsupervised cross domain (UCD) learning by transferring the knowledge from the labeled source domain to the unlabeled target domain, which results in poor performance due to the severe shift. To address this problem, in this paper, we utilize the tracklet and camera information as weak supervision to propose a distribution discrepancy minimization learning (DDML) model for UCD person Re-ID. In addition to aligning data distributions from the perspective of domain adaptation learning, two losses are developed from the view of neighborhood invariance exploration to optimize matching results. Specifically, to bridge the gap between domains, we propose a camera-distribution-based (CDB) loss to align pair-wise distance distributions. Furthermore, to alleviate the biased search within the target domain, we propose a ranking-confidence-based (RCB) loss to perform the mined neighborhood for intra-camera and inter-camera separately to explore a high degree of confidence neighbor relations. Extensive experiments on three challenging datasets demonstrate that applying our method to unlabeled target domain outperforms current weakly supervised methods for person Re-ID.

Effect of Reconstruction Losses in Discriminative and Generative Learning based Networks for the Person Re-identification

The Person Re-identification (Re-ID) task has gained popularity in recent times. Researchers are continuously looking to improve the accuracy of the existing person Re-ID systems. Identifying the person from the surveillance footage can be essential to security concerns. Currently, there are many state-of-art Person Re-ID systems available. Deep learning frameworks are also adopted for designing Re-ID systems. Apart from deep learning-based approaches, the Generative Adversarial Networks (GAN) based approach also gained substantial interest in Person Re-ID tasks. Augmentation of training data has significantly improved the performance of the system. Our primary objective is to analyze the effect of applying different reconstruction losses and their combinations on the GAN-based approach. The Discriminative and Generative Learning (DG-Net) approach is chosen for carrying out this study from other existing GAN-based systems. DG-Net is currently considered benchmarked in the GAN-based method for person Re-ID. Analysis shows that the proposed idea of using a variety of reconstruction losses simultaneously significantly improves the existing system's performance. Using the proposed technique of fusing multiple Losses simultaneously, we achieved a massive performance gain of 20.57% over the current benchmarked approach on the Market1501 dataset. This paper includes a thorough study of different loss functions and their effect on the generated images for performing Person Re-ID tasks.

From Collective Attribute Association of Groups to Precise Attribute Association of Individuals

Obscured person re-identification (Re-ID) aims to match an obscured image with a complete image of the same person captured by other cameras. As a major challenge in person identification, occlusion severely affects the effectiveness of most traditional person Re-ID methods. To solve this problem, this study proposes a trajectory association method, which, as a pre-processing technique for person Re-ID, can narrow the search range and reduce the problem of degradation caused by mixing. We investigate the method of converting the fuzzy association between sets into the precise association between elements for M video objects and N phone objects (trajectory information) with fuzzy group association relationships at the crime scene. First, we decompose the M-N precise association problem and analyze the similarity of the video objects in the source point and on the trajectories. Then, we define high-similarity points, study their distribution characteristics in different trajectories, and find that there is a significant difference between the distribution of high-similarity points in correct and incorrect matching trajectories. We simplify the full-path association problem into a partial-path high-similarity point distribution difference problem, which effectively reduces the difficulty in accurate association relationship construction. The association experiments in simple and mixed scenarios as well as Re-ID experiments on the PRPW and Market1501 demonstrate the effectiveness of our method.

A Real-Time Memory Updating Strategy for Unsupervised Person Re-Identification

Recently, clustering-based methods have been the dominant solution for unsupervised person re-identification (ReID). Memory-based contrastive learning is widely used for its effectiveness in unsupervised representation learning. However, we find that the inaccurate cluster proxies and the momentum updating strategy do harm to the contrastive learning system. In this paper, we propose a real-time memory updating strategy (RTMem) to update the cluster centroid with a randomly sampled instance feature in the current mini-batch without momentum. Compared to the method that calculates the mean feature vectors as the cluster centroid and updating it with momentum, RTMem enables the features to be up-to-date for each cluster. Based on RTMem, we propose two contrastive losses, i.e., sample-to-instance and sample-to-cluster, to align the relationships between samples to each cluster and to all outliers not belonging to any other clusters. On the one hand, sample-to-instance loss explores the sample relationships of the whole dataset to enhance the capability of density-based clustering algorithm, which relies on similarity measurement for the instance-level images. On the other hand, with pseudo-labels generated by the density-based clustering algorithm, sample-to-cluster loss enforces the sample to be close to its cluster proxy while being far from other proxies. With the simple RTMem contrastive learning strategy, the performance of the corresponding baseline is improved by 9.3% on Market-1501 dataset. Our method consistently outperforms state-of-the-art unsupervised learning person ReID methods on three benchmark datasets. Code is made available at:https://github.com/PRIS-CV/RTMem.

Learning to Purification for Unsupervised Person Re-Identification.

Unsupervised person re-identification is a challenging and promising task in computer vision. Nowadays unsupervised person re-identification methods have achieved great progress by training with pseudo labels. However, how to purify feature and label noise is less explicitly studied in the unsupervised manner. To purify the feature, we take into account two types of additional features from different local views to enrich the feature representation. The proposed multi-view features are carefully integrated into our cluster contrast learning to leverage more discriminative cues that the global feature easily ignored and biased. To purify the label noise, we propose to take advantage of the knowledge of teacher model in an offline scheme. Specifically, we first train a teacher model from noisy pseudo labels, and then use the teacher model to guide the learning of our student model. In our setting, the student model could converge fast with the supervision of the teacher model thus reduce the interference of noisy labels as the teacher model greatly suffered. After carefully handling the noise and bias in the feature learning, our purification modules are proven to be very effective for unsupervised person re-identification. Extensive experiments on two popular person re-identification datasets demonstrate the superiority of our method. Especially, our approach achieves a state-of-the-art accuracy 85.8% @mAP and 94.5% @Rank-1 on the challenging Market-1501 benchmark with ResNet-50 under the fully unsupervised setting. Code has been available at: https://github.com/tengxiao14/Purification_ReID.

A Double Stream Person Re-Identification Method Based on Attention Mechanism and Multi-Scale Feature Fusion

Person re-identification is the key technology for designing a intelligent video surveillance system to retrieve and recongise a specific person in a series of videos. It mainly plays a role in multiple non-overlapping camera monitoring environments to determine whether the target person of interest that has appeared under a camera appears again under others. However, the image data, in the real scene, taken by the surveillance camera may be occluded or blurred, which increased the difficulty of identifying the pedestrian posturn and then lead to dramatically decrease the accuracy of recognition. To solve the above problems, we propose a dualbranch multi-scale feature fusion network, which improves the expression ability of pedestrian features under partial occlusion by learning discriminative pedestrian features. The feature erasure mechanism is introduced into the branch network to effectively enhance the fine-grained feature learning ability of the network. By embedding the lightweight attention module into Resnet-50, the image sequence features of the channel dimension will be extracted, and the interference caused by the cluttered back ground information will be suppressed. In the training phase, the average pooling layer and the maximum pooling layer of different kernels and strides are utilized for different residual stages, and the mixed pooling strategy and mixed loss function of different kernels are designed. Finally, a novel hybrid loss function which combines triplet loss and ID loss is presented to train the network model. By comparison with the existing representative methods on Market1501, DukeMTMC reID and MTMS17 datasets, the experimental results show that the features extracted by the proposed method are more discriminative and with high recognition accuracy.

MTNet: Mutual tri-training network for unsupervised domain adaptation on person re-identification

The existing unsupervised domain adaptation (UDA) methods on person re-identification (re-ID) often employ clustering to assign pseudo labels for unlabeled target domain samples. However, it is difficult to give accurate pseudo labels to unlabeled samples in the clustering process. To solve this problem, we propose a novel mutual tri-training network, termed MTNet, for UDA person re-ID. The MTNet method can avoid noisy labels and enhance the complementarity of multiple branches by collaboratively training the three different branch networks. Specifically, the high-confidence pseudo labels are used to update each network branch according to the joint decisions of the other two branches. Moreover, inspired by self-paced learning, we employ a sample filtering scheme to feed unlabeled samples into the network from easy to hard, so as to avoid trapping in the local optimal solution. Extensive experiments show that the proposed method can achieve competitive performance compared with the state-of-the-art person re-ID methods.

Progressive unsupervised video person re-identification with accumulative motion and tracklet spatial–temporal correlation

Person re-identification (re-id) methods based on supervised learning require a large number of manually labeled samples for training, resulting in poor scalability in actual re-id applications. Existing unsupervised video person re-id methods typically focus on extracting appearance features from pedestrian videos, ignoring motion information and the fact that people usually move in groups, i.e., pedestrian spatio–temporal co-occurrence patterns. The key factor for unsupervised video person re-id is to effectively exploit both spatio–temporal clues from video sequences and cross-camera tracklet association. In this work, we propose a progressive deep learning method for unsupervised person re-id via tracklet association with spatio–temporal correlation (TASTC). Specifically, we first divide uniformly each tracklet into multiple temporally localized slices according to a time pyramid structure. Then, an initial re-id model is trained based on a two-stream convolutional architecture, and the accumulative motion context information of temporally localized slices of the tracklets per camera is learned. Finally, combining accumulative motion and tracklet spatial–temporal correlation, we associate tracklets across cameras and update the re-id model. The above steps are iterated to optimize the re-id model progressively. Experiment results demonstrate that the proposed method significantly outperforms the current state-of-the-art unsupervised video person re-identification methods on three video-based benchmark datasets, ILIDS-VID, MARS, and DukeMTMC-VideoReID.

Multiscale Global-Aware Channel Attention for Person Re-identification

Most person re-identification methods are researched under various assumptions. However, viewpoint variations or occlusions are often encountered in practical scenarios. These are prone to intra-class variance. In this paper, we propose a multiscale global-aware channel attention (MGCA) model to solve this problem. It imitates the process of human visual perception, which tends to observe things from coarse to fine. The core of our approach is a multiscale structure containing two key elements: the global-aware channel attention (GCA) module for capturing the global structural information and the adaptive selection feature fusion (ASFF) module for highlighting discriminative features. Moreover, we introduce a bidirectional guided pairwise metric triplet (BPM) loss to reduce the effect of outliers. Extensive experiments on Market-1501, DukeMTMC-reID, and MSMT17, and achieve the state-of-the-art results on mAP. Especially, our approach exceeds the current best method by 2.0% on the most challenging MSMT17 dataset.

Multi-granularity Pseudo-label Collaboration for unsupervised person re-identification

Unsupervised person re-identification (ReID) aims at training a model through using fully unlabeled training images. Most successful approaches combine clustering-based pseudo-label prediction with ReID model learning and perform the two steps in an alternating fashion. However, there are some person images, which have common appearance but different identities or have same identity but large intra-variations, and they are prone to be assigned wrong pseudo-labels. Existing methods typically infer person image pseudo-labels in a single pseudo-label space, which is insufficient due to losing the accurate labels of these hard images. This limits model’s power to cope with varying changes. To address this problem, we propose a Multi-granularity Pseudo-label Collaboration (MPC) method for unsupervised person ReID. Firstly, a multi-granularity pseudo-label prediction (MPP) method is proposed to predict the pseudo-labels of person images. MPP generates multiple pseudo-label spaces with each one inferring the person image pseudo-labels under a different granularity at cluster size. Secondly, the multiple pseudo-label spaces are applied to learn a mixture of experts (MoE) model. Each expert model is supervised by the pseudo-labels in a specific pseudo-label space, enabling MoE model to handle different variations. The output of these expert models are fused critically by a bilateral attention aggregation (BAA) to generate aggregation image features. Meanwhile, a label mix-up method is designed for refining pseudo-labels. It fuses labels across different pseudo-label spaces based on label co-occurrence to promote aggregation feature learning. We extensively evaluate the proposed MPC method under both image-based and video-based unsupervised person ReID settings. MPC significantly improves the current baseline method by a margin of 5.4% and 3.7% in mAP, and 3.4% and 3.9% in Rank1 accuracy on the image-based datasets Market-1501 and DukeMTMC-ReID, respectively. In particular, MPC method achieves state-of-the-art performance on video-based datasets with mAP of 71.4% and 87.3% on MARS and DukeMTMC-VideoReID, respectively.

A Person Re-Identification Method with Multi-Scale and Multi-Feature Fusion

In order to solve the problem of insufficient expression of pedestrian features and ignoring the semantic relevance of adjacent areas in person re-identification, a multi-scale and multi-feature fusion person re-identification model is proposed. Firstly, feature maps containing different information are obtained by multi-branch structure. Secondly, the semantic correlation between local features is emphasized by combining adjacent local regions. Finally, combining the advantages of maximum pooling and average pooling, the model can learn more comprehensive feature information from different directions. Experiments are carried out on Market-1501, DukeMTMC-reID and MSMT17 respectively, it can be evaluated that the model can effectively extract pedestrian feature and has high recognition accuracy under different lighting and camera angles.