Deep Person Re-identification Research Articles

Towards Robust Person Re-Identification by Defending Against Universal Attackers.

Recent studies show that deep person re-identification (re-ID) models are vulnerable to adversarial examples, so it is critical to improving the robustness of re-ID models against attacks. To achieve this goal, we explore the strengths and weaknesses of existing re-ID models, i.e., designing learning-based attacks and training robust models by defending against the learned attacks. The contributions of this paper are three-fold: First, we build a holistic attack-defense framework to study the relationship between the attack and defense for person re-ID. Second, we introduce a combinatorial adversarial attack that is adaptive to unseen domains and unseen model types. It consists of distortions in pixel and color space (i.e., mimicking camera shifts). Third, we propose a novel virtual-guided meta-learning algorithm for our attack-defense system. We leverage a virtual dataset to conduct experiments under our meta-learning framework, which can explore the cross-domain constraints for enhancing the generalization of the attack and the robustness of the re-ID model. Comprehensive experiments on three large-scale re-ID benchmarks demonstrate that: 1) Our combinatorial attack is effective and highly universal in cross-model and cross-dataset scenarios; 2) Our meta-learning algorithm can be readily applied to different attack and defense approaches, which can reach consistent improvement; 3) The defense model trained on the learning-to-learn framework is robust to recent SOTA attacks that are not even used during training.

A deep person re-identification model with multi visual-semantic information embedding

The local features of different body parts have been widely used to learn more discriminative representation for person re-identification, which act as either extra visual semantic information or auxiliary means to deal with the issue of misalignment and background bias. However, the existing person re-identification works mainly focuses on the common impact of multiple body parts while failing to explicitly explore the influence of body edge contour. As the edge contour is one of the most significant visual-semantic clues for object detection and person identification in the blurred scene, this paper intentionally explores the effect of edge contour clues on person re-identification and proposes a deep learning framework with multi visual-semantic information embedding, including body parts and edge contour. Meanwhile, we conceive a practical strategy which can effectively fuse the different body part features and reduce the dimensionality of features. Extensive experimental results on four benchmark data sets show that our model has achieved competitive accuracy compared to the state-of-the-art models.

A Strong Baseline and Batch Normalization Neck for Deep Person Re-Identification

This study explores a simple but strong baseline for person re-identification (ReID). Person ReID with deep neural networks has progressed and achieved high performance in recent years. However, many state-of-the-art methods design complex network structures and concatenate multi-branch features. In the literature, some effective training tricks briefly appear in several papers or source codes. The present study collects and evaluates these effective training tricks in person ReID. By combining these tricks, the model achieves 94.5% rank-1 and 85.9% mean average precision on Market1501 with only using the global features of ResNet50. The performance surpasses all existing global- and part-based baselines in person ReID. We propose a novel neck structure named as batch normalization neck (BNNeck). BNNeck adds a batch normalization layer after global pooling layer to separate metric and classification losses into two different feature spaces because we observe they are inconsistent in one embedding space. Extended experiments show that BNNeck can boost the baseline, and our baseline can improve the performance of existing state-of-the-art methods. Our codes and models are available at: https://github.com/michuanhaohao/reid-strong-baseline.

Adaptive Exploration for Unsupervised Person Re-identification

Due to domain bias, directly deploying a deep person re-identification (re-ID) model trained on one dataset often achieves considerably poor accuracy on another dataset. In this article, we propose an Adaptive Exploration (AE) method to address the domain-shift problem for re-ID in an unsupervised manner. Specifically, in the target domain, the re-ID model is inducted to (1) maximize distances between all person images and (2) minimize distances between similar person images. In the first case, by treating each person image as an individual class, a non-parametric classifier with a feature memory is exploited to encourage person images to move far away from each other. In the second case, according to a similarity threshold, our method adaptively selects neighborhoods for each person image in the feature space. By treating these similar person images as the same class, the non-parametric classifier forces them to stay closer. However, a problem of the adaptive selection is that, when an image has too many neighborhoods, it is more likely to attract other images as its neighborhoods. As a result, a minority of images may select a large number of neighborhoods while a majority of images has only a few neighborhoods. To address this issue, we additionally integrate a balance strategy into the adaptive selection. We evaluate our methods with two protocols. The first one is called “target-only re-ID”, in which only the unlabeled target data is used for training. The second one is called “domain adaptive re-ID”, in which both the source data and the target data are used during training. Experimental results on large-scale re-ID datasets demonstrate the effectiveness of our method. Our code has been released at https://github.com/dyh127/Adaptive-Exploration-for-Unsupervised-Person-Re-Identification.

Visualization of spatial matching features during deep person re-identification

Person re-identification (Re-ID) based on deep learning has made great progress and achieved state-of-the-art performance in recent years. However, the end-to-end properties of deep neural networks allow us to directly feedback the output results based on its input, making the inner working mechanism of the deep person Re-ID model and its decision reasons lack of transparency and explainability. This further impedes improvements to pedestrian recognition performance. As feature visualization has been proven to be an effective method for characterizing the middle layer of a neural network, we propose a novel gradient-based visualization method to interpret the internal features learned by deep person Re-ID. Based on the idea of transfer learning, this model regards the pretrained ResNet-50 on the ImageNet dataset as a basic network for deep person Re-ID. First, the network is fine-tuned on the person Re-ID dataset to achieve pedestrian classification, and then, the gradient-based visualization of the trained network is performed to highlight important regions contributing to image similarity. Experiments conducted on the Market-1501 dataset verify that our model can not only enable the network to identify key features of an individual across different images, but also provide visual interpretation for the pedestrian classification results to improve the reliability of person Re-ID and foster trust from users regarding its decisions.

Deep person re-identification in UAV images

The person re-identification is one of the most significant problems in computer vision and surveillance systems. The recent success of deep convolutional neural networks in image classification has inspired researchers to investigate the application of deep learning to the person re-identification. However, the huge amount of research on this problem considers classical settings, where pedestrians are captured by static surveillance cameras, although there is a growing demand for analyzing images and videos taken by drones. In this paper, we aim at filling this gap and provide insights on the person re-identification from drones. To our knowledge, it is the first attempt to tackle this problem under such constraints. We present the person re-identification dataset, named DRone HIT (DRHIT01), which is collected by using a drone. It contains 101 unique pedestrians, which are annotated with their identities. Each pedestrian has about 500 images. We propose to use a combination of triplet and large-margin Gaussian mixture (L-GM) loss to tackle the drone-based person re-identification problem. The proposed network equipped with multi-branch design, channel group learning, and combination of loss functions is evaluated on the DRHIT01 dataset. Besides, transfer learning from the most popular person re-identification datasets is evaluated. Experiment results demonstrate the importance of transfer learning and show that the proposed model outperforms the classic deep learning approach.

Learning deep embedding with mini-cluster loss for person re-identification

Recently, the triplet loss is commonly used in many deep person re-identification (ReID) frameworks to learn an embedding space in which similar data points are close and dissimilar data points are far away. However, the triplet loss simply focuses on the relative orders of points. This may lead to a relatively large intra-class variance and then a weak generalization capacity on the test set. In this paper, we propose a mini-cluster loss, which regards images belonging to the same identity as a mini-cluster and treats them as a whole during the training instead of considering them separately. For each mini-cluster in a batch, we define the largest distance between points in a mini-cluster as its inner divergence and the shortest distance with outer points as its outer divergence. By constraining the outer divergence larger than the inner divergence, our framework with the mini-cluster loss achieves the more compact mini-clusters while keeping the diversity distributions of the classes. As a result, a better generalization ability and a higher performance can be obtained. In the extensive experiments, our proposed framework achieves a state-of-the-art performance on two large-scale person ReID datasets (Market1501, DukeMTMC-reID) which clearly demonstrates its effectiveness. Specifically, 72.44% mAP and 87.05% rank-1 score are achieved on the Market1501 dataset with single query setting, 78.17% mAP and 91.05% rank-1 score with multiply query setting, and on the DukeMTMC-reID dataset, 60.19% mAP and 77.20% rank-1 score are obtained.

Pseudo-positive regularization for deep person re-identification

An intrinsic challenge of person re-identification (re-ID) is the annotation difficulty. This typically means 1) few training samples per identity, and 2) thus the lack of diversity among the training samples. Consequently, we face high risk of over-fitting when training the convolutional neural network (CNN), a state-of-the-art method in person re-ID. To reduce the risk of over-fitting, this paper proposes a Pseudo Positive Regularization (PPR) method to enrich the diversity of the training data. Specifically, unlabeled data from an independent pedestrian database is retrieved using the target training data as query. A small proportion of these retrieved samples are randomly selected as the Pseudo Positive samples and added to the target training set for the supervised CNN training. The addition of Pseudo Positive samples is therefore a data augmentation method to reduce the risk of over-fitting during CNN training. We implement our idea in the identification CNN models (i.e., CaffeNet, VGGNet-16 and ResNet-50). On CUHK03 and Market-1501 datasets, experimental results demonstrate that the proposed method consistently improves the baseline and yields competitive performance to the state-of-the-art person re-ID methods.