Person Re-identification Models Research Articles

Complementary Pseudo Labels for Unsupervised Domain Adaptation On Person Re-Identification.

In recent years, supervised person re-identification (re-ID) models have received increasing studies. However, these models trained on the source domain always suffer dramatic performance drop when tested on an unseen domain. Existing methods are primary to use pseudo labels to alleviate this problem. One of the most successful approaches predicts neighbors of each unlabeled image and then uses them to train the model. Although the predicted neighbors are credible, they always miss some hard positive samples, which may hinder the model from discovering important discriminative information of the unlabeled domain. In this paper, to complement these low recall neighbor pseudo labels, we propose a joint learning framework to learn better feature embeddings via high precision neighbor pseudo labels and high recall group pseudo labels. The group pseudo labels are generated by transitively merging neighbors of different samples into a group to achieve higher recall. However, the merging operation may cause subgroups in the group due to imperfect neighbor predictions. To utilize these group pseudo labels properly, we propose using a similarity-aggregating loss to mitigate the influence of these subgroups by pulling the input sample towards the most similar embeddings. Extensive experiments on three large-scale datasets demonstrate that our method can achieve state-of-the-art performance under the unsupervised domain adaptation re-ID setting.

Back-propagation of the Mahalanobis istance through a deep triplet learning model for person Re-Identification

The automatization of the Re-Identification of an individual across different video-surveillance cameras poses a significant challenge due to the presence of a vast number of potential candidates with a similar appearance. This task requires the learning of discriminative features from person images and a distance metric to properly compare them and decide whether they belong to the same person or not. Nevertheless, the fact of acquiring images of the same person from different, distant and non-overlapping views produces changes in illumination, perspective, background, resolution and scale between the person’s representations, resulting in appearance variations that hamper his/her re-identification. This article focuses the feature learning on automatically finding discriminative descriptors able to reflect the dissimilarities mainly due to the changes in actual people appearance, independently from the variations introduced by the acquisition point. With that purpose, such variations have been implicitly embedded by the Mahalanobis distance. This article presents a learning algorithm to jointly model features and the Mahalanobis distance through a Deep Neural Re-Identification model. The Mahalanobis distance learning has been implemented as a novel neural layer, forming part of a Triplet Learning model that has been evaluated over PRID2011 dataset, providing satisfactory results.

Person Re-IDentification Based on Mutual Learning With Embedded Noise Block

Some person re-identification(Re-ID) algorithms based on deep learning utilizes a baseline as basis to modify, and add some strategies to achieve better performance. Different from the conventional methods, this work combines distillation with mutual learning to construct a person Re-ID model of mutual learning. In training, in view of the characteristics of metric learning, we introduce a mutual loss <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$L_{M2}$ </tex-math></inline-formula> in the mutual learning network, so as to better promote the student networks to mine complementary information. In order to overcome the coupling problem in mutual learning, we designed a lightweight noise block and embedded it into mutual learning, which greatly improves the complementarity between networks. It should be added that the improvement achieved on the poor baseline can’t strictly prove the effectiveness of the research, so this paper constructs a person Re-ID baseline with relatively good performance, which is used as the student networks in mutual learning. Experiments demonstrate that the proposed person Re-ID algorithm based on mutual learning with embedded noise block achieves competitive performance on the Market1501, DukeMTMC-ReID, and CUHK-03 datasets.

Beyond Universal Person Re-Identification Attack

Deep learning-based person re-identification (Re-ID) has made great progress and achieved high performance recently. In this paper, we make the first attempt to examine the vulnerability of current person Re-ID models against a dangerous attack method, i.e. , the universal adversarial perturbation (UAP) attack, which has been shown to fool classification models with a little overhead. We propose a more universal adversarial perturbation (MUAP) method for both image-agnostic and model-insensitive person Re-ID attack. Firstly, we adopt a list-wise attack objective function to disrupt the similarity ranking list directly. Secondly, we propose a model-insensitive mechanism for cross-model attack. Extensive experiments show that the proposed attack approach achieves high attack performance and outperforms other state of the arts by large margin in cross-model scenario. The results also demonstrate the vulnerability of current Re-ID models to MUAP and further suggest the need of designing more robust Re-ID models.

End-to-End Domain Adaptive Attention Network for Cross-Domain Person Re-Identification

Person re-identification (re-ID) remains challenging in a real-world scenario, as it requires a trained network to generalise to totally unseen target data in the presence of variations across domains. Recently, generative adversarial models have been widely adopted to enhance the diversity of training data. These approaches, however, often fail to generalise to other domains, as existing generative person re-identification models have a disconnect between the generative component and the discriminative feature learning stage. To address the on-going challenges regarding model generalisation, we propose an end-to-end domain adaptive attention network to jointly translate images between domains and learn discriminative re-id features in a single framework. To address the domain gap challenge, we introduce an attention module for image translation from source to target domains without affecting the identity of a person. More specifically, attention is directed to the background instead of the entire image of the person, ensuring identifying characteristics of the subject are preserved. The proposed joint learning network results in a significant performance improvement over state-of-the-art methods on several challenging benchmark datasets.

Learning Person Re-Identification Models From Videos With Weak Supervision.

Most person re-identification methods, being supervised techniques, suffer from the burden of massive annotation requirement. Unsupervised methods overcome this need for labeled data, but perform poorly compared to the supervised alternatives. In order to cope with this issue, we introduce the problem of learning person re-identification models from videos with weak supervision. The weak nature of the supervision arises from the requirement of video-level labels, i.e. person identities who appear in the video, in contrast to the more precise frame-level annotations. Towards this goal, we propose a multiple instance attention learning framework for person re-identification using such video-level labels. Specifically, we first cast the video person re-identification task into a multiple instance learning setting, in which person images in a video are collected into a bag. The relations between videos with similar labels can be utilized to identify persons, on top of that, we introduce a co-person attention mechanism which mines the similarity correlations between videos with person identities in common. The attention weights are obtained based on all person images instead of person tracklets in a video, making our learned model less affected by noisy annotations. Extensive experiments demonstrate the superiority of the proposed method over the related methods on two weakly labeled person re-identification datasets.

A deep person re-identification model with multi visual-semantic information embedding

The local features of different body parts have been widely used to learn more discriminative representation for person re-identification, which act as either extra visual semantic information or auxiliary means to deal with the issue of misalignment and background bias. However, the existing person re-identification works mainly focuses on the common impact of multiple body parts while failing to explicitly explore the influence of body edge contour. As the edge contour is one of the most significant visual-semantic clues for object detection and person identification in the blurred scene, this paper intentionally explores the effect of edge contour clues on person re-identification and proposes a deep learning framework with multi visual-semantic information embedding, including body parts and edge contour. Meanwhile, we conceive a practical strategy which can effectively fuse the different body part features and reduce the dimensionality of features. Extensive experimental results on four benchmark data sets show that our model has achieved competitive accuracy compared to the state-of-the-art models.

Bodyprint—A Meta-Feature Based LSTM Hashing Model for Person Re-Identification

Person re-identification is concerned with matching people across disjointed camera views at different places and different time instants. This task results of great interest in computer vision, especially in video surveillance applications where the re-identification and tracking of persons are required on uncontrolled crowded spaces and after long time periods. The latter aspects are responsible for most of the current unsolved problems of person re-identification, in fact, the presence of many people in a location as well as the passing of hours or days give arise to important visual appearance changes of people, for example, clothes, lighting, and occlusions; thus making person re-identification a very hard task. In this paper, for the first time in the state-of-the-art, a meta-feature based Long Short-Term Memory (LSTM) hashing model for person re-identification is presented. Starting from 2D skeletons extracted from RGB video streams, the proposed method computes a set of novel meta-features based on movement, gait, and bone proportions. These features are analysed by a network composed of a single LSTM layer and two dense layers. The first layer is used to create a pattern of the person’s identity, then, the seconds are used to generate a bodyprint hash through binary coding. The effectiveness of the proposed method is tested on three challenging datasets, that is, iLIDS-VID, PRID 2011, and MARS. In particular, the reported results show that the proposed method, which is not based on visual appearance of people, is fully competitive with respect to other methods based on visual features. In addition, thanks to its skeleton model abstraction, the method results to be a concrete contribute to address open problems, such as long-term re-identification and severe illumination changes, which tend to heavily influence the visual appearance of persons.

Segmentation mask-guided person image generation

Background clutters and pose variation are the key factors which prevents the network from learning a robust Person re-identification (Re-ID) model. To address the problem above, we first introduce the binary segmentation mask to construct the body region served as the input of the generator, then design a segmentation mask-guided person image generation network for the pose transfer. The binary segmentation mask has the capability of removing the background clutters in pixel-level, and contains more details about the edge information, where better shape consistency can be achieved for the generated image with the input image. Compared with the previous methods, the proposed method can dramatically improve the model adaptive ability and deal with the diversity of postures. In addition, we design a lightweight attention mechanism module as a guider module, which can assist the generator to focus on the discriminative features of pedestrians. The experiment results are introduced to demonstrate the effectiveness of the proposed method and the superiority performance over most state-of-the-art methods without over-computing in the design process of the Re-ID model. It is worth mentioning that our ideas can be easily combined with other fields to solve the phenomenon of the current situation with insufficient pose variations in the datasets.

Appearance and Motion Enhancement for Video-Based Person Re-Identification

In this paper, we propose an Appearance and Motion Enhancement Model (AMEM) for video-based person re-identification to enrich the two kinds of information contained in the backbone network in a more interpretable way. Concretely, human attribute recognition under the supervision of pseudo labels is exploited in an Appearance Enhancement Module (AEM) to help enrich the appearance and semantic information. A Motion Enhancement Module (MEM) is designed to capture the identity-discriminative walking patterns through predicting future frames. Despite a complex model with several auxiliary modules during training, only the backbone model plus two small branches are kept for similarity evaluation which constitute a simple but effective final model. Extensive experiments conducted on three popular video-based person ReID benchmarks demonstrate the effectiveness of our proposed model and the state-of-the-art performance compared with existing methods.

An Improved Deep Mutual-Attention Learning Model for Person Re-Identification

Person re-identification is the task of matching pedestrian images across a network of non-overlapping camera views. It poses aggregated challenges resulted from random human pose, clutter from the background, illumination variations, and other factors. There has been a vast number of studies in recent years with promising success. However, key challenges have not been adequately addressed and continue to result in sub-optimal performance. Attention-based person re-identification gains more popularity in identifying discriminatory features from person images. Its potential in terms of extracting features common to a pair of person images across the feature extraction pipeline has not been be fully exploited. In this paper, we propose a novel attention-based Siamese network driven by a mutual-attention module decomposed into spatial and channel components. The proposed mutual-attention module not only leads feature extraction to the discriminative part of individual images, but also fuses mutual features symmetrically across pairs of person images to get informative regions common to both input images. Our model simultaneously learns feature embedding for discriminative cues and the similarity measure. The proposed model is optimized with multi-task loss, namely classification and verification loss. It is further optimized by a learnable mutual-attention module to facilitate an efficient and adaptive learning. The proposed model is thoroughly evaluated on extensively used large-scale datasets, Market-1501 and Duke-MTMC-ReID. Our experimental results show competitive results with the state-of-the-art works and the effectiveness of the mutual-attention module.

Adaptive Exploration for Unsupervised Person Re-identification

Due to domain bias, directly deploying a deep person re-identification (re-ID) model trained on one dataset often achieves considerably poor accuracy on another dataset. In this article, we propose an Adaptive Exploration (AE) method to address the domain-shift problem for re-ID in an unsupervised manner. Specifically, in the target domain, the re-ID model is inducted to (1) maximize distances between all person images and (2) minimize distances between similar person images. In the first case, by treating each person image as an individual class, a non-parametric classifier with a feature memory is exploited to encourage person images to move far away from each other. In the second case, according to a similarity threshold, our method adaptively selects neighborhoods for each person image in the feature space. By treating these similar person images as the same class, the non-parametric classifier forces them to stay closer. However, a problem of the adaptive selection is that, when an image has too many neighborhoods, it is more likely to attract other images as its neighborhoods. As a result, a minority of images may select a large number of neighborhoods while a majority of images has only a few neighborhoods. To address this issue, we additionally integrate a balance strategy into the adaptive selection. We evaluate our methods with two protocols. The first one is called “target-only re-ID”, in which only the unlabeled target data is used for training. The second one is called “domain adaptive re-ID”, in which both the source data and the target data are used during training. Experimental results on large-scale re-ID datasets demonstrate the effectiveness of our method. Our code has been released at https://github.com/dyh127/Adaptive-Exploration-for-Unsupervised-Person-Re-Identification.

Visualization of spatial matching features during deep person re-identification

Person re-identification (Re-ID) based on deep learning has made great progress and achieved state-of-the-art performance in recent years. However, the end-to-end properties of deep neural networks allow us to directly feedback the output results based on its input, making the inner working mechanism of the deep person Re-ID model and its decision reasons lack of transparency and explainability. This further impedes improvements to pedestrian recognition performance. As feature visualization has been proven to be an effective method for characterizing the middle layer of a neural network, we propose a novel gradient-based visualization method to interpret the internal features learned by deep person Re-ID. Based on the idea of transfer learning, this model regards the pretrained ResNet-50 on the ImageNet dataset as a basic network for deep person Re-ID. First, the network is fine-tuned on the person Re-ID dataset to achieve pedestrian classification, and then, the gradient-based visualization of the trained network is performed to highlight important regions contributing to image similarity. Experiments conducted on the Market-1501 dataset verify that our model can not only enable the network to identify key features of an individual across different images, but also provide visual interpretation for the pedestrian classification results to improve the reliability of person Re-ID and foster trust from users regarding its decisions.

Spatial Preserved Graph Convolution Networks for Person Re-identification

Person Re-identification is a very challenging task due to inter-class ambiguity caused by similar appearances, and large intra-class diversity caused by viewpoints, illuminations, and poses. To address these challenges, in this article, a graph convolution network based model for person re-identification is proposed to learn more discriminative feature embeddings, where a graph-structured relationship between person images and person parts are together integrated. Graph convolution networks extract common characteristics of the same person, while pyramid feature embedding exploits parts relations and learns stable representation with each person image. We achieve a very competitive performance respectively on three widely used datasets, indicating that the proposed approach significantly outperforms the baseline methods and achieves the state-of-the-art performance.

Pair-based Uncertainty and Diversity Promoting Early Active Learning for Person Re-identification

The effective training of supervised Person Re-identification (Re-ID) models requires sufficient pairwise labeled data. However, when there is limited annotation resource, it is difficult to collect pairwise labeled data. We consider a challenging and practical problem called Early Active Learning, which is applied to the early stage of experiments when there is no pre-labeled sample available as references for human annotating. Previous early active learning methods suffer from two limitations for Re-ID. First, these instance-based algorithms select instances rather than pairs, which can result in missing optimal pairs for Re-ID. Second, most of these methods only consider the representativeness of instances, which can result in selecting less diverse and less informative pairs. To overcome these limitations, we propose a novel pair-based active learning for Re-ID. Our algorithm selects pairs instead of instances from the entire dataset for annotation. Besides representativeness, we further take into account the uncertainty and the diversity in terms of pairwise relations. Therefore, our algorithm can produce the most representative, informative, and diverse pairs for Re-ID data annotation. Extensive experimental results on five benchmark Re-ID datasets have demonstrated the superiority of the proposed pair-based early active learning algorithm.

Unsupervised domain adaption for image-to-video person re-identification

Recently, person re-identification technique has been successfully applied to many fields, such as suspect tracking and lost human location. As video always contains more valuable information, more and more researchers focus on video based person re-identification, especially in image-to-video person re-identification (IVPR). However, most of existing IVPR models are under the supervised framework. In fact, marking enough training samples will cost numbers of labors, which limits the practical value of them. At the same time, the 2D features extracted from pedestrian image and 3D features extracted from pedestrian video are heterogeneous, which brings significant challenge for IVPR task. To effective solve the above problems, we propose an unsupervised domain adaption image-to-video person re-identification model by cross-modal feature generating and target information preserving transfer network (CMGTN). On one hand, the designed generator in our model can not only transform target domain unlabeled sample features into source domain feature space, but also can preserve target identity information. On the other hand, we eliminate the gap between pedestrian images and videos by embedding a cross-modal loss term. To evaluate the performance of our approach, we conduct extensive experiments on PRID-2011, iLIDS-VID and MARS datasets, and compare our approach with existing state-of-the-art IVPR models including four unsupervised methods and three supervised methods. Experimental results demonstrate the effectiveness of our approach.

An Effective Adversarial Attack on Person Re-Identification in Video Surveillance via Dispersion Reduction

Person re-identification across a network of cameras, with disjoint views, has been studied extensively due to its importance in wide-area video surveillance. This is a challenging task due to several reasons including changes in illumination and target appearance, and variations in camera viewpoint and camera intrinsic parameters. The approaches developed to re-identify a person across different camera views need to address these challenges. More recently, neural network-based methods have been proposed to solve the person re-identification problem across different camera views, achieving state-of-the-art performance. In this paper, we present an effective and generalizable attack model that generates adversarial images of people, and results in very significant drop in the performance of the existing state-of-the-art person re-identification models. The results demonstrate the extreme vulnerability of the existing models to adversarial examples, and draw attention to the potential security risks that might arise due to this in video surveillance. Our proposed attack is developed by decreasing the dispersion of the internal feature map of a neural network to degrade the performance of several different state-of-the-art person re-identification models. We also compare our proposed attack with other state-of-the-art attack models on different person re-identification approaches, and by using four different commonly used benchmark datasets. The experimental results show that our proposed attack outperforms the state-of-art attack models on the best performing person re-identification approaches by a large margin, and results in the most drop in the mean average precision values.

Part-Based Enhanced Super Resolution Network for Low-Resolution Person Re-Identification

Person re-identification (REID) is an important task in video surveillance and forensics applications. Many previous works often build models on the assumption that they have same resolution cross different camera views, while it is divorced from reality. To increase the adaptability of person REID models, this paper focuses on the low-resolution person REID task to relax the impractical assumption when traditional low-resolution person REID models are under pixel-to-pixel supervision in low and high resolution pedestrian image pairs. In addition, they are easily influenced by the global background, illumination or pose variations across camera views. Therefore, we propose a Part-based Enhanced Super Resolution (PESR) network by employing a part division strategy and an enhanced generative adversarial network to boost the unpaired pedestrian image super resolution process. Specifically, the part-based super resolution network transforms low resolution image in probe into high resolution without any pixel-to-pixel supervision and the part-based synthetic feature extractor module can learn discriminative pedestrian feature representation for the generated high resolution images, which employ a part feature connection loss as constraint to conduct matching for person re-identification. Furthermore, evaluations on four public person REID datasets demonstrate the advantages of our method over the state-of-the-art ones.

A Discriminative Person Re-Identification Model With Global-Local Attention and Adaptive Weighted Rank List Loss

At present, occlusion and appearance similarity pose severe challenges to person re-identification tasks. Although many robust deep convolutional neural networks alleviate these problems, convolutional layers with limited receptive fields cannot model global semantic information well. In addition, in the person re-identification model, many metric losses ignore or destroy the intra-class structure of the sample, which makes the model difficult to be optimized. Therefore, we design a discriminative Re-identification model with global-local attention and adaptive weighted rank list loss (GLWR). Specifically, our global-local attention (GL-Attention) learns the semantic context in the channel and spatial dimensions. By learning the dependencies between features, GL-Attention integrates global semantic information into local features to extract discriminative features. Unlike rank list loss, our adaptive weighted rank list loss (WRLL) adaptively assigns weights according to the metric distance between the negative sample and the input image, which further improves the performance of the model. Experimental studies on three public datasets (Market-1501, DukeMTMC-ReID and CUHK03) indicate that the performance of our GLWR is significantly superior to many of the latest algorithms.

Learning to Align via Wasserstein for Person Re-Identification

Existing successful person re-identification (Re-ID) models often employ the part-level representation to extract the fine-grained information, but commonly use the loss that is particularly designed for global features, ignoring the relationship between semantic parts. In this paper, we present a novel triplet loss that emphasizes the salient parts and also takes the consideration of alignment. This loss is based on the crossing-bing matching metric that also known as Wasserstein Distance. It measures how much effort it would take to move the embeddings of local features to align two distributions, such that it is able to find an optimal transport matrix to re-weight the distance of different local parts. The distributions in support of local parts is produced via a new attention mechanism, which is calculated by the inner product between high-level global feature and local features, representing the importance of different semantic parts w.r.t. identification. We show that the obtained optimal transport matrix can not only distinguish the relevant and misleading parts, and hence assign different weights to them, but also rectify the original distance according to the learned distributions, resulting in an elegant solution for the mis-alignment issue. Besides, the proposed method is easily implemented in most Re-ID learning system with end-to-end training style, and can obviously improve their performance. Extensive experiments and comparisons with recent Re-ID methods manifest the competitive performance of our method.