Person Re-identification Problem Research Articles

A Subspace Learning Approach to Multishot Person Reidentification

This paper addresses the challenging problem of multishot person reidentification (Re-ID) in real world uncontrolled surveillance systems. A key issue is how to effectively represent and process the multiple data with various appearance information due to the variations of pose, occlusions, and viewpoints. To this end, this paper develops a novel subspace learning approach, which pursues regularized low-rank and sparse representation for multishot person Re-ID. For the images of a person crossing a certain camera, we assume that the appearances of those subset images with similar viewpoints against a camera draw from the same low-rank subspace, and all the images of a person under a camera lie on a union of low-rank subspaces. Based on this assumption, we propose to learn a nonnegative low-rank and sparse graph to represent the person images. Moreover, the recurring pattern prior is integrated into our model to refine the affinities among images. Extensive experiments on four public benchmark datasets yield impressive performance by improving 22.9% on imagery library for intelligent detection systems video re identification (iLIDS-VID), 42.4% on person RE-ID (PRID) dataset 2011, 39.7% and 30.6% on speech, audio, image, and video technology-SoftBio camera 3/8 and camera 5/8, respectively, and 1.6% on motion analysis and re identification set compared to the state-of-the-art methods.

An Effective Adversarial Attack on Person Re-Identification in Video Surveillance via Dispersion Reduction

Person re-identification across a network of cameras, with disjoint views, has been studied extensively due to its importance in wide-area video surveillance. This is a challenging task due to several reasons including changes in illumination and target appearance, and variations in camera viewpoint and camera intrinsic parameters. The approaches developed to re-identify a person across different camera views need to address these challenges. More recently, neural network-based methods have been proposed to solve the person re-identification problem across different camera views, achieving state-of-the-art performance. In this paper, we present an effective and generalizable attack model that generates adversarial images of people, and results in very significant drop in the performance of the existing state-of-the-art person re-identification models. The results demonstrate the extreme vulnerability of the existing models to adversarial examples, and draw attention to the potential security risks that might arise due to this in video surveillance. Our proposed attack is developed by decreasing the dispersion of the internal feature map of a neural network to degrade the performance of several different state-of-the-art person re-identification models. We also compare our proposed attack with other state-of-the-art attack models on different person re-identification approaches, and by using four different commonly used benchmark datasets. The experimental results show that our proposed attack outperforms the state-of-art attack models on the best performing person re-identification approaches by a large margin, and results in the most drop in the mean average precision values.

Bayesian Inferred Self-Attentive Aggregation for Multi-Shot Person Re-Identification

Person re-identification is a challenging retrieval task that aims to match pedestrians from multiple non-overlapping cameras. In this paper, we introduce a deep multi-instance learning framework to aggregate instance-level images to boost retrieval performance. Considerable annotation inconsistency inevitably happens in many current person re-identification datasets due to unconcerned of annotations or dramatic varieties in surveillance scenarios, thereby leading to model drifting. To alleviate this issue, we formulate the person re-identification problem in a weakly supervised setting, and propose a self-inspired attention model based on Bayesian inference, to adaptively evaluate regional features with their global dependencies across instances, which we refer to as Bayesian Inferred Self-Attentive Aggregation (BISAA). The evaluation mechanism is parameterized by neural networks to provide an insight into the contribution of each instance and semantic human part to set-level labels. Furthermore, to facilitate aggregation across a set of instances, we propose a new collective aggregation function to make the model more robust to outliers, by adjusting the activation threshold, to allow some non-informative instances to be ignored while paying more attention to the discriminative ones. Extensive experiments with ablation analysis show the effectiveness of our method and the proposed method outperforms many related state-of-the-art techniques on four benchmark datasets: PRID2011, iLIDS-VID, Market-1501 and MSMT17.

Scale-invariant batch-adaptive residual learning for person re-identification

The problem of person re-identification (re-ID) deals with matching two similar persons in probe and gallery sets. The underlying pattern matching task can become more complex as similar persons can appear in different scales in the two sets. In this paper, we address this challenging problem of scale-invariant person re-ID. As a solution, we propose two scale-invariant residual networks with a new loss function for deep metric learning. The first network, termed as Scale Invariant Triplet Network (SI-TriNet), is deeper and is trained from the pre-trained weights. In contrast, the second network, named Scale-Invariant Siamese Resnet-32 (SISR-32), is shallower and uses training from the scratch. Deep metric learning for both the networks are realized through a batch adaptive triplet loss function. Extensive comparisons and ablation studies on the benchmark Market-1501 and CUHK03 datasets clearly demonstrate the effectiveness of the proposed formulation.

Deep person re-identification in UAV images

The person re-identification is one of the most significant problems in computer vision and surveillance systems. The recent success of deep convolutional neural networks in image classification has inspired researchers to investigate the application of deep learning to the person re-identification. However, the huge amount of research on this problem considers classical settings, where pedestrians are captured by static surveillance cameras, although there is a growing demand for analyzing images and videos taken by drones. In this paper, we aim at filling this gap and provide insights on the person re-identification from drones. To our knowledge, it is the first attempt to tackle this problem under such constraints. We present the person re-identification dataset, named DRone HIT (DRHIT01), which is collected by using a drone. It contains 101 unique pedestrians, which are annotated with their identities. Each pedestrian has about 500 images. We propose to use a combination of triplet and large-margin Gaussian mixture (L-GM) loss to tackle the drone-based person re-identification problem. The proposed network equipped with multi-branch design, channel group learning, and combination of loss functions is evaluated on the DRHIT01 dataset. Besides, transfer learning from the most popular person re-identification datasets is evaluated. Experiment results demonstrate the importance of transfer learning and show that the proposed model outperforms the classic deep learning approach.

Sharp Attention Network via Adaptive Sampling for Person Re-Identification

In this paper, we present novel sharp attention networks by adaptively sampling feature maps from convolutional neural networks (CNNs) for person re-identification (re-ID) problem. Due to the introduction of sampling-based attention models, the proposed approach can adaptively generate sharper attention-aware feature masks. This greatly differs from the gating-based attention mechanism that relies soft gating functions to select the relevant features for person re-ID. In contrast, the proposed sampling-based attention mechanism allows us to effectively trim irrelevant features by enforcing the resultant feature masks to focus on the most discriminative features. It can produce sharper attentions that are more assertive in localizing subtle features relevant to re-identifying people across cameras. For this purpose, a differentiable Gumbel-Softmax sampler is employed to approximate the Bernoulli sampling to train the sharp attention networks. Extensive experimental evaluations demonstrate the superiority of this new sharp attention model for person re-ID over the other state-of-the-art methods on three challenging benchmarks including CUHK03, Market-1501, and DukeMTMC-reID.

ILRA: Novelty Detection in Face-Based Intervener Re-Identification

Transparency laws facilitate citizens to monitor the activities of political representatives. In this sense, automatic or manual diarization of parliamentary sessions is required, the latter being time consuming. In the present work, this problem is addressed as a person re-identification problem. Re-identification is defined as the process of matching individuals under different camera views. This paper, in particular, deals with open world person re-identification scenarios, where the captured probe in one camera is not always present in the gallery collected in another one, i.e., determining whether the probe belongs to a novel identity or not. This procedure is mandatory before matching the identity. In most cases, novelty detection is tackled applying a threshold founded in a linear separation of the identities. We propose a threshold-less approach to solve the novelty detection problem, which is based on a one-class classifier and therefore it does not need any user defined threshold. Unlike other approaches that combine audio-visual features, an Isometric LogRatio transformation of a posteriori (ILRA) probabilities is applied to local and deep computed descriptors extracted from the face, which exhibits symmetry and can be exploited in the re-identification process unlike audio streams. These features are used to train the one-class classifier to detect the novelty of the individual. The proposal is evaluated in real parliamentary session recordings that exhibit challenging variations in terms of pose and location of the interveners. The experimental evaluation explores different configuration sets where our system achieves significant improvement on the given scenario, obtaining an average F measure of 71.29% for online analyzed videos. In addition, ILRA performs better than face descriptors used in recent face-based closed world recognition approaches, achieving an average improvement of 1.6% with respect to a deep descriptor.

Learned versus Handcrafted Features for Person Re-identification

Person re-identification is one of the indispensable elements for visual surveillance. It assigns consistent labeling for the same person within the field of view of the same camera or even across multiple cameras. While handcrafted feature extraction is certainly one way of approaching this problem, in many cases, these features are becoming more and more complex. Besides, training a deep convolutional neural network (CNN) from scratch is difficult because it requires a large amount of labeled training data and a great deal of expertise to ensure proper convergence. This paper explores the following three main strategies for solving the person re-identification problem: (i) using handcrafted features, (ii) using transfer learning based on a pre-trained deep CNN (trained for object categorization) and (iii) training a deep CNN from scratch. Our experiments consistently demonstrated that: (1) The handcrafted features may still have favorable characteristics and benefits especially in cases where the learning database is not sufficient to train a deep network. (2) A fully trained Siamese CNN outperforms handcrafted approaches and the combination of pre-trained CNN with different re-identification processes. (3) Moreover, our experiments demonstrated that pre-trained features and handcrafted features perform equally well. These experiments have also revealed the most discriminative parts in the human body.

Multi-level Similarity Perception Network for Person Re-identification

In this article, we propose a novel deep Siamese architecture based on a convolutional neural network (CNN) and multi-level similarity perception for the person re-identification (re-ID) problem. According to the distinct characteristics of diverse feature maps, we effectively apply different similarity constraints to both low-level and high-level feature maps during training stage. Due to the introduction of appropriate similarity comparison mechanisms at different levels, the proposed approach can adaptively learn discriminative local and global feature representations, respectively, while the former is more sensitive in localizing part-level prominent patterns relevant to re-identifying people across cameras. Meanwhile, a novel strong activation pooling strategy is utilized on the last convolutional layer for abstract local-feature aggregation to pursue more representative feature representations. Based on this, we propose final feature embedding by simultaneously encoding original global features and discriminative local features. In addition, our framework has two other benefits: First, classification constraints can be easily incorporated into the framework, forming a unified multi-task network with similarity constraints. Second, as similarity-comparable information has been encoded in the network’s learning parameters via back-propagation, pairwise input is not necessary at test time. That means we can extract features of each gallery image and build an index in an off-line manner, which is essential for large-scale real-world applications. Experimental results on multiple challenging benchmarks demonstrate that our method achieves splendid performance compared with the current state-of-the-art approaches.

Multi-target Tracking in Multiple Non-overlapping Cameras Using Fast-Constrained Dominant Sets

In this paper, a unified three-layer hierarchical approach for solving tracking problem in a multiple non-overlapping cameras setting is proposed. Given a video and a set of detections (obtained by any person detector), we first solve within-camera tracking employing the first two layers of our framework and then, in the third layer, we solve across-camera tracking by associating tracks of the same person in all cameras simultaneously. To best serve our purpose, we propose fast-constrained dominant set clustering (FCDSC), a novel method which is several orders of magnitude faster (close to real time) than existing methods. FCDSC is a parameterized family of quadratic programs that generalizes the standard quadratic optimization problem. In our method, we first build a graph where nodes of the graph represent short-tracklets, tracklets and tracks in the first, second and third layer of the framework, respectively. The edge weights reflect the similarity between nodes. FCDSC takes as input a constrained set, a subset of nodes from the graph which need to be included in the extracted cluster. Given a constrained set, FCDSC generates compact clusters by selecting nodes from the graph which are highly similar to each other and with elements in the constrained set. We have tested this approach on a very large and challenging dataset (namely, MOTchallenge DukeMTMC) and show that the proposed framework outperforms the state-of-the-art approaches. Even though the main focus of this paper is on multi-target tracking in non-overlapping cameras, the proposed approach can also be applied to solve video-based person re-identification problem. We show that when the re-identification problem is formulated as a clustering problem, FCDSC can be used in conjunction with state-of-the-art video-based re-identification algorithms, to increase their already good performances. Our experiments demonstrate the general applicability of the proposed framework for multi-target multi-camera tracking and person re-identification tasks.

Deep asymmetric video-based person re-identification

In this paper, we investigate the problem of video-based person re-identification (re-id) which matches people’s video clips across non-overlapping camera views at different time. A key challenge of video-based person re-id is a person’s appearance and motion would always display differently and take effects unequally at disjoint camera views due to the change of lighting, viewpoint, background and etc., which we call the “view-bias” problem. However, many previous video-based person re-id approaches have not quantified the importance of different types of features at different camera views, so that the two types of important features (i.e. appearance and motion features) do not collaborate effectively and thus the “view-bias” problem remains unsolved. To address this problem, we propose a Deep Asymmetric Metric learning(DAM) method that embeds a proposed asymmetric distance metric learning loss into a two-stream deep neural network for jointly learning view-specific and feature-specific transformations to overcome the “view-bias” problem in video-based person re-id. As learning these view-specific transformations become expensive when there are large amount of camera views, a clustering-based DAM method is developed to make our DAM scalable. Extensive evaluations have been carried out on three public datasets: PRID2011, iLIDS-VID and MARS. Our results verify that learning view-specific and feature-specific transformations are beneficial, and the presented DAM has empirically performed more effectively overall for video-based person re-id on challenging benchmarks.

Discriminative Feature Learning with Foreground Attention for Person Re-identification.

The performance of person re-identification (Re-ID) has been seriously effected by the large cross-view appearance variations caused by mutual occlusions and background clutters. Hence learning a feature representation that can adaptively emphasize the foreground persons becomes very critical to solve the person Re-ID problem. In this paper, we propose a simple yet effective foreground attentive neural network (FANN) to learn a discriminative feature representation for person Re-ID, which can adaptively enhance the positive side of foreground and weaken the negative side of background. Specifically, a novel foreground attentive subnetwork is designed to drive the network's attention, in which a decoder network is used to reconstruct the binary mask by using a novel local regression loss function, and an encoder network is regularized by the decoder network to focus its attention on the foreground persons. The resulting feature maps of encoder network are further fed into the body part subnetwork and feature fusion subnetwork to learn discriminative features. Besides, a novel symmetric triplet loss function is introduced to supervise feature learning, in which the intra-class distance is minimized and the inter-class distance is maximized in each triplet unit, simultaneously. Training our FANN in a multi-task learning framework, a discriminative feature representation can be learned to find out the matched reference to each probe among various candidates in the gallery. Extensive experimental results on several public benchmark datasets are evaluated, which have shown clear improvements of our method over the state-of-the-art approaches.

Urgent image-to-video person reidentification by cross-media transfer cycle generative adversarial networks

Recently, image-to-video person reidentification (IVPR) has attracted enormous research interest, and various models are proposed. IVPR is often applied to urgent situations, such as suspect tracking and lost-human locating. Existing IVPR models are under supervised frameworks, which require a large number of labeled image-to-video pairs. This severely limits their real-time efficiency in urgent situations, because annotation is much more time-consuming. To solve the urgent image-to-video person reidentification (UIVPR) problem, we propose a cross-media transfer cycle generative adversarial networks (CTC-GAN) network. Our model aims to alleviate the “media-gap” between image-to-video pairs without newly labeled pairs. We make an existing completely labeled dataset as guidance for CTC-GAN to achieve domain adaptation and make urgent image-to-video matching easier for person reidentification. We introduce cycle GANs for image(video)-to-video(image) translation and extract cross-media features using a triplet constraint in the source domain for different media features. Furthermore, we train the model in the labeled source domain by reconstructing the image (video) as its related video (image). Then, train the model in the unlabeled target domain by reconstructing itself along with source data, so as to ensure that the discriminative model can be used in target domain. Through CTC-GAN, our network can retain pedestrian discriminative information as much as possible, to ensure the matching rate in the target domain. To validate the effectiveness of our approach, we implement substantial experiments on two large-scale person reidentification datasets compared with six existing state-of-the-art unsupervised revised person reidentification models, and experimental results demonstrate that our method can solve UIVPR effectively.

Multi-Instance Convolutional Neural Network for multi-shot person re-identification

This paper tackles the challenging problem of multi-shot person re-identification with Convolutional Neural Network (CNN). As no prior information about how importance each instance plays, it is non-trivial to exploit the interaction information shared by the multi-shot images to help identification. Traditional CNN is in single-shot architecture, then how to utilize the interaction information provided by multi-shot images becomes an important problem to solve. Furthermore, as data augmentation methods are not strictly label-preserving, it increases the difficulty to select discriminative instance for CNN training. In this paper, we propose a weakly supervised CNN framework named Multi-Instance Convolutional Neural Network (MICNN) to solve the aforementioned problem. We develop two paradigms, i.e., Embedding-Space paradigm and Instance-Space paradigm, which re-formulate the person re-identification problem as a multi-instance verification problem with part-based features extracted by neural network. We respectively devise a specific bag-level loss function which incorporates the characteristics of the multi-instance problem for each paradigm. Experiments show that the proposed IS method outperforms many related state-of-the-art techniques on four benchmark datasets: CUHK03, SYSUm, RAiD and Market-1501.

Person Re-Identification via Group Symmetry Theory

In recent years, deep learning represented by convolutional neural networks (CNNs) has developed rapidly. The development of deep learning has also led to rapid progress in the field of person re-identification. Many related researchers have begun to use deep learning to solve the problem of person re-identification. The existing deep learning methods for person re-identification mainly use the convolutional neural networks to extract features. The middle layers of the convolutional neural networks contain a wealth of structural information but the previous methods have not fully exploited them. This paper proposes a network named ResGroupNet that uses group symmetry theory to constrain the middle structure of ResNet-50. In detail, we added a branch at the fourth layer of the backbone, the branch was implemented based on theory, at each tail to the backbone and branch, we use sphere loss and triplet loss, respectively. According to our survey, we are the first to introduce the group theory into the ReID task. The experiments show that the proposed method is effective and have achieved good results on the Market-1501, DukeMTMC-reID, and CUHK03-NP datasets.

A Survey on Deep Learning-Based Person Re-Identification Systems

Person re-identification systems (person Re-ID) have recently gained more attention between computer vision researchers. They are playing a key role in intelligent visual surveillance systems and have widespread applications like applications for public security. The person Re-ID systems can identify if a person has been seen by a non-overlapping camera over large camera network in an unconstrained environment. It is a challenging issue since a person appears differently under different camera views and faces many challenges such as pose variation, occlusion and illumination changes. Many methods had been introduced for generating handcrafted features aimed to handle the person Re-ID problem. In recent years, many studies have started to apply deep learning methods to enhance the person Re-ID performance due the deep learning yielded significant results in computer vision issues. Therefore, this paper is a survey of the recent studies that proposed to improve the person Re-ID systems using deep learning. The public datasets that are used for evaluating these systems are discussed. Finally, the paper addresses future directions and current issues that must be considered toward improving the person Re-ID systems.

Cross-Media Body-Part Attention Network for Image-to-Video Person Re-Identification

Person re-identification (re-id) is a significant application in public security and attracts much more research interest due to its significant application in reality. Most person re-id models focus on image-based or video-based re-id problems. In fact, image-to-video person re-id has important significance in lost-human location, criminal-tracking, and pedestrian video retrieval. In image-to-video person re-id task, the key challenge of this issue is how to build an accurate connection between appearance image features and spatio-temporal video features due to the huge cross-media gap in different modalities. Although existing image-to-video person re-id models have achieved good effectiveness, there is still a large distance away from practical application. These methods only consider the similarity measurement of cross-media features, which are extracted from the original whole image/video without any importance. However, the main useful and discriminative information is always contained in human body parts (torso, elbow, wrist, knee, and ankle), while pedestrian image/video backgrounds retain lots of useless information. In this paper, we present a Cross-media Body-part Attention Network (CBAN) for image-to-video person re-id, which can extract the cross-media body part attention features from images/videos (by CNN/LSTM), and simultaneously ignore the useless information in the background by using a part attention mechanism. Besides, our network can alleviate the inherent cross-media gap by a novel media-pulling constraint term. The extensive experiments are conducted on three large scale datasets (Market1501, Mars and CUHK03) and two small datasets (PRID-2011, iLIDS-VID), and the results show our CBAN approach can solve the image-to-video person re-id problem effectively with a body-part attention mechanism.

Night Person Re-Identification and a Benchmark

Person re-identification is an important problem in computer vision fields due to its widely application. However, most of existing person re-identification methods are evaluated in daytime scenarios which is still far from real applications. In this paper, we pay attention to the night scenario person re-identification problem which most of works are not focused on. For this purpose, we contribute a large and real-scenario person re-identification dataset for night scenario named KnightReid, which aims to bridge the gap between theoretical research and practical application. To the best of our knowledge, the KnightReid dataset is the first night scenario dataset for the person re-identification which distinguishes existing works. Furthermore, by carefully examining the properties of night scenario data, we propose to combine image denoising networks with common used person re-identification networks to adapt to this kind of problem. Besides, we provide a comprehensive benchmark result that is evaluated on the dataset. The extensive experiments convince the effectiveness of the proposed model.

Person re-identification based on re-ranking with expanded k-reciprocal nearest neighbors

In this paper, a robust re-ranking method based on expanded k-reciprocal neighbors is proposed. Our method assumes that if a gallery image is the probe image of the expanded k-reciprocal nearest neighbors, these images are more likely to be of the same person. Specifically, given a probe image, we replace the probe with its expanded reciprocal nearest neighbor and the final distance is computed by the mean value of the corresponding neighbor set. The proposed method is unsupervised, automatic and applicable to other person re-identification problems. Moreover, our method can perform well even with a simple direct rank list where the Euclidean distance was used to compute the distances between the images. Experiments on many public datasets demonstrate the effectiveness and robustness of our re-ranking method. The proposed method achieves 4.9% improvement in Rank-1 on the CUHK03 dataset and a significant improvement of 18.6% in mAP on the Duke dataset.

Unsupervised Joint Subspace and Dictionary Learning for Enhanced Cross-Domain Person Re-Identification

Person re-identification (Re-ID) has drawn increasing attention from both academia and industry due to its great potentials in surveillance applications. Most existing research efforts have attempted to tackle cross-view variation in single-domain person Re-ID. However, there is still a lack of effective approaches to cross-domain person Re-ID problem. In this paper, an unsupervised joint subspace and dictionary learning (UJSDL) framework is proposed to address the cross-domain person Re-ID problem, where both cross-view (i.e., across different cameras in the same network of cameras) and cross-domain (i.e., across different network of cameras) variation are jointly addressed. In particular, to reduce the impact of cross-view distribution variation, the graph Laplacian approach is used to project the images from different camera views in each domain into a shared subspace. To alleviate the impact of cross-domain distribution variation, a shared dictionary is learned from all the projection subspaces such that the discriminative information from both the labeled source datasets and the unlabeled target dataset are well encoded. To efficiently solve the joint subspace and dictionary learning task, an alternating optimization algorithm is presented. We used multiple different feature sets and conducted experiments on multiple benchmark datasets as the target domain. The results demonstrate that UJSDL outperforms the state-of-the-art approaches.