Person Re-identification Datasets Research Articles

Multi-type attributes driven multi-camera person re-identification

One of the major challenges in person Re-Identification (ReID) is the inconsistent visual appearance of a person. Current works on visual feature and distance metric learning have achieved significant achievements, but still suffer from the limited robustness to pose variations, viewpoint changes, etc., and the high computational complexity. This makes person ReID among multiple cameras still challenging. This work is motivated to learn mid-level human attributes which are robust to visual appearance variations and could be used as efficient features for person matching. We propose a weakly supervised multi-type attribute learning framework which considers the contextual cues among attributes and progressively boosts the accuracy of attributes only using a limited number of labeled data. Specifically, this framework involves a three-stage training. A deep Convolutional Neural Network (dCNN) is first trained on an independent dataset labeled with attributes. Then it is fine-tuned on another dataset only labeled with person IDs using our defined triplet loss. Finally, the updated dCNN predicts attribute labels for the target dataset, which is combined with the independent dataset for the final round of fine-tuning. The predicted attributes, namely deep attributes exhibit promising generalization ability across different datasets. By directly using the deep attributes with simple Cosine distance, we have obtained competitive accuracy on four person ReID datasets. Experiments also show that a simple distance metric learning modular further boosts our method, making it outperform many recent works.

Kernelized random KISS metric learning for person re-identification

Abstract Person re-identification is critical for human tracking in the video surveillance which has attracted more and more attention in recent years. Various recent approaches have made great progress in re-identification performance using metric learning techniques and among them, Keep It Simple and Straightforward (KISS) metric learning method has shown remarkably importance because of its simpleness and high-efficiency. The KISS method is based on an assumption that the differences between feature pairs obey the Gaussian distribution. However, for most existing features of person re-identification, the distributions of differences between feature pairs are irregular and undulant. Therefore, prior to the Guassian based metric learning step, it's important to augment the Guassian distribution of data without losing discernment. Moreover, most metric learning methods were greatly influenced by the small sample size (SSS) problem and the KISS method is no exception, which causing the inexistence of inverse of covariance matrices. To solve the above two problems, we present Kernelized Random KISS (KRKISS) metric learning method. By transforming the original features into kernelized features, the differences between feature pairs can better fit the Gaussian distribution and thus they can be more suitable for the Guassian assumption based models. To solve the inverse of covariance matrix estimation problem, we apply a random subspace ensemble method to obtain exact estimation of covariance matrix by randomly selecting and combining several different subspaces. In each subspace, the influence of SSS problem can be minimized. Experimental results on three challenging person re-identification datasets demonstrate that the KRKISS method significantly improves the KISS method and achieves better performance than most existing metric learning approaches.

Perceptual hash-based feature description for person re-identification

Person re-identification is one of the most important and challenging problems in video surveillance systems. For person re-identification, feature description is a fundamental problem. While many approaches focus on exploiting low-level features to describe person images, most of them are not robust enough to illumination and viewpoint changes. In this paper, we propose a simple yet effective feature description method for person re-identification. Starting from low-level features, the proposed method uses perceptual hashing to binarize low-level feature maps and combines several feature channels for feature encoding. Then, an image pyramid is built, and three regional statistics are computed for hierarchical feature description. To some extent, the perceptual hash algorithm (PHA) can encode invariant macro structures of person images to make the representation robust to both illumination and viewpoint changes. On the other hand, while a rough hashing may be not discriminative enough, the combination of several different feature channels and regional statistics is able to exploit complementary information and enhance the discriminability. The proposed approach is evaluated on seven major person re-identification datasets. The results of comprehensive experiments show the effectiveness of the proposed method and notable improvements over the state-of-the-art approaches.

Multi-modal uniform deep learning for RGB-D person re-identification

In this paper, we propose a multi-model uniform deep learning (MMUDL) method for RGB-D person re-identification. Unlike most existing person re-identification methods which only use RGB images, our approach recognizes people from RGB-D images so that more information such as anthropometric measures and body shapes can be exploited for re-identification. In order to exploit useful information from depth images, we use the deep network to extract efficient anthropometric features from processed depth images which also have three channels. Moreover, we design a multi-modal fusion layer to combine these features extracted from both depth images and RGB images through the network with a uniform latent variable which is robust to noise, and optimize the fusion layer with two CNN networks jointly. Experimental results on two RGB-D person re-identification datasets are presented to show the efficiency of our proposed approach.

Deep multi-instance learning for end-to-end person re-identification

In this paper, we introduce a deep multi-instance learning framework to boost the instance-level person re-identification performance. Motivated by the observation of considerably dramatic and complex varieties of visual appearances in many current person re-identification datasets, we explicitly represent a deep feature representation learning method for the final person re-identification task. However, most public datasets for person re-identification are usually small, that usually make deep learning model suffer from seriously over-fitting problem. To alleviate this matter, we formulate the problem of person re-identification as a deep multi-instance learning (DMIL) task. More specifically, We build a novel end-to-end person re-identification system by unifying DMIL with the convolutional feature learning. For well capturing these intra-class diversities and inter-class ambiguities of input visual samples across cameras, a multi-scale convolutional feature learning method is proposed by optimizing the Contrastive Loss function. Comprehensive evaluations over three public benchmark datasets (including VIPeR, ETHZ, and CUHK01 datasets) well demonstrate the encouraging performance of our proposed person re-identification framework on small datasets.

PaMM: Pose-Aware Multi-Shot Matching for Improving Person Re-Identification.

Person re-identification is the problem of recognizing people across different images or videos with non-overlapping views. Although a significant progress has been made in person re-identification over the last decade, it remains a challenging task because the appearances of people can seem extremely different across diverse camera viewpoints and person poses. In this paper, we propose a novel framework for person re-identification by analyzing camera viewpoints and person poses called pose-aware multi-shot matching. It robustly estimates individual poses and efficiently performs multi-shot matching based on the pose information. The experimental results obtained by using public person re-identification data sets show that the proposed methods outperform the current state-of-the-art methods, and are promising for accomplishing person re-identification under diverse viewpoints and pose variances.

End-to-End Comparative Attention Networks for Person Re-Identification.

Person re-identification across disjoint camera views has been widely applied in video surveillance yet it is still a challenging problem. One of the major challenges lies in the lack of spatial and temporal cues, which makes it difficult to deal with large variations of lighting conditions, viewing angles, body poses, and occlusions. Recently, several deep-learning-based person re-identification approaches have been proposed and achieved remarkable performance. However, most of those approaches extract discriminative features from the whole frame at one glimpse without differentiating various parts of the persons to identify. It is essentially important to examine multiple highly discriminative local regions of the person images in details through multiple glimpses for dealing with the large appearance variance. In this paper, we propose a new soft attention-based model, i.e., the end-to-end comparative attention network (CAN), specifically tailored for the task of person re-identification. The end-to-end CAN learns to selectively focus on parts of pairs of person images after taking a few glimpses of them and adaptively comparing their appearance. The CAN model is able to learn which parts of images are relevant for discerning persons and automatically integrates information from different parts to determine whether a pair of images belongs to the same person. In other words, our proposed CAN model simulates the human perception process to verify whether two images are from the same person. Extensive experiments on four benchmark person re-identification data sets, including CUHK01, CHUHK03, Market-1501, and VIPeR, clearly demonstrate that our proposed end-to-end CAN for person re-identification outperforms well established baselines significantly and offer the new state-of-the-art performance.

Joint distance and similarity measure learning based on triplet-based constraints

Distance and similarity measures usually are complementary to pattern classification. With pairwise constraints, several approaches have been proposed to combine distance and similarity measures. However, it remains less investigated to use triplets of samples for joint learning of distance and similarity measures. Moreover, the kernel extension of triplet-based model is also nontrivial and computationally expensive. In this paper, we propose a novel method to learn a combined distance and similarity measure (CDSM). By incorporating with the max-margin model, we suggest a triplet-based CDSM learning model with a unified regularizer of the Frobenius norm. A support vector machine (SVM)-based algorithm is then adopted to solve the optimization problem. Furthermore, we extend CDSM for learning nonlinear measures via the kernel trick. Two effective strategies are adopted to speed up training and testing of kernelized CDSM. Experiments on the UCI, handwritten digits and person re-identification datasets demonstrate that CDSM and kernelized CDSM outperform several state-of-the-art metric learning methods.

An ensemble of visual features for Gaussians of local descriptors and non-binary coding for texture descriptors

This paper presents an improved version of a recent state-of-the-art texture descriptor called Gaussians of Local Descriptors (GOLD), which is based on a multivariate Gaussian that models the local feature distribution that describes the original image. The full rank covariance matrix, which lies on a Riemannian manifold, is projected on the tangent Euclidean space and concatenated to the mean vector for representing a given image. In this paper, we test the following features for describing the original image: scale-invariant feature transform (SIFT), histogram of gradients (HOG), and weber's law descriptor (WLD). To improve the baseline version of GOLD, we describe the covariance matrix using a set of visual features that are fed into a set of Support Vector Machines (SVMs). The SVMs are combined by sum rule. The scores obtained by an SVM trained using the original GOLD approach and the SVMs trained with visual features are then combined by sum rule. Experiments show that our proposed variant outperforms the original GOLD approach. The superior performance of the proposed system is validated across a large set of datasets. Particularly interesting is the performance obtained in two widely used person re-identification datasets, CAVIAR4REID and IAS, where the proposed GOLD variant is coupled with a state-of-the-art ensemble to obtain an improvement of performance on these two datasets. Moreover, we performed further tests that combine GOLD with non-binary features (local ternary/quinary patterns) and deep transfer learning. The fusion among SVMs trained with deep features and the SVMs trained using the ternary/quinary coding ensemble is demonstrated to obtain a very high performance across datasets. The MATLAB code for the ensemble of classifiers and for the extraction of the features will be publicly available11https://www.dei.unipd.it/node/2357 (+Pattern Recognition and Ensemble Classifiers) to other researchers for future comparisons.

Kernelised reference‐wise metric learning

Unlike the doublet or triplet constraints, a novel kernelised reference-wise metric learning is proposed by constructing reference-wise constraints, which contain similarity information of each sample to all reference samples. After selecting several representative training samples as the reference set, the training data are first mapped into a projected space by the reference sample matrix. Then the problem of metric learning is casted as a multi-label classification problem under the reference-wise constraints, in which an l 2-norm regularised least squares canonical correlation analysis can be used. Besides, the formulation is generalised to kernelised version for further boosting the performance. Experiments on two benchmark person re-identification datasets demonstrate that the approach clearly outperforms the state-of-the-art methods.

Unsupervised Learning of Multi-Level Descriptors for Person Re-Identification

In this paper, we propose a novel coding method named weighted linear coding (WLC) to learn multi-level (e.g., pixel-level, patch-level and image-level) descriptors from raw pixel data in an unsupervised manner. It guarantees the property of saliency with a similarity constraint. The resulting multi-level descriptors have a good balance between the robustness and distinctiveness. Based on WLC, all data from the same region can be jointly encoded. Consequently, when we extract the holistic image features, it is able to preserve the spatial consistency. Furthermore, we apply PCA to these features and compact person representations are then achieved. During the stage of matching persons, we exploit the complementary information resided in multi-level descriptors via a score-level fusion strategy. Experiments on the challenging person re-identification datasets - VIPeR and CUHK 01, demonstrate the effectiveness of our method.

Person Re-Identification by Camera Correlation Aware Feature Augmentation.

The challenge of person re-identification (re-id) is to match individual images of the same person captured by different non-overlapping camera views against significant and unknown cross-view feature distortion. While a large number of distance metric/subspace learning models have been developed for re-id, the cross-view transformations they learned are view-generic and thus potentially less effective in quantifying the feature distortion inherent to each camera view. Learning view-specific feature transformations for re-id (i.e., view-specific re-id), an under-studied approach, becomes an alternative resort for this problem. In this work, we formulate a novel view-specific person re-identification framework from the feature augmentation point of view, called Camera coR relation Aware Feature augmenTation (CRAFT). Specifically, CRAFT performs cross-view adaptation by automatically measuring camera correlation from cross-view visual data distribution and adaptively conducting feature augmentation to transform the original features into a new adaptive space. Through our augmentation framework, view-generic learning algorithms can be readily generalized to learn and optimize view-specific sub-models whilst simultaneously modelling view-generic discrimination information. Therefore, our framework not only inherits the strength of view-generic model learning but also provides an effective way to take into account view specific characteristics. Our CRAFT framework can be extended to jointly learn view-specific feature transformations for person re-id across a large network with more than two cameras, a largely under-investigated but realistic re-id setting. Additionally, we present a domain-generic deep person appearance representation which is designed particularly to be towards view invariant for facilitating cross-view adaptation by CRAFT. We conducted extensively comparative experiments to validate the superiority and advantages of our proposed framework over state-of-the-art competitors on contemporary challenging person re-id datasets.

Fully-automated person re-identification in multi-camera surveillance system with a robust kernel descriptor and effective shadow removal method

In this paper, a fully-automated person Re-ID (Re-identification) system is proposed for real scenarios of human tracking in non-overlapping camera network. The system includes two phases of human detection and Re-ID. The human ROIs (Regions of Interest) are extracted from human detection phase and then feature extraction is done on these ROIs in order to build human descriptor for Re-ID. Unlike other approaches which deal with manually-cropped human ROIs for person Re-ID, in this system, the person identity is determined based on the human ROIs extracted automatically by a combined method of human detection. Two main contributions are proposed on both phases of human detection and Re-ID in order to enhance the performance of person Re-ID system. First, an effective shadow removal method based on score fusion of density matching is proposed to get better human detection results. Second, a robust KDES (Kernel DEScriptor) is extracted from human ROI for person classification. Additionally, a new person Re-ID dataset is built in real surveillance scenarios from multiple cameras. The experiments on benchmark datasets and our own dataset show that the person Re-ID results using the proposed solutions outperform some of the state-of-the-art methods.

Discriminant quaternion local binary pattern embedding for person re-identification through prototype formation and color categorization

Re-identifying objects is one of the fundamental elements for visual surveillance, in the sense that images of the same object at different time or places should be assigned with the same label. In this work, we propose a new embedding scheme for person re-identification under nonoverlapping target cameras. Inspired by the prototype approach derived from cognition field, we propose to use prototype images as a reference set to achieve a discriminative representation of a person's appearance. To enhance the discrimination between different persons, we learn a linear subspace in a training phase during which person correspondences are assumed to be known. The robustness of the algorithm against results that are counterintuitive to a human operator is improved by proposing the Color Categorization procedure. By doing so, our method becomes very flexible when tracing a person in a camera network even under large illumination changes. The proposed framework was tested on VIPeR, the most challenging dataset for person re-identification. Results confirm that our method outperforms the state of the art techniques.

Continuous adaptation of multi-camera person identification models through sparse non-redundant representative selection

The problem of image-base person identification/recognition is to provide an identity to the image of an individual based on learned models that describe his/her appearance. Most traditional person identification systems rely on learning a static model on tediously labeled training data. Though labeling manually is an indispensable part of a supervised framework, for a large scale identification system labeling huge amount of data is a significant overhead. For large multi-sensor data as typically encountered in camera networks, labeling a lot of samples does not always mean more information, as redundant images are labeled several times. In this work, we propose a convex optimization based iterative framework that progressively and judiciously chooses a sparse but informative set of samples for labeling, with minimal overlap with previously labeled images. We also use a structure preserving sparse reconstruction based classifier to reduce the training burden typically seen in discriminative classifiers. The two stage approach leads to a novel framework for online update of the classifiers involving only the incorporation of new labeled data rather than any expensive training phase. We demonstrate the effectiveness of our approach on multi-camera person re-identification datasets, to demonstrate the feasibility of learning online classification models in multi-camera big data applications. Using three benchmark datasets, we validate our approach and demonstrate that our framework achieves superior performance with significantly less amount of manual labeling.

Person Re-Identification by Dual-Regularized KISS Metric Learning.

Person re-identification aims to match the images of pedestrians across different camera views from different locations. This is a challenging intelligent video surveillance problem that remains an active area of research due to the need for performance improvement. Person re-identification involves two main steps: feature representation and metric learning. Although the keep it simple and straightforward (KISS) metric learning method for discriminative distance metric learning has been shown to be effective for the person re-identification, the estimation of the inverse of a covariance matrix is unstable and indeed may not exist when the training set is small, resulting in poor performance. Here, we present dual-regularized KISS (DR-KISS) metric learning. By regularizing the two covariance matrices, DR-KISS improves on KISS by reducing overestimation of large eigenvalues of the two estimated covariance matrices and, in doing so, guarantees that the covariance matrix is irreversible. Furthermore, we provide theoretical analyses for supporting the motivations. Specifically, we first prove why the regularization is necessary. Then, we prove that the proposed method is robust for generalization. We conduct extensive experiments on three challenging person re-identification datasets, VIPeR, GRID, and CUHK 01, and show that DR-KISS achieves new state-of-the-art performance.

Person re-identification via hypergraph-based matching

Person re-identification which aims to match people across non-overlapping cameras has become an important research topic due to the increasing demand in many important applications such as video surveillance and security monitoring. Matching people in different cameras is a challenging task since the appearance of the same subject may change dramatically in different views due to variations in pose, lighting condition, etc. In order to reduce the feature discrepancy caused by view change, most of the existing methods focus either on robust feature extraction or view-invariant feature transformation. During matching, a subject to be identified, i.e., a probe, is compared with each subject in the gallery with known identities. The returned ranked list is generated based on the similarity scores. However, such a matching process only considers pairwise similarity between the probe and a gallery subject while higher order relationships between the probe and the gallery or even among the gallery subjects are ignored. To address this issue, we propose a hypergraph-based matching scheme in which both pairwise and higher order relationships for the probe and gallery subjects are discovered through hypergraph learning. In this way, improved similarity scores are obtained as compared to the conventional pairwise similarity measure. We conduct experiments on two widely used person re-identification datasets and the results demonstrate that matching through hypergraph learning leads to superior performance compared with state-of-the-arts. Furthermore, the proposed approach can be easily incorporated into any existing approach where similarities between probe and gallery are to be computed.

Person re-identification over camera networks using multi-task distance metric learning.

Person reidentification in a camera network is a valuable yet challenging problem to solve. Existing methods learn a common Mahalanobis distance metric by using the data collected from different cameras and then exploit the learned metric for identifying people in the images. However, the cameras in a camera network have different settings and the recorded images are seriously affected by variability in illumination conditions, camera viewing angles, and background clutter. Using a common metric to conduct person reidentification tasks on different camera pairs overlooks the differences in camera settings; however, it is very time-consuming to label people manually in images from surveillance videos. For example, in most existing person reidentification data sets, only one image of a person is collected from each of only two cameras; therefore, directly learning a unique Mahalanobis distance metric for each camera pair is susceptible to over-fitting by using insufficiently labeled data. In this paper, we reformulate person reidentification in a camera network as a multitask distance metric learning problem. The proposed method designs multiple Mahalanobis distance metrics to cope with the complicated conditions that exist in typical camera networks. We address the fact that these Mahalanobis distance metrics are different but related, and learned by adding joint regularization to alleviate over-fitting. Furthermore, by extending, we present a novel multitask maximally collapsing metric learning (MtMCML) model for person reidentification in a camera network. Experimental results demonstrate that formulating person reidentification over camera networks as multitask distance metric learning problem can improve performance, and our proposed MtMCML works substantially better than other current state-of-the-art person reidentification methods.