Person Re-identification Framework Research Articles

Distributed person re-identification through network-wise rank fusion consensus

The problem of re-identify persons across single disjoint camera-pairs has received great attention from the community. Despite this, when the re-identification process has to be carried out on a wide camera network additional problems arise and deny the direct application of existing solutions. Thus, a different approach has to be considered. In particular, existing approaches have neglected the importance of the network topology (i.e., the configuration of the monitored area) in such a process. To try filling such a gap, we propose a distributed person re-identification framework which brings in the following contributions: (i) a weighted camera matching cost that measures the re-identification performance between cameras in the network; (ii) a derivation of the distance vector algorithm that yields to network topology learning and allows us to prioritize and limit the cameras inquired for the re-identification; (iii) a network consensus weighted rank fusion solution that allows us to perform the re-identification in a robust fashion. Results on four benchmark datasets show that the proposed approach brings to significant network-wise re-identification improvements.

Engineering Hand-designed and Deeply-learned features for person Re-identification

Person re-identification is still a useful and challenging task in pattern recognition and computer vision. Based on a probe image, the task focuses on identifying a set of matching images from a gallery set. In this work, a novel multi-feature fusion-based person re-identification framework is proposed. The key technique is to intelligently fuze hand-crafted feature and deeply-learned feature. More specifically, hand-crafted features from both local and global region are extracted from each image. Afterward, a novel Convolutional Neural Networks (CNN) is trained based on combining the three datasets, wherein robust deep features can be obtained. Finally, the above hand-crafted feature as well as the deep features are optimally fuzed for calculating a robust re-identification result. Extensive experimental results on three state-of-the-art data sets have demonstrated the effectiveness of our method.

Video Person Re-identification by Temporal Residual Learning.

In this paper, we propose a novel feature learning framework for video person re-identification (re-ID). The proposed framework largely aims to exploit the adequate temporal information of video sequences and tackle the poor spatial alignment of moving pedestrians. More specifically, for exploiting the temporal information, we design a temporal residual learning (TRL) module to simultaneously extract the generic and specific features of consecutive frames. The TRL module is equipped with two bi-directional LSTM (BiLSTM), which are respectively responsible to describe a moving person in different aspects, providing complementary information for better feature representations. To deal with the poor spatial alignment in video re- ID datasets, we propose a spatial-temporal transformer network (ST2N) module. Transformation parameters in the ST2N module are learned by leveraging the high-level semantic information of the current frame as well as the temporal context knowledge from other frames. The proposed ST2N module with less learnable parameters allows effective person alignments under significant appearance changes. Extensive experimental results on the largescale MARS, PRID2011, ILIDS-VID and SDU-VID datasets demonstrate that the proposed method achieves consistently superior performance and outperforms most of the very recent state-of-the-art methods.

Deep and low-level feature based attribute learning for person re-identification

In video surveillance, pedestrian attributes are defined as semantic descriptors like gender, clothing or accessories. In this paper, we propose a CNN-based pedestrian attribute-assisted person re-identification framework. First we perform the attribute learning by a part-specific CNN to model attribute patterns related to different body parts and fuse them with low-level robust Local Maximal Occurrence (LOMO) features to address the problem of the large variation of visual appearance and location of attributes due to different body poses and camera views. Our experiments on three public benchmarks show that the proposed method improves the state of the art on attribute recognition. Then we merge the learned attribute CNN embedding with another identification CNN embedding in a triplet structure to perform the person re-identification task. Both CNNs are pre-trained in a supervised way on attributes and person identities respectively, and then continue the training with a combined architecture for re-identification. We experimentally show that this fusion of “identity and attributes features” improves the overall re-identification.

Multishot person reidentification through inlier orientation and consensus clusters

Reidentifying people across disjoint camera views, known as person reidentification (PRId), has gained attention in many surveillance applications. This letter presents a framework for PRId that jointly exploits orientation and appearance cues to filter a subset of gallery candidates, where the probability of finding an exact match for a given probe remains very high. In addition, a similarity measure is formulated using the bipartite graph matching together with the weights of feature channels to estimate the correspondence within the filtered set. Experimental evaluation on standard datasets alongside comparisons with state-of-the-art validates the efficacy of our approach.

A person re-identification framework by inlier-set group modeling for video surveillance

Recognizing the same person across different camera views is crucial and yet difficult task in video surveillance. The difficulty lies in finding the matched image-pair against drastic variations in appearances and structures of the individuals. In this article, we present a three stage person re-identification framework to establish the correspondence among persons observed across non-overlapping camera views. In first stage, we propose to apply an algorithm for handling the illumination variations in image pairs. A pyramidal body partitioning scheme is then introduced to handle the viewpoint variations, in which it segments the pedestrian image into several logical parts. In second stage, we formulate an ensemble weighted hypergraph partitioning strategy that divides the gallery candidates into a set of groups with high intra-group and low inter-group commonality. A weighing scheme is suggested to find the contribution of each feature channel towards defining a group. Furthermore, we generate a set of inlier groups for each probe, where the probability of finding the desired match pair is high. In final stage, contributory weights are fused with the correlation-based similarity measure to find the corresponding match within the inlier group. Extensive experiments are carried out on three challenging datasets to evaluate the effectiveness of our proposed framework. The experimental results demonstrate that the proposed framework can achieve better performance compared with the existing methods.

Person re-identification by order-induced metric fusion

Abstract This paper presents a novel two-pronged framework for person re-identification. Its idea articulates over the fact that distinct descriptors manifest different ranking scores for the same probe pattern. Thus, if conveniently fused, the descriptors in hand are ought to compensate each other, leading to significant improvements. In this respect, this paper proposes a learning-free weighting method that penalizes and averages the re-identification estimates (e.g., distances) pointed out by different descriptors according to their confidence in evidencing the correct match, to a given probe person, among a given gallery. We particularly show that tangible improvements can be attained with respect to utilizing each descriptor individually. Moreover, we consider a confidence measure mechanism that treats the mutual pairwise distances within the gallery, in order to raise the scores obtained at the fusion stage, and we show that interesting improvements can be achieved. We evaluate the proposed framework on four benchmark datasets and advance late works by large margins.

Deep multi-instance learning for end-to-end person re-identification

In this paper, we introduce a deep multi-instance learning framework to boost the instance-level person re-identification performance. Motivated by the observation of considerably dramatic and complex varieties of visual appearances in many current person re-identification datasets, we explicitly represent a deep feature representation learning method for the final person re-identification task. However, most public datasets for person re-identification are usually small, that usually make deep learning model suffer from seriously over-fitting problem. To alleviate this matter, we formulate the problem of person re-identification as a deep multi-instance learning (DMIL) task. More specifically, We build a novel end-to-end person re-identification system by unifying DMIL with the convolutional feature learning. For well capturing these intra-class diversities and inter-class ambiguities of input visual samples across cameras, a multi-scale convolutional feature learning method is proposed by optimizing the Contrastive Loss function. Comprehensive evaluations over three public benchmark datasets (including VIPeR, ETHZ, and CUHK01 datasets) well demonstrate the encouraging performance of our proposed person re-identification framework on small datasets.

Illumination and scale invariant relevant visual features with hypergraph-based learning for multi-shot person re-identification

Person re-identification which aims at matching people across disjoint cameras has received increasing attention due to the widespread use of video surveillance applications. Existing methods concentrate either on robust feature extraction or view-invariant feature transformation. However, the extracted features suffer from various limitations such as color inconsistency and scale variations. Besides, during matching, a probe is compared against each gallery instance which represents only the pairwise relationship and ignores the high order relationship among them. To address these issues, we propose a multi-shot person re-identification framework that first performs a preprocessing task on images to address illumination variations for maintaining the color consistency. Subsequently, we formulate an approach to handle scale variations in the pedestrian appearances for keeping them with relatively a fixed scale ratio. Overlapped visual patches representing appearance cues are then extracted from the processed images. A structured multi-class feature selection approach is employed to select a set of relevant patches that simultaneously discriminates all distinct persons. These selected patches use a hypergraph to represent the visual association among a probe and gallery images. Finally, for matching, we formulate a hypergraph-based learning scheme, which considers both the pairwise and high-order association among the probe and gallery images. The hypergraph structure is then optimized to yield an improved similarity score for a probe against each gallery instance. The effectiveness of our proposed framework is validated on three public datasets and comparison with state-of-the-art methods shows the superior performance of our framework.

PaMM: Pose-Aware Multi-Shot Matching for Improving Person Re-Identification.

Person re-identification is the problem of recognizing people across different images or videos with non-overlapping views. Although a significant progress has been made in person re-identification over the last decade, it remains a challenging task because the appearances of people can seem extremely different across diverse camera viewpoints and person poses. In this paper, we propose a novel framework for person re-identification by analyzing camera viewpoints and person poses called pose-aware multi-shot matching. It robustly estimates individual poses and efficiently performs multi-shot matching based on the pose information. The experimental results obtained by using public person re-identification data sets show that the proposed methods outperform the current state-of-the-art methods, and are promising for accomplishing person re-identification under diverse viewpoints and pose variances.

Person re-identification via integrating patch-based metric learning and local salience learning

In this paper, aiming at improving the generalization capability, we propose a cross-dataset person re-identification framework via integrating patch-based metric learning and local salience learning. Firstly, Convolution Neural Network(CNN) features are extracted to represent patches of a person. Secondly, only two positive patch-pairs are chosen and input into a Large Margin Nearest Neighbour(LMNN) network to learn two patch-based metric matrices for feature projection respectively. Thirdly, according to projected new features, a local salience learning algorithm based on Kmeans clustering is proposed to train the weights of patches. Finally, the similarity of image-pair is computed by a weighted summing of all patches. The experimental results indicate that the proposed method outperforms existing conventional approaches based on hand-crafted features and achieves a comparable performance with most recent CNN-based methods, which demonstrates our method’s effectiveness and practicality. It does not need a large-scale labeled training dataset, and has a high matching rate with a low computation complexity.

Person Re-Identification by Camera Correlation Aware Feature Augmentation.

The challenge of person re-identification (re-id) is to match individual images of the same person captured by different non-overlapping camera views against significant and unknown cross-view feature distortion. While a large number of distance metric/subspace learning models have been developed for re-id, the cross-view transformations they learned are view-generic and thus potentially less effective in quantifying the feature distortion inherent to each camera view. Learning view-specific feature transformations for re-id (i.e., view-specific re-id), an under-studied approach, becomes an alternative resort for this problem. In this work, we formulate a novel view-specific person re-identification framework from the feature augmentation point of view, called Camera coR relation Aware Feature augmenTation (CRAFT). Specifically, CRAFT performs cross-view adaptation by automatically measuring camera correlation from cross-view visual data distribution and adaptively conducting feature augmentation to transform the original features into a new adaptive space. Through our augmentation framework, view-generic learning algorithms can be readily generalized to learn and optimize view-specific sub-models whilst simultaneously modelling view-generic discrimination information. Therefore, our framework not only inherits the strength of view-generic model learning but also provides an effective way to take into account view specific characteristics. Our CRAFT framework can be extended to jointly learn view-specific feature transformations for person re-id across a large network with more than two cameras, a largely under-investigated but realistic re-id setting. Additionally, we present a domain-generic deep person appearance representation which is designed particularly to be towards view invariant for facilitating cross-view adaptation by CRAFT. We conducted extensively comparative experiments to validate the superiority and advantages of our proposed framework over state-of-the-art competitors on contemporary challenging person re-id datasets.

A Neuromorphic Person Re-Identification Framework for Video Surveillance

This paper presents a neuromorphic person re-identification (NPReId) framework to establish the correspondence among individuals observed across two disjoint camera views. The proposed framework comprises three modules ( observation , cognition , and contemplation ), inspired by the form-and-color-and-depth (FACADE) theory model of object recognition system. In the observation module, a semantic partitioning scheme is introduced to segment a pedestrian into several logical parts, and an exhaustive set of experiments have been carried out to select the best possible complementary feature cues. In the cognition module, an unsupervised procedure is suggested to partition the gallery candidates into multiple consensus clusters with high intra-cluster and low inter-cluster similarity. A supervised classifier is then deployed to learn the relationship between each gallery candidate and its associated cluster, which is subsequently used to identify a set of inlier consensus clusters. This module also includes weighing of contribution of each feature channel toward defining a consensus cluster. Finally, in the contemplation module, the contributory weights are employed in a correlation-based similarity measure to find the corresponding match within the inlier set. The proposed framework is compared with several state-of-the-art methods on three challenging data sets: VIPeR, iLIDS-VID, and CUHK01. The experimental results, with respect to recognition rates, demonstrate that the proposed framework can obtain superior performance as compared with the counterparts. The proposed framework, along with its low-rank bound property, further establishes its suitability in practical scenarios through yielding high cluster hit rate with low database penetration.

Person Reidentification via Ranking Aggregation of Similarity Pulling and Dissimilarity Pushing

Person reidentification is a key technique to match different persons observed in nonoverlapping camera views. Many researchers treat it as a special object-retrieval problem, where ranking optimization plays an important role. Existing ranking optimization methods mainly utilize the similarity relationship between the probe and gallery images to optimize the original ranking list, but seldom consider the important dissimilarity relationship. In this paper, we propose to use both similarity and dissimilarity cues in a ranking optimization framework for person reidentification. Its core idea is that the true match should not only be similar to those strongly similar galleries of the probe, but also be dissimilar to those strongly dissimilar galleries of the probe. Furthermore, motivated by the philosophy of multiview verification, a ranking aggregation algorithm is proposed to enhance the detection of similarity and dissimilarity based on the following assumption: the true match should be similar to the probe in different baseline methods. In other words, if a gallery blue image is strongly similar to the probe in one method, while simultaneously strongly dissimilar to the probe in another method, it will probably be a wrong match of the probe. Extensive experiments conducted on public benchmark datasets and comparisons with different baseline methods have shown the great superiority of the proposed ranking optimization method.

Resolution adaptive feature extracting and fusing framework for person re-identification

Person re-identification has gained an increasing attention in both of academic research communities and industrial labs in recent years. It remains as a challenging problem due to the extracting and matching of reliable and distinctive features under different camera views across a wide spatial and temporal scope. To address these issues, we propose a resolution adaptive method by extracting and fusing the global and local features within a unified framework. Specifically, global features and local features are extracted separately in two image scales which are constructed in a cascaded way. After extracting HS and HOG features and ranking in low scale, we choose top k percent as a candidate subset, meanwhile we obtain another person subset by LPQ (Local Phase Quantization) Face detection. The union of these two candidate subsets is used for high scale processing in which wHSV (weighted Hue Saturation Value), LSCF (Local Spatial Constrain Feature) and MSCR (Maximally Stable Colour Regions) local features are adopted. Afterwards, both the global and local features extracted are fused with an unsupervised query adaptive method, based on which person re-identification is conducted with a high accuracy. Experiments are conducted on two real world datasets: ETHZ1, 2, 3, and our own dataset from high resolution cameras in real roads and campus scenes. Experimental results demonstrate that the proposed method outperforms the conventional methods in terms of both accuracy and efficiency.

Person Reidentification in a Distributed Camera Network Framework.

Plenty of research has been conducted to obtain the best reidentification performance between a single camera-pairs. None of the current approaches has addressed the reidentification in a camera network by considering the network topology (i.e., the structure of the monitored environment). We introduce a distributed network person reidentification framework which introduces the following contributions. 1) a camera matching cost to measure the reidentification performance between nodes of the network and 2) a derivation of the distance vector algorithm which allows to learn the network topology thus to prioritize and limit the cameras inquired for the matching of the probe. Results on three benchmark datasets show that the network topology can be learned in an unsupervised fashion and network-wise reidentification performance improves. As a side effect, we obtain that the communication bandwidth usage is reduced.