Person Re-identification Datasets Research Articles

Person re-identification via semi-supervised adaptive graph embedding.

Video surveillance is an indispensable part of the smart city for public safety and security. Person Re-Identification (Re-ID), as one of elementary learning tasks for video surveillance, is to track and identify a given pedestrian in a multi-camera scene. In general, most existing methods has firstly adopted a CNN based detector to obtain the cropped pedestrian image, it then aims to learn a specific distance metric for retrieval. However, unlabeled gallery images are generally overlooked and not utilized in the training. On the other hands, Manifold Embedding (ME) has well been applied to Person Re-ID as it is good to characterize the geometry of database associated with the query data. However, ME has its limitation to be scalable to large-scale data due to the huge computational complexity for graph construction and ranking. To handle this problem, we in this paper propose a novel scalable manifold embedding approach for Person Re-ID task. The new method is to incorporate both graph weight construction and manifold regularized term in the same framework. The graph we developed is discriminative and doubly-stochastic so that the side information has been considered so that it can enhance the clustering performances. The doubly-stochastic property can also guarantee the graph is highly robust and less sensitive to the parameters. Meriting from such a graph, we then incorporate the graph construction, the subspace learning method in the unified loss term. Therefore, the subspace results can be utilized into the graph construction, and the updated graph can in turn incorporate discriminative information for graph embedding. Extensive simulations is conducted based on three benchmark Person Re-ID datasets and the results verify that the proposed method can achieve better ranking performance compared with other state-of-the-art graph-based methods.

Joint generative and camera-aware clustering for unsupervised domain adaptation on person re-identification

Existing supervised person re-identification (Re-ID) methods demonstrate excellent performance. However, their performances suffer degradation when tested on an unseen different distributed domain. Generative adversarial network-based (GAN-based) and clustering- or pseudolabel-based methods are proposed to alleviate this problem. Due to the transfer scheme and the ignorance of correlations between the style-transferred and the original target images, the performance of GAN-based methods is unsatisfactory. We resolve these problems by jointly employing a generative strategy and performing camera-aware clustering for the target domain. Style-transferred images are generated from source cameras to target cameras, and then they are merged into the target domain selectively after exploiting their domain-specific discriminative information. To reduce the noise in generated images, we propose a domain-level boundary separation loss to group the transferred images and push them away from the original target images. The camera-level neighborhood-based clustering is proposed to learn well-clustered features in a camera-aware manner. Extensive experiments on two commonly used person Re-ID datasets demonstrate that our proposed method can achieve state-of-the-art performance.

Occlusion-Sensitive Person Re-Identification via Attribute-Based Shift Attention

Occluded person re-identification is one of the most challenging tasks in security surveillance. Most existing methods focus on extracting human body features from occluded pedestrian images. This paper prioritizes a difference between occluded and non-occluded person re-ID: When computing the similarity between a holistic pedestrian image and an occluded pedestrian image, a certain part of the human body in this holistic image can be distractive for pedestrian retrieval. To solve this problem, we propose an occluded person re-ID framework named attribute-based shift attention network (ASAN). First, unlike other methods that use off-the-shelf tools to locate pedestrian body parts in the occluded images, we design an attribute-guided occlusion-sensitive pedestrian segmentation (AOPS) module. AOPS is a weakly supervised method that leverages the semantic-level attribute annotations in person re-ID datasets. Second, guided by the pedestrian masks provided by AOPS, a shift feature adaption (SFA) module extracts the visible part of the human body feature in a part-based manner. After that, a visible region matching (VRM) algorithm is proposed to filter out the interfer-ence information in the holistic person images during the retrieval phase and further purify the representation of pedestrian features. Extensive experiments with ablation analysis demonstrate our method’s effectiveness. And the state-of-the-art results are achieved on four occluded datasets Partial-REID, Partial-iLIDS, Occluded-DukeMTMC, and Occluded REID. Moreover, the experiments on two holistic person re-ID datasets Market-1501 and DukeMTMC-reID, and a vehicle re-ID dataset VeRi-776 show that ASAN also has a good generality.

An Episodic Learning Network for Text Detection on Human Bodies in Sports Images

Due to the proliferation of sports-related multimedia content on the WWW, effective visual search and retrieval present interesting research challenges. These are caused by poor image quality, a wide range of possible camera points of view, pose variations on the part of athletes engaged in playing a sport, deformations of text appearing on sports person’s clothing and uniforms in motion, occlusions caused by other objects, etc. To address these challenges, this paper presents a new method for detecting text on human bodies in sports images. Unlike most existing methods, which attempt to exploit locations of a player’s torso, face, and skin, we propose an end-to-end episodic learning approach that employs inductive learning criteria for detecting clothing regions in an image, which are, in turn, then used for text detection. Our method integrates a Residual Network (ResNet) and Pyramidal Pooling Module (PPM) for generating a spatial attention map. The Progressive Scalable Expansion Algorithm (PSE) is adapted for text detection from these regions. Experimental results on our own dataset as well as several benchmarks (like RBNR and MMM which contain images of runners in marathons, and Re-ID which is a person re-identification dataset) demonstrate that the proposed method outperforms existing methods in terms of precision and F1-score. We also present results for sports images chosen from natural scene text detection datasets such as CTW1500 and MS-COCO to show the proposed method is effective and reliable across a range of inputs.

Multi‐stage attention network for video‐based person re‐identification

Video-based person re-identification (Re-ID) has received increasing attention in video surveillance analysis in recent years. To extract relevant information of the target, many existing methods utilise the attention mechanism in the residual block of the ResNet. However, these methods only focus on the residual block and ignore the output of the shortcut part, which also contains rich information about the person. To solve this problem, a different aspect of network design is investigated: the insert position of the attention module. To simultaneously explore the discriminative information in both the residual block and the shortcut, a novel multi-stage attention method is proposed by inserting the attention mechanism between stages of ResNet. Using this method can effectively extract the rich discriminative features of the target to better distinguish different pedestrians and improve the feature extraction capabilities of the model. Extensive experiments are conducted on four popular video-based person Re-ID datasets to demonstrate the effectiveness of the authors’ proposed method and display its superiority with the existing video-based person Re-ID methods.

Consistent attentive dual branch network for person re-identification

Several recent person re-identification methods are focusing on learning discriminative representations by designing efficient metric learning loss functions. Other approaches design part based architectures to compute an informative descriptor based on local features from semantically coherent parts. Few efforts learn the relationship between distant similar regions and parts by adjusting them to their most feasible positions with the help of soft attention. However, they focus on calibrating distant similar parts features and ignore to learn the noise (blur) free and distinct feature representations as the person re-identification datasets contain degraded images. To tackle these issues, we propose a novel Consistent Attention Dual Branch Network (CadNet) that has ability to model long-range dependencies (correlations) between channels as well as feature maps. We adopt multiple classifiers trained to learn the most discriminative global features for a unique representation of a person. Correlation between channels are consistently computed by using channel attention mechanism to make the learned feature noise free and distict from noisy and blurry data. Feature correlations interpret the relationship between distant similarities in the images computed by the self attention mechanism. The proposed CadNet significantly enhances the performance with respect to the baseline on the person re-identification benchmarks.

Relation-based global-partial feature learning network for video-based person re-identification

Video-based person re-identification (Re-ID) aims to match the same pedestrian from the video sequences captured by non-overlapping cameras. It is the key to fully extracting abundant spatial and temporal information from the video frames in video-based Re-ID. In this paper, a novel Relation-Based Global-Partial Feature Learning framework is proposed to explore discriminative spatiotemporal features with the help of the global and partial relationship between frames. Specifically, we propose a Relation-Based Global Feature Learning Module (RGL) to obtain global references for generating features correlation maps between frames in the video sequence and determine the importance of frame-level features. As the supplementary of the global relation-based features, a Relation-Based Partial Feature Learning Module (RPL) is also proposed to obtain the relationship between partial features of the same spatial position in different frames to enhance the frame-level partial representation. Moreover, we design a multi-level training scheme to deeply supervise our model. Extensive experiments are conducted on three public video-based person Re-ID datasets, and the results indicate that our framework achieves state-of-the-art performance on three benchmarks.

Refining pseudo labels for unsupervised Domain Adaptive Re-Identification

Our paper focuses on the topic of unsupervised domain adaptation for person re-identification (UDA re-ID) in an end-to-end framework. Currently, most existing methods tackle the problem by mining pseudo labels, and they rely heavily on the reliability of these pseudo labels. However, the generation of these labels mainly depends on clustering-based algorithms, which may inevitably introduce noise into the framework. Such noise substantially hinders the capability of the model to further improve feature representations in the target domain. We called these clustering-based labels HARD pseudo labels. To decrease the impacts of these noisy hard labels, in this paper, we directly consider the relative relationships of sample pair distances as another kind of pseudo label instead of clustering results, called SOFT pseudo labels. Furthermore, considering that the constructed relative relationships are not always correct and the wrong relationships may cause negative effects, it is not appropriate to assign the same weight to each sample pair. To alleviate the negative influence caused by tiny distances and strengthen the positive effect due to large distances, we assign different weights according to the different distances to adjust these relationships. By integrating the HARD labels and SOFT labels, the proposed method achieves considerable improvements on several popular person re-ID datasets, e.g, Market1501-to-Duke, Duke-to-Market1501, Market-to-MSMT17 and Duke-to-MSMT17 UDA tasks.

A Survey on Generative Adversarial Network Based Person Re-Identification Method

In recent years, with the growing demand for public safety and the rapid development of intelligent surveillance networks, person re-identification (Re-ID) has become one of the hottest research topics in the field of computer vision, which aims at retrieving a person of interest through multiple cameras with non-overlapping views. First, the current classical person Re-ID datasets are introduced. Second, the latest GAN-based person Re-ID methods are focused on sorting out. These methods are classified into three categories: image-image style transfer, data enhancement and invariant feature learning according to the characteristics and application scenarios of GAN. And basic principles, advantages, and disadvantages are summarized for each category. Then, the states of the art of the classic algorithms mentioned are compared on the mainstream datasets. Finally, the present challenging problems of person Re-ID and directions for future research are discussed.

Person Re-Identification Combined with Style Transfer and Pose Generation

The number of existing person re-identification datasets is limited, and there are a series of changes such as illumination, background occlusion and pose among each dataset, which makes it difficult for the existing methods to learn robust feature representation, leading to a decline in recognition performance. To solve these problems, a person re-identification method combining style and pose generation is proposed in this paper. First with the impact of camera style differences in collecting images from different cameras, a style transformation method based on generative adversarial network is introduced into a person re-identification model, and cyclic generative adversarial networks (CycleGAN) is used to realize style transfer and reduce the influence of camera differences. Second, in view of the problem that when pedestrian pose changes greatly, easy to ignore identity-sensitive related information, AlphaPose is introduced to implement pose estimation. Combining style and posture for the first time and using improved deep convolution generative adversarial network (DCGAN) structure enrich the input sample information and generate unified style pose image; while using new synthetic data to train person re-identification network model improves the recognition performance of the model. Finally, further introducing random erasure method during data enhancement, in order to reduce the overfitting phenomena, improves the generalization ability of the network simultaneously and solves partial occlusion. The experimental results show that the proposed method outperforms typical style-based or pose-based methods. The accuracy of rank-1 and mAP on Market-1501 dataset is 90.4% and 74.5%, respectively, which are 2.28% and 5.78% higher, respectively. To a certain extent, the performance of person re-identification is improved.

Learning Feature Recovery Transformer for Occluded Person Re-Identification.

One major issue that challenges person re-identification (Re-ID) is the ubiquitous occlusion over the captured persons. There are two main challenges for the occluded person Re-ID problem, i.e. , the interference of noise during feature matching and the loss of pedestrian information brought by the occlusions. In this paper, we propose a new approach called Feature Recovery Transformer (FRT) to address the two challenges simultaneously, which mainly consists of visibility graph matching and feature recovery transformer. To reduce the interference of the noise during feature matching, we mainly focus on visible regions that appear in both images and develop a visibility graph to calculate the similarity. In terms of the second challenge, based on the developed graph similarity, for each query image, we propose a recovery transformer that exploits the feature sets of its k -nearest neighbors in the gallery to recover the complete features. Extensive experiments across different person Re-ID datasets, including occluded, partial and holistic datasets, demonstrate the effectiveness of FRT. Specifically, FRT significantly outperforms state-of-the-art results by at least 6.2% Rank- 1 accuracy and 7.2% mAP scores on the challenging Occluded-Duke dataset.

Offline-Online Associated Camera-Aware Proxies for Unsupervised Person Re-Identification.

Recently, unsupervised person re-identification (Re-ID) has received increasing research attention due to its potential for label-free applications. A promising way to address unsupervised Re-ID is clustering-based, which generates pseudo labels by clustering and uses the pseudo labels to train a Re-ID model iteratively. However, most clustering-based methods take each cluster as a pseudo identity class, neglecting the intra-cluster variance mainly caused by the change of cameras. To address this issue, we propose to split each single cluster into multiple proxies according to camera views. The camera-aware proxies explicitly capture local structures within clusters, by which the intra-ID variance and inter-ID similarity can be better tackled. Assisted with the camera-aware proxies, we design two proxy-level contrastive learning losses that are, respectively, based on offline and online association results. The offline association directly associates proxies according to the clustering and splitting results, while the online strategy dynamically associates proxies in terms of up-to-date features to reduce the noise caused by the delayed update of pseudo labels. The combination of two losses enables us to train a desirable Re-ID model. Extensive experiments on three person Re-ID datasets and one vehicle Re-ID dataset show that our proposed approach demonstrates competitive performance with state-of-the-art methods. Code will be available at: https://github.com/Terminator8758/O2CAP.

Multi-Modal Context Propagation for Person Re-Identification With Wireless Positioning

Existing person re-identification methods mainly rely on thevisualappearance captured by camerasfor identity matching.However, due to the sensitivity of visual data to occlusion, blur, clothing change, etc.,existing methodsstruggleto distinguish pedestriansin challenging scenarios. Inspired by the fact that most pedestrians carry around smart wireless devices, e.g. mobile phones that can be sensed by WiFi or cellular networks as wireless positioning signals, we propose to exploit the free yet informative wireless signalsto assist person re-identification. It is well recognized thatwireless signals are robust to visual noisesmentioned above, which perform as a good complement to the visual data. To make full use of these multi-modal clues for person re-identification, we propose a multi-modal context propagation framework MCPF that contains a recurrent context propagation module RCPM and an unsupervised multi-modal cross-domain method UMM-ReID. RCPM enables context information to be continuously propagatedand fusedbetween visual data and wireless data. UMM-ReID utilizes wireless signals to constrain the estimation of pseudo labels. To evaluate our approach, we contribute a new wireless positioning person re-identification (WP-ReID) dataset. Extensive experimentsdemonstrate the effectiveness of the proposed method. Benefitingfrom the collaboration of RCPM and UMM-ReID, the proposed framework MCPF achieves a significant performance improvement over existing methods.

Robust Label Rectifying With Consistent Contrastive-Learning for Domain Adaptive Person Re-Identification

Domain adaptive person re-identification (Re-ID) is challenging due to the domain gap between the source and target domains. Existing methods have recently shown great promise by training models with contrastive learning and assigning pseudo labels by clustering, in which a memory bank is utilized to keep features for contrast. However, two main problems lead to sub-optimal generalization ability for existing methods. First, there is no constraint on the updating for memory kept features in existing methods, resulting in inaccurate contrastive learning. Second, the inevitable noisy labels during clustering are usually ignored. To alleviate these problems, we propose a Label Rectifying with Consistent Contrastive-learning (LRCC) framework with two strategies. (1) The consistent contrastive-learning (CC) strategy works with a memory bank which stores the source domain class centroids and all the target domain image features. With the CC strategy, the contrast is conducted across the source and target domains simultaneously. More specifically, we design and maintain consistent clustering during model iteration, thus the classes of memory kept target features are invariable in one epoch. (2) The label rectifying (LR) strategy introduces an auxiliary classifier into the LRCC framework. Thus the pseudo labels are rectified by minimizing the prediction variance between the primary classifier and the auxiliary classifier. To verify the effectiveness of LRCC, we conduct experiments on three public person Re-ID datasets under the domain adaptive setting, DukeMTMC-reID, Market-1501, and MSMT17. The experimental results demonstrate that the proposed LRCC can obtain reliable pseudo labels and achieves state-of-the-art adaptation performance.

Beyond Triplet Loss: Meta Prototypical N-Tuple Loss for Person Re-identification

Person Re-identification (ReID) aims at matching a person of interest across images. In convolutional neural network (CNN) based approaches, loss design plays a vital role in pulling closer features of the same identity and pushing far apart features of different identities. In recent years, triplet loss achieves superior performance and is predominant in ReID. However, triplet loss considers only three instances of two classes in per-query optimization (with an anchor sample as query) and it is actually equivalent to a two-class classification. There is a lack of loss design which enables the joint optimization of multiple instances (of multiple classes) within per-query optimization for person ReID. In this paper, we introduce a multi-class classification loss, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i</i> . <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">e</i> ., N-tuple loss, to jointly consider multiple ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$N$</tex-math></inline-formula> ) instances for per-query optimization. This in fact aligns better with the ReID test/inference process, which conducts the ranking/comparisons among multiple instances. Furthermore, for more efficient multi-class classification, we propose a new meta prototypical N-tuple loss. With the multi-class classification incorporated, our model achieves the state-of-the-art performance on the benchmark person ReID datasets

Learning Global Average Attention Pooling (GAAP) on Resnet50 Backbone for Person Re-identification Problem

Person re-identification has been an extremely chal-lenging task in computer vision which has been seen as a success with deep learning approaches. Despite successful models, there are gaps in the form of unbalanced labels, poor resolution, uncertain bounding box annotations, occlusions, and unlabelled datasets. Previous methods applied deep learning approaches based on feature representation, metric learning, and ranking optimization. In this work, we propose Global Average Attention Pooling (GAAP) on Resnet50 applied on four benchmark Re-ID datasets for classification tasks. We also perform an extensive evaluation on the proposed Attention module with different deep learning pipelines as backbone architecture. The four benchmark person Re-ID datasets used is Market-1501, RAiD, Partial-iLIDS, and RPIfield. We computed cumulative matching characteristics (CMC) and mean Average Precision (mAP) as the performance evaluation parameters of the proposed against the state of the art. The results obtained have shown that the added attention layer has improved the overall recognition precision over the baselines.

Revisiting Dropout Regularization for Cross-Modality Person Re-Identification

This paper investigated targeted dropout regularization for improving convolutional neural network classifier performance on cross-modality person re-identification problems. The dropout regularization is carefully applied on specific layers (targeted dropout regularization) or regions of the features map (spatially targeted dropout regularization). The intuition behind the spatially targeted dropout regularization is that the importantness of feature map regions is not equally the same, and the specific object is more likely placed at the center. We experimented heavily with PKU-Sketch-ReID and multi-modality person re-identification datasets with SwinTransformer deep neural network architecture. Three different targeted dropout regularizations are used for the experiments, including block-wise dropout, horizontal block-wise dropout, and vertical-horizontal block-wise dropout. Experiments on three different sketch re-identification datasets show that the proposed spatially targeted dropout regularization can improve the performance of the deep neural network classifiers with the best rank-1 of 73.20% on the PKU-Sketch-ReID, 52.73% on the SYSU-MM01 dataset, and 72.74% on the RegDB dataset.

Adaptive multi-task learning for cross domain and modal person re-identification

Person re-identification (re-ID) aims at matching a person-of-interest across various non-overlap cameras with distinguished visual appearance variances. Pre-existing research methods mainly employ deep neural models to train large-scale person re-ID datasets, achieving good performance. However, these methods are primarily deployed only on visual data, which can be easily influenced by the environment variances (e.g., viewpoints, poses, and illuminations). In this paper, we propose an adaptive multi-task learning (MTL) scheme for cross domain and modal person re-ID. It can effectively utilize the visual and language information from multiple datasets for improving learning performance. Comprehensive experiments are also conducted on the widely-used person re-ID datasets, i.e., Market-1501 and DukeMTMC-reID, validating the effectiveness of the proposed method. It can model the domain difference and the relationship between the vision and language modalities and achieve state-of-the-art performance. The source code of our proposed method will be available athttps://github.com/emdata-ailab/Multitask_Learning_ReID.

Impact of Kernel-PCA on Different Features for Person Re-Identification

In the driving field of computer vision, re-identification of an individual in a camera network is very challenging task. Existing methods mainly focus on strategies based on feature learning, which provide feature space and force the same person to be closer than separate individuals. These methods rely to a large extent on high-dimensional feature vectors to achieve high re-identification accuracy. Due to computational cost and efficiency, they are difficult to achieve in practical applications. We comprehensively analyzed the effect of kernel-based principal component analysis (PCA) on some existing high-dimensional person re-identification feature extractors to solve these problems. We initially formulate a kernel function on the extracted features and then apply PCA, significantly reducing the feature dimension. After that, we have proved that the kernel is very effective on different state-of-the-art high-dimensional feature descriptors. Finally, a thorough experimental evaluation of the reference person re-identification data set determined that the prediction method was significantly superior to more advanced techniques and computationally feasible.

Deep progressive attention for person re-identification

Person re-identification (Re-ID) aims to retrieve specific individuals across non-overlapping camera views. In recent years, attention-based models contribute to many computer vision tasks due to their great ability for learning discriminative features. We propose the deep progressive attention (DPA) in a more natural manner for person Re-ID. Similar to human visual attention mechanism, the proposed DPA progressively selects the most discriminative parts of a specific individual and formulates feature representation for comparison purpose. Concretely, on the one hand, the proposed DPA uses a long-term reward to optimize the discriminative feature selection. On the other hand, a deep convolutional architecture is integrated into a recurrent model for feature representation learning. Extensive experiments on three person Re-ID benchmarks Market-1501, DukeMTMC-reID, and CUHK03-NP demonstrate the proposed DPA is on par with the state-of-the-art. Moreover, the experiments on partial person Re-ID datasets indicate the proposed DPA is competitive with the specially designed partial person Re-ID methods.