Re-identification Research Articles

Complete solution for vehicle Re-ID in surround-view camera system

Vehicle re-identification (Re-ID) is a critical component of the autonomous driving perception system, and research in this area has accelerated in recent years. However, there is yet no perfect solution to the vehicle re-identification issue associated with the car’s surround-view camera system. Our analysis identifies two significant issues in the aforementioned scenario: (1) It is difficult to identify the same vehicle in many picture frames due to the unique construction of the fisheye camera. (2) The appearance of the same vehicle when seen via the surround vision system’s several cameras is rather different. To overcome these issues, we suggest an integrative vehicle Re-ID solution method. On the one hand, we provide a technique for determining the consistency of the tracking box drift with respect to the target. On the other hand, we combine a Re-ID network based on the attention mechanism with spatial limitations to increase performance in situations involving multiple cameras. Finally, our approach combines state-of-the-art accuracy with real-time performance. We will soon make the source code and annotated fisheye dataset available.

An Adaptive Partitioning and Multi-Granularity Network for Video-Based Person Re-Identification

Person re-identification (Re-ID) is a key technology used in the field of intelligent surveillance. The existing Re-ID methods are mainly realized by using convolutional neural networks (CNNs), but the feature information is easily lost in the operation process due to the down-sampling structure design in CNNs. Moreover, CNNs can only process one local neighbourhood at a time, which makes the global perception of the network poor. To overcome these shortcomings, in this study, we apply a pure transformer to a video-based Re-ID task by proposing an adaptive partitioning and multi-granularity (APMG) network framework. To enable the pure transformer structure better at adapting to the Re-ID task, we propose a new correlation-adaptive partitioning (CAP) of feature embedding modules that can adaptively partition person images according to structural correlations and thus retain the structure and semantics of local feature information in the images. To improve the Re-ID performance of the network, we also propose a multi-granularity (MG) module to better capture people feature information at different levels of granularity. We performed validation trials on three video-based benchmark datasets. The results show that the network structure based on the pure transformer can adapt to Re-ID tasks well, and our APMG network outperforms other state-of-the-art methods.

Multi-granularity Pseudo-label Collaboration for unsupervised person re-identification

Unsupervised person re-identification (ReID) aims at training a model through using fully unlabeled training images. Most successful approaches combine clustering-based pseudo-label prediction with ReID model learning and perform the two steps in an alternating fashion. However, there are some person images, which have common appearance but different identities or have same identity but large intra-variations, and they are prone to be assigned wrong pseudo-labels. Existing methods typically infer person image pseudo-labels in a single pseudo-label space, which is insufficient due to losing the accurate labels of these hard images. This limits model’s power to cope with varying changes. To address this problem, we propose a Multi-granularity Pseudo-label Collaboration (MPC) method for unsupervised person ReID. Firstly, a multi-granularity pseudo-label prediction (MPP) method is proposed to predict the pseudo-labels of person images. MPP generates multiple pseudo-label spaces with each one inferring the person image pseudo-labels under a different granularity at cluster size. Secondly, the multiple pseudo-label spaces are applied to learn a mixture of experts (MoE) model. Each expert model is supervised by the pseudo-labels in a specific pseudo-label space, enabling MoE model to handle different variations. The output of these expert models are fused critically by a bilateral attention aggregation (BAA) to generate aggregation image features. Meanwhile, a label mix-up method is designed for refining pseudo-labels. It fuses labels across different pseudo-label spaces based on label co-occurrence to promote aggregation feature learning. We extensively evaluate the proposed MPC method under both image-based and video-based unsupervised person ReID settings. MPC significantly improves the current baseline method by a margin of 5.4% and 3.7% in mAP, and 3.4% and 3.9% in Rank1 accuracy on the image-based datasets Market-1501 and DukeMTMC-ReID, respectively. In particular, MPC method achieves state-of-the-art performance on video-based datasets with mAP of 71.4% and 87.3% on MARS and DukeMTMC-VideoReID, respectively.

Global Relation-Aware Contrast Learning for Unsupervised Person Re-Identification

The goal of unsupervised person re-identification (Re-ID) is to use unlabeled person images to learn discriminative features. In recent years, many approaches have adopted clustered pseudo labels to construct proxies for contrastive learning, and have thereby achieved great success. However, existing methods of this kind only utilize local structures within IDs to design their proxies while ignoring the relations between samples of different IDs, which limits the improvement for inter-ID discriminative ability. To resolve this issue, we propose a Global Relation-Aware Contrast Learning (GRACL) method for the task of unsupervised Re-ID. Our method first sets up two proxies for each cluster to capture the inter- and intra-ID relations respectively, which enables us to both effectively increase inter-ID variances and reduce the intra-ID discrepancies. Specifically, the samples that are most different from those in different clusters are selected as inter-ID relation-aware proxies, while those that are least similar to samples from the same clusters are employed as intra-ID relation-aware proxies. With the aid of these proxies, we design both inter- and intra-ID relation-aware contrastive learning modules to facilitate model learning. By pulling each sample close to the positive proxy, we can obtain identity-invariant discriminative features. Experiments on five widely-used Re-ID datasets prove that our GRACL model outperforms current state-of-the-art approaches to a remarkable extent.

Weakly-Supervised Part-Attention and Mentored Networks for Vehicle Re-Identification

Vehicle re-identification (Re-ID) aims to retrieve images with the same vehicle ID across different cameras. Current part-level feature learning methods typically detect vehicle parts via uniform division, outside tools, or attention modeling. However, such part features often require expensive additional annotations and cause sub-optimal performance in case of unreliable part mask predictions. In this paper, we propose a weakly-supervised Part-Attention Network (PANet) and Part-Mentored Network (PMNet) for Vehicle Re-ID. Firstly, PANet localizes vehicle parts via part-relevant channel recalibration and cluster-based mask generation without vehicle part supervisory information. Secondly, PMNet leverages teacher-student guided learning to distill vehicle part-specific features from PANet and performs multi-scale global-part feature extraction. During inference, PMNet can adaptively extract discriminative part features without part localization by PANet, preventing unstable part mask predictions. We address this Re-ID issue as a multi-task problem and adopt Homoscedastic Uncertainty to learn the optimal weighing of ID losses. Experiments are conducted on two public benchmarks, showing that our approach outperforms recent methods, which require no extra annotations by an average increase of 3.0% in CMC@5 on VehicleID and over 1.4% in mAP on VeRi776. Moreover, our method can extend to the occluded vehicle Re-ID task and exhibits good generalization ability.

Bi‐level deep mutual learning assisted multi‐task network for occluded person re‐identification

An occluded person re-identification (ReID) approach is presented by constructing a Bi-level deep Mutual learning assisted Multi-task network (BMM), where the holistic and occluded person ReID tasks are treated as two related but not identical tasks. This is inspired by the human perception characteristic that there exist both similarities and differences when human views a holistic image and the occluded one. Specifically, a multi-task network with two branches is designed, where the convolutional neural network based feature representation part shares the weights by two tasks for commonality extraction, while the following output layers have respective weights for difference representation. Furthermore, as the non-occluded regions convey discriminative information, a bi-level mutual learning strategy is proposed and applied mutually on two branches to obtain more effective information from the non-occluded regions in the occluded images for better identity recognition. This is achieved by both feature-level and output-level mutual loss functions. Extensive experiments prove the advantages of the BMM for person ReID.

Person Tracking by Detection Using Dual Visible-Infrared Cameras

We study the problem of cross-modality person reidentification (ReID) and tracking with dual visible-infrared (VI) cameras, while most existing efforts on tracking-by-detection have been paid on single-modality visible ReID which is inapplicable for poor-light environments. The major difficulties for cross-modality (e.g., VI) ReID stem from the large modality gap between three-channel visible images and one-channel infrared images and such unknown environmental factors as background clutter, occlusions, etc. To tackle these issues, we propose to enrich the diversities of visible and infrared images for intra- and cross-modality matching by using both the channel-aware data augmentation (DA) techniques (e.g., channel exchanged augmentation and random occlusions) and standard DA techniques. On top of these DA techniques, we incorporate ResNet50 and vision transformer (ViT) into the feature extraction backbone network and apply the dynamic weight average (DWA) strategy for learning loss weights by regarding the minimization of identity loss and triplet loss as a multitask learning problem. We then apply the proposed ReID approach for person tracking in the field of interests. The experiments on two public data sets, i.e., RegDB and SYSU-MM01, show that our approach can improve the performance of state-of-the-art rank-1, mAP, and mINP for cross-modality matching. In addition, the experiments on our data set show that tracking by VI-ReID using dual VI cameras can achieve an accuracy of around 0.24 m.

V2ReID: Vision-Outlooker-Based Vehicle Re-Identification.

With the increase of large camera networks around us, it is becoming more difficult to manually identify vehicles. Computer vision enables us to automate this task. More specifically, vehicle re-identification (ReID) aims to identify cars in a camera network with non-overlapping views. Images captured of vehicles can undergo intense variations of appearance due to illumination, pose, or viewpoint. Furthermore, due to small inter-class similarities and large intra-class differences, feature learning is often enhanced with non-visual cues, such as the topology of camera networks and temporal information. These are, however, not always available or can be resource intensive for the model. Following the success of Transformer baselines in ReID, we propose for the first time an outlook-attention-based vehicle ReID framework using the Vision Outlooker as its backbone, which is able to encode finer-level features. We show that, without embedding any additional side information and using only the visual cues, we can achieve an 80.31% mAP and 97.13% R-1 on the VeRi-776 dataset. Besides documenting our research, this paper also aims to provide a comprehensive walkthrough of vehicle ReID. We aim to provide a starting point for individuals and organisations, as it is difficult to navigate through the myriad of complex research in this field.

Camera-aware representation learning for person re-identification

Person re-identification (ReID) aims to associate the same person across non-overlapping cameras. However, most of existing works neglect the issue of camera-imbalanced data distribution. Consequently, pedestrian representation learning gives preference to the head cameras, which have comparatively more training data, and disregards the tail cameras, which have relatively less training data. In this paper, we propose a novel framework for camera-aware representation learning to overcome this issue, named CARL. On the proxy level representation learning, CARL presents a multiple-center softmax loss to correct the head camera bias and presents a hard sub-center mining strategy to improve the discrimination of tail camera samples. On the pair-wise level representation learning, CARL builds a camera-balanced memory bank (CBM) to re-balance the sample pair distribution and proposes a camera-paired loss for pair-wise metric learning. Extensive experiments and ablation studies on MSMT17, the current largest ReID dataset with massive camera-imbalanced data distribution, demonstrate that our CARL is superior to previous metric learning based ReID methods and achieves state-of-the-art performance.

Similarity learning with deep CRF for person re-identification

The core of person re-identification (Re-ID) lies in robustly estimating similarities for each probe-gallery image pair. A common practice in existing works is to calculate the similarity of each image pair independently, ignoring relations between different probe-gallery pairs. In this paper, we present a deep learning conditional random field (Deep-CRF) graph to model group-wise similarities within a batch of images, and regard the Re-ID task as a CRF node labeling problem. Unlike the existing deep CRF based approach where the CRF inference is only involved in the training stage, our method intends to fully exploit the potential of CRF model, exhibiting inference consistency in both training and testing. Specifically, we design unary potentials for computing each probe-gallery similarity separately. To efficiently encode relationships between different probe-gallery pairs, pairwise potentials are built on an arbitrary node pair whose learning is achieved by a joint matching strategy using bidirectional LSTM. We pose the CRF inference as a RNN learning process, where unary and pairwise potentials are jointly optimized in an end-to-end manner. Extensive experiments on three large-scale person Re-ID datasets demonstrate the effectiveness of the proposed method. Our Deep-CRF achieves the best results compared with the previous graph-based deep learning approaches and substantially exceeds the existing deep CRF framework by 8% in Rank1 accuracy on CUHK03 dataset. It also behaves competitive among the current state-of-the-art methods.

Video Person Re-Identification Using Attribute-Enhanced Features

In this work we propose to boost video-based person re-identification (Re-ID) by using attribute-enhanced feature presentation. To this end, we not only try to use the ID-relevant attributes more effectively, but also for the first time in literature harness the ID-irrelevant attributes to help model training. The former mainly include gender, age, clothing characteristics, etc., which contain rich and supplementary information about the pedestrian; the latter include viewpoint, action, etc., which are seldom used for identification previously. In particular, we use the attributes to enhance the significant areas of the image with a novel Attribute Salient Region Enhance (ASRE) module that can attend more accurately to the body of the pedestrian, so as to better separate the target from the background. Furthermore, we find that many ID-irrelevant but subject-relevant factors, like the view angle and movement of the target pedestrian, have great impact on the two-dimensional appearance of a pedestrian. We then propose to exploit both the ID-relevant and the ID-irrelevant attributes via a novel triplet loss called the Viewpoint and Action-Invariant (VAI) triplet loss. Based on the above, we design an Attribute Salience Assisted Network (ASA-Net) to perform attribute recognition along with identity recognition, and use the attributes for feature enhancement and hard sample mining. Extensive experiments on MARS and DukeMTMC-VideoReID datasets show that our method outperforms the state-of-the-arts. Also, the visualizations of learning results further prove the effectiveness of the proposed method.

Multitask Multigranularity Aggregation With Global-Guided Attention for Video Person Re-Identification

The goal of video-based person re-identification (Re-ID) is to identify the same person across multiple non-overlapping cameras. The key to accomplishing this challenging task is to sufficiently exploit both spatial and temporal cues in video sequences. However, most current methods are incapable of accurately locating semantic regions or efficiently filtering discriminative spatio-temporal features; so it is difficult to handle issues such as spatial misalignment and occlusion. Thus, we propose a novel feature aggregation framework, multi-task and multi-granularity aggregation with global-guided attention (MMA-GGA), which aims to adaptively generate more representative spatio-temporal aggregation features. Specifically, we develop a multi-task multi-granularity aggregation (MMA) module to extract features at different locations and scales to identify key semantic-aware regions that are robust to spatial misalignment. Then, to determine the importance of the multi-granular semantic information, we propose a global-guided attention (GGA) mechanism to learn weights based on the global features of the video sequence, allowing our framework to identify stable local features while ignoring occlusions. Therefore, the MMA-GGA framework can efficiently and effectively capture more robust and representative features. Extensive experiments on four benchmark datasets demonstrate that our MMA-GGA framework outperforms current state-of-the-art methods. In particular, our method achieves a rank-1 accuracy of 91.0% on the MARS dataset, the most widely used database, significantly outperforming existing methods.

Dually Distribution Pulling Network for Cross-Resolution Person Reidentification.

Person reidentification (Re-ID) aims at recognizing the same identity across different camera views. However, the cross resolution of images [high resolution (HR) and low resolution (LR)] is unavoidable in a realistic scenario due to the various distances among cameras and pedestrians of interest, thus leading to cross-resolution person Re-ID problems. Recently, most cross-resolution person Re-ID methods focus on solving the resolution mismatch problem, while the distribution mismatch between HR and LR images is another factor that significantly impacts the person Re-ID performance. In this article, we propose a dually distribution pulling network (DDPN) to tackle the distribution mismatch problem. DDPN is composed of two modules, that is: 1) super-resolution module and 2) person Re-ID module. They attempt to pull the distribution of LR images closer to the distribution of HR images from image and feature aspects, respectively, through optimizing the maximum mean discrepancy losses. Extensive experiments have been conducted on three benchmark datasets and the results demonstrate the effectiveness of DDPN. Remarkably, DDPN shows a great advantage when compared to the state-of-the-art methods, for instance, we achieve rank-1 accuracy of 76.9% on VR-Market1501, which outperforms the best existing cross-resolution person Re-ID method by 10%.

MSTN: A Multi-granular Spatial–Temporal Network for video-based person re-identification

With the popularization of surveillance cameras, public-safety related applications requiring the functionality of video-based person re-identification (Re-ID) thrive. Re-ID aims at accurately identifying a person-of-interest across video sequences from multiple cameras. Existing methods usually focus on either spatially salient regions, or temporal features among frames of fixed intervals (i.e., either short- or long-term temporal features), resulting in the under-utilization of neglected features and hence moderate identification accuracy. To achieve high Re-ID accuracy, we propose a novel framework termed Multi-granular Spatial–Temporal Network (MSTN), that facilitates full utilization of spatial–temporal features for video-based person Re-ID. Within MSTN, a Temporal Kernel Attention (TKA) module is proposed to adaptively capture both short- and long-term temporal relationships; a Feature Disentanglement Spatial Attention (FDSA) module is further proposed to mine spatially salient and subtle features. Extensive experiments on the MARS dataset demonstrate that MSTN can achieve high identification accuracy, exhibiting 86.1% in terms of mAP and 91.0% in terms of Rank-1, notably higher than state-of-the-art comparison schemes.

ICMiF: Interactive cascade microformers for cross-domain person re-identification

Person re-identification (ReID) arises in many applications, such as video surveillance and intelligent security. A challenge of cross-domain person ReID is the notorious distribution drift problem. Improving the accuracy of pseudo labels can promote the model to fit the target domain, so it is crucial to extract discriminative person features. The transformer can extract discriminative person features due to its ability to capture long distance dependencies, but its poor generalization ability may aggravate the influence of noise pseudo labels. The isolation between local information modeling and global information modeling further prevents the extraction of the discriminative person features. Therefore, this paper proposes an interactive cascade transformer framework. Firstly, a lightweight transformer structure (microformer) with better generalization is proposed. It maintains the advantages of traditional transformers on extracting discriminative features, and avoids the aggravation of the influence of noise labels. Secondly, an interactive cascade microformer framework (ICMiF) is proposed, which promotes the interaction between local information modeling and global information modeling to enhance the person feature representation. Local information facilitates mining global attribute dependencies, and global information improves the accuracy of local information. The experimental results demonstrate that the proposed ICMiF outperforms state-of-the-art methods for the cross-domain person ReID tasks.

Human instance guided two-stream network for person reidentification

Since the person reidentification (Re-ID) technique was proposed, it has successively produced successful research. The performance of this technique still has room for improvement due to the challenges of cluttered background and spatial misalignment. Recently, researchers have attempted to integrate human parsing or pose estimation results to capture person regions for mitigating these problems. However, many important and recognizable cues remain in the background regions. To mine these useful cues while still noticing the informative human body parts, this work proposes a simple yet effective two-stream model that utilizes one stream from human instance to learn discriminative human part features and enriches the representation with the other stream from the original image. Moreover, to alleviate the spatial misalignment problem, this paper rectifies the feature extraction regions with the assistance of human segmentation results. Experimental results demonstrate that the proposed method significantly improves the performance of person Re-ID.

Learning Identity-Consistent Feature for Cross-Modality Person Re-Identification via Pixel and Feature Alignment

RGB-IR cross-modality person re-identification (ReID) can be seen as a multicamera retrieval problem that aims to match pedestrian images captured by visible and infrared cameras. Most of the existing methods focus on reducing modality differences through feature representation learning. However, they ignore the huge difference in pixel space between the two modalities. Unlike these methods, we utilize the pixel and feature alignment network (PFANet) to reduce modal differences in pixel space while aligning features in feature space in this paper. Our model contains three components, including a feature extractor, a generator, and a joint discriminator. Like previous methods, the generator and the joint discriminator are used to generate high-quality cross-modality images; however, we make substantial improvements to the feature extraction module. Firstly, we fuse batch normalization and global attention (BNG) which can pay attention to channel information while conducting information interaction between channels and spaces. Secondly, to alleviate the modal difference in feature space, we propose the modal mitigation module (MMM). Then, by jointly training the entire model, our model is able to not only mitigate the cross-modality and intramodality variations but also learn identity-consistent features. Finally, extensive experimental results show that our model outperforms other methods. On the SYSU-MM01 dataset, our model achieves a rank-1 accuracy of 40.83 % and an mAP of 39.84 % .

Mini-transformer with pooling for unsupervised domain adaptation person reidentification

Person reidentification (Re-ID) aims to match specific pedestrians across nonoverlapping camera views. Due to the dramatic disparities between different datasets, transferring a Re-ID model trained on the source domain to the target domain is challenging. Some outstanding unsupervised domain adaptation (UDA) Re-ID methods use clustering to generate pseudolabels, optimizing the model on the target domain, but the pseudolabels are inevitably noisy. To address the above issues, we propose a framework named mini-transformer with pooling (MTP) to facilitate the generation of superior quality pseudolabels by improving the model’s feature representation capability. First, we introduce an effective mini-transformer (MT) that can be placed directly behind the CNNs as a feature extractor to capture long-range dependency. Then, we design two delicate pooling methods named global hybrid pooling (GHP) and global subvalue pooling (GSVP) to suit mini-transformer’s tremendous capability without increasing computational complexity. Specifically, GHP can keep more global information and GSVP can keep more discriminative information. Finally, experiments on four mainstream UDA Re-ID tasks demonstrate that MTP achieves competitive mAP and rank-1 accuracy to the current state-of-the-art methods, suggesting that our technique is simple but effective. In addition to UDA Re-ID, our MTP can be extended to other supervised retrieval tasks.

Learning diverse and deep clues for person reidentification

Extracting robust features has been the core of person re-identification(ReID). The existing convolutional neural network-based methods pay more attention to local features rather than the connection between local features. Given that human bodies possess certain structural information, it is absolutely necessary to strengthen the connection of local features for the ReID task. This paper proposes a two-stage attention network termed Width and Depth Channel Attention Network (WDC-Net) for ReID. Unlike conventional attention-based methods, which only focus on the single local features, our network exploits diverse feature representations to alleviate the missing information problem caused by occlusion. Precisely, for the first stage, it splits the local associations of the feature map through a multi-scale perspective to extract relatively independent multi-level local features of the human body. For the second stage, the correlation of multi-level local features is reconstructed through grouped pyramid structure to obtain a more robust global feature representation. We also propose an adaptive margin weight adjustment strategy to enhance the adaptability of the attention weights. Large-scale ReID datasets are tested to evaluate our method. On Market1501 and DukeMTMC, the proposed method achieves 90.7 % /96.4% mAP/R-1 and 81.8 % /90.8% mAP/R-1, respectively. It is worth highlighting that the proposed method also achieves 55.3 % /65.3% mAP/R-1 on the challenging Occluded-Duke dataset. Extensive experimental results demonstrate the superiority of our method, which achieves state-of-the-art performance on ReID. • An efficient framework is used to handle both historic ReID and occluded ReID. • Exploring diverse local features through the width channel attention. • Constructing global context-aware features with pyramid attention. • Enhancing the adaptability of the attention weights.

Posture Calibration Based Cross-View & Hard-Sensitive Metric Learning for UAV-Based Vehicle Re-Identification

Machine vision based vehicle re-identification (ReID) plays an important role in some Intelligent Transportation Systems (ITS). Yet, most previous methods mainly focus on fixed surveillance cameras instead of Unmanned Aerial Vehicle (UAV). With high flexibility, the UAV-based vehicle ReID problem has some special challenges including complicated shooting angles, low discrimination of top-down features, and large variance in vehicle scales, etc. To overcome these challenges, we propose a novel structure for UAV-based vehicle ReID without license plates. Firstly, a triple-head segmentation net is proposed for segmenting UAV-captured vehicles under different heights and directions. Secondly, a posture calibration model is designed to uniform the vehicle postures based on the segmentation results, with the purpose of reducing the influence of different postures. Thirdly, the novel Cross-View & Hard-Sensitive Metric Learning (CHSML) method is proposed to train a ReID network with cross-view training constraint and hard sensitive principles; the mechanism of CHSML takes the cross-view samples of same ID as a training unit to learn the potential visual relationship in cross-view and builds a hard sensitive weight matrix to make learning more focus on hard samples, which improves the low ReID accuracy brought by cross-view or hard samples. Moreover, to facilitate the research of UAV-based vehicle ReID, a large-scale UAV-based vehicle ReID dataset called VeRi-UAV is released with 17 516 vehicles of 453 IDs. The experiments show that the proposed structure gains a better performance compared with the representative methods in the UAV-based vehicle ReID task.