Re-identification Performance Research Articles

Self-selective receptive field network for person re-identification

AbstractPerson Re-identification (Re-ID) technology aims to solve the matching problem of the same pedestrians at different times and places, which has important application value in the field of public safety. At present, most scholars focus on designing complex models to improve the accuracy of Re-ID, but the high complexity of the model further restricts the practical application of Re-ID algorithm. To solve the above problems, this paper designs a lightweight Self-selective Receptive Field (SRF) block instead of directly designing complex models. Specifically, the module can be plug-and-play on the general backbone network, so as to significantly improve the performance of Re-ID while effectively controlling the amount of its own parameter and calculation: (1) the SRF block encodes pedestrian targets and image contexts at different scales by constructing pyramidal convolution group and allows the module to independently select the size of the receptive field through training by means of self-adaptive weighting; (2) in order to reduce the complexity of SRF block, we introduce a "channel scaling factor" and design a "grouped convolution operation" by constraining the channels of the feature map and changing the structure of the convolution kernel respectively. Experiments on multiple datasets show that SRF Network (SRFNet) for Re-ID can achieve a good balance between performance and complexity, which fully demonstrates the effectiveness of SRF block.

IGIE-net: Cross-modality person re-identification via intermediate modality image generation and discriminative information enhancement

Given an RGB image (or an IR image) of a pedestrian, the task of cross-modality person re-identification aims to retrieve images of a specific pedestrian from an IR (or RGB) gallery. However, the huge modality discrepancy between RGB and IR images significantly affects the performance of cross-modality person re-identification. For the purpose of reducing the impact of modality discrepancy and extracting more discriminative pedestrian features, a new cross-modality person re-identification network is proposed by us, which is based on intermediate modality image generation and discriminative information enhancement (IGIE-Net). Specifically, we design an intermediate modality image generation module (IMIGM) to effectively weaken the impact of modality discrepancy, which first extracts the modality-specific and modality-shared information contained in RGB and IR images separately, and then generates intermediate RGB and intermediate IR images by adaptively fusing original RGB and original IR images with their corresponding modality-specific and modality-shared information separately. In addition, we also design a discriminative information enhancement module (DIEM) to obtain more discriminative pedestrian representations by enhancing the discriminative information contained in the deep features of pedestrians. Extensive experiments on the publicly available SYSU-MM01 and RegDB datasets indicate that the performance of IGIE-Net reaches the current advanced level.

Mask-guided discriminative feature network for occluded person re-identification

In recent years, although research on person re-identification (ReID) has made significant progress, occluded person ReID remains a major challenge. In real-world scenes, persons are often occluded by various obstacles such as vehicles, umbrellas, and other persons. This leads to the noisy interference and the loss of visual information, which result in poor ReID performance of occluded persons. To address this issue, we propose an end-to-end Mask-guided Discriminative Feature Network (MDFNet). First, MDFNet adopts a dual-branch architecture with a shared encoder as the Feature Extraction module for paired images. Each pair of images consist of one image from the training set and its corresponding occluded image generated through an occlusion augmentation strategy. Second, MDFNet utilizes a Mask-guided Discriminative Feature Enhancement and Fusion (MDFEF) module to achieve the fusion and enhancement of global and local features for high-quality person representations. MDFEF module effectively suppresses the interference of occlusion, enriches the representation capacity of person features, and enables the model to focus more on the important features in non-occluded regions. Furthermore, MDFNet introduces a sparse pairwise loss for enabling the model to dynamically adapt to intra-class variations and reduce the negative impact of complex occlusions. The experimental results on four challenging person ReID datasets demonstrate the effectiveness of the proposed method.

MP2PMatch: A Mask-guided Part-to-Part Matching network based on transformer for occluded person re-identification

Occluded person re-identification (ReID) remains challenging due to the misaligned body parts. Existing works, mainly utilizing extra clues, excel in predicting holistic person images but falter when confronting substantial occlusion. This paper proposes a transformer-based Mask-guided Part-to-Part Matching (MP2PMatch) network for fine-grained matching. Firstly, the Consistency Occlusion Augmentation (COA) processes holistic person images and corresponding body part masks to construct occluded “image-mask” pairs. Next, we introduce learnable part tokens to capture semantic features of various body parts, performing “pull close” and “push apart” operations based on identity labels and part visibility, ensuring the one-to-one correspondence between part features and body parts. Additionally, the proposed Body Region Attention (BRA) utilizes the overall attention on body regions to guide the network to overcome interference from both occlusion and background. Extensive experiments demonstrate that MP2PMatch achieves exceptional occluded ReID performance.

Distillation embedded absorbable pruning for fast object re-identification

Combining knowledge distillation (KD) and network pruning (NP) shows promise in learning a light network to accelerate object re-identification. However, KD requires an untrained student network to establish more critical connections in early epochs, but NP demands a well-trained student network to avoid destroying critical connections. This presents a dilemma, potentially leading to a collapse of the student network and harming object Re-ID performance. For that, we propose a distillation embedded absorbable pruning (DEAP) method. We design a pruner-convolution-pruner (PCP) unit to resolve the dilemma by loading NP’s sparse regularization on extra untrained pruners. Additionally, we propose an asymmetric relation knowledge distillation method. It readily transfers feature representation knowledge and asymmetric pairwise similarity knowledge without using additional adaptation modules. Finally, we apply re-parameterization to absorb pruners of PCP units to simplify student networks. Experiments demonstrate the superiority of DEAP, such as on the VeRi-776 dataset, with ResNet-101 as a teacher, DEAP saves 73.24% of model parameters and 71.98% of floating-point operations without sacrificing accuracy.

Patient Re-Identification Based on Deep Metric Learning in Trunk Computed Tomography Images Acquired from Devices from Different Vendors

During radiologic interpretation, radiologists read patient identifiers from the metadata of medical images to recognize the patient being examined. However, it is challenging for radiologists to identify “incorrect” metadata and patient identification errors. We propose a method that uses a patient re-identification technique to link correct metadata to an image set of computed tomography images of a trunk with lost or wrongly assigned metadata. This method is based on a feature vector matching technique that uses a deep feature extractor to adapt to the cross-vendor domain contained in the scout computed tomography image dataset. To identify “incorrect” metadata, we calculated the highest similarity score between a follow-up image and a stored baseline image linked to the correct metadata. The re-identification performance tests whether the image with the highest similarity score belongs to the same patient, i.e., whether the metadata attached to the image are correct. The similarity scores between the follow-up and baseline images for the same “correct” patients were generally greater than those for “incorrect” patients. The proposed feature extractor was sufficiently robust to extract individual distinguishable features without additional training, even for unknown scout computed tomography images. Furthermore, the proposed augmentation technique further improved the re-identification performance of the subset for different vendors by incorporating changes in width magnification due to changes in patient table height during each examination. We believe that metadata checking using the proposed method would help detect the metadata with an “incorrect” patient identifier assigned due to unavoidable errors such as human error.

Exploring biometric domain adaptation in human action recognition models for unconstrained environments

AbstractIn conventional machine learning (ML), a fundamental assumption is that the training and test sets share identical feature distributions, a reasonable premise drawn from the same dataset. However, real-world scenarios often defy this assumption, as data may originate from diverse sources, causing disparities between training and test data distributions. This leads to a domain shift, where variations emerge between the source and target domains. This study delves into human action recognition (HAR) models within an unconstrained, real-world setting, scrutinizing the impact of input data variations related to contextual information and video encoding. The objective is to highlight the intricacies of model performance and interpretability in this context. Additionally, the study explores the domain adaptability of HAR models, specifically focusing on their potential for re-identifying individuals within uncontrolled environments. The experiments involve seven pre-trained backbone models and introduce a novel analytical approach by linking domain-related (HAR) and domain-unrelated (re-identification (re-ID)) tasks. Two key analyses addressing contextual information and encoding strategies reveal that maintaining the same encoding approach during training results in high task correlation while incorporating richer contextual information enhances performance. A notable outcome of this study is the comprehensive evaluation of a novel transformer-based architecture driven by a HAR backbone, which achieves a robust re-ID performance superior to state-of-the-art (SOTA). However, it faces challenges when other encoding schemes are applied, highlighting the role of the HAR classifier in performance variations.

Image-based human re-identification: Which covariates are actually (the most) important?

Human re-identification (re-ID) is nowadays among the most popular topics in computer vision, due to the increasing importance given to safety/security in modern societies. Being expected to sun in totally uncontrolled data acquisition settings (e.g., visual surveillance) automated re-ID not only depends on various factors that may occur in non-controlled data acquisition settings, but - most importantly - performance varies with respect to different subject features (e.g., gender, height, ethnicity, clothing, and action being performed), which may result in highly biased and undesirable automata. While many efforts have been putted in increase the robustness of identification to uncontrolled settings, a systematic assessment of the actual variations in performance with respect to each subject feature remains to be done. Accordingly, the contributions of this paper are threefold: 1) we report the correlation between the performance of three state-of-the-art re-ID models and different subject features; 2) we discuss the most concerning features and report valuable insights about the roles of the various features in re-ID performance, which can be used to develop more effective and unbiased re-ID systems; and 3) we leverage the concept of biometric menagerie, in order to identify the groups of individuals that typically fall into the most common menagerie families (e.g., goats, lambs, and wolves). Our findings not only contribute to a better understanding of the factors affecting re-ID performance, but also may offer practical guidance for researchers and practitioners concerned on human re-identification development.

Enhanced visible–infrared person re-identification based on cross-attention multiscale residual vision transformer

Visible–infrared (VI) person re-identification (Re-ID) is a critical identification task that involves retrieving and matching images of an individual using both infrared and visible imaging modalities. To improve the performance, researchers have developed methods to obtain implicit feature information; however, this degrades with fewer discriminative features. To address this issue, we propose a weighted fused cross-attention multi-scale residual vision transformer (WF-CAMReViT) approach to re-identify the appropriate person from visible–infrared modality images by integrating the cross-attention multi-scale residual vision transformer architecture with Opposition-based Dove Swarm Optimization (ODSO). The proposed framework aims to bridge the domain gap between the visible and infrared modalities and significantly improve the re-identification performance. RGB (visible) and infrared (IR) images of persons are gathered from standard datasets, subjected to a cross-attention multi-scale residual vision transformer network to collect features, and then fuse using minimal weight. We also propose Opposition-based DSO to find the minimal weight. The weighted fused features are then subjected to the final decoder layer of CAMReViT to perceive the characteristics of each modality. In this study, model-aware enhancement (MAE) loss is develop to improve the modality information capacity of modality-shared features. Then, the experimental results on the SYSU-MM01 and RegDB datasets are compared with state-of-the-art transformer-based visible–infrared person Re-ID tasks to verify the efficacy of the proposed model.

Dual-stream feature fusion network for person re-identification

Person re-identification (Re-ID) has made significant progress in recent years. However, it still faces numerous challenges in real scenarios. Although researchers have proposed various solutions, the issue of similar clothing colors remains an obstacle in improving the performance of person re-identification. To solve this issue, we propose a dual-stream feature fusion network (DSFF-Net) to extract discriminative features from pedestrian images in two color spaces. Specifically, a dual-stream network is designed to extract RGB global features, grayscale global features, and local features of pedestrian images to increase the richness of pedestrian representations. A channel attention module is designed to direct the network to focus on the salient features of pedestrians. An embedding mixed pooling is designed, which integrates the outputs of global average pooling (GAP) and global max pooling (GMP) to obtain more discriminative global features. Besides, it can also remove redundant information and increase the discrimination of pedestrian representations. A fine-grained local feature embedding fusion operation is designed to obtain more discriminative local features by embedding and fusing fine-grained local features of RGB and grayscale pedestrian images. Since the final pedestrian representation fuses both global features and fine-grained discriminative features in RGB and grayscale spaces, DSFF-Net increases the discriminative capability and richness of pedestrian representations. Moreover, we conduct extensive experiments on three datasets, Market-1501 DukeMTMC-Reid, and CUHK03, and our method achieves the Rank-1/mAP of 95.9%/89.1%, 89.0%/79.2%, and 81.2%/78.7%, respectively. Experimental results show that the performance of DSFF-Net is better than those of most of the state-of-the-art person Re-ID methods.

Vehicle re-identification method based on multi-attribute dense linking network combined with distance control module.

Vehicle re-identification is a crucial task in intelligent transportation systems, presenting enduring challenges. The primary challenge involves the inefficiency of vehicle re-identification, necessitating substantial time for recognition within extensive datasets. A secondary challenge arises from notable image variations of the same vehicle due to differing shooting angles, lighting conditions, and diverse camera equipment, leading to reduced accuracy. This paper aims to enhance vehicle re-identification performance by proficiently extracting color and category information using a multi-attribute dense connection network, complemented by a distance control module. We propose an integrated vehicle re-identification approach that combines a multi-attribute dense connection network with a distance control module. By merging a multi-attribute dense connection network that encompasses vehicle HSV color attributes and type attributes, we improve classification rates. The integration of the distance control module widens inter-class distances, diminishes intra-class distances, and boosts vehicle re-identification accuracy. To validate the feasibility of our approach, we conducted experiments using multiple vehicle re-identification datasets. We measured various quantitative metrics, including accuracy, mean average precision, and rank-n. Experimental results indicate a significant enhancement in the performance of our method in vehicle re-identification tasks. The findings of this study provide valuable insights into the application of multi-attribute neural networks and deep learning in the field of vehicle re-identification. By effectively extracting color information from the HSV color space and vehicle category information using a multi-attribute dense connection network, coupled with the utilization of a distance control module to process vehicle features, our approach demonstrates improved performance in vehicle re-identification tasks, contributing to the advancement of smart city systems.

Enhancing Person Re-Identification Performance Through In Vivo Learning.

This research investigates the potential of in vivo learning to enhance visual representation learning for image-based person re-identification (re-ID). Compared to traditional self-supervised learning (which require external data), the introduced in vivo learning utilizes supervisory labels generated from pedestrian images to improve re-ID accuracy without relying on external data sources. Three carefully designed in vivo learning tasks, leveraging statistical regularities within images, are proposed without the need for laborious manual annotations. These tasks enable feature extractors to learn more comprehensive and discriminative person representations by jointly modeling various aspects of human biological structure information, contributing to enhanced re-ID performance. Notably, the method seamlessly integrates with existing re-ID frameworks, requiring minimal modifications and no additional data beyond the existing training set. Extensive experiments on diverse datasets, including Market1501, CUHK03-NP, Celeb-reID, Celeb-reid-light, PRCC, and LTCC, demonstrate substantial enhancements in rank-1 precision compared to state-of-the-art methods.

Weakly supervised end-to-end domain adaptation for person re-identification

Unsupervised domain adaptation (UDA) approaches for person re-identification (Re-ID) heavily rely on accurate pedestrian location, which is often unavailable in real-world scenarios. With respect to this situation, a two-step approach is commonly employed. However, this approach involves a complex process of disentangling UDA person Re-ID and pedestrian location, which consumes significant time and resources. This paper introduces a novel approach called Weakly Supervised End-to-End Domain Adaptation for Person Re-ID (WS-PreID) to address this problem. WS-PreID is designed to perform weakly supervised end-to-end person Re-ID by integrating pedestrian location and UDA person Re-ID in a unified manner. Our framework is a weakly supervised end-to-end network (WS-net) that operates solely on pedestrian bounding boxes. This approach mitigates the impact of background noise on the overall performance. Additionally, WS-PreID introduces a pseudo-label generation strategy with feature invariance (PLFI) to calculate pseudo-labels for person Re-ID training. PLFI utilizes both current features and comprehensive features to improve the precision of the assigned pseudo-labels. To evaluate the effectiveness of WS-PreID, we conducted comparative evaluations using the PRW and CUHK-SYSU datasets. The experimental results validate that WS-PreID achieves an end-to-end solution and significantly improves the performance of person Re-ID.

Identifiable, But Not Visible: A Privacy-Preserving Person Reidentification Scheme

Person re-identification (Person Re-ID) is widely regarded as a promising technique to identify a target person through surveillance cameras in the wild. Nevertheless, person Re-ID leads to severe personal image privacy concerns as personal images are stipulated by laws and guidelines as private data. To address these concerns, this article explores the first solution for building a privacy-preserving person Re-ID system. Specifically, this article formulizes privacy-preserving person Re-ID as similarity metrics of encrypted feature vectors because the underlying operation of person Re-ID is to compute the similarity of feature vectors that are extracted from person images by a machine learning model. However, feature vectors are generally denoted by floating-point numbers. To this end, this article exploits a series of new encoding mechanisms and secure batch computing protocols to encrypt floating-point feature vectors and achieve the underlying operation of person Re-ID. Rigorous theoretical analyses demonstrate that this work achieves person Re-ID without compromising any personal image privacy. Furthermore, the proposed secure batch protocols significantly enhance the performance of privacy-preserving person Re-ID while outputting the same precision as the previous method.

Multi-modal person re-identification based on transformer relational regularization

For robust multi-modal person re-identification (re-ID) models, it is crucial to effectively utilize the complementary information and constraint relationships among different modalities. However, current multi-modal methods often overlook the correlation between modalities at the feature fusion stage. To address this issue, we propose a novel multimodal person re-ID method called Transformer Relation Regularization (TRR). Firstly, we introduce an adaptive collaborative matching module that facilitates the exchange of useful information by mining feature correspondences between modalities. This module allows for the integration of complementary information, enhancing the re-ID performance. Secondly, we propose an enhanced embedded module that corrects general information using discriminative information within each modality. By leveraging this approach, we improve the model’s stability in challenging multi-modal environments. Lastly, we propose an adaptive triple loss to enhance sample utilization efficiency and mitigate the problem of inconsistent representation among multimodal samples. This loss function optimizes the model’s ability to distinguish between different individuals, leading to improved re-ID accuracy. Experimental results on several challenging visible-infrared person re-ID benchmark datasets demonstrate that our proposed TRR method achieves optimal performance. Additionally, extensive ablation studies validate the effective contribution of each component to the overall model. In summary, our proposed TRR method effectively leverages complementary information, addresses the correlation between modalities, and improves the re-ID performance in multi-modal scenarios. The results obtained from various benchmark datasets and the comprehensive analysis support the efficacy of our approach.

Semantic Map Guided Identity Transfer GAN for Person Re-identification

Generative adversarial networks (GANs)-based person re-identification (re-id) schemes provide potential ways to augment data in practical applications. However, existing solutions perform poorly because of the separation of data generation and re-id training and a lack of diverse data in real-world scenarios. In this paper, a person re-id model (IDGAN) based on semantic map guided identity transfer GAN is proposed to improve the person re-id performance. With the aid of the semantic map, IDGAN generates pedestrian images with varying poses, perspectives, and backgrounds efficiently and accurately, improving the diversity of training data. To increase the visual realism, IDGAN utilizes a gradient augmentation method based on local quality attention to refine the generated image locally. Then, a two-stage joint training framework is employed to allow the GAN and the person re-id network to learn from each other to better use the generated data. Detailed experimental results demonstrate that, compared with the existing state-of-the-art methods, IDGAN is capable of producing high-quality images and significantly enhancing re-id performance, with the FID of generated images on the Market-1501 dataset being reduced by 1.15, and mAP on the Market-1501 and DukeMTMC-reID datasets being increased by 3.3% and 2.6%, respectively.

Research on Key Technology of Ship Re-Identification Based on the USV-UAV Collaboration

Distinguishing ship identities is critical in ensuring the safety and supervision of the marine agriculture and transportation industry. In this paper, we present a comprehensive investigation and validation of the progression of ship re-identification technology within a cooperative framework predominantly governed by UAVs. Our research revolves around the creation of a ship ReID dataset, the creation of a ship ReID dataset, the development of a feature extraction network, ranking optimization, and the establishment of a ship identity re-identification system built upon the collaboration of unmanned surface vehicles (USVs) and unmanned aerial vehicles (UAVs). We introduce a ship ReID dataset named VesselID-700, comprising 56,069 images covering seven classes of typical ships. We also simulated the multi-angle acquisition state of UAVs to categorize the ship orientations within this dataset. To address the challenge of distinguishing between ships with small inter-class differences and large intra-class variations, we propose a fine-grained feature extraction network called FGFN. FGFN enhances the ResNet architecture with a self-attentive mechanism and generalized mean pooling. We also introduce a multi-task loss function that combines classification and triplet loss, incorporating hard sample mining. Ablation experiments on the VesselID-700 dataset demonstrate that the FGFN network achieves outstanding performance, with a Rank-1 accuracy of 89.78% and mAP of 65.72% at a state-of-the-art level. Generalization experiments on pedestrian and vehicle ReID datasets reveal that FGFN excels in recognizing other rigid body targets and diverse viewpoints. Furthermore, to further enhance the advantages of UAV-USV synergy in ship ReID performance, we propose a ranking optimization method based on the homologous fusion of multi-angle UAVs and heterologous fusion of USV-UAV collaborative architecture. This optimization leads to a significant 3% improvement in Rank-1 performance, accompanied by a 73% reduction in retrieval time cost.

Multi-Scale Semantic and Detail Extraction Network for Lightweight Person Re-Identification

Exploring multi-level information to obtain fine-grained features is the key factor to improve the performance of person re-identification (Re-ID). However, existing models for person Re-ID only focus on learning the high-level semantic information while neglecting the low-level detail information. To alleviate this issue, we propose a lightweight person Re-ID method termed Multi-Scale Semantic and Detail Extraction Network (MSDENet) to obtain robustness and discriminative feature representation for the Re-ID task. Specifically, we design a Series Channel-Spatial Attention (SCSA) and embed it into the lightweight backbone network to focus on the key parts of pedestrian images. Meanwhile, we propose a Multi-Scale Semantic and Detail Extraction (MSDE) method to extract multi-scale features of semantic information and detail information, which can effectively capture the feature diversity of pedestrian images. Furthermore, we design a Feature Enhancement Fusion (FEF), which enhances and fuses the fine-grained features of semantic extraction and detail extraction branches to better obtain the discriminative feature representation. Extensive experiments conducted on popular datasets Market1501, MSMT17, and CUHK03 demonstrate that the proposed MSDENet has competitive performance compared with the state-of-the-art methods.

Influence of tracking duration on the privacy of individual mobility graphs

ABSTRACT Location graphs, compact representations of human mobility without geocoordinates, can be used to personalise location-based services. While they are more privacy-preserving than raw tracking data, it was shown that they still hold a considerable risk for users to be re-identified solely by the graph topology. However, it is unclear how this risk depends on the tracking duration. Here, we consider a scenario where the attacker wants to match the new tracking data of a user to a pool of previously recorded mobility profiles, and we analyse the dependence of the re-identification performance on the tracking duration. We find that the re-identification accuracy varies between 0.41% and 20.97% and is affected by both the pool duration and the test-user tracking duration, it is greater if both have the same duration, and it is not significantly affected by socio-demographics such as age or gender, but can to some extent be explained by different mobility and graph features. Overall, the influence of tracking duration on user privacy has clear implications for data collection and storage strategies. We advise data collectors to limit the tracking duration or to reset user IDs regularly when storing long-term tracking data.

Dynamic Weighting Network for Person Re-Identification.

Recently, hybrid Convolution-Transformer architectures have become popular due to their ability to capture both local and global image features and the advantage of lower computational cost over pure Transformer models. However, directly embedding a Transformer can result in the loss of convolution-based features, particularly fine-grained features. Therefore, using these architectures as the backbone of a re-identification task is not an effective approach. To address this challenge, we propose a feature fusion gate unit that dynamically adjusts the ratio of local and global features. The feature fusion gate unit fuses the convolution and self-attentive branches of the network with dynamic parameters based on the input information. This unit can be integrated into different layers or multiple residual blocks, which will have varying effects on the accuracy of the model. Using feature fusion gate units, we propose a simple and portable model called the dynamic weighting network or DWNet, which supports two backbones, ResNet and OSNet, called DWNet-R and DWNet-O, respectively. DWNet significantly improves re-identification performance over the original baseline, while maintaining reasonable computational consumption and number of parameters. Finally, our DWNet-R achieves an mAP of 87.53%, 79.18%, 50.03%, on the Market1501, DukeMTMC-reID, and MSMT17 datasets. Our DWNet-O achieves an mAP of 86.83%, 78.68%, 55.66%, on the Market1501, DukeMTMC-reID, and MSMT17 datasets.