Re-identification Performance Research Articles

Take good care of your fish: fish re-identification with synchronized multi-view camera system

IntroductionFish re-identification (re-ID) is of great significance for fish monitoring and can contribute to aquaculture and fish breeding. Synchronizing information from different cameras is beneficial for optimizing re-ID performance.MethodsWe constructed the first underwater fish re-identification benchmark dataset (FS48) under three camera conditions. FS48 encompasses 48 different fish identities, 10,300 frames, and 39,088 bounding boxes, covering various lighting conditions and background environments. Additionally, we developed the first robust and accurate fish re-identification baseline, FSNet, which fuses information from three camera positions by extracting features from synchronized video frames of each position and combining the synchronized information.ResultsThe experimental results show that FS48 is universal and of high quality. FSNet has an effective network design and demonstrates good performance, achieving better re-identification performance by combining information from three positions, helping improve overall re-test accuracy, and evaluating the effectiveness of re-identification among detectors.DiscussionOur dataset will be released upon acceptance of this paper, which is expected to further promote the development of underwater fish re-identification.

Weak saliency ensemble network for person Re-identification using infrared light images

In recent years, person re-identification (re-id) has primarily been studied using visible light (VL) images. However, the challenges of employing VL images in nighttime environments have prompted research into using infrared light (IR) images. Yet, the utilization of both VL and IR images in person re-id has resulted in increased computational cost and processing time in multi-modality systems, leading to studies focusing solely on IR images. Nevertheless, IR images, lacking color and texture information, generally yield lower recognition performance in existing person re-id studies. In addition, previous studies have shown that person re-id performance suffers in the presence of complex background noise. To tackle these challenges, this study proposes a new weak saliency ensemble network (WSE-Net) for person re-id using IR images. WSE-Net incorporates a channel reduction of feature (CRF) method to reduce computational cost in the ensemble network, a technique for converting input images into group of patch images and feeding them into the ensemble model to enhance the reduced feature information, and a grouped convolution ensemble network (GCE-Net) that enables the fusion of features extracted from original and attention-guided ensemble models.The performance of person re-id using WSE-Net was evaluated on the Dongguk body-based person recognition database version 1 (DBPerson-Recog-DB1) and the Sun Yat-sen university multiple modality re-identification version 1 (SYSU-MM01). Experimental results demonstrated that on DBPerson-Recog-DB1, WSE-Net achieved 93.65% in rank 1, 95.28% in mean average precision (mAP), and 93.52% in the harmonic mean of precision and recall. Additionally, on SYSU-MM01, WSE-Net achieved 86.85% in rank 1, 44.58% in mAP, and 40.06% in the harmonic mean of precision and recall. Furthermore, the accuracy of WSE-Net on both datasets surpassed that of state-of-the-art methods.

Research on Pseudo-Label Improvement Based on Probabilistic Uncertainty

Abstract The pedestrian re-identification task explored in this study has strong application scenarios in real life. Generally, in actual scenarios, the pedestrian samples taken by the camera are unlabeled, so pseudo-labeling is used. However, low-quality image feature extraction and imperfect clustering will produce low-quality and incorrect clustering results, which will have a negative impact on the re-identification performance. This article proposes a dual extraction method (PLR). The core idea of this study is to explore the distribution differences and similarities between two data sets through the pseudo-label method, so that the model can learn the distribution in one data set and apply it to another data set and accurately identify the target person to be found. Therefore, this article proposes a dual refinement method based on probability uncertainty. PLR first removes unqualified clustering results through probability uncertainty, and then continues to refine the remaining ones. Experiments have shown that compared to newer methods, PLR is approximately 1% higher in most tasks.

Learning a generalizable re-identification model from unlabelled data with domain-agnostic expert

In response to real-world scenarios, the domain generalization (DG) problem has spurred considerable research in person re-identification (ReID). This challenge arises when the target domain, which is significantly different from the source domains, remains unknown. However, the performance of current DG ReID relies heavily on labor-intensive source domain annotations. Considering the potential of unlabeled data, we investigate unsupervised domain generalization (UDG) in ReID. Our goal is to create a model that can generalize from unlabeled source domains to semantically retrieve images in an unseen target domain. To address this, we propose a new approach that trains a domain-agnostic expert (DaE) for unsupervised domain-generalizable person ReID. This involves independently training multiple experts to account for label space inconsistencies between source domains. At the same time, the DaE captures domain-generalizable information for testing. Our experiments demonstrate the effectiveness of this method for learning generalizable features under the UDG setting. The results demonstrate the superiority of our method over state-of-the-art techniques. We will make our code and models available for public use.

Improving Re-Identification by Estimating and Utilizing Diverse Uncertainty Types for Embeddings

In most re-identification approaches, embedding vectors are compared to identify the best match for a given query. However, this comparison does not take into account whether the encoded information in the embedding vectors was extracted reliably from the input images. We propose the first attempt that illustrates how all three types of uncertainty, namely model uncertainty (also known as epistemic uncertainty), data uncertainty (also known as aleatoric uncertainty), and distributional uncertainty, can be estimated for embedding vectors. We provide evidence that we do indeed estimate these types of uncertainty, and that each type has its own value for improving re-identification performance. In particular, while the few state-of-the-art approaches that employ uncertainty for re-identification during inference utilize only data uncertainty to improve single-shot re-identification performance, we demonstrate that the estimated model uncertainty vector can be utilized to modify the feature vector. We explore the best method for utilizing the estimated model uncertainty based on the Market-1501 dataset and demonstrate that we are able to further enhance the performance above the already strong baseline UAL. Additionally, we show that the estimated distributional uncertainty resembles the degree to which the current sample is out-of-distribution. To illustrate this, we divide the distractor set of the Market-1501 dataset into four classes, each representing a different degree of out-of-distribution. By computing a score based on the estimated distributional uncertainty vector, we are able to correctly order the four distractor classes and to differentiate them from an in-distribution set to a significant extent.

Personalized Federated Mutual Learning for Unsupervised Camera-aware Person Re-identification

Person re-identification (ReID) is essential for enhancing security and tracking in multi-camera surveillance systems. To achieve effective person re-identification (ReID) performance across diverse datasets, the Federated Unsupervised Person Re-identification via Camera-aware Clustering (FedUCA) approach has made strides in utilizing distributed datasets while ensuring data privacy. Nevertheless, its uniform model may not adequately cater to the specific characteristics of each participant’s data, given the diversity in camera perspectives and client-specific data variances, thus obtaining degraded results. To address this issue, we propose an advanced framework, Personalized Federated Dual-Model Learning for Camera-Aware Person Re-Identification (PerFedDual), which introduces knowledge-sharing techniques inherent to mutual learning for FedUCA with the camera-centric clustering process. PerFedDual supports a dual-model training approach that creates a cooperative learning space that improves both the global model and client-specific models by exchanging knowledge both ways. The methodology adopted is precisely adjusted to the distinctive data environment of each client, ensuring the protection of privacy while simultaneously enhancing the accuracy and flexibility of ReID models in diverse camera configurations. The empirical evaluation reveals that PerFedDual outperforms FedUCA and alternative federated learning strategies, highlighting the benefits of our technique that leverage collective intelligence to enhance unsupervised person re-identification.

A review on video person re-identification based on deep learning

Person Re-Identification (ReID) is an essential technology for matching a person across non-overlapping cameras. It has attracted increasing attention in recent years due to its wide range of applications in various real-world scenarios such as security surveillance and criminal investigation. Different from other person ReID tasks, video-based ReID uses a video clip as the retrieval input, which can provide more promising ReID performance because that the video has rich information on appearance, motion cues and pose variations on temporal pipeline. Over the last few years, many deep learning-based video person ReID have been proposed to address various challenges, such as illumination variation, complex background, occlusion, etc. To provide a more comprehensive and readable review on existing video-based person ReID methods, we propose a novel taxonomy method that observes existing methods from four perspectives: data, algorithms, computing power, and applications. Specifically, we first introduce some popular datasets and evaluation criterion used for video-based person ReID. Next, from limited data and little annotation view, we introduce data augmentation and unsupervised learning ReID. From algorithm view, we focus on reviewing supervised methods including spatial feature learning, temporal feature learning and spatio-temporal feature learning, and further discuss and conduct a systematic comparison among these approaches. From complex open-world application view, we mainly summarized domain adaption and multimodal ReID. From insufficient GPU computing power view, we mainly discuss modality-agnostic unified large-scale ReID and their lightweighting. Finally, we provide a discussion of open problems and potential research directions for the community.

In-Depth Analysis of GAF-Net: Comparative Fusion Approaches in Video-Based Person Re-Identification

This study provides an in-depth analysis of GAF-Net, a novel model for video-based person re-identification (Re-ID) that matches individuals across different video sequences. GAF-Net combines appearance-based features with gait-based features derived from skeletal data, offering a new approach that diverges from traditional silhouette-based methods. We thoroughly examine each module of GAF-Net and explore various fusion methods at the both score and feature levels, extending beyond initial simple concatenation. Comprehensive evaluations on the iLIDS-VID and MARS datasets demonstrate GAF-Net’s effectiveness across scenarios. GAF-Net achieves state-of-the-art 93.2% rank-1 accuracy on iLIDS-VID’s long sequences, while MARS results (86.09% mAP, 89.78% rank-1) reveal challenges with shorter, variable sequences in complex real-world settings. We demonstrate that integrating skeleton-based gait features consistently improves Re-ID performance, particularly with long, more informative sequences. This research provides crucial insights into multi-modal feature integration in Re-ID tasks, laying a foundation for the advancement of multi-modal biometric systems for diverse computer vision applications.

Self-selective receptive field network for person re-identification

Person Re-identification (Re-ID) technology aims to solve the matching problem of the same pedestrians at different times and places, which has important application value in the field of public safety. At present, most scholars focus on designing complex models to improve the accuracy of Re-ID, but the high complexity of the model further restricts the practical application of Re-ID algorithm. To solve the above problems, this paper designs a lightweight Self-selective Receptive Field (SRF) block instead of directly designing complex models. Specifically, the module can be plug-and-play on the general backbone network, so as to significantly improve the performance of Re-ID while effectively controlling the amount of its own parameter and calculation: (1) the SRF block encodes pedestrian targets and image contexts at different scales by constructing pyramidal convolution group and allows the module to independently select the size of the receptive field through training by means of self-adaptive weighting; (2) in order to reduce the complexity of SRF block, we introduce a "channel scaling factor" and design a "grouped convolution operation" by constraining the channels of the feature map and changing the structure of the convolution kernel respectively. Experiments on multiple datasets show that SRF Network (SRFNet) for Re-ID can achieve a good balance between performance and complexity, which fully demonstrates the effectiveness of SRF block.

Learning the Meta Feature Transformer for Unsupervised Person Re-Identification

Although unsupervised person re-identification (Re-ID) has drawn increasing research attention, it still faces the challenge of learning discriminative features in the absence of pairwise labels across disjoint camera views. To tackle the issue of label scarcity, researchers have delved into clustering and multilabel learning using memory dictionaries. Although effective in improving unsupervised Re-ID performance, these methods require substantial computational resources and introduce additional training complexity. To address this issue, we propose a conceptually simple yet effective and learnable module effective block, named the meta feature transformer (MFT). MFT is a streamlined, lightweight network architecture that operates without the need for complex networks or feature memory bank storage. It primarily focuses on learning interactions between sample features within small groups using a transformer mechanism in each mini-batch. It then generates a new sample feature for each group through a weighted sum. The main benefits of MFT arise from two aspects: (1) it allows for the use of numerous new samples for training, which significantly expands the feature space and enhances the network’s generalization capabilities; (2) the trainable attention weights highlight the importance of samples, enabling the network to focus on more useful or distinguishable samples. We validate our method on two popular large-scale Re-ID benchmarks, where extensive evaluations show that our MFT outperforms previous methods and significantly improves Re-ID performances.

IGIE-net: Cross-modality person re-identification via intermediate modality image generation and discriminative information enhancement

Given an RGB image (or an IR image) of a pedestrian, the task of cross-modality person re-identification aims to retrieve images of a specific pedestrian from an IR (or RGB) gallery. However, the huge modality discrepancy between RGB and IR images significantly affects the performance of cross-modality person re-identification. For the purpose of reducing the impact of modality discrepancy and extracting more discriminative pedestrian features, a new cross-modality person re-identification network is proposed by us, which is based on intermediate modality image generation and discriminative information enhancement (IGIE-Net). Specifically, we design an intermediate modality image generation module (IMIGM) to effectively weaken the impact of modality discrepancy, which first extracts the modality-specific and modality-shared information contained in RGB and IR images separately, and then generates intermediate RGB and intermediate IR images by adaptively fusing original RGB and original IR images with their corresponding modality-specific and modality-shared information separately. In addition, we also design a discriminative information enhancement module (DIEM) to obtain more discriminative pedestrian representations by enhancing the discriminative information contained in the deep features of pedestrians. Extensive experiments on the publicly available SYSU-MM01 and RegDB datasets indicate that the performance of IGIE-Net reaches the current advanced level.

Mask-guided discriminative feature network for occluded person re-identification

In recent years, although research on person re-identification (ReID) has made significant progress, occluded person ReID remains a major challenge. In real-world scenes, persons are often occluded by various obstacles such as vehicles, umbrellas, and other persons. This leads to the noisy interference and the loss of visual information, which result in poor ReID performance of occluded persons. To address this issue, we propose an end-to-end Mask-guided Discriminative Feature Network (MDFNet). First, MDFNet adopts a dual-branch architecture with a shared encoder as the Feature Extraction module for paired images. Each pair of images consist of one image from the training set and its corresponding occluded image generated through an occlusion augmentation strategy. Second, MDFNet utilizes a Mask-guided Discriminative Feature Enhancement and Fusion (MDFEF) module to achieve the fusion and enhancement of global and local features for high-quality person representations. MDFEF module effectively suppresses the interference of occlusion, enriches the representation capacity of person features, and enables the model to focus more on the important features in non-occluded regions. Furthermore, MDFNet introduces a sparse pairwise loss for enabling the model to dynamically adapt to intra-class variations and reduce the negative impact of complex occlusions. The experimental results on four challenging person ReID datasets demonstrate the effectiveness of the proposed method.

MP2PMatch: A Mask-guided Part-to-Part Matching network based on transformer for occluded person re-identification

Occluded person re-identification (ReID) remains challenging due to the misaligned body parts. Existing works, mainly utilizing extra clues, excel in predicting holistic person images but falter when confronting substantial occlusion. This paper proposes a transformer-based Mask-guided Part-to-Part Matching (MP2PMatch) network for fine-grained matching. Firstly, the Consistency Occlusion Augmentation (COA) processes holistic person images and corresponding body part masks to construct occluded “image-mask” pairs. Next, we introduce learnable part tokens to capture semantic features of various body parts, performing “pull close” and “push apart” operations based on identity labels and part visibility, ensuring the one-to-one correspondence between part features and body parts. Additionally, the proposed Body Region Attention (BRA) utilizes the overall attention on body regions to guide the network to overcome interference from both occlusion and background. Extensive experiments demonstrate that MP2PMatch achieves exceptional occluded ReID performance.

Distillation embedded absorbable pruning for fast object re-identification

Combining knowledge distillation (KD) and network pruning (NP) shows promise in learning a light network to accelerate object re-identification. However, KD requires an untrained student network to establish more critical connections in early epochs, but NP demands a well-trained student network to avoid destroying critical connections. This presents a dilemma, potentially leading to a collapse of the student network and harming object Re-ID performance. For that, we propose a distillation embedded absorbable pruning (DEAP) method. We design a pruner-convolution-pruner (PCP) unit to resolve the dilemma by loading NP’s sparse regularization on extra untrained pruners. Additionally, we propose an asymmetric relation knowledge distillation method. It readily transfers feature representation knowledge and asymmetric pairwise similarity knowledge without using additional adaptation modules. Finally, we apply re-parameterization to absorb pruners of PCP units to simplify student networks. Experiments demonstrate the superiority of DEAP, such as on the VeRi-776 dataset, with ResNet-101 as a teacher, DEAP saves 73.24% of model parameters and 71.98% of floating-point operations without sacrificing accuracy.

Patient Re-Identification Based on Deep Metric Learning in Trunk Computed Tomography Images Acquired from Devices from Different Vendors

During radiologic interpretation, radiologists read patient identifiers from the metadata of medical images to recognize the patient being examined. However, it is challenging for radiologists to identify “incorrect” metadata and patient identification errors. We propose a method that uses a patient re-identification technique to link correct metadata to an image set of computed tomography images of a trunk with lost or wrongly assigned metadata. This method is based on a feature vector matching technique that uses a deep feature extractor to adapt to the cross-vendor domain contained in the scout computed tomography image dataset. To identify “incorrect” metadata, we calculated the highest similarity score between a follow-up image and a stored baseline image linked to the correct metadata. The re-identification performance tests whether the image with the highest similarity score belongs to the same patient, i.e., whether the metadata attached to the image are correct. The similarity scores between the follow-up and baseline images for the same “correct” patients were generally greater than those for “incorrect” patients. The proposed feature extractor was sufficiently robust to extract individual distinguishable features without additional training, even for unknown scout computed tomography images. Furthermore, the proposed augmentation technique further improved the re-identification performance of the subset for different vendors by incorporating changes in width magnification due to changes in patient table height during each examination. We believe that metadata checking using the proposed method would help detect the metadata with an “incorrect” patient identifier assigned due to unavoidable errors such as human error.

Exploring biometric domain adaptation in human action recognition models for unconstrained environments

AbstractIn conventional machine learning (ML), a fundamental assumption is that the training and test sets share identical feature distributions, a reasonable premise drawn from the same dataset. However, real-world scenarios often defy this assumption, as data may originate from diverse sources, causing disparities between training and test data distributions. This leads to a domain shift, where variations emerge between the source and target domains. This study delves into human action recognition (HAR) models within an unconstrained, real-world setting, scrutinizing the impact of input data variations related to contextual information and video encoding. The objective is to highlight the intricacies of model performance and interpretability in this context. Additionally, the study explores the domain adaptability of HAR models, specifically focusing on their potential for re-identifying individuals within uncontrolled environments. The experiments involve seven pre-trained backbone models and introduce a novel analytical approach by linking domain-related (HAR) and domain-unrelated (re-identification (re-ID)) tasks. Two key analyses addressing contextual information and encoding strategies reveal that maintaining the same encoding approach during training results in high task correlation while incorporating richer contextual information enhances performance. A notable outcome of this study is the comprehensive evaluation of a novel transformer-based architecture driven by a HAR backbone, which achieves a robust re-ID performance superior to state-of-the-art (SOTA). However, it faces challenges when other encoding schemes are applied, highlighting the role of the HAR classifier in performance variations.

Image-based human re-identification: Which covariates are actually (the most) important?

Human re-identification (re-ID) is nowadays among the most popular topics in computer vision, due to the increasing importance given to safety/security in modern societies. Being expected to sun in totally uncontrolled data acquisition settings (e.g., visual surveillance) automated re-ID not only depends on various factors that may occur in non-controlled data acquisition settings, but - most importantly - performance varies with respect to different subject features (e.g., gender, height, ethnicity, clothing, and action being performed), which may result in highly biased and undesirable automata. While many efforts have been putted in increase the robustness of identification to uncontrolled settings, a systematic assessment of the actual variations in performance with respect to each subject feature remains to be done. Accordingly, the contributions of this paper are threefold: 1) we report the correlation between the performance of three state-of-the-art re-ID models and different subject features; 2) we discuss the most concerning features and report valuable insights about the roles of the various features in re-ID performance, which can be used to develop more effective and unbiased re-ID systems; and 3) we leverage the concept of biometric menagerie, in order to identify the groups of individuals that typically fall into the most common menagerie families (e.g., goats, lambs, and wolves). Our findings not only contribute to a better understanding of the factors affecting re-ID performance, but also may offer practical guidance for researchers and practitioners concerned on human re-identification development.

Enhanced visible–infrared person re-identification based on cross-attention multiscale residual vision transformer

Visible–infrared (VI) person re-identification (Re-ID) is a critical identification task that involves retrieving and matching images of an individual using both infrared and visible imaging modalities. To improve the performance, researchers have developed methods to obtain implicit feature information; however, this degrades with fewer discriminative features. To address this issue, we propose a weighted fused cross-attention multi-scale residual vision transformer (WF-CAMReViT) approach to re-identify the appropriate person from visible–infrared modality images by integrating the cross-attention multi-scale residual vision transformer architecture with Opposition-based Dove Swarm Optimization (ODSO). The proposed framework aims to bridge the domain gap between the visible and infrared modalities and significantly improve the re-identification performance. RGB (visible) and infrared (IR) images of persons are gathered from standard datasets, subjected to a cross-attention multi-scale residual vision transformer network to collect features, and then fuse using minimal weight. We also propose Opposition-based DSO to find the minimal weight. The weighted fused features are then subjected to the final decoder layer of CAMReViT to perceive the characteristics of each modality. In this study, model-aware enhancement (MAE) loss is develop to improve the modality information capacity of modality-shared features. Then, the experimental results on the SYSU-MM01 and RegDB datasets are compared with state-of-the-art transformer-based visible–infrared person Re-ID tasks to verify the efficacy of the proposed model.

Dual-stream feature fusion network for person re-identification

Person re-identification (Re-ID) has made significant progress in recent years. However, it still faces numerous challenges in real scenarios. Although researchers have proposed various solutions, the issue of similar clothing colors remains an obstacle in improving the performance of person re-identification. To solve this issue, we propose a dual-stream feature fusion network (DSFF-Net) to extract discriminative features from pedestrian images in two color spaces. Specifically, a dual-stream network is designed to extract RGB global features, grayscale global features, and local features of pedestrian images to increase the richness of pedestrian representations. A channel attention module is designed to direct the network to focus on the salient features of pedestrians. An embedding mixed pooling is designed, which integrates the outputs of global average pooling (GAP) and global max pooling (GMP) to obtain more discriminative global features. Besides, it can also remove redundant information and increase the discrimination of pedestrian representations. A fine-grained local feature embedding fusion operation is designed to obtain more discriminative local features by embedding and fusing fine-grained local features of RGB and grayscale pedestrian images. Since the final pedestrian representation fuses both global features and fine-grained discriminative features in RGB and grayscale spaces, DSFF-Net increases the discriminative capability and richness of pedestrian representations. Moreover, we conduct extensive experiments on three datasets, Market-1501 DukeMTMC-Reid, and CUHK03, and our method achieves the Rank-1/mAP of 95.9%/89.1%, 89.0%/79.2%, and 81.2%/78.7%, respectively. Experimental results show that the performance of DSFF-Net is better than those of most of the state-of-the-art person Re-ID methods.

Vehicle re-identification method based on multi-attribute dense linking network combined with distance control module.

Vehicle re-identification is a crucial task in intelligent transportation systems, presenting enduring challenges. The primary challenge involves the inefficiency of vehicle re-identification, necessitating substantial time for recognition within extensive datasets. A secondary challenge arises from notable image variations of the same vehicle due to differing shooting angles, lighting conditions, and diverse camera equipment, leading to reduced accuracy. This paper aims to enhance vehicle re-identification performance by proficiently extracting color and category information using a multi-attribute dense connection network, complemented by a distance control module. We propose an integrated vehicle re-identification approach that combines a multi-attribute dense connection network with a distance control module. By merging a multi-attribute dense connection network that encompasses vehicle HSV color attributes and type attributes, we improve classification rates. The integration of the distance control module widens inter-class distances, diminishes intra-class distances, and boosts vehicle re-identification accuracy. To validate the feasibility of our approach, we conducted experiments using multiple vehicle re-identification datasets. We measured various quantitative metrics, including accuracy, mean average precision, and rank-n. Experimental results indicate a significant enhancement in the performance of our method in vehicle re-identification tasks. The findings of this study provide valuable insights into the application of multi-attribute neural networks and deep learning in the field of vehicle re-identification. By effectively extracting color information from the HSV color space and vehicle category information using a multi-attribute dense connection network, coupled with the utilization of a distance control module to process vehicle features, our approach demonstrates improved performance in vehicle re-identification tasks, contributing to the advancement of smart city systems.