Re-identification Research Articles

스마트 축사내 상황인지 자율이동형 환경센서 개발 및 가축행동 분석에 관한 연구

This study aims to develop a system that predicts the health status of cattle based on behavior patterns and environmental data within a smart barn using an autonomous driving system. Maintaining a unique ID for each cow using only a camera, without external devices (such as RFID tags), is essential. This enables the tracking of behavior patterns such as standing, sitting, and lying for each cow over time. Additionally, environmental data such as temperature and humidity are integrated to comprehensively assess the cows' health conditions. To achieve this, we propose a unique ID retention algorithm that combines object detection using YOLO, tracking with Deep SORT, and re-identification (ReID). Experimental results show that the YOLO + Deep SORT + ReID algorithm delivers the best performance in maintaining unique IDs, and the LSTM-based behavior analysis model demonstrates high accuracy in predicting behavior patterns. This system can serve as an effective tool for real-time prediction of livestock health conditions, such as disease or stress, through comprehensive analysis of environmental data and behavior patterns inside the barn.

A survey on person and vehicle re‐identification

AbstractPerson/vehicle re‐identification aims to use technologies such as cross‐camera retrieval to associate the same person (same vehicle) in the surveillance videos at different locations, different times, and images captured by different cameras so as to achieve cross‐surveillance image matching, person retrieval and trajectory tracking. It plays an extremely important role in the fields of intelligent security, criminal investigation etc. In recent years, the rapid development of deep learning technology has significantly propelled the advancement of re‐identification (Re‐ID) technology. An increasing number of technical methods have emerged, aiming to enhance Re‐ID performance. This paper summarises four popular research areas in the current field of re‐identification, focusing on the current research hotspots. These areas include the multi‐task learning domain, the generalisation learning domain, the cross‐modality domain, and the optimisation learning domain. Specifically, the paper analyses various challenges faced within these domains and elaborates on different deep learning frameworks and networks that address these challenges. A comparative analysis of re‐identification tasks from various classification perspectives is provided, introducing mainstream research directions and current achievements. Finally, insights into future development trends are presented.

CLIP-DFGS: A Hard Sample Mining Method for CLIP in Generalizable Person Re-Identification

Recent advancements in pre-trained vision-language models like CLIP have shown promise in person re-identification (ReID) applications. However, their performance in generalizable person re-identification tasks remains suboptimal. The large-scale and diverse image-text pairs used in CLIP's pre-training may lead to a lack or insufficiency of certain fine-grained features. In light of these challenges, we propose a hard sample mining method called DFGS (Depth-First Graph Sampler), based on depth-first search, designed to offer sufficiently challenging samples to enhance CLIP's ability to extract fine-grained features. DFGS can be applied to both the image encoder and the text encoder in CLIP. By leveraging the powerful cross-modal learning capabilities of CLIP, we aim to apply our DFGS method to extract challenging samples and form mini-batches with high discriminative difficulty, providing the image model with more efficient and challenging samples that are difficult to distinguish, thereby enhancing the model's ability to differentiate between individuals. Our results demonstrate significant improvements over other methods, confirming the effectiveness of DFGS in providing challenging samples that enhance CLIP's performance in generalizable person re-identification.

ADT: Person re-identification based on efficient attention mechanism and single-channel dual-channel fusion with transformer features aggregation

In person re-identification(Re-ID) tasks, using only convolutional neural network(CNNs) makes it challenging to fully exploit the global key information in person images, while using only the Transformer structure fails to adequately capture the multi-scale local detail information in person images. Therefore, to address the more challenging person Re-ID tasks, effectively integrating CNN and Transformer into an algorithmic framework capable of fusing multi-dimensional features has become one of the current focal points of research. Due to the ability of the attention mechanism to suppress the extraction of non-salient features by convolutional neural networks, and the capability of the dual-channel multi-scale mechanism to facilitate the extraction of finer-grained deep information by the Transformer, we propose a network Based on Efficient Attention Mechanism and Single-channel Dual-channel Fusion with Transformer Features Aggregation (ADT). First, we introduce the Convolutional Block Attention Module(CBAM) and Spatial Reconstruction Unit (SRU) into the CNN network to construct a Multi-dimensional Attention Features Re-fusion Network(MCF). By weighting the local features extracted in the convolutional network, attention to person detail features in the model is enhanced. Then, we propose the Single and Dual-channel Integrated Transformer Features Aggregator(SDT) and overlap it in MCF. This integration is designed to more effectively aggregate the multi-scale high-dimensional global features from each layer of the CNN network. Experimental results on the Market1501, DukeMTMC, MSMT17, and CUHK03 datasets demonstrate that our model significantly enriches features representation compared to most methods.

Learning a generalizable re-identification model from unlabelled data with domain-agnostic expert

In response to real-world scenarios, the domain generalization (DG) problem has spurred considerable research in person re-identification (ReID). This challenge arises when the target domain, which is significantly different from the source domains, remains unknown. However, the performance of current DG ReID relies heavily on labor-intensive source domain annotations. Considering the potential of unlabeled data, we investigate unsupervised domain generalization (UDG) in ReID. Our goal is to create a model that can generalize from unlabeled source domains to semantically retrieve images in an unseen target domain. To address this, we propose a new approach that trains a domain-agnostic expert (DaE) for unsupervised domain-generalizable person ReID. This involves independently training multiple experts to account for label space inconsistencies between source domains. At the same time, the DaE captures domain-generalizable information for testing. Our experiments demonstrate the effectiveness of this method for learning generalizable features under the UDG setting. The results demonstrate the superiority of our method over state-of-the-art techniques. We will make our code and models available for public use.

Instant pose extraction based on mask transformer for occluded person re-identification

Re-Identification (Re-ID) of obscured pedestrians is a daunting task, primarily due to the frequent occlusion caused by various obstacles like buildings, vehicles, and even other pedestrians. To address this challenge, we propose a novel approach named Instant Pose Extraction based on Mask Transformer (MTIPE), tailored specifically for occluded person Re-ID. MTIPE consists of several new modules: a Mask Aware Module (MAM) for alignment between the overall prototype and the occluded image; a Multi-headed Attention Constraint Module (MACM) to enrich the feature representation; a Pose Aggregation Module (PAM) to separate useful human information from the occlusion noise; a Feature Matching Module (FMM) in matching non-occluded parts; introduction of learnable local prototypes in the defined local prototype-based transformer decoder; a Pooling Attention Module (PAM) instead of traditional self-attention module to better extract and propagate local contextual information; and Pose Key-points Loss to better match non-occluded body parts. Through comprehensive experimental evaluations and comparisons, MTIPE demonstrates encouraging performance improvements in both occluded and holistic person Re-ID tasks. Its results surpass or at least match those of current state-of-the-art methods in various aspects, highlighting its potential advantages and promising application prospects.

Logical Relation Inference and Multiview Information Interaction for Domain Adaptation Person Re-Identification.

Domain adaptation person re-identification (Re-ID) is a challenging task, which aims to transfer the knowledge learned from the labeled source domain to the unlabeled target domain. Recently, some clustering-based domain adaptation Re-ID methods have achieved great success. However, these methods ignore the inferior influence on pseudo-label prediction due to the different camera styles. The reliability of the pseudo-label plays a key role in domain adaptation Re-ID, while the different camera styles bring great challenges for pseudo-label prediction. To this end, a novel method is proposed, which bridges the gap of different cameras and extracts more discriminative features from an image. Specifically, an intra-to-intermechanism is introduced, in which samples from their own cameras are first grouped and then aligned at the class level across different cameras followed by our logical relation inference (LRI). Thanks to these strategies, the logical relationship between simple classes and hard classes is justified, preventing sample loss caused by discarding the hard samples. Furthermore, we also present a multiview information interaction (MvII) module that takes features of different images from the same pedestrian as patch tokens, obtaining the global consistency of a pedestrian that contributes to the discriminative feature extraction. Unlike the existing clustering-based methods, our method employs a two-stage framework that generates reliable pseudo-labels from the views of the intracamera and intercamera, respectively, to differentiate the camera styles, subsequently increasing its robustness. Extensive experiments on several benchmark datasets show that the proposed method outperforms a wide range of state-of-the-art methods. The source code has been released at https://github.com/lhf12278/LRIMV.

Personalized Federated Mutual Learning for Unsupervised Camera-aware Person Re-identification

Person re-identification (ReID) is essential for enhancing security and tracking in multi-camera surveillance systems. To achieve effective person re-identification (ReID) performance across diverse datasets, the Federated Unsupervised Person Re-identification via Camera-aware Clustering (FedUCA) approach has made strides in utilizing distributed datasets while ensuring data privacy. Nevertheless, its uniform model may not adequately cater to the specific characteristics of each participant’s data, given the diversity in camera perspectives and client-specific data variances, thus obtaining degraded results. To address this issue, we propose an advanced framework, Personalized Federated Dual-Model Learning for Camera-Aware Person Re-Identification (PerFedDual), which introduces knowledge-sharing techniques inherent to mutual learning for FedUCA with the camera-centric clustering process. PerFedDual supports a dual-model training approach that creates a cooperative learning space that improves both the global model and client-specific models by exchanging knowledge both ways. The methodology adopted is precisely adjusted to the distinctive data environment of each client, ensuring the protection of privacy while simultaneously enhancing the accuracy and flexibility of ReID models in diverse camera configurations. The empirical evaluation reveals that PerFedDual outperforms FedUCA and alternative federated learning strategies, highlighting the benefits of our technique that leverage collective intelligence to enhance unsupervised person re-identification.

A Multi-Scale Graph Attention-Based Transformer for Occluded Person Re-Identification

The objective of person re-identification (ReID) tasks is to match a specific individual across different times, locations, or camera viewpoints. The prevalent issue of occlusion in real-world scenarios affects image information, rendering the affected features unreliable. The difficulty and core challenge lie in how to effectively discern and extract visual features from human images under various complex conditions, including cluttered backgrounds, diverse postures, and the presence of occlusions. Some works have employed pose estimation or human key point detection to construct graph-structured information to counteract the effects of occlusions. However, this approach introduces new noise due to issues such as the invisibility of key points. Our proposed module, in contrast, does not require the use of additional feature extractors. Our module employs multi-scale graph attention for the reweighting of feature importance. This allows features to concentrate on areas genuinely pertinent to the re-identification task, thereby enhancing the model’s robustness against occlusions. To address these problems, a model that employs multi-scale graph attention to reweight the importance of features is proposed in this study, significantly enhancing the model’s robustness against occlusions. Our experimental results demonstrate that, compared to baseline models, the method proposed herein achieves a notable improvement in mAP on occluded datasets, with increases of 0.5%, 31.5%, and 12.3% in mAP scores.

AGS: Transferable adversarial attack for person re-identification by adaptive gradient similarity attack

Person re-identification (Re-ID) has achieved tremendous success in the fields of computer vision and security. However, Re-ID models are susceptible to adversarial examples, which are crafted by introducing imperceptible perturbations to benign person images. These adversarial examples often display high success rates in white-box settings but their transferability to black-box settings is relatively low. To improve the transferability of adversarial examples, this paper proposes a novel approach called the adaptive gradient similarity attack (AGS), which encompasses two essential components: gradient similarity and enhanced second moment. Specifically, a gradient similarity modulation is established to better harness the information in the neighborhood of the adjacent input, which can adaptively correct the update direction. Additionally, this paper formulates an enhanced second moment to adjust the update step at each iteration to address the issue of poor transferability. Extensive experiments confirm that AGS achieves the best performance compared with the state-of-the-art gradient-based attacks. Moreover, AGS is a versatile approach that can be integrated with existing input transformation attack techniques. Code is available at https://github.com/ZezeTao/similar_Attack4.

Data compensation and feature fusion for sketch based person retrieval

Sketch re-identification (Re-ID) aims to retrieve pedestrian photo in the gallery dataset by a query sketch drawn by professionals. The sketch Re-ID task has not been adequately studied because collecting such sketches is difficult and expensive. In addition, the significant modality difference between sketches and images makes extracting the discriminative feature information difficult. To address above issues, we introduce a novel sketch-style pedestrian dataset named Pseudo-Sketch dataset. Our proposed dataset maximizes the utilization of the existing person dataset resources and is freely available, thus effectively reducing the expenses associated with the training and deployment phases. Furthermore, to mitigate the modality gap between sketches and visible images, a cross-modal feature fusion network is proposed that incorporates information from each modality. Experiment results show that the proposed Pseudo-Sketch dataset can effectively complement the real sketch dataset, and the proposed network obtains competitive results than SOTA methods. The dataset will be released later.

Learning adaptive shift and task decoupling for discriminative one-step person search

Mainstream person search models aim to jointly optimize person detection and re-identification (ReID) in a one-step manner. Despite notable progress, existing one-step person search models still face three major challenges in extracting discriminative features: 1) incomplete feature extraction and fusion hinder the effective utilization of multiscale information, 2) the models struggle to capture critical features in complex occlusion scenarios, and 3) the optimization objectives of person detection and ReID are in conflict in the shared feature space. To address these issues, this study proposes a novel adaptive shift and task decoupling (ASTD) method that aims to enhance the accuracy and robustness of extracting discriminative features within the region of interest. In particular, we introduce a scale-aware transformer to handle scale/pose variations and occlusions. This transformer incorporates scale-aware modulation to enhance the utilization of multiscale information and adaptive shift augmentation to learn adaptation to occlusions dynamically. In addition, we design a task decoupling mechanism to hierarchically learn independent task representations using orthogonal loss to decouple two subtasks during training. Experimental results show that ASTD achieves state-of-the-art performance on the CUHK-SYSU and PRW datasets. Our code is accessible at https://github.com/zqx951102/ASTD.

Vehicle trajectory extraction and integration from multi-direction video on urban intersection

With the gradual maturity of computer vision technology, using intersection surveillance videos for vehicle trajectory extraction has become a popular method to analyze vehicle conflicts and safety in urban intersection. However, many intersection surveillance videos have blind spots, failing to fully cover the entire intersection. Vehicles may also obstruct each other, resulting in incomplete vehicle trajectories. The angle of surveillance videos can also lead to inaccurate trajectory extraction. In response to these challenges, this study proposes an vehicle trajectory extraction and integration framework using surveillance videos collected from four entrance of urban intersection. The framework first employs the improved YOLOv5s model to detect the positions of vehicles. Then, we proposed an object tracking model MS-SORT to extract the trajectories in each surveillance video. Subsequently, the trajectories of each surveillance video are mapped into the same coordinate system. Then the integration of trajectories is achieved using space–time information and re-identification (ReID) methods. The framework extracts and integrates trajectories from four intersection surveillance videos, obtaining trajectories with significantly broader temporal and spatial coverage compared to those obtained from any single direction of surveillance video. Our detection model improved mAP by 1.3 percentage points compared to the basic YOLOv5s, and our object tracking model improved MOTA and IDF1 by 2.6 and 2.1 percentage points compared to DeepSORT. The trojectory integration method achieved 94.7 % of F1-Score and RMSE of 0.51 m. The average length and number of the extracted trajectories has increased by at least 47.6 % and 24.2 % respectively compared to trajectories extracted from a single video.

NEFELI: A deep-learning detection and tracking pipeline for enhancing autonomy in advanced air mobility

Efficient detection and accurate collision estimation for non-cooperative aerial vehicles are crucial for the realization of fully autonomous aircraft and Advanced Air Mobility (AAM). This paper introduces NEFELI, a machine learning software utilizing optical sensors to detect and track non-cooperative aerial vehicles. NEFELI's detector employs an enhanced YOLOv5-large model, strengthened with a sliced inference step to enhance detection capabilities for distant, diminutive objects. Furthermore, NEFELI introduces several innovations in its tracking component. A key advancement lies in the creation and utilization of the first large-scale re-identification (Re-ID) dataset of aerial objects. This dataset is used to train the deep learning appearance (Re-ID) model of the tracking module and integrates appearance information into the detection and tracking pipeline, resulting in more robust and reliable tracking performance. Moreover, the tracking model combines the deep learning appearance model with a Kalman Filter-based motion model to address the challenge of precisely tracking distant aerial objects. Notably, an extensive comparative analysis that was conducted showed that NEFELI outperforms state-of-the-art detection and tracking models in terms of Higher Order Tracking Accuracy (HOTA) metric, ID switches, and Association Accuracy (AssA) by a wide margin. A crucial aspect of this work is NEFELI's software architecture design, which enables efficient implementation on a low SWaP (Size, Weight, and Power) edge Graphic Processing Unit (GPU). To further showcase NEFELI's generalization capabilities and edge implementation performance, real-world flight experiments with small UAVs were carried out. The experimental results demonstrate NEFELI's ability to detect and track small UAVs at distances of up to 145 m in real-time speed of 6.7 fps.

A review on video person re-identification based on deep learning

Person Re-Identification (ReID) is an essential technology for matching a person across non-overlapping cameras. It has attracted increasing attention in recent years due to its wide range of applications in various real-world scenarios such as security surveillance and criminal investigation. Different from other person ReID tasks, video-based ReID uses a video clip as the retrieval input, which can provide more promising ReID performance because that the video has rich information on appearance, motion cues and pose variations on temporal pipeline. Over the last few years, many deep learning-based video person ReID have been proposed to address various challenges, such as illumination variation, complex background, occlusion, etc. To provide a more comprehensive and readable review on existing video-based person ReID methods, we propose a novel taxonomy method that observes existing methods from four perspectives: data, algorithms, computing power, and applications. Specifically, we first introduce some popular datasets and evaluation criterion used for video-based person ReID. Next, from limited data and little annotation view, we introduce data augmentation and unsupervised learning ReID. From algorithm view, we focus on reviewing supervised methods including spatial feature learning, temporal feature learning and spatio-temporal feature learning, and further discuss and conduct a systematic comparison among these approaches. From complex open-world application view, we mainly summarized domain adaption and multimodal ReID. From insufficient GPU computing power view, we mainly discuss modality-agnostic unified large-scale ReID and their lightweighting. Finally, we provide a discussion of open problems and potential research directions for the community.

Health Data Re-Identification: Assessing Adversaries and Potential Harms.

Sharing biomedical data for research can help to improve disease understanding and support the development of preventive, diagnostic, and therapeutic methods. However, it is vital to balance the amount of data shared and the sharing mechanism chosen with the privacy protection provided. This requires a detailed understanding of potential adversaries who might attempt to re-identify data and the consequences of their actions. The aim of this paper is to present a comprehensive list of potential types of adversaries, motivations, and harms to targeted individuals. A group of 13 researchers performed a three-step process in a one-day workshop, involving the identification of adversaries, the categorization by motivation, and the deduction of potential harms. The group collected 28 suggestions and categorized them into six types, each associated with several of six distinct harms. The findings align with previous efforts in structuring threat actors and outcomes and we believe that they provide a robust foundation for evaluating re-identification risks and developing protection measures in health data sharing scenarios.

Deep learning large-scale drug discovery and repurposing.

Large-scale drug discovery and repurposing is challenging. Identifying the mechanism of action (MOA) is crucial, yet current approaches are costly and low-throughput. Here we present an approach for MOA identification by profiling changes in mitochondrial phenotypes. By temporally imaging mitochondrial morphology and membrane potential, we established a pipeline for monitoring time-resolved mitochondrial images, resulting in a dataset comprising 570,096 single-cell images of cells exposed to 1,068 United States Food and Drug Administration-approved drugs. A deep learning model named MitoReID, using a re-identification (ReID) framework and an Inflated 3D ResNet backbone, was developed. It achieved 76.32% Rank-1 and 65.92% mean average precision on the testing set and successfully identified the MOAs for six untrained drugs on the basis of mitochondrial phenotype. Furthermore, MitoReID identified cyclooxygenase-2 inhibition as the MOA of the natural compound epicatechin in tea, which was successfully validated in vitro. Our approach thus provides an automated and cost-effective alternative for target identification that could accelerate large-scale drug discovery and repurposing.

FRCE: Transformer-based feature reconstruction and cross-enhancement for occluded person re-identification

Occluded person re-identification (ReID) aims to capture the same person from different camera views with partial occlusion. In real-world scenarios, any object can become a random occluder, which is still an unsettled challenge for person ReID. Previous methods used auxiliary models to locate or reconstruct the occluded parts for ReID, but these methods are affected by external models or unable to model well, leaving the difficulty unresolved. Currently, Transformer has been widely applied to occluded person ReID tasks, but due to the lack of attention to local features, these methods still have limitations. Therefore, we propose a Feature Reconstruction and Cross-Enhancement (FRCE) network. Firstly, a Cross-level Feature Reconstruction Distiller (CFRD) is designed to improve the feature richness in the recognition scope and generate higher quality identity feature mapping by reconstructing the effective recognition region of the target. Secondly, we introduce a Multi-Dimensional Local Refinement (MDLR) module to deeply excavate local details in spatial and channel dimensions, seek subtle target cues and similarity relationships at local angles, and achieve more efficient ReID. Extensive experiments on five challenging occluded and holistic person ReID datasets demonstrate the effectiveness of our approach.

In-Depth Analysis of GAF-Net: Comparative Fusion Approaches in Video-Based Person Re-Identification

This study provides an in-depth analysis of GAF-Net, a novel model for video-based person re-identification (Re-ID) that matches individuals across different video sequences. GAF-Net combines appearance-based features with gait-based features derived from skeletal data, offering a new approach that diverges from traditional silhouette-based methods. We thoroughly examine each module of GAF-Net and explore various fusion methods at the both score and feature levels, extending beyond initial simple concatenation. Comprehensive evaluations on the iLIDS-VID and MARS datasets demonstrate GAF-Net’s effectiveness across scenarios. GAF-Net achieves state-of-the-art 93.2% rank-1 accuracy on iLIDS-VID’s long sequences, while MARS results (86.09% mAP, 89.78% rank-1) reveal challenges with shorter, variable sequences in complex real-world settings. We demonstrate that integrating skeleton-based gait features consistently improves Re-ID performance, particularly with long, more informative sequences. This research provides crucial insights into multi-modal feature integration in Re-ID tasks, laying a foundation for the advancement of multi-modal biometric systems for diverse computer vision applications.

Euclidean-Distance-Preserved Feature Reduction for efficient person re-identification

Person Re-identification (Re-ID) aims to match person images across non-overlapping cameras. The existing approaches formulate this task as fine-grained representation learning with deep neural networks, which involves extracting image features using a deep convolutional network, followed by mapping the features into a discriminative space through another smaller network, in order to make full use of all possible cues. However, recent Re-ID methods that strive to capture every cue and make the space more discriminative have resulted in longer features, ranging from 1024 to 14336, leading to higher time (distance computation) and space (feature storage) complexities. There are two potential solutions: reduction-after-training methods (such as Principal Component Analysis and Linear Discriminant Analysis) and reduction-during-training methods (such as 1 × 1 Convolution). The former utilizes a statistical approach aiming for a global optimum but lacking end-to-end optimization of large data and deep neural networks. The latter lacks theoretical guarantees and may be vulnerable to training noise such as dataset noise or initialization seed.To address these limitations, we propose a method called Euclidean-Distance-Preserving Feature Reduction (EDPFR) that combines the strengths of both reduction-after-training and reduction-during-training methods. EDPFR first formulates the feature reduction process as a matrix decomposition and derives a condition to preserve the Euclidean distance between features, thus ensuring accuracy in theory. Furthermore, the method integrates the matrix decomposition process into a deep neural network to enable end-to-end optimization and batch training, while maintaining the theoretical guarantee. The result of the EDPFR is a reduction of the feature dimensions from fa and fb to fa′ and fb′, while preserving their Euclidean distance, i.e.L2(fa,fb)=L2(fa′,fb′). In addition to its Euclidean-Distance-Preserving capability, EDPFR also features a novel feature-level distillation loss. One of the main challenges in knowledge distillation is dimension mismatch. While previous distillation losses, usually project the mismatched features to matched class-level, spatial-level, or similarity-level spaces, this can result in a loss of information and decrease the flexibility and efficiency of distillation. Our proposed feature-level distillation leverages the benefits of the Euclidean-Distance-Preserving property and performs distillation directly in the feature space, resulting in a more flexible and efficient approach. Extensive on three Re-ID datasets, Market-1501, DukeMTMC-reID and MSMT demonstrate the effectiveness of our proposed Euclidean-Distance-Preserving Feature Reduction.