Object Re-identification Research Articles

Quality of tracking methods with object reidentification

Objective. This paper presents a study of object tracking algorithms for analyzing sports competitions with a minimum permissible number of frames per second without critical loss of quality of operational systems. The purpose of the study is to determine the optimal tracking method and reidentification model with an increase in the time interval between frames. Method. The study used the Re-Identification MSMT17, Market1501, DukeMTMCreID datasets; specialized neural networks OsNet, ResNet and MobileNet. Tracking algorithms BotSort, Bytetrack, OcSort, DeepOcSort, StrongSort. The NOTA and IDF1 quality metrics were used. Result. The metrics of the experiments are generally quite low, which is due to the nonlinearity of athletes' movement, unlike pedestrians' movement, and a large number of changes in their posture. Conclusion. Depending on the degree of information compression, the optimal tracking methods when using pre-trained reidentification models are ByteTrack and OcSort.

Detection of Novel Objects without Fine-Tuning in Assembly Scenarios by Class-Agnostic Object Detection and Object Re-Identification

Object detection is a crucial capability of autonomous agents for human–robot collaboration, as it facilitates the identification of the current processing state. In industrial scenarios, it is uncommon to have comprehensive knowledge of all the objects involved in a given task. Furthermore, training during deployment is not a viable option. Consequently, there is a need for a detector that is able to adapt to novel objects during deployment without the necessity of retraining or fine-tuning on novel data. To achieve this, we propose to exploit the ability of discriminative embeddings learned by an object re-identification model to generalize to unknown categories described by a few shots. To do so, we extract object crops with a class-agnostic detector and then compare the object features with the prototypes of the novel objects. Moreover, we demonstrate that the embedding is also effective for predicting regions of interest, which narrows the search space of the class-agnostic detector and, consequently, increases processing speed. The effectiveness of our approach is evaluated in an assembly scenario, wherein the majority of objects belong to categories distinct from those present in the training datasets. Our experiments demonstrate that, in this scenario, our approach outperforms the current best few-shot object-detection approach DE-ViT, which also does not perform fine-tuning on novel data, in terms of both detection capability and inference speed.

MambaReID: Exploiting Vision Mamba for Multi-Modal Object Re-Identification.

Multi-modal object re-identification (ReID) is a challenging task that seeks to identify objects across different image modalities by leveraging their complementary information. Traditional CNN-based methods are constrained by limited receptive fields, whereas Transformer-based approaches are hindered by high computational demands and a lack of convolutional biases. To overcome these limitations, we propose a novel fusion framework named MambaReID, integrating the strengths of both architectures with the effective VMamba. Specifically, our MambaReID consists of three components: Three-Stage VMamba (TSV), Dense Mamba (DM), and Consistent VMamba Fusion (CVF). TSV efficiently captures global context information and local details with low computational complexity. DM enhances feature discriminability by fully integrating inter-modality information with shallow and deep features through dense connections. Additionally, with well-aligned multi-modal images, CVF provides more granular modal aggregation, thereby improving feature robustness. The MambaReID framework, with its innovative components, not only achieves superior performance in multi-modal object ReID tasks, but also does so with fewer parameters and lower computational costs. Our proposed MambaReID's effectiveness is validated by extensive experiments conducted on three multi-modal object ReID benchmarks.

Other tokens matter: Exploring global and local features of Vision Transformers for Object Re-Identification

Object Re-Identification (Re-ID) aims to identify and retrieve specific objects from images captured at different places and times. Recently, object Re-ID has achieved great success with the advances of Vision Transformers (ViT). However, the effects of the global–local relation have not been fully explored in Transformers for object Re-ID. In this work, we first explore the influence of global and local features of ViT and then further propose a novel Global–Local Transformer (GLTrans) for high-performance object Re-ID. We find that the features from last few layers of ViT already have a strong representational ability, and the global and local information can mutually enhance each other. Based on this fact, we propose a Global Aggregation Encoder (GAE) to utilize the class tokens of the last few Transformer layers and learn comprehensive global features effectively. Meanwhile, we propose the Local Multi-layer Fusion (LMF) which leverages both the global cues from GAE and multi-layer patch tokens to explore the discriminative local representations. Extensive experiments demonstrate that our proposed method achieves superior performance on four object Re-ID benchmarks. The code is available at https://github.com/AWangYQ/GLTrans.

Pairwise difference relational distillation for object re-identification

Most relationship knowledge distillation methods individually optimize pairwise similarities to improve the accuracy performance of a lightweight student network. However, this optimization approach may not be optimal for object re-identification (Re-ID) which prioritizes ranking. This is because it does not guarantee consistent ranking results between a lightweight student network and a large teacher network. For that, we propose a novel method called pairwise difference relational distillation (PDRD) for object Re-ID. First, we theoretically prove that minimizing the difference relationship between pairwise similarities resulting from student and teacher networks ensures consistent ranking results between the two networks. Second, based on this theoretical foundation, we combine non-linear activation functions on pairwise similarity discrepancies to create a non-linear pairwise difference rational knowledge loss function, which enhances knowledge transfer. Extensive experiments on four public datasets demonstrate that our method achieves state-of-the-art performance. For example, on Market-1501, using ResNet18 as a lightweight student network, our method acquires a rank-1 identification rate of 93.62%.

TOP-ReID: Multi-Spectral Object Re-identification with Token Permutation

Multi-spectral object Re-identification (ReID) aims to retrieve specific objects by leveraging complementary information from different image spectra. It delivers great advantages over traditional single-spectral ReID in complex visual environment. However, the significant distribution gap among different image spectra poses great challenges for effective multi-spectral feature representations. In addition, most of current Transformer-based ReID methods only utilize the global feature of class tokens to achieve the holistic retrieval, ignoring the local discriminative ones. To address the above issues, we step further to utilize all the tokens of Transformers and propose a cyclic token permutation framework for multi-spectral object ReID, dubbled TOP-ReID. More specifically, we first deploy a multi-stream deep network based on vision Transformers to preserve distinct information from different image spectra. Then, we propose a Token Permutation Module (TPM) for cyclic multi-spectral feature aggregation. It not only facilitates the spatial feature alignment across different image spectra, but also allows the class token of each spectrum to perceive the local details of other spectra. Meanwhile, we propose a Complementary Reconstruction Module (CRM), which introduces dense token-level reconstruction constraints to reduce the distribution gap across different image spectra. With the above modules, our proposed framework can generate more discriminative multi-spectral features for robust object ReID. Extensive experiments on three ReID benchmarks (i.e., RGBNT201, RGBNT100 and MSVR310) verify the effectiveness of our methods. The code is available at https://github.com/924973292/TOP-ReID.

Catalyst for Clustering-Based Unsupervised Object Re-identification: Feature Calibration

Clustering-based methods are emerging as a ubiquitous technology in unsupervised object Re-Identification (ReID), which alternate between pseudo-label generation and representation learning. Recent advances in this field mainly fall into two groups: pseudo-label correction and robust representation learning. Differently, in this work, we improve unsupervised object ReID from feature calibration, a completely different but complementary insight from the current approaches. Specifically, we propose to insert a conceptually simple yet empirically powerful Feature Calibration Module (FCM) before pseudo-label generation. In practice, FCM calibrates the features using a nonparametric graph attention network, enforcing similar instances to move together in the feature space while allowing dissimilar instances to separate. As a result, we can generate more reliable pseudo-labels using the calibrated features and further improve subsequent representation learning. FCM is simple, effective, parameter-free, training-free, plug-and-play, and can be considered as a catalyst, increasing the ’chemical reaction’ between pseudo-label generation and representation learning. Moreover, it maintains the efficiency of testing time with negligible impact on training time. In this paper, we insert FCM into a simple baseline. Experiments across different scenarios and benchmarks show that FCM consistently improves the baseline (e.g., 8.2% mAP gain on MSMT17), and achieves the new state-of-the-art results. Code is available at: https://github.com/lhf12278/FCM-ReID.

Distillation embedded absorbable pruning for fast object re-identification

Combining knowledge distillation (KD) and network pruning (NP) shows promise in learning a light network to accelerate object re-identification. However, KD requires an untrained student network to establish more critical connections in early epochs, but NP demands a well-trained student network to avoid destroying critical connections. This presents a dilemma, potentially leading to a collapse of the student network and harming object Re-ID performance. For that, we propose a distillation embedded absorbable pruning (DEAP) method. We design a pruner-convolution-pruner (PCP) unit to resolve the dilemma by loading NP’s sparse regularization on extra untrained pruners. Additionally, we propose an asymmetric relation knowledge distillation method. It readily transfers feature representation knowledge and asymmetric pairwise similarity knowledge without using additional adaptation modules. Finally, we apply re-parameterization to absorb pruners of PCP units to simplify student networks. Experiments demonstrate the superiority of DEAP, such as on the VeRi-776 dataset, with ResNet-101 as a teacher, DEAP saves 73.24% of model parameters and 71.98% of floating-point operations without sacrificing accuracy.

Dynamic Screening Strategy Based on Feature Graphs for UAV Object and Group Re-Identification

In contemporary times, owing to the swift advancement of Unmanned Aerial Vehicles (UAVs), there is enormous potential for the use of UAVs to ensure public safety. Most research on capturing images by UAVs mainly focuses on object detection and tracking tasks, but few studies have focused on the UAV object re-identification task. In addition, in the real-world scenarios, objects frequently get together in groups. Therefore, re-identifying UAV objects and groups poses a significant challenge. In this paper, a novel dynamic screening strategy based on feature graphs framework is proposed for UAV object and group re-identification. Specifically, the graph-based feature matching module presented aims to enhance the transmission of group contextual information by using adjacent feature nodes. Additionally, a dynamic screening strategy designed attempts to prune the feature nodes that are not identified as the same group to reduce the impact of noise (other group members but not belonging to this group). Extensive experiments have been conducted on the Road Group, DukeMTMC Group and CUHK-SYSU-Group datasets to validate our framework, revealing superior performance compared to most methods. The Rank-1 on CUHK-SYSU-Group, Road Group and DukeMTMC Group datasets reaches 71.8%, 86.4% and 57.8%, respectively. Meanwhile, our method performance is explored on the UAV datasets of PRAI-1581 and Aerial Image, the infrared datasets of SYSU-MM01 and CM-Group and the NIR dataset of RBG-NIR Scene dataset; the unexpected findings demonstrate the robustness and wide applicability of our method.

An enhanced Swin Transformer for soccer player reidentification

The re-identification (ReID) of objects in images is a widely studied topic in computer vision, with significant relevance to various applications. The ReID of players in broadcast videos of team sports is the focus of this study. We specifically focus on identifying the same player in images taken at any given moment during a game from various camera angles. This work varies from other person ReID apps since the same team wears very similar clothes, there are few samples for each identification, and image resolutions are low. One of the hardest parts of object ReID is robust feature representation extraction. Despite the great success of current convolutional neural network-based (CNN) methods, most studies only consider learning representations from images, neglecting long-range dependency. Transformer-based model studies are increasing and yielding encouraging results. Transformers still have trouble extracting features from small objects and visual cues. To address these issues, we enhanced the Swin Transformer with the levering of CNNs. We created a regional feature extraction Swin Transformer (RFES) backbone to increase local feature extraction and small-scale object feature extraction. We also use three loss functions to handle imbalanced data and highlight challenging situations. Re-ranking with k-reciprocal encoding was used in this study's retrieval phase, and its assessment findings were provided. Finally, we conducted experiments on the Market-1501 and SoccerNet-v3 ReID datasets. Experimental results show that the proposed re-ID method reaches rank-1 accuracy of 96.2% with mAP: 89.1 and rank-1 accuracy of 84.1% with mAP: 86.7 on the Market-1501 and SoccerNet-v3 datasets, respectively, outperforming the state-of-the-art approaches.

Expanding and Refining Hybrid Compressors for Efficient Object Re-Identification.

Recent object re-identification (Re-ID) methods gain high efficiency via lightweight student models trained by knowledge distillation (KD). However, the huge architectural difference between lightweight students and heavy teachers causes students to have difficulties in receiving and understanding teachers' knowledge, thus losing certain accuracy. To this end, we propose a refiner-expander-refiner (RER) structure to enlarge a student's representational capacity and prune the student's complexity. The expander is a multi-branch convolutional layer to expand the student's representational capacity to understand a teacher's knowledge comprehensively, which does not require any feature-dimensional adapter to avoid knowledge distortions. The two refiners are 1×1 convolutional layers to prune the input and output channels of the expander. In addition, in order to alleviate the competition accuracy-related and pruning-related gradients, we design a common consensus gradient resetting (CCGR) method, which discards unimportant channels according to the intersection of each sample's unimportant channel judgment. Finally, the trained RER can be simplified into a slim convolutional layer via re-parameterization to speed up inference. As a result, we propose an expanding and refining hybrid compressing (ERHC) method. Extensive experiments show that our ERHC has superior inference speed and accuracy, e.g., on the VeRi-776 dataset, given the ResNet101 as a teacher, ERHC saves 75.33% model parameters (MP) and 74.29% floating-point of operations (FLOPs) without sacrificing accuracy.

Pose-Guided Modality-Invariant Feature Alignment for Visible–Infrared Object Re-Identification

Pose-Guided Modality-Invariant Feature Alignment for Visible–Infrared Object Re-Identification

Linkage-Based Object Re-Identification via Graph Learning

Linkage-Based Object Re-Identification via Graph Learning

Attribute Descent: Simulating Object-Centric Datasets on the Content Level and Beyond.

This article aims to use graphic engines to simulate a large number of training data that have free annotations and possibly strongly resemble to real-world data. Between synthetic and real, a two-level domain gap exists, involving content level and appearance level. While the latter is concerned with appearance style, the former problem arises from a different mechanism, i.e., content mismatch in attributes such as camera viewpoint, object placement and lighting conditions. In contrast to the widely-studied appearance-level gap, the content-level discrepancy has not been broadly studied. To address the content-level misalignment, we propose an attribute descent approach that automatically optimizes engine attributes to enable synthetic data to approximate real-world data. We verify our method on object-centric tasks, wherein an object takes up a major portion of an image. In these tasks, the search space is relatively small, and the optimization of each attribute yields sufficiently obvious supervision signals. We collect a new synthetic asset VehicleX, and reformat and reuse existing the synthetic assets ObjectX and PersonX. Extensive experiments on image classification and object re-identification confirm that adapted synthetic data can be effectively used in three scenarios: training with synthetic data only, training data augmentation and numerically understanding dataset content.

Object Re-identification Based on Federated Incremental Subgradient Proximal Optimization

Object Re-identification Based on Federated Incremental Subgradient Proximal Optimization

DLAReID: double-layer attention network for object re-identification

DLAReID: double-layer attention network for object re-identification

Unsupervised Vehicle Re-Identification Method Based on Source-Free Knowledge Transfer

The unsupervised domain-adaptive vehicle re-identification approach aims to transfer knowledge from a labeled source domain to an unlabeled target domain; however, there are knowledge differences between the target domain and the source domain. To mitigate domain discrepancies, existing unsupervised domain-adaptive re-identification methods typically require access to source domain data to assist in retraining the target domain model. However, for security reasons, such as data privacy, data exchange between different domains is often infeasible in many scenarios. To this end, this paper proposes an unsupervised domain-adaptive vehicle re-identification method based on source-free knowledge transfer. First, by constructing a source-free domain knowledge migration module, the target domain is consistent with the source domain model output to train a generator to generate the “source-like samples”. Then, it can effectively reduce the model knowledge difference and improve the model’s generalization performance. In the experiment, two mainstream public datasets in this field, VeRi776 and VehicleID, are tested experimentally, and the obtained rank-k (the cumulative matching features) and mAP (the mean Average Precision) indicators are both improved, which are suitable for object re-identification tasks when data between domains cannot be interoperated.

Enhanced camera-based individual pig detection and tracking for smart pig farms

Negative social interactions are harmful for animal health and welfare. It is increasingly important to employ a continuous and effective monitoring system for detecting and tracking individual animals in large-scale farms. Such a system can provide timely alarms for farmers to intervene when damaging behavior occurs. Deep learning combined with camera-based monitoring is currently arising in agriculture. In this work, deep neural networks are employed to assist individual pig detection and tracking, which enables further analyzing behavior at the individual pig level. First, three state-of-the-art deep learning-based Multi-Object Tracking (MOT) methods are investigated, namely Joint Detection and Embedding (JDE), FairMOT, and YOLOv5s with DeepSORT. All models facilitate automated and continuous individual detection and tracking. Second, weighted-association algorithms are proposed for each MOT method, in order to optimize the object re-identification (re-ID), and improve the individual animal-tracking performance, especially for reducing the number of identity switches. The proposed weighted-association methods are evaluated on a large manually annotated pig dataset, and compared with the state-of-the-art methods. FairMOT with the proposed weighted association achieves the highest IDF1, the least number of identity switches, and the fastest execution rate. YOLOv5s with DeepSORT results in the highest MOTA and MOTP tracking metrics. These methods show high accuracy and robustness for individual pig tracking, and are promising candidates for continuous multi-object tracking for real use in commercial farms.

Non-Contrastive Nearest Neighbor Identity-Guided Method for Unsupervised Object Re-Identification

Recently, self-paced contrastive learning has emerged as a promising method for unsupervised object re-identification. These methods generate pseudo labels, store centroid features in the memory bank, and periodically update them. However, affected by the performance of the clustering method, within each cluster exists inevitably noisy instances, and self-paced contrastive learning usually requires a large number of negative samples from various classes, where false-negative samples give rise to the class collision issue. These lead to performing incorrect model optimization. In this paper, we propose a non-contrastive nearest neighbor identity-guided (NNNI) method to overcome these challenges. The advantage of NNNI is to provide the model with a highly accurate prior. Specifically, this method relies on the random identity sampler commonly used in re-identification tasks to provide the network with a regression target of the nearest neighbors of the same identity within a mini-batch. It encodes more and more information in an iterative process through a Siamese network with an exponential moving average to train high-quality representations. NNNI alleviates the negative effects of noise instances and corrects class collision issues during training. Extensive experiments show that our method is effective on unsupervised object re-identification and achieves state-of-the-art performance on three large-scale person re-identification datasets and one large-scale vehicle re-identification dataset, which is competitive with even supervised methods.

Reidentification of Objects From Aerial Photos With Hybrid Siamese Neural Networks

In this article, we consider the task of reidentifying the same object in different photos taken from separate positions and angles during aerial reconnaissance, which is a crucial task for the maintenance and surveillance of critical large-scale infrastructure. To effectively hybridize deep neural networks with available domain expertise for a given scenario, we propose a customized pipeline, wherein a domain-dependent object detector is trained to extract the assets (i.e., subcomponents) present on the objects, and a siamese neural network learns to reidentify the objects, exploiting both visual features (i.e., the image crops corresponding to the assets) and the graphs describing the relations among their constituting assets. We describe a real-world application concerning the reidentification of electric poles in the Italian energy grid, showing our pipeline to significantly outperform siamese networks trained from visual information alone. We also provide a series of ablation studies of our framework to underline the effect of including topological asset information in the pipeline, learnable positional embeddings in the graphs, and the effect of different types of graph neural networks on the final accuracy.