Non-overlapping Cameras Research Articles

Debiased hybrid contrastive learning with hard negative mining for unsupervised person re-identification

The goal of unsupervised person re-identification is to retrieve a specific person across several non-overlapping cameras without the aid of manual labeling information. In recent times, contrastive learning has found extensive application in undertaking the complexities of unsupervised person Re-ID. Nevertheless, prevailing approaches often ignore the bias in negative proxy sampling and the significance of hard negatives in contrastive learning. These limitations have constrained the performance of existing methods. To solve these issues, we introduce a Debiased Hybrid Contrastive Learning with Hard Negative Mining (DHCL-HNM) approach. Particularly, the proposed approach employs an instance-level memory bank to save the class prototypes for all training images. In each training epoch, the memory bank undergoes clustering, dividing the dataset into un-clustered outliers and clustered images with pseudo labels. Then, the debiasing of negative proxies and the hard negative mining are integrated into a hybrid contrastive learning process to enhance the intra-class similarity and the instance discrimination. The debiasing operation is realized during the sampling of negative proxies to reduce the negative effects of false negatives. In the meantime, the hard negative mining can guide the Re-ID model to concentrate on the hard negatives by reweighting negative proxies based on their similarities to the anchor sample. The efficiency of the proposed method in the realm of unsupervised person Re-ID is demonstrated through comprehensive experiment outcomes conducted on several datasets.

A review on video person re-identification based on deep learning

Person Re-Identification (ReID) is an essential technology for matching a person across non-overlapping cameras. It has attracted increasing attention in recent years due to its wide range of applications in various real-world scenarios such as security surveillance and criminal investigation. Different from other person ReID tasks, video-based ReID uses a video clip as the retrieval input, which can provide more promising ReID performance because that the video has rich information on appearance, motion cues and pose variations on temporal pipeline. Over the last few years, many deep learning-based video person ReID have been proposed to address various challenges, such as illumination variation, complex background, occlusion, etc. To provide a more comprehensive and readable review on existing video-based person ReID methods, we propose a novel taxonomy method that observes existing methods from four perspectives: data, algorithms, computing power, and applications. Specifically, we first introduce some popular datasets and evaluation criterion used for video-based person ReID. Next, from limited data and little annotation view, we introduce data augmentation and unsupervised learning ReID. From algorithm view, we focus on reviewing supervised methods including spatial feature learning, temporal feature learning and spatio-temporal feature learning, and further discuss and conduct a systematic comparison among these approaches. From complex open-world application view, we mainly summarized domain adaption and multimodal ReID. From insufficient GPU computing power view, we mainly discuss modality-agnostic unified large-scale ReID and their lightweighting. Finally, we provide a discussion of open problems and potential research directions for the community.

Euclidean-Distance-Preserved Feature Reduction for efficient person re-identification

Person Re-identification (Re-ID) aims to match person images across non-overlapping cameras. The existing approaches formulate this task as fine-grained representation learning with deep neural networks, which involves extracting image features using a deep convolutional network, followed by mapping the features into a discriminative space through another smaller network, in order to make full use of all possible cues. However, recent Re-ID methods that strive to capture every cue and make the space more discriminative have resulted in longer features, ranging from 1024 to 14336, leading to higher time (distance computation) and space (feature storage) complexities. There are two potential solutions: reduction-after-training methods (such as Principal Component Analysis and Linear Discriminant Analysis) and reduction-during-training methods (such as 1 × 1 Convolution). The former utilizes a statistical approach aiming for a global optimum but lacking end-to-end optimization of large data and deep neural networks. The latter lacks theoretical guarantees and may be vulnerable to training noise such as dataset noise or initialization seed.To address these limitations, we propose a method called Euclidean-Distance-Preserving Feature Reduction (EDPFR) that combines the strengths of both reduction-after-training and reduction-during-training methods. EDPFR first formulates the feature reduction process as a matrix decomposition and derives a condition to preserve the Euclidean distance between features, thus ensuring accuracy in theory. Furthermore, the method integrates the matrix decomposition process into a deep neural network to enable end-to-end optimization and batch training, while maintaining the theoretical guarantee. The result of the EDPFR is a reduction of the feature dimensions from fa and fb to fa′ and fb′, while preserving their Euclidean distance, i.e.L2(fa,fb)=L2(fa′,fb′). In addition to its Euclidean-Distance-Preserving capability, EDPFR also features a novel feature-level distillation loss. One of the main challenges in knowledge distillation is dimension mismatch. While previous distillation losses, usually project the mismatched features to matched class-level, spatial-level, or similarity-level spaces, this can result in a loss of information and decrease the flexibility and efficiency of distillation. Our proposed feature-level distillation leverages the benefits of the Euclidean-Distance-Preserving property and performs distillation directly in the feature space, resulting in a more flexible and efficient approach. Extensive on three Re-ID datasets, Market-1501, DukeMTMC-reID and MSMT demonstrate the effectiveness of our proposed Euclidean-Distance-Preserving Feature Reduction.

Model-aware privacy-preserving with start trigger method for person re-identification

Person Re-identification (ReID) could search for the same pedestrian from non-overlapping cameras, which completes the pedestrian location and search purpose. However, the process contains much sensitive pedestrian information and raises serious privacy problems. Conventional methods mainly remove identity-related features from pedestrian images to alleviate the privacy issue. Unfortunately, these strategies cause pedestrian information loss and poor data utility. In the paper, we propose a novel Model-Aware Privacy-Preserving with Start Trigger (MPST) method, which not only prevents personal identity for third parties but also achieves accurate pedestrian location for authorized organizations. The core idea is that authorized organizations obtain the start trigger to activate the ReID model that has the ability to search for target pedestrians, while third parties (i.e., hackers) cannot employ the ReID model to complete the pedestrian matching task without the start trigger. To this end, we develop a universal adversarial algorithm to learn an ingenious start trigger for the person ReID system. Afterwards, we further design a model-aware training strategy to facilitate our deep model to perceive issued instructions by synthetically utilizing the start trigger and original pedestrian samples. As a result, we successfully install an activate button to change the ReID model state for deciding whether the deep model has the ability to search for pedestrians or not. Abundant experiments demonstrate that the proposed MPST is effective for pedestrian identity anonymization. Our study achieves superior performance for authorized organizations and completes the privacy protection goal.

Deep video-based person re-identification (Deep Vid-ReID): comprehensive survey

Person re-identification (ReID) aims to find the person of interest across multiple non-overlapping cameras. It is considered an essential step for person tracking applications which is vital for surveillance. Person ReID could be investigated either using image-based or video-based. Video-based person ReID is considered more discriminating and realistic than image-based ReID due to the massive information extracted for each person. Different deep-learning techniques have been used for video-based ReID. In this survey, recently published articles are reviewed according to video-based ReID system pipeline: deep features learning, deep metric learning, and deep learning approaches. The deep feature learning approaches are categorized into spatial and temporal approaches, while deep metric learning is divided into metric and metric learning approaches. The deep learning approaches are differentiated into: supervised, unsupervised, weakly-supervised, and one-shot learning. A detailed analysis is held for the architectures of the state-of-the-art deep learning approaches. And their performance on four benchmark datasets is compared.

PLoPS: Localization-aware person search with prototypical normalization

Person search involves localizing and re-identifying persons of interest captured by multiple, non-overlapping cameras. Recent approaches to person search are typically built on object detection frameworks to learn joint person representations for detection and re-identification. To this end, the features extracted from pedestrian proposals are projected onto a unit hypersphere using L2 normalization, and positive proposals that sufficiently overlap with the ground truth are equally incorporated for training by exploiting an external lookup table (LUT). We have found that (1) using the L2 normalization technique, without considering feature distributions, can degenerate the discriminative power of person representations, (2) positive proposals often depict distracting details, such as background clutter and person overlaps, and (3) person features in the LUT are not often updated during training. To address these limitations, we propose a novel framework for person search, dubbed PLoPS, using a prototypical normalization layer, ProtoNorm, that calibrates features while considering the long-tail distribution across person IDs. PLoPS also entails a localization-aware learning scheme that prioritizes better-aligned proposals w.r.t the ground truth. We further introduce a LUT calibration technique to continuously adjust the person features in the LUT. Experimental results and analysis on standard benchmarks demonstrate the effectiveness of PLoPS.

Semantic-Oriented Feature Coupling Transformer for Vehicle Re-Identification in Intelligent Transportation System

More robust intelligent transportation systems including autonomous driving systems are in full flourish with the revolution of deep learning and the 6G wireless communication network. Vehicle Re-Identification, an indispensable branch of the intelligent transportation system, aims to retrieve specific vehicles captured from non-overlapping cameras. However, this is fundamentally challenging with the substantial inter-class similarity and substantial intra-class divergence. Embedding semantic information into vehicle re-identification task has gained ample interest, but the performance needs to be further improved. This work proposes a semantic-oriented feature coupling transformer (SOFCT) for vehicle re-identification as a solution. Specifically, the knowledge-based transformer is first embedded to model images with discriminative attributes. Second, original patches are divided into five semantic groups via semantics-patches coupling, and the feature extractions for different semantics are performed in the semantic feature extraction (SFE) transformer. Third, patch features are weighted via semantics-patches coupling in the patch feature weighting (PFW) transformer, the weighted feature is fed into subsequent encoders to excavate information. Finally, two groups of learnable semantics are embedded to automatically learn semantic features in the learnable semantic extraction (LSE) transformer. Experiments demonstrate that the proposed SOFCT method surpasses other state-of-the-arts with the mAP/Rank-1 of 80.7%/96.6%, 89.8%/84.5%, 86.4%/80.9%, and 84.3%/78.7% on VeRi776 and VehicleID.

A Generative Approach to Person Reidentification.

Person Re-identification is the task of recognizing comparable subjects across a network of nonoverlapping cameras. This is typically achieved by extracting from the source image a vector of characteristic features of the specific person captured by the camera. Learning a good set of robust, invariant and discriminative features is a complex task, often leveraging contrastive learning. In this article, we explore a different approach, learning the representation of an individual as the conditioning information required to generate images of the specific person starting from random noise. In this way we decouple the identity of the individual from any other information relative to a specific instance (pose, background, etc.), allowing interesting transformations from one identity to another. As generative models, we use the recent diffusion models that have already proven their sensibility to conditioning in many different contexts. The results presented in this article serve as a proof-of-concept. While our current performance on common benchmarks is lower than state-of-the-art techniques, the approach is intriguing and rich of innovative insights, suggesting a wide range of potential improvements along various lines of investigation.

Vehicle re-identification based on dimensional decoupling strategy and non-local relations.

Vehicle re-identification (Re-ID) is a challenging task that aims to recognize the same vehicle across different non-overlapping cameras. Existing attention mechanism-based methods for vehicle Re-ID often suffer from significant intra-class variation and inter-class variation due to various factors such as illumination, occlusion, viewpoint, etc. In this paper, we propose a novel network architecture for vehicle Re-ID, named Dimensional Decoupling Strategy and Non-local Relationship Network (DMNR-Net), which uses three modules to extract complementary features: global feature extraction module, non-local relationship capture module(NRCM), and dimensional decoupling module (DDS). The global feature extraction module captures complete and coarse-grained features from the whole image; the NRCM module extracts saliency information from feature maps in both spatial and channel dimensions; and the DDS decouples spatial and channel features into two branches to extract fine-grained features and focus on specific subspaces. We conduct extensive experiments on two popular publicly datasets, VeRi-776 and VehicleID, to evaluate the effectiveness of our method. The experimental results show that our DMNR-Net outperforms state-of-the-art methods by a large margin on both datasets.

Camera Topology Graph Guided Vehicle Re-Identification

Vehicle re-identification (Re-ID) aims to retrieve vehicles across non-overlapping cameras. Most studies consider representation learning from single appearance information of the vehicle images. Some works adopt the spatio-temporal information to remove unreasonable vehicles to refine the results in the testing phase. However, they ignore the potential topological relations among cameras under the Closed Circuit Television (CCTV) camera systems in the training phase, which usually leads to suboptimal results due to the high intra-identity variations. To handle this problem, we propose a novel vehicle re-identification framework, which explicitly models the camera topological relations of all input images to aggregate neighbor images and thus acquires camera-independent representations. Specifically, we first construct a Camera Topology Graph (CTG) to elucidate the topological relations among cameras. It takes different cameras as nodes and constructs edges from four levels of the camera system, position, orientation, and individual. Then, we introduce a Camera Topology-based Graph Convolutional Network (CT-GCN), which suppresses irrelevant neighbor images and learns different camera representation functions. Finally, we propose a topological cross-entropy loss to obtain the more discriminative vehicle representations. The whole network is trained in an end-to-end manner. Extensive experiments on three benchmark datasets demonstrate the effectiveness of the proposed method against state-of-the-art vehicle Re-ID methods.

Prototype-guided Attention Distillation for Discriminative Person Search.

Person search aims to localize a person of interest in a large image gallery captured by multiple, non-overlapping cameras. Prevalent unified methods have suffered from (1) noisy proposals with mis-detection and occlusion, and (2) large appearance variation within a class, which deteriorates the prototype-based metric learning. To address these problems, we introduce a Prototype-guided Attention Distillation, shortly PAD, which exploits a prototype (a typical representation of an identity) as a guidance to the attention module to consistently highlight identity-inherent regions across different poses. To utilize the knowledge encoded in prototypes for matching unseen IDs, PAD conducts attention distillation to guide student Re-ID queries by deeply mimicking attention maps from the prototype query. Additionally, to address large intra-class variation induced by pose or camera views, we extend PAD with multiple part prototypes representing consistent local regions across different instances. Furthermore, we exploit an adaptive momentum strategy for robust attention distillation in PAD to update more distinct prototypes. Extensive experiments conducted on CUHK-SYSU and PRW demonstrate the effectiveness of PAD, showcasing state-of-the-art performance. Moreover, our distilled attention surprisingly highlights distinguished multiple regions for person search.

A Versatile Framework for Multi-Scene Person Re-Identification.

Person Re-identification (ReID) has been extensively developed for a decade in order to learn the association of images of the same person across non-overlapping camera views. To overcome significant variations between images across camera views, mountains of variants of ReID models were developed for solving a number of challenges, such as resolution change, clothing change, occlusion, modality change, and so on. Despite the impressive performance of many ReID variants, these variants typically function distinctly and cannot be applied to other challenges. To our best knowledge, there is no versatile ReID model that can handle various ReID challenges at the same time. This work contributes to the first attempt at learning a versatile ReID model to solve such a problem. Our main idea is to form a two-stage prompt-based twin modeling framework called VersReID. Our VersReID firstly leverages the scene label to train a ReID Bank that contains abundant knowledge for handling various scenes, where several groups of scene-specific prompts are used to encode different scene-specific knowledge. In the second stage, we distill a V-Branch model with versatile prompts from the ReID Bank for adaptively solving the ReID of different scenes, eliminating the demand for scene labels during the inference stage. To facilitate training VersReID, we further introduce the multi-scene properties into self-supervised learning of ReID via a multi-scene prioris data augmentation (MPDA) strategy. Through extensive experiments, we demonstrate the success of learning an effective and versatile ReID model for handling ReID tasks under multi-scene conditions without manual assignment of scene labels in the inference stage, including general, low-resolution, clothing change, occlusion, and cross-modality scenes. Codes and models will be made publicly available.

Learning feature contexts by transformer and CNN hybrid deep network for weakly supervised person search

Person search is a computer vision task that aims to locate and re-identify specific pedestrians in images captured by non-overlapping cameras. However, the identity annotation in person search is labor-intensive, especially as the amount of data increases. Therefore, more and more studies consider training person search models using weakly supervised learning with only location annotations. The context information is useful to improve feature representations in the absence of pedestrian identity as supervision. Existing weakly supervised person search methods focus on logic-driven contexts while ignoring feature contexts. In this paper, we propose a hybrid deep network for weakly supervised person search. The hybrid architecture consists of a Transformer-based feature extraction network and a fully convolution-based region recognition head network. The purpose is to enable the model to learn feature contexts at different levels. In our network, hierarchical vision Transformers are used to extract features in order to obtain discriminative representations of scene images. The context-enhanced head network is designed to integrate different features for candidate pedestrians. In addition, a pedestrian proposal network is proposed to improve the quality of predicted proposals. Experiments are conducted on the CUHK-SYSU and the PRW benchmarks to evaluate the effectiveness of the proposed method.

Global-local attention with triplet loss and label smoothed crossentropy for person re-identification

Person re-identification (Person Re-ID) is a research direction on tracking and identifying people in surveillance camera systems with non-overlapping camera perspectives. Despite much research on this topic, there are still some practical problems that Person Re-ID has not yet solved, in reality, human objects can easily be obscured by obstructions such as other people, trees, luggage, umbrellas, signs, cars, motorbikes. In this paper, we propose a multibranch deep learning network architecture. In which one branch is for the representation of global features and two branches are for the representation of local features. Dividing the input image into small parts and changing the number of parts between the two branches helps the model to represent the features better. In addition, we add an attention module to the ResNet50 backbone that enhances important human characteristics and eliminates irrelevant information. To improve robustness, the model is trained by combining triplet loss and label smoothing cross-entropy loss (LSCE). Experiments are carried out on datasets Market1501, and duke multi-target multi-camera (DukeMTMC) datasets, our method achieved 96.04% rank-1, 88,11% mean average precision (mAP) on the Market1501 dataset, and 88.78% rank-1, 78,6% mAP on the DukeMTMC dataset. This method achieves performance better than some state-of-the-art methods.

An intelligent correlation learning system for person Re-identification

Person re-identification (PRe-id) aims to retrieve a target person's images captured across multiple/single non-overlapping cameras. To this end, significant techniques have been implemented that extract handcrafted, deep, part-based, and ensemble features to get more refined patterns for matching. But due to the limited focus on the multi-grained, view-consistent, and semantic correlation among different views, these approaches show low performance. Therefore, we present an attention-based multi-view correlation learning framework named (ACLS), which enables to learn multi-grained spatiotemporal features from individuals. The ACLS is mainly composed of three key steps: First, multi-view correlated visual features of pedestrians are extracted using a correlation vision transformer (CVIT) and a pyramid dilated network (PDN), followed by the person attention mechanism. Next, we employ convolutional long short-term memory (ConvLSTM) to extract spatiotemporal information from pedestrian images captured in different time frames. Finally, a deep fusion strategy is employed, which intelligently integrates features for the final matching task. Extensive evaluations are conducted over three famous datasets: Market-1501, DukeMCMT-reID, and CUHK03, results show tremendous ranking performance including 93.7%, 90.4%, and 85.7%. Thus, concluded remarks that our learning mechanism beats the current state-of-the-art (SOTA) methods.

Tracking people across ultra populated indoor spaces by matching unreliable Wi-Fi signals with disconnected video feeds

Tracking in dense indoor environments where several thousands of people move around is an extremely challenging problem. In this paper, we present a system — DenseTrack for tracking people in such environments. DenseTrack leverages data from the sensing modalities that are already present in these environments — Wi-Fi (from enterprise network deployments) and Video (from surveillance cameras). We combine Wi-Fi information with video data to overcome the individual errors induced by these modalities. More precisely, the locations derived from video are used to overcome the localization errors inherent in using Wi-Fi signals where precise Wi-Fi MAC IDs are used to locate the same devices across different levels and locations inside a building. Typically, localization in dense environments is a computationally expensive process when done with just video data; hence hard to scale. DenseTrack combines Wi-Fi and video data to improve the accuracy of tracking people that are represented by video objects from non-overlapping video feeds. DenseTrack is a scalable and device-agnostic solution as it does not require any app installation on user smartphones or modifications to the Wi-Fi system. At the core of DenseTrack, is our algorithm — inCremental Association of Independent Variables under Uncertainty (CAIVU). CAIVU is inspired by the multi-armed bandits model and is designed to handle various complex features of practical real-world environments. CAIVU matches the devices reported by an off-the-shelf Wi-Fi system using connectivity information to specific video blobs obtained through a computationally efficient analysis of video data. By exploiting data from heterogeneous sources, DenseTrack offers an effective real-time solution for individual tracking in heavily populated indoor environments. We emphasize that no other previous system targeted nor was validated in such dense indoor environments. We tested DenseTrack extensively using both simulated data, as well as two real-world validations using data from an extremely dense convention center and a moderately dense university environment. Our simulation results show that DenseTrack achieves an average video-to-Wi-Fi matching accuracy of up to 90% in dense environments with a matching latency of 60 s on the simulator. When tested in a real-world extremely dense environment with over 500,000 people moving between different non-overlapping camera feeds, DenseTrack achieved an average match accuracy of 83% to within a 2-people distance with an average latency of 48 s.

Extrinsic calibration of non-overlapping multi-camera system with high precision using circular encoded point ruler

In the field of visual metrology, achieving coordinate system unification among non-overlapping multi-camera systems in wide field-of-view scenarios poses significant challenges due to the absence of overlapping fields of view. This paper presents a novel calibration method for such multi-camera systems utilizing a ruler equipped with double-layer circular encoded landmarks. The fabrication of the ruler is simple, and its length covers the entire multi-camera measurement system, rendering the calibration process convenient. To calibrate the cameras, the ruler is positioned in front of the target cameras, and images of the ruler are captured by the multi-cameras. By establishing the initial transformation relationships between the non-overlapping cameras and the ruler's coordinate system, the proposed method efficiently determines the inter-camera transformations. Subsequently, by moving the ruler to different positions and capturing corresponding images, the distances between the encoded center points on the ruler and other encoded points, as captured by the non-overlapping cameras, are kept constant. An optimization-based objective function is formulated to minimize the three-dimensional distances between the measured encoded center points, thereby refining the initial transformation relationships. Combining the coplanarity constraint of the coding center points with the extrinsic results obtained through distance optimization, a secondary optimization was conducted to obtain the final transformation parameters. Experimental results demonstrate the simplicity and versatility of this method, which does not rely on complex constraints. Moreover, it exhibits high calibration accuracy, with maximum distance errors between ruler points and a fitting standard plane error both below 0.275 mm and 0.132 mm, respectively. Following multiple experimental iterations, the average disparity between the radius of the standard cylinder measured via line laser scanning and its true value was found to be 0.022 mm. The average maximum distance from the point cloud to the fitted cylinder surface was determined to be 0.281 mm, while the average distance, computed across multiple instances, was 0.048 mm. Consequently, the proposed approach proves effective for calibrating non-overlapping camera systems in various applications.

CMOT: A cross-modality transformer for RGB-D fusion in person re-identification with online learning capabilities

Person re-identification (reID) is a crucial aspect of intelligent surveillance systems, enabling the recognition of individuals across non-overlapping camera views. As compared to the RGB modality, RGB-D based reID has the potential to achieve robust and high performance by leveraging rich complementary features of both modalities, making it applicable in various occluded scenarios. However, current multimodal reID approaches often rely on late fusion or feature-level fusion techniques to combine multiple modalities, which limits their ability to proficiently exploit complementary visual and depth-related semantic information and capture complex interactions between unimodal features. To address these limitations, this paper introduces a cross-modality online transformer (CMOT) for RGB-D based person reID with online learning capabilities, which effectively utilizes both RGB and depth modalities for the extraction of spatio-temporal features and fuses across modalities. Our CMOT is composed of three main components: (1) a hypothesis generation module based on a person detector and tracker, (2) dual-stream feature extractors via convolutional neural networks (CNNs), (3) and a fusion transformer based on a self-attention-driven self-attentive modality refinement module (SAMR) and a cross-attention-driven cross-attentive modality interaction module (CAMI) to refine and fuse RGB-D complementary features extracted from the dual-stream, RGB and depth stream, CNNs. Additionally, we introduce a bottleneck enhancement feed-forward block to enhance the model’s representation capability within SAMR and CAMI, significantly reducing parameters and computations compared to the traditional feed-forward network. Moreover, we design the triplet loss function with distance measuring ability for incorporating online learning and finally CMOT works as a few-shot network for reID. Experimental results on three RGB-D person re-identification datasets, namely BIWI RGBD-ID, RobotPKU RGBD-ID, and TVPR2, demonstrate the effectiveness and robustness of CMOT.

Graph-Based Progressive Fusion Network for Multi-Modality Vehicle Re-Identification

Vehicle re-identification (Re-ID) is a critical task in intelligent transportation, aiming to match vehicle images of the same identity captured by non-overlapping cameras. However, it is difficult to achieve satisfactory results based on RGB images alone in darkness. Therefore, it is of great importance to consider multi-modality vehicle re-identification. Currently, the proposed works deal with different modality features through direct summation and fusion based on heat map, which however ignores the relationship between them. Meanwhile, there is a huge gap between the different modalities, which needs to be reduced. In this paper, to solve the above two problems, we propose a Graph-based Progressive Fusion Network (GPFNet) using a graph convolutional network to adaptively fuse multi-modality features in an end-to-end learning framework. GPFNet consists of a CNN feature extraction module (FEM), a GCN feature fusion module (FFM), and a loss function module (LFM). Firstly, in FEM, we employ a multi-stream network architecture to extract single-modality features and common-modality features and employ a random modality substitution module to extract mixed-modality features. Secondly, in FFM, we design an efficient graph structure to associate the features of different modalities and adopt a progressive two-stage strategy to fuse them. Finally, in LFM, we use GCN-aware multi-modality loss to constrain the features. For reducing modality differences and contributing better initial mixed-modality features to FFM, we propose random modality substitution as a data enhancement method for multi-modality datasets. Extensive experiments on multi-modality vehicle Re-ID datasets RGBN300 and RGBNT100 show that our model achieves state-of-the-art performance.

Deep Learning Based Occluded Person Re-Identification: A Survey

Occluded person re-identification (Re-ID) focuses on addressing the occlusion problem when retrieving the person of interest across non-overlapping cameras. With the increasing demand for intelligent video surveillance and the application of person Re-ID technology, the real-world occlusion problem draws considerable interest from researchers. Although a large number of occluded person Re-ID methods have been proposed, there are few surveys that focus on occlusion. To fill this gap and help boost future research, this article provides a systematic survey of occluded person Re-ID. In this work, we review recent deep learning based occluded person Re-ID research. First, we summarize the main issues caused by occlusion as four groups: position misalignment, scale misalignment, noisy information, and missing information. Second, we categorize existing methods into six solution groups: matching, image transformation, multi-scale features, attention mechanism, auxiliary information, and contextual recovery. We also discuss the characteristics of each approach, as well as the issues they address. Furthermore, we present the performance comparison of recent occluded person Re-ID methods on four public datasets: Partial-ReID, Partial-iLIDS, Occluded-ReID, and Occluded-DukeMTMC. We conclude the study with thoughts on promising future research directions.