Features For Re-identification Research Articles

Boosting person ReID feature extraction via dynamic convolution

Extraction of discriminative features is crucial in person re-identification (ReID) which aims to match a query image of a person to her/his images, captured by different cameras. The conventional deep feature extraction methods on ReID employ CNNs with static convolutional kernels, where the kernel parameters are optimized during the training and remain constant in the inference. This approach limits the network's ability to model complex contents and decreases performance, particularly when dealing with occlusions or pose changes. In this work, to improve the performance without a significant increase in parameter size, we present a novel approach by utilizing a channel fusion-based dynamic convolution backbone network, which enables the kernels to change adaptively based on the input image, within two existing ReID network architectures. We replace the backbone network of two ReID methods to investigate the effect of dynamic convolution on both simple and complex networks. The first one called Baseline, is a simpler network with fewer layers, while the second, CaceNet represents a more complex architecture with higher performance. Evaluation results demonstrate that both of the designed dynamic networks improve identification accuracy compared to the static counterparts. A significant increase in accuracy is reported under occlusion tested on Occluded-DukeMTMC. Moreover, our approach achieves a performance comparable to the state-of-the-art on Market1501, DukeMTMC-reID, and CUHK03 with a limited computational load. These findings validate the effectiveness of the dynamic convolution in enhancing the person ReID networks and push the boundaries of performance in this domain.

A novel dual-pooling attention module for UAV vehicle re-identification

Vehicle re-identification (Re-ID) involves identifying the same vehicle captured by other cameras, given a vehicle image. It plays a crucial role in the development of safe cities and smart cities. With the rapid growth and implementation of unmanned aerial vehicles (UAVs) technology, vehicle Re-ID in UAV aerial photography scenes has garnered significant attention from researchers. However, due to the high altitude of UAVs, the shooting angle of vehicle images sometimes approximates vertical, resulting in fewer local features for Re-ID. Therefore, this paper proposes a novel dual-pooling attention (DpA) module, which achieves the extraction and enhancement of locally important information about vehicles from both channel and spatial dimensions by constructing two branches of channel-pooling attention (CpA) and spatial-pooling attention (SpA), and employing multiple pooling operations to enhance the attention to fine-grained information of vehicles. Specifically, the CpA module operates between the channels of the feature map and splices features by combining four pooling operations so that vehicle regions containing discriminative information are given greater attention. The SpA module uses the same pooling operations strategy to identify discriminative representations and merge vehicle features in image regions in a weighted manner. The feature information of both dimensions is finally fused and trained jointly using label smoothing cross-entropy loss and hard mining triplet loss, thus solving the problem of missing detail information due to the high height of UAV shots. The proposed method’s effectiveness is demonstrated through extensive experiments on the UAV-based vehicle datasets VeRi-UAV and VRU.

Camera-aware re-identification feature for multi-target multi-camera tracking

Multi-target multi-camera tracking is an important research topic in intelligent surveillance to achieve re-identification (Re-ID) of moving targets across different cameras. However, Re-ID faces significant challenges owing to variations in cameras/viewpoints, making it difficult to learn discriminative feature representations for targets captured from different cameras/viewpoints. A new pipeline is introduced by balancing the across-cameras sample feature space in camera-aware Re-ID framework. Specifically, different sampling strategies play a crucial role on the performance under the same baseline and identify the feature discrepancy between cameras/viewpoints as a crucial factor. This proposed sampling strategy is called camera equalization sampling to learn enhanced feature disparities, which balances cameras under identity rather than randomly sampling in a batch. The sampled images are combined with non visual cues(cameras position encoding) to reduce intra class variance. For the mechanism of camera equalization sampling, the improved camera centric loss function can better reduce the negative impact of individual samples and provide stable learnable features.Our proposed method consists of three modules. (i) Camera equalization (CE) ensures that each batch collects at least one image from each camera for every identity, thereby enabling robust features. (ii) Camera position embedding (CPE) mitigates the scene bias caused by different cameras/viewpoints by encoding camera indices. (iii) Camera Center triplet loss(CCL) based on CE improves higher robustness to outliers and noisy labels. We demonstrated the proposed method's effectiveness on popular datasets such as DukeMTMC-reID, Market-1501, and MSMT17, achieving state-of-the-art performance.

Cross-Camera Tracking Model and Method Based on Multi-Feature Fusion

Multi-camera video surveillance has been widely applied in crowd statistics and analysis in smart city scenarios. Most existing studies rely on appearance or motion features for cross-camera trajectory tracking, due to the changing asymmetric perspectives of multiple cameras and occlusions in crowded scenes, resulting in low accuracy and poor tracking performance. This paper proposes a tracking method that fuses appearance and motion features. An implicit social model is used to obtain motion features containing spatio-temporal information and social relations for trajectory prediction. The TransReID model is used to obtain appearance features for re-identification. Fused features are derived by integrating appearance features, spatio-temporal information and social relations. Based on the fused features, multi-round clustering is adopted to associate cross-camera objects. Exclusively employing robust pedestrian reidentification and trajectory prediction models, coupled with the real-time detector YOLOX, without any reliance on supplementary information, an IDF1 score of 70.64% is attained on typical datasets derived from AiCity2023.

VoxelTrack: Multi-Person 3D Human Pose Estimation and Tracking in the Wild.

We present VoxelTrack for multi-person 3D pose estimation and tracking from a few cameras which are separated by wide baselines. It employs a multi-branch network to jointly estimate 3D poses and re-identification (Re-ID) features for all people in the environment. In contrast to previous efforts which require to establish cross-view correspondence based on noisy 2D pose estimates, it directly estimates and tracks 3D poses from a 3D voxel-based representation constructed from multi-view images. We first discretize the 3D space by regular voxels and compute a feature vector for each voxel by averaging the body joint heatmaps that are inversely projected from all views. We estimate 3D poses from the voxel representation by predicting whether each voxel contains a particular body joint. Similarly, a Re-ID feature is computed for each voxel which is used to track the estimated 3D poses over time. The main advantage of the approach is that it avoids making any hard decisions based on individual images. The approach can robustly estimate and track 3D poses even when people are severely occluded in some cameras. It outperforms the state-of-the-art methods by a large margin on four public datasets including Shelf, Campus, Human3.6 M and CMU Panoptic.

Person re-identification in indoor videos by information fusion using Graph Convolutional Networks

Information contained in visual appearance and gait features can play an important role in designing computer vision assisted person re-identification (ReID) systems. Fusion of appearance and gait features has not yet been tested in this domain of research despite its great potential to solve some of the intriguing challenges faced by the research community due to viewpoint variations, illumination change, varying recording setups, etc. This paper proposes a new deep learning framework for person re-identification in videos. The framework referred to as Fused Graph Network (FGN-ReID) uses an information fusion strategy to deal with the aforementioned variations. We have used visual appearance and gait features independently and fused these features for re-identification of persons in close shot indoor video recordings. A feature similarity-based score-level fusion strategy has been adopted to fuse spatio-temporal and gait features using Graph Convolutional Networks (GCN). The ReID problem has been mapped to an inherent graph searching problem. In our proposed framework, appearance and gait features represent the nodes. And the relations between spatio-temporal segments denote the edges. Experiments using CASIA-B dataset reveal that the proposed method is more effective as compared to existing methods. We have improved the rank-1 accuracy by a margin of 8%–12% as compared to the baseline algorithms. The mean average precision (mAP) has improved significantly. The method has several applications including visual surveillance, biometric authentication, etc.

Impact of Kernel-PCA on Different Features for Person Re-Identification

In the driving field of computer vision, re-identification of an individual in a camera network is very challenging task. Existing methods mainly focus on strategies based on feature learning, which provide feature space and force the same person to be closer than separate individuals. These methods rely to a large extent on high-dimensional feature vectors to achieve high re-identification accuracy. Due to computational cost and efficiency, they are difficult to achieve in practical applications. We comprehensively analyzed the effect of kernel-based principal component analysis (PCA) on some existing high-dimensional person re-identification feature extractors to solve these problems. We initially formulate a kernel function on the extracted features and then apply PCA, significantly reducing the feature dimension. After that, we have proved that the kernel is very effective on different state-of-the-art high-dimensional feature descriptors. Finally, a thorough experimental evaluation of the reference person re-identification data set determined that the prediction method was significantly superior to more advanced techniques and computationally feasible.

FT-MDnet: A Deep-Frozen Transfer Learning Framework for Person Search

Matching manually cropped pedestrian images between queries and candidates, termed as person re-identification, has achieved significant progress with deep convolutional neural networks. Recently, a topic called ‘person search’ is proposed for the end-to-end application of re-identification technologies. It integrates object detection and person re-identification and aims to both locate and match pedestrians on a gallery of raw images. However, the design and implementation of such kind of hybrid network are difficult and computationally consuming in real practical situations. In order to fasten the design and ease the implementation, this paper proposes a deep-frozen transfer learning framework, named FT-MDnet, to extract re-identification features from a pre-trained detection network in two steps. First, using a channel-wise attention mechanism, a network called adaptive transfer learning network (ATLnet) is used to convert the sharing data of the underlying detection network to a re-identification feature map. Then, a multi-branch feature representation network called multiple descriptor network (MDnet) is proposed to extract re-identification features from the re-identification feature map. Our proposed solution has been verified on different types of mainstream detection networks, including YOLOv3, YOLOv4, Mask RCNN, and CenterNet. The experimental results show that our solution outperforms all other person search solutions by a large margin. It proves that the feature representations of detection networks are highly compatible with re-identification, and the proposed framework effectively extracts these features out. To encourage further research, we have made our framework open source.

Attention-Aligned Network for Person Re-Identification

Currently, attention mechanism receives enormous interest and has been extensively employed in the fields of Person Re-Identification (RE-ID), as it gains superior performance in learning discriminative feature representations. However, most off-the-shelf attention methods are still vulnerable to cross-view inconsistency problem. Besides, they merely exploit imprecise channel attention information and coarse-grained spatial attention of homogeneous scales, being insufficient to capture subtle differences among highly-similar individuals. To this end, we propose a novel Attention-Aligned Network (AANet) to address the aforementioned problems, in which a novel Omnibearing Foreground-aware Attention (OFA) module, Attention Alignment Mechanism (AAM) and an improved triplet loss with hard mining are proposed to learn foreground attentive features for RE-ID. Specifically, AANet firstly leverages OFA module to exploit heterogeneous-scale spatial attention and foreground-aware channel attention information. Then AANet further reduces the impact of background clutter and learns camera-invariant and background-invariant representations by virtue of AAM. Last but not least, an improved triplet loss with hard mining is also introduced to enhance the feature learning capability, which can jointly minimize the intra-class distance and maximize the inter-class distance in each triplet unit. Extensive experiments are carried out to demonstrate that the proposed method outperforms most current methods on three main RE-ID benchmarks.

Weighted bilinear coding over salient body parts for person re-identification

Deep convolutional neural networks (CNNs) have demonstrated dominant performance in person re-identification (Re-ID). Existing CNN based methods utilize global average pooling (GAP) to aggregate intermediate convolutional features for Re-ID. However, this strategy only considers the first-order statistics of local features and treats local features at different locations equally important, leading to sub-optimal feature representation. To deal with these issues, we propose a novel weighted bilinear coding (WBC) framework for local feature aggregation in CNN networks to pursue more representative and discriminative feature representations, which can adapt to other advanced methods and improve their performance. In specific, bilinear coding is used to encode the channel-wise feature correlations to capture richer feature interactions. Meanwhile, a weighting scheme is applied on the bilinear coding to adaptively adjust the weights of local features at different locations based on their importance in recognition, further improving the discriminability of feature aggregation. To handle the spatial misalignment issue, we use a salient part net to derive salient body parts, and apply the WBC model on each part. The final representation, formed by concatenating the WBC encoded features of each part, is both discriminative and resistant to spatial misalignment. Experiments on three benchmarks including Market-1501, DukeMTMC-reID and CUHK03 evidence the favorable performance of our method against other outstanding methods.

Improve Person Re-Identification With Part Awareness Learning

Person re-identification (ReID) aims to predict whether two images from different cameras belong to the same person. Due to low image quality and variance in view point and body pose, it remains a difficult task. To solve the task, a model is supposed to appropriately capture features that describe body regions for identification. With the simple intuition that explicitly incorporating ReID model with part awareness could be beneficial for learning a more discriminative feature space, we propose part segmentation as an assistant body perception task during the training of a ReID model. Specifically, we add a lightweight segmentation head to the backbone of ReID model during training, which is supervised with part labels. Note that our segmentation head is only introduced during training and that it does not change network input or the way of extracting ReID feature. Experiments show that part segmentation considerably improves the performance of ReID. Through quantitative and qualitative analyses, we further reveal that body part perception helps ReID model to capture a set of more diverse features from the body, with decreased similarity between part features and increased focus on different body regions. We experiment with various representative ReID models and achieve consistent improvement on several large-scale datasets including Market1501, CUHK03, DukeMTMC-reID and MSMT17. E.g . on MSMT17, our method increases Rank-1 Accuracy of GlobalPool-ResNet-50, PCB and MGN by 2.3%, 2.9% and 3.9%, respectively. Incorporated with MGN, our model achieves state-of-the-art performance, with Rank-1 Accuracy 95.8%, 78.8%, 90.0% and 84.0% on four datasets, respectively.

Person Re-Identification over Encrypted Outsourced Surveillance Videos

Person re-identification (Re-ID) has attracted extensive attention due to its potential to identify a person of interest from different surveillance videos. With the increasing amount of the surveillance videos, high computation and storage costs have posed a great challenge for the resource-constrained users. In recent years, the cloud storage services have made a large volume of video data outsourcing become possible. However, person Re-ID over outsourced surveillance videos could lead to a security threat, i.e., the privacy leakage of the innocent person in these videos. Therefore, we propose an ef F icient priv A cy-prese R ving pe R son Re- I D S cheme (FARRIS) over outsourced surveillance videos, which can ensure the privacy of the detected person while providing the person Re-ID service. Specifically, FARRIS exploits the convolutional neural network (CNN) and kernels based supervised hashing (KSH) to extract the efficient person Re-ID feature. Then, we design a secret sharing based Hamming distance computation protocol to allow cloud servers to calculate similarities among obfuscated feature indexes. Furthermore, a dual Merkle hash trees based verification is proposed, which permits users to validate the correctness of the matching results. The extensive experimental results and security analysis demonstrate that FARRIS can work efficiently, without compromising the privacy of the involved person.

Deep Deformable Patch Metric Learning for Person Re-Identification

The methodology for finding the same individual in a network of cameras must deal with significant changes in appearance caused by variations in illumination, viewing angle, and a person’s pose. Re-identification requires solving two fundamental problems: 1) determining a distance measure between features extracted from different cameras that cope with illumination changes (metric learning) and 2) ensuring that matched features refer to the same body part (correspondence). Most metric learning approaches focus on finding a robust distance measure between bounding box images, neglecting the alignment aspects. In this paper, we propose to learn appearance measures for patches that are combined using deformable models. Learning metrics for patches avoids strong dimensionality reduction, thus keeping more information. Additionally, we allow patches to change their locations, directly addressing the correspondence problem. As patches from different locations may share the same metric, our method effectively multiplies the amount of training data and allows patch metrics to be learned on the smaller amounts of labeled images. Different metric learning approaches (KISSME, XQDA, and LSSL) together with different deformable models (spring constraints and one-to-one matching constraints) are investigated and compared. For describing patches, we propose to learn a deep feature representation with convolutional neural networks, thus obtaining highly effective features for re-identification. We demonstrate that our approach significantly outperforms state-of-the-art methods on multiple data sets.

Survey on Colour, Texture and Shape Features for Person Re-Identification

Background: Re-identification is the pattern recognition problem which helps to identify the person in surveillance network. Feature plays an important role in person re-identification. Objective: The main objective of this paper is to analyze the properties of few existing features for person re-identification. Methods/Statistical Analysis: The features for re-identification is categorized into three groups namely colour, texture and shape. In this work various methods for extracting these features and their challenges are considered for survey and their properties, advantages disadvantages and applications are presented as a summary. Findings: From the analysis we found that colour and texture features outperform the shape feature. Application/Improvements: This analysis helps to understand the characteristics of the existing features and develop the new robust feature for improving the matching rate for person re-identification.

Sparse tree structured representation for re-identification

Finding substantial features for image representation is one of the keys to cope with the challenges of person re-identification given video streams. The important features for re-identification can be found by image saliency computation and subject appearance modeling. State-of-the-art models explore this direction by balancing between the global low-level features and the features from local patches. We proposed a novel nested patch tree for a tree structured feature representation, and the feature representation is used to match between a probe image and a gallery image to solve the re-identification problem. The feature representation is learned based on an unsupervised approach, which is different from the majority of the community when they work on finding similar subjects. Usually, the video streams for the same figure may have highly repetitive information, and the pseudo repetitiveness should be useful for a center-learning based method. We further improve the prediction accuracy by learning components by components for the same subject and working in the multi-color space. We evaluate the proposed method for person re-identification on the VIPeR and GRID datasets. The result shows that the proposed method is indeed superior to other state-of-the-art methods.

Need for a Temporal Feature in Re-identification

We demonstrate the effectiveness of space-time features incorporated in person re-identification (Re-ID). The space-time representation, improved dense trajectories, effectively extracts multiple features on a large number of trajectories. The experiments show that our proposal outperforms convolution neural network features on the iLIDS-VID dataset. Moreover, we customize the space-time feature for Re-ID. The numbers of temporal accumulations and visual words are highly tuned to classify individuals.

Cost-Effective Features for Reidentification in Camera Networks

Networks of smart cameras share large amounts of data to accomplish tasks such as re-identification. We propose a feature selection method that minimizes the data needed to represent the appearance of objects by learning the most appro- priate feature set for the task at hand (person re-identification). The computational cost for feature extraction and the cost for storing the feature descriptor are considered jointly with feature performance in order to select cost-effective good features. This selection allows us to improve inter-camera re-identification while reducing the bandwidth that is necessary to share data across the camera network. We also rank the selected features in the order of effectiveness for the task to enable a further reduction of the feature set by dropping the least effective features when application constraints require this adaptation. We compare the proposed approach with state-of-the-art methods on the i- LIDS and VIPeR datasets and show that the proposed approach considerably reduces network traffic due to inter-camera feature sharing while keeping the re-identification performance at an equivalent or better level compared with the state of the art.