CUHK03 Datasets Research Articles

A Novel Approach to Pedestrian Re-Identification in Low-Light and Zero-Shot Scenarios: Exploring Transposed Convolutional Reflectance Decoders

In recent years, pedestrian re-identification technology has made significant progress, with various neural network models performing well under normal conditions, such as good weather and adequate lighting. However, most research has overlooked extreme environments, such as rainy weather and nighttime. Additionally, the existing pedestrian re-identification datasets predominantly consist of well-lit images. Although some studies have started to address these issues by proposing methods for enhancing low-light images to restore their original features, the effectiveness of these approaches remains limited. We noted that a method based on Retinex theory designed a reflectance representation learning module aimed at restoring image features as much as possible. However, this method has so far only been applied in object detection networks. In response to this, we improved the method and applied it to pedestrian re-identification, proposing a transposed convolution reflectance decoder (TransConvRefDecoder) to better restore details in low-light images. Extensive experiments on the Market1501, CUHK03, and MSMT17 datasets demonstrated that our approach delivered superior performance.

ADT: Person re-identification based on efficient attention mechanism and single-channel dual-channel fusion with transformer features aggregation

In person re-identification(Re-ID) tasks, using only convolutional neural network(CNNs) makes it challenging to fully exploit the global key information in person images, while using only the Transformer structure fails to adequately capture the multi-scale local detail information in person images. Therefore, to address the more challenging person Re-ID tasks, effectively integrating CNN and Transformer into an algorithmic framework capable of fusing multi-dimensional features has become one of the current focal points of research. Due to the ability of the attention mechanism to suppress the extraction of non-salient features by convolutional neural networks, and the capability of the dual-channel multi-scale mechanism to facilitate the extraction of finer-grained deep information by the Transformer, we propose a network Based on Efficient Attention Mechanism and Single-channel Dual-channel Fusion with Transformer Features Aggregation (ADT). First, we introduce the Convolutional Block Attention Module(CBAM) and Spatial Reconstruction Unit (SRU) into the CNN network to construct a Multi-dimensional Attention Features Re-fusion Network(MCF). By weighting the local features extracted in the convolutional network, attention to person detail features in the model is enhanced. Then, we propose the Single and Dual-channel Integrated Transformer Features Aggregator(SDT) and overlap it in MCF. This integration is designed to more effectively aggregate the multi-scale high-dimensional global features from each layer of the CNN network. Experimental results on the Market1501, DukeMTMC, MSMT17, and CUHK03 datasets demonstrate that our model significantly enriches features representation compared to most methods.

Person re-identification by utilizing hierarchical spatial relation reasoning

Deep learning-based methods have become increasingly popular in person re-identification (re-id) as they enhance the discriminative features while suppressing extraneous or irrelevant details. However, these methods face a potential limitation in overlooking the relationships between semantic regions focusing solely on salient features or pixel-level matching information. As a result, the model performance becomes susceptible to adverse factors such as pose variations, misalignment, and background clutter. To overcome this issue, we propose a novel deep relational learning network (DRLNet) that learns relationship knowledge between semantic regions and “local–global” information to improve discriminative feature learning effectively. We manually establish the relation between local regions using a simple partition strategy that captures global structural information and improves relation-aware attention learning. Dividing images into uniform partitions allows each part-level feature to represent itself and the entire image area, facilitating the learning of more refined and discerning relationships. Our approach's efficacy is evaluated on three datasets, where experimental results exhibit its superiority. Significantly, our approach achieves a new state-of-the-art Rank-1 accuracy of 84.7%, 80.9%, and 83.5% on CUHK03 (Labeled), CUHK03 (Detected), and MSMT17 datasets, respectively, outperforming current state-of-the-art methods. In conclusion, our proposed DRLNet demonstrates its potential as a highly effective solution to overcoming the limitations of deep learning-based methods by providing superior means of learning discriminative features while incorporating relationships between semantic regions.

Multiple-local feature and attention fused person re-identification method

Person re-identification (ReID) is widely used in intelligent security, monitoring, criminal investigation and other fields. Aiming at the problems of local occlusion, scale misalignment and attitude change of pedestrian images in actual scenes, we propose a Multi-local Feature and Attention fused network (MFA) used for person re-identification task. Firstly, Channel Point Affinity Attention module (CPAA) is embedded in the backbone network to enhance the ability of the network for extracting local details. The feature map output from the backbone network is horizontally segmented into four local feature maps, and further four branch networks are concatenated to the feature map of the backbone network. The four local feature maps are used to guide the four branch networks to pay more attention on different areas of pedestrians through Global Local Aligned loss (GLA) function. Finally, the pedestrian feature vector containing multi-local features is obtained. The mAP of the network on Market-1501, DukeMTMC-reID,CUHK03 and MSMT17 datasets were 88.6%, 81.4%, 79.5% and 64.7%, and the Rank-1 was 95.8%, 90.1%, 81.2% and 84.1% respectively. In addition, the model also obtained 73.2% and 68.1% of Rank-1 on partial dataset Patial-REID and Patial-iLIDS, respectively. Recently, The MFA model parameter is 28.3M and the inference efficiency is approximately 32 fps to an image with a resulation of 256 × 128. Compared with other ReID methods, our proposed methods achieved a competitive performance for ReID task. The code was available at github:git@github.com:ISCLab-Bistu/MFA.git.

Semi-supervised LDA pedestrian re-identification algorithm based on K-nearest neighbor resampling

Person re-identification identify a specific person in surveillance network by similarity measurement between images of different camera views. However, existing metric learning based methods suffer from over-fitting problem. To solve this problem, a resampled linear discriminant analysis (LDA) method was proposed based on the statistical and topological characteristics of pedestrian images. This method utilized the k-nearest neighbours to form potential positive sample pairs. The potential positive pairs are used to improve the metric model and generalize the metric model to the test data. By minimizing the inter-class divergence of potential positive sample pairs, a semi-supervised re-sampling LDA person re-identification algorithm was established. It was then tested on the VIPeR, CUHK01 and Market 1501datasets. The results show that the proposed method achieves the best performance compared to some available methods. Especially, the proposed method outplays the best comparison method by 0.6% and 5.76% at rank-1 identification rate on the VIPeR and CUHK01 datasets respectively. At the same time, the improved LDA algorithm has improved the rank-1 identification accuracy of traditional LDA method by 9.36% and 32.11% on these two datasets respectively. Besides, the proposed method is limited to Market-1501 dataset when the test data is of large size.

Combining Human Parsing with Analytical Feature Extraction and Ranking Schemes for High-Generalization Person Reidentification

Person reidentification (re-ID) has been receiving increasing attention in recent years due to its importance for both science and society. Machine learning (particularly Deep Learning (DL)) has become the main re-ID tool that has allowed to achieve unprecedented accuracy levels on benchmark datasets. However, there is a known problem of poor generalization in respect of DL models. That is, models that are trained to achieve high accuracy on one dataset perform poorly on other ones and require re-training. In order to address this issue, we present a model without trainable parameters. This, in turn, results in a great potential for high generalization. This approach combines a fully analytical feature extraction and similarity ranking scheme with DL-based human parsing wherein human parsing is used to obtain the initial subregion classification. We show that such combination, to a high extent, eliminates the drawbacks of existing analytical methods. In addition, we use interpretable color and texture features that have human-readable similarity measures associated with them. In order to verify the proposed method we conduct experiments on Market1501 and CUHK03 datasets, thus achieving a competitive rank-1 accuracy comparable with that of DL models. Most importantly, we show that our method achieves 63.9% and 93.5% rank-1 cross-domain accuracy when applied to transfer learning tasks, while also being completely re-ID dataset agnostic. We also achieve a cross-domain mean average precision (mAP) that is higher than that of DL models in some experiments. Finally, we discuss the potential ways of adding new features to further improve the model. We also show the advantages of interpretable features for the purposes of constructing human-generated queries from verbal descriptions in order to conduct searches without a query image.

Diff attention: A novel attention scheme for person re-identification

The Person Re-ID has made great progress benefiting from the advance of deep neural networks and the attention mechanism in recent years. However, existing models and attentions for person Re-ID only focus on learning the robust features while neglecting the difference between features of pairs, which is the core of feature similarity matching tasks. To address this problem, we propose the Diff Attention idea and design Diff Attention Module to guide the network to learn a more discriminative attention map based on the difference of feature vectors. Taking the Diff Attention Module, we develop Diff Attention Framework to match various backbone Re-ID models and train them. To efficiently train Diff Attention Framework, we also propose two training strategies: Joint Training and Separate Training. Our framework and strategies have achieved excellent performance on Market-1501, CUHK03, and MSMT17 datasets. Our code will be made publicly available at https://github.com/linxin98/ReID-DiffAttention.

Self-Regulation Feature Network for Person Reidentification

The performance of person reidentification (ReID) has been significantly improved with the development of convolutional neural networks in the past few years. Most existing ReID methods focus on modifying the backbone network architecture to learn feature representations of persons; however, there are only a few methods that explore the potential performance of the network architecture to relearn features. In this article, we propose a self-regulation feature (SRF) scheme based on the network model, called SRFnet, to address the problem of the limited ability of the model for learning person features. The proposed SRFnet utilizes global branches to supervise local branches and deeply mine potential features of the network from the perspective of optimization to obtain more distinguished feature representations of persons. Different from methods that add an attention mechanism, our SRFnet can self-adjust feature learning by only adding loss functions between the local and global features of the network, and it does not require any additional convolution parameters. Extensive experiments on three public datasets show that the proposed SRFnet method achieves the results of mean average precision (mAP)/Rank-1 as 93.65%/95.28% on Market1501, 89.65%/91.79% on DuckMTMC-reID, 83.79%/82.00% on CUHK03 (labeled), and 81.92%/80.21% on CUHK03 (detected) datasets, respectively. Furthermore, experimental results on cross-domain datasets also demonstrate the effectiveness of the proposed SRFnet.

Deep Learning-Based SOLO Architecture for Re-Identification of Single Persons by Locations

Analyzing and judging of captured and retrieved images of the targets from the surveillance video cameras for person re-identification have been a herculean task for computer vision that is worth further research. Hence, re-identification of single persons by locations based on single objects by locations (SOLO) model is proposed in this paper. To achieve the re-identification goal, we based the training of the re-identification model on synchronized stochastic gradient descent (SGD). SOLO is capable of exploiting the contextual cues and segmenting individual persons by their motions. The proposed approach consists of the following steps: (1) reformulating the person instance segmentation as: (a) prediction of category and (b) mask generation tasks for each person instance, (2) dividing the input person image into a uniform grids, i.e., G×G grid cells in such a way that a grid cell can predict the category of the semantic and masks of the person instances provided the center of the person falls into the grid cell and (3) conducting person segmentation. Discriminating features of individual persons are obtained by extraction using convolution neural networks. On person re-identification Market-1501 dataset, SOLO model achieved mAP of 84.1% and 93.8% rank-1 identification rate, higher than what is achieved by other comparative algorithms such as PL-Net, SegHAN, Siamese, GoogLeNet, and M 3 L (IBN-Net50). On person re-identification CUHK03 dataset, SOLO model achieved mAP of 82.1 % and 90.1% rank-1 identification rate, higher than what is achieved by other comparative algorithms such as PL-Net, SegHAN, Siamese, GoogLeNet, and M 3 L (IBN-Net50). These results show that SOLO model achieves best results for person re-identification, indicating high effectiveness of the model. The research contributions are: (1) Application of synchronized stochastic gradient descent (SGD) to SOLO training for person re-identification and (2) Single objects by locations using semantic category branch and instance mask branch instead of detect-then-segment method, thereby converting person instance segmentation into a solvable problem of single-shot classification.

Similarity learning with deep CRF for person re-identification

The core of person re-identification (Re-ID) lies in robustly estimating similarities for each probe-gallery image pair. A common practice in existing works is to calculate the similarity of each image pair independently, ignoring relations between different probe-gallery pairs. In this paper, we present a deep learning conditional random field (Deep-CRF) graph to model group-wise similarities within a batch of images, and regard the Re-ID task as a CRF node labeling problem. Unlike the existing deep CRF based approach where the CRF inference is only involved in the training stage, our method intends to fully exploit the potential of CRF model, exhibiting inference consistency in both training and testing. Specifically, we design unary potentials for computing each probe-gallery similarity separately. To efficiently encode relationships between different probe-gallery pairs, pairwise potentials are built on an arbitrary node pair whose learning is achieved by a joint matching strategy using bidirectional LSTM. We pose the CRF inference as a RNN learning process, where unary and pairwise potentials are jointly optimized in an end-to-end manner. Extensive experiments on three large-scale person Re-ID datasets demonstrate the effectiveness of the proposed method. Our Deep-CRF achieves the best results compared with the previous graph-based deep learning approaches and substantially exceeds the existing deep CRF framework by 8% in Rank1 accuracy on CUHK03 dataset. It also behaves competitive among the current state-of-the-art methods.

SSMTReID-Net: Multi-Target Unsupervised Domain Adaptation for Person Re-Identification

• Person re-identification problem statement. • Novel unsupervised single-source multiple-target domain adaptation setup. • Loss function modeled around triplet loss. • Tackling catastrophic forgetting in source domain using domain adaptive methods. • Ensuring competitive performance across source and all target domains. The task of unsupervised domain adaptive (DA) person re-identification (reID) has gained prominence in recent years. Current reID works mostly leverage off-the-shelf single-target domain adaptive (STDA) techniques to reduce any domain gap between a labeled source dataset and an unlabeled target dataset in a single-source single-target training setup in their reID framework. However, directly extending such STDA techniques to multiple target domains suffers from two inherent drawbacks: a) the performance on the source domain deteriorates once the model is fine-tuned for a target domain, and b) posing the single-source multi-target (SSMT) setup as an independent single-source single-target case by blending all target domains to form a single target domain fails to utilize the complementary information present in the different target domains. Hence, to tackle this problem of unsupervised multi-target domain adaptation (MTDA), we propose a novel architecture called SSMTReID-Net. SSMTReID-Net employs the elastic weight consolidation (EWC) regularizer to ensure competitive performance on the source domain after adaptation, and the notion of information bottleneck (IB) to highlight domain invariant target features while suppressing any domain-irrelevant artifacts . Our model is end-to-end trainable, and extensive results on different single-source multi-target combinations of challenging person reID datasets like DukeMTMC-reID, Market-1501 and CUHK03 datasets confirm the superiority of SSMTReID-Net over the other baselines.

Multinetwork Collaborative Feature Learning for Semisupervised Person Reidentification.

Person reidentification (Re-ID) aims at matching images of the same identity captured from the disjoint camera views, which remains a very challenging problem due to the large cross-view appearance variations. In practice, the mainstream methods usually learn a discriminative feature representation using a deep neural network, which needs a large number of labeled samples in the training process. In this article, we design a simple yet effective multinetwork collaborative feature learning (MCFL) framework to alleviate the data annotation requirement for person Re-ID, which can confidently estimate the pseudolabels of unlabeled sample pairs and consistently learn the discriminative features of input images. To keep the precision of pseudolabels, we further build a novel self-paced collaborative regularizer to extensively exchange the weight information of unlabeled sample pairs between different networks. Once the pseudolabels are correctly estimated, we take the corresponding sample pairs into the training process, which is beneficial to learn more discriminative features for person Re-ID. Extensive experimental results on the Market1501, DukeMTMC, and CUHK03 data sets have shown that our method outperforms most of the state-of-the-art approaches.

Integration of Multi-Head Self-Attention and Convolution for Person Re-Identification.

Person re-identification is essential to intelligent video analytics, whose results affect downstream tasks such as behavior and event analysis. However, most existing models only consider the accuracy, rather than the computational complexity, which is also an aspect to consider in practical deployment. We note that self-attention is a powerful technique for representation learning. It can work with convolution to learn more discriminative feature representations for re-identification. We propose an improved multi-scale feature learning structure, DM-OSNet, with better performance than the original OSNet. Our DM-OSNet replaces the convolutional stream in OSNet with multi-head self-attention. To maintain model efficiency, we use double-layer multi-head self-attention to reduce the computational complexity of the original multi-head self-attention. The computational complexity is reduced from the original to . To further improve the model performance, we use SpCL to perform unsupervised pre-training on the large-scale unlabeled pedestrian dataset LUPerson. Finally, our DM-OSNet achieves an mAP of 87.36%, 78.26%, 72.96%, and 57.13% on the Market1501, DukeMTMC-reID, CUHK03, and MSMT17 datasets.

3D Skeleton and Two Streams Approach to Person Re-identification Using Optimized Region Matching

Person re-identification (Re-ID) is a challenging and arduous task due to non-overlapping views, complex background, and uncontrollable occlusion in video surveillance. An existing method for capturing pedestrian local region information is to divide person regions into horizontal stripes, which may lead to invalid features and erroneous learning. To solve this problem, this paper proposes a 3D skeleton and a two-stream approach to person Re-ID. The first stream of the method uses the 3D skeleton for background filtering and region segmentation. The second stream uses Siamese net to extract the global descriptor. The features of the two streams are fused to preserve the integrity of the person. An optimized region matching method for metric learning is designed. Extensive comparing experiments were conducted with state-of-the-art Re-ID methods on the Market-1501, CUHK03, and DukeMTMC-reID datasets. Experimental results show that the proposed method outperforms the existing methods in recognition accuracy.

Person Re‐identification Based on Feature Erasure and Diverse Feature Learning

Robust and comprehensive modelling of targets is the key to successful person re-identification. However, some useful information may be ignored, since CNNs tend to learn from the most distinctive feature region of the human body. In the present study, a multi-branch lightweight network structure that can enhance the ability of diverse feature retrieval is introduced. The proposed network consists of three branches. In the feature erasure branch, a drop block model is added to remove the horizontal region with the highest activation degree from feature vectors so as to allow the network to learn relatively low discrimination features. The global branch is used as an essential supplement to the feature erasure branch. A unified horizontal segmentation strategy is adopted in the local branch to avoid the influence of feature dislocation. Finally, diverse feature learning is achieved through the branch network structure. The proposed method can achieve state-of-the-art results on Market-1501, CUHK03 and DukeMTMC-Reid data sets, thereby demonstrating the effectiveness of the method.

Learning to rank: An intelligent system for person reidentification

Person reidentification (P-Reid) is an emerging research domain in the field of information retrieval that has gained exponential growth due to its wide range of applications in pedestrian tracking and crime prevention. The primary goal of P-Reid is to recognize a person based on previous appearance in multiview surveillance videos. The mainstream approaches apply fully supervised learning techniques that have poor scalability when deployed in complex real-world scenes, due to the overfitting problem, caused by the lack of sufficient annotated data. Further, optimization of these models for unlabeled data in real-time surveillance is a challenging task. To tackle these issues, an intelligent framework (LR-Net) is proposed, consisting of three tiers including fine-tuning (FT), siamese network (SN), and fusion strategy (FS). In the first tier, a deep learning model is fine-tuned for P-Reid that can handle both labeled and unlabeled data. Next, with the assistance of transfer learning, an SN is proposed that has a strong discriminative capability in terms of similarity between a pair of images. Finally, a learning-to-rank strategy is applied to optimize the learning capability of the SN, in which a triplet network extracts spatial-temporal patterns from unlabeled samples. In addition, a bayesian fusion model (BFM) is introduced to integrate the spatiotemporal and visual features, which yields 4.4%, 9.3%, and 0.8% improvement in the matching score over Market-1501, DukeMCMT-reID, and CUHK03 data sets, respectively. The conducted experiments and ablation study on the benchmark data sets empirically validate the proposed system, which obtains a high Rank-1 score as compared with the state-of-the-art (SOTA) methods.

Two‐level salient feature complementary network for person re‐identification

Given a query image from a camera, person re-identification (Re-ID) can retrieve the images of the same identity from a gallery, the images of which are captured by the other cameras. Therefore, person Re-ID has been widely used in the field of video surveillance. However, person Re-ID still suffers from a series of challenges, such as illumination changes, pose variations, and occlusions. Although the person Re-ID methods based on attention mechanism give an effective and feasible solution for the above challenges, attention mechanism may make a network focus too much on the most salient discriminative features and ignore other potential discriminative features. To solve this problem, we propose a two-level salient feature complementary network (TSFC-Net) to extract the most salient discriminative features and the secondary salient discriminative features of pedestrian images for person Re-ID. Specifically, TSFC-Net first extracts the most salient discriminative features of pedestrian images by embedding the spatial and channel attention modules in the backbone network, and then extracts the secondary salient discriminative features of pedestrian images by a secondary salient feature mining module (SSFM). Since the final features of pedestrian images fuse the most salient discriminative features and the secondary salient discriminative features, TSFC-Net can significantly improve the richness and discrimination capability of pedestrian representations. In addition, we conduct extensive experiments on the Market-1501, DukeMTMC-reID, and CUHK03 data sets, and the experimental results indicate that our TSFC-Net has a better performance compared with most of the state-of-the-art person Re-ID methods.

Semi-supervised LDA and Multi-distance Metric Learning for Person Re-identification

The problem of person re-identification has attracted a lot of attention in the field of machine vision. In practice, the non-overlapping sample images change drastically and the sample size is small, which makes the metric model overfitting phenomenon. In this paper, based on the k-NN and the sample normality property, we propose a resampling linear discriminant analysis (LDA) algorithm to suppress the local constraints caused by small samples, then train it to obtain the person re-identification metric learning model. A semi-supervised LDA algorithm with semi-supervised characteristics is developed by optimizing the inter-class scatter for weighting. A joint distance metric-based approach is also proposed to learn both the Mahalanobis distance and Euclidean distance. The improved algorithm is tested on the VIPeR and CUHK01 datasets, and the results indicate that, despite the change in the total number of training samples, the algorithm in this paper shows high recognition accuracy.

Progressive Transfer Learning.

Model fine-tuning is a widely used transfer learning approach in person Re-identification (ReID) applications, which fine-tuning a pre-trained feature extraction model into the target scenario instead of training a model from scratch. It is challenging due to the significant variations inside the target scenario, e.g., different camera viewpoint, illumination changes, and occlusion. These variations result in a gap between each mini-batch's distribution and the whole dataset's distribution when using mini-batch training. In this paper, we study model fine-tuning from the perspective of the aggregation and utilization of the dataset's global information when using mini-batch training. Specifically, we introduce a novel network structure called Batch-related Convolutional Cell (BConv-Cell), which progressively collects the dataset's global information into a latent state and uses it to rectify the extracted feature. Based on BConv-Cells, we further proposed the Progressive Transfer Learning (PTL) method to facilitate the model fine-tuning process by jointly optimizing BConv-Cells and the pre-trained ReID model. Empirical experiments show that our proposal can greatly improve the ReID model's performance on MSMT17, Market-1501, CUHK03, and DukeMTMC-reID datasets. Moreover, we extend our proposal to the general image classification task. The experiments in several image classification benchmark datasets demonstrate that our proposal can significantly improve baseline models' performance. The code has been released at https://github.com/ZJULearning/PTL.

Person Re-Identification Using LBPH and K-Reciprocal Encoding

Individual re-identification proof (Re-ID) targets recovering an individual of interest across different non-covering cameras. With the recent development of technological algorithm and expanding request of intelligence video observation, it has acquired fundamentally expanded interest in the computer vision. Person re-identification is characterized as the issue of perceiving an individual caught in different occasions and additionally areas more than a few nonoverlapping camera sees, thinking about a huge arrangement of up-and-comers. This issue influences essentially the administration of disseminated, multiview observation frameworks, in which subjects should be followed across better places, either deduced or on-the-fly when they travel through various areas. Re-identification proof is a truly challenging issue, as more often than not individuals can be caught by a few low goal cameras, under impediment conditions, severely (and not quite the same as view to see) enlightened, and in differing presents. In this context an encoding technique K-reciprocal results using the LBPH (Local Binary Patterns Histogram) Algorithm has been proposed. This work aims to obtain a genuine image match more prone to the probe in the K-corresponding closest neighbour. When probe image is given, complementary is encoded with the k-equal nearest neighbours into a vector to rerank using the Jaccard matrix. The obtained result is a combination of a Mahalanobis metric, the Jaccard metric, and the LBPH algorithm. The reranking activity needs no Human interference in producing an appropriate enormous scale dataset. The performance of rank-1 metrics 77.27, 61.90, 76.34 &.55.11 percentage is achieved for large-scale Market-1501, CUHK03, MARS, and PRW datasets. The other metrics used for person re-id named mAP recorded 65.01%, 61.21%, 68.21% and 38.13% for the same dataset in that order.