Human Re-identification Research Articles

Generative Adversarial-Based Ubiquitous Data Integration Model for Human Re-Identification

Generative Adversarial-Based Ubiquitous Data Integration Model for Human Re-Identification

Image-based human re-identification: Which covariates are actually (the most) important?

Human re-identification (re-ID) is nowadays among the most popular topics in computer vision, due to the increasing importance given to safety/security in modern societies. Being expected to sun in totally uncontrolled data acquisition settings (e.g., visual surveillance) automated re-ID not only depends on various factors that may occur in non-controlled data acquisition settings, but - most importantly - performance varies with respect to different subject features (e.g., gender, height, ethnicity, clothing, and action being performed), which may result in highly biased and undesirable automata. While many efforts have been putted in increase the robustness of identification to uncontrolled settings, a systematic assessment of the actual variations in performance with respect to each subject feature remains to be done. Accordingly, the contributions of this paper are threefold: 1) we report the correlation between the performance of three state-of-the-art re-ID models and different subject features; 2) we discuss the most concerning features and report valuable insights about the roles of the various features in re-ID performance, which can be used to develop more effective and unbiased re-ID systems; and 3) we leverage the concept of biometric menagerie, in order to identify the groups of individuals that typically fall into the most common menagerie families (e.g., goats, lambs, and wolves). Our findings not only contribute to a better understanding of the factors affecting re-ID performance, but also may offer practical guidance for researchers and practitioners concerned on human re-identification development.

Person re-identification from multiple surveillance cameras combining face and body feature matching

In this work, a new method for human re-identification (Re-ID) from multiple surveillance cameras is proposed. Unlike traditional methods in which only the body features are used for matching, our proposed method uses both body and facial features for the Re-ID process. This combination allows us to re-identify people in challenging conditions, such as people with a uniform or similar-looking outfit, a partial or occluded body, appearance changes, or illumination. The face and body feature extraction models were developed using the state-of-the-art deep neural backbones and the synthesis of existing datasets in the literature. The performance of the method was evaluated on a self-generated dataset, which contains images under challenging conditions. The evaluation results show that our method outperforms traditional methods, in which the accuracy Rank1 reaches 91.30% while the traditional ones have a Rank1 of only 86.96%. This newly introduced method can be used for many practical applications in security surveillance of buildings and offices where challenging conditions often appear.

Review on deep learning based techniques for person re-identification

In-depth study has recently been concentrated on human re-identification, which is a crucial component of automated video surveillance. Re-identification is the act of identifying someone in photos or videos acquired from other cameras after they have already been recognized in an image or video from one camera. Re-identification, which involves generating consistent labelling between several cameras, or even just one camera, is required to reconnect missing or interrupted tracks. In addition to surveillance, it may be used in forensics, multimedia, and robotics.Re-identification of the person is a difficult problem since their look fluctuates across many cameras with visual ambiguity and spatiotemporal uncertainty. These issues can be largely caused by inadequate video feeds or lowresolution photos that are full of unnecessary facts and prevent re-identification. The geographical or temporal restrictions of the challenge are difficult to capture. The computer vision research community has given the problem a lot of attention because of how widely used and valuable it is. In this article, we look at the issue of human re-identification and discuss some viable approaches.

Cross Vision-RF Gait Re-identification with Low-cost RGB-D Cameras and mmWave Radars

Human identification is a key requirement for many applications in everyday life, such as personalized services, automatic surveillance, continuous authentication, and contact tracing during pandemics, etc. This work studies the problem of cross-modal human re-identification (ReID), in response to the regular human movements across camera-allowed regions (e.g., streets) and camera-restricted regions (e.g., offices) deployed with heterogeneous sensors. By leveraging the emerging low-cost RGB-D cameras and mmWave radars, we propose the first-of-its-kind vision-RF system for cross-modal multi-person ReID at the same time. Firstly, to address the fundamental inter-modality discrepancy, we propose a novel signature synthesis algorithm based on the observed specular reflection model of a human body. Secondly, an effective cross-modal deep metric learning model is introduced to deal with interference caused by unsynchronized data across radars and cameras. Through extensive experiments in both indoor and outdoor environments, we demonstrate that our proposed system is able to achieve ~ 92.5% top-1 accuracy and ~ 97.5% top-5 accuracy out of 56 volunteers. We also show that our proposed system is able to robustly reidentify subjects even when multiple subjects are present in the sensors' field of view.

Human re-identification by local maximal occurrence of color and scale-Invariant Channel integrated statistical pattern

Human re-identification by local maximal occurrence of color and scale-Invariant Channel integrated statistical pattern

Novel Architecture for Human Re-Identification with a Two-Stream Neural Network and Attention Mechanism

This paper proposes a novel architecture that utilises an attention mechanism in conjunction with multi-stream convolutional neural networks (CNN) to obtain high accuracy in human re-identification (Reid). The proposed architecture consists of four blocks. First, the pre-processing block prepares the input data and feeds it into a spatial-temporal two-stream CNN (STC) with two fusion points that extract the spatial-temporal features. Next, the spatial-temporal attentional LSTM block (STA) automatically fine-tunes the extracted features and assigns weight to the more critical frames in the video sequence by using an attention mechanism. Extensive experiments on four of the most popular datasets support our architecture. Finally, the results are compared with the state of the art, which shows the superiority of this approach.

HMMN: Online metric learning for human re-identification via hard sample mining memory network

Effective metric learning is important in various applications, especially for re-identification. Compared with most existing re-identification methods which are not suitable for a real-time update mode, we exploit a novel memory-based strategy for mining hard triplets in online metric learning. This strategy is realized with an end-to-end deep learning based framework using an external memory pool. Our proposed pipeline is able to explicitly provide hard negative and positive samples to generate effective triplets, which are important for online metric learning due to the representative triplets could provide distinctive information to help understand the concept of metric learning between categories. In addition, a “focal-triplet loss” function is proposed to deal with the lack of positive or negative samples for one anchor, and the imbalance between easy and hard triplets for mini-batch. Experimental results on Market-1501, CUHK03 and DukeMTMC-reID demonstrate the effectiveness of our method, and its performance even outperforms that of some existing offline methods.

Pedestrian Reidentification Algorithm Based on Deconvolution Network Feature Extraction‐Multilayer Attention Mechanism Convolutional Neural Network

Pedestrian reidentification is a key technology in large‐scale distributed camera systems. It can quickly and efficiently detect and track target people in large‐scale distributed surveillance networks. The existing traditional pedestrian reidentification methods have problems such as low recognition accuracy, low calculation efficiency, and weak adaptive ability. Pedestrian reidentification algorithms based on deep learning have been widely used in the field of pedestrian reidentification due to their strong adaptive ability and high recognition accuracy. However, the pedestrian recognition method based on deep learning has the following problems: first, during the learning process of the deep learning model, the initial value of the convolution kernel is usually randomly assigned, which makes the model learning process easily fall into a local optimum. The second is that the model parameter learning method based on the gradient descent method exhibits gradient dispersion. The third is that the information transfer of pedestrian reidentification sequence images is not considered. In view of these issues, this paper first examines the feature map matrix from the original image through a deconvolution neural network, uses it as a convolution kernel, and then performs layer‐by‐layer convolution and pooling operations. Then, the second derivative information of the error function is directly obtained without calculating the Hessian matrix, and the momentum coefficient is used to improve the convergence of the backpropagation, thereby suppressing the gradient dispersion phenomenon. At the same time, to solve the problem of information transfer of pedestrian reidentification sequence images, this paper proposes a memory network model based on a multilayer attention mechanism, which uses the network to effectively store image visual information and pedestrian behavior information, respectively. It can solve the problem of information transmission. Based on the above ideas, this paper proposes a pedestrian reidentification algorithm based on deconvolution network feature extraction‐multilayer attention mechanism convolutional neural network. Experiments are performed on the related data sets using this algorithm and other major popular human reidentification algorithms. The results show that the pedestrian reidentification method proposed in this paper not only has strong adaptive ability but also has significantly improved average recognition accuracy and rank‐1 matching rate compared with other mainstream methods.

Giant Panda Identification.

The lack of automatic tools to identify giant panda makes it hard to keep track of and manage giant pandas in wildlife conservation missions. In this paper, we introduce a new Giant Panda Identification (GPID) task, which aims to identify each individual panda based on an image. Though related to the human re-identification and animal classification problem, GPID is extraordinarily challenging due to subtle visual differences between pandas and cluttered global information. In this paper, we propose a new benchmark dataset iPanda-50 for GPID. The iPanda-50 consists of 6, 874 images from 50 giant panda individuals, and is collected from panda streaming videos. We also introduce a new Feature-Fusion Network with Patch Detector (FFN-PD) for GPID. The proposed FFN-PD exploits the patch detector to detect discriminative local patches without using any part annotations or extra location sub-networks, and builds a hierarchical representation by fusing both global and local features to enhance the inter-layer patch feature interactions. Specifically, an attentional cross-channel pooling is embedded in the proposed FFN-PD to improve the identify-specific patch detectors. Experiments performed on the iPanda-50 datasets demonstrate the proposed FFN-PD significantly outperforms competing methods. Besides, experiments on other fine-grained recognition datasets (i.e., CUB-200-2011, Stanford Cars, and FGVC-Aircraft) demonstrate that the proposed FFN-PD outperforms existing state-of-the-art methods.

Deep Learning Technique Based Surveillance Video Analysis for the Store

ABSTRACT AI technology has developed so fast, and it has been applied to the commercial area. In order to predict the customer preference and adjust the placement of product or advertisement, etc., the intelligent surveillance video analysis technique has been proposed to gather the sufficient customer information and realize crowd counting and density map drawing. In this paper, a series of deep learning techniques are adopted to realize surveillance video analysis. This work covers different subproblems such as object detection, tracking and human identification. A skeleton recognition algorithm is adopted instead of object detection algorithm to overcome the severe occlusion problem. A multiple human tracking algorithm combing the human re-identification technology is adopted to realize the human tracking and counting. Finally, the density map and statistics information are obtained which can be used to evaluate and adjust the current business plan. A real store surveillance video is analyzed by the algorithm, and the results show the advantage of the algorithm.

A Multi-Modal Person Perception Framework for Socially Interactive Mobile Service Robots.

In order to meet the increasing demands of mobile service robot applications, a dedicated perception module is an essential requirement for the interaction with users in real-world scenarios. In particular, multi sensor fusion and human re-identification are recognized as active research fronts. Through this paper we contribute to the topic and present a modular detection and tracking system that models position and additional properties of persons in the surroundings of a mobile robot. The proposed system introduces a probability-based data association method that besides the position can incorporate face and color-based appearance features in order to realize a re-identification of persons when tracking gets interrupted. The system combines the results of various state-of-the-art image-based detection systems for person recognition, person identification and attribute estimation. This allows a stable estimate of a mobile robot’s user, even in complex, cluttered environments with long-lasting occlusions. In our benchmark, we introduce a new measure for tracking consistency and show the improvements when face and appearance-based re-identification are combined. The tracking system was applied in a real world application with a mobile rehabilitation assistant robot in a public hospital. The estimated states of persons are used for the user-centered navigation behaviors, e.g., guiding or approaching a person, but also for realizing a socially acceptable navigation in public environments.

Human Re-Identification with a Robot Thermal Camera Using Entropy-Based Sampling

Human re-identification is an important feature of domestic service robots, in particular for elderly monitoring and assistance, because it allows them to perform personalized tasks and human-robot interactions. However vision-based re- identification systems are subject to limitations due to human pose and poor lighting conditions. This paper presents a new re-identification method for service robots using thermal images. In robotic applications, as the number and size of thermal datasets is limited, it is hard to use approaches that require huge amount of training samples. We propose a re-identification system that can work using only a small amount of data. During training, we perform entropy-based sampling to obtain a thermal dictionary for each person. Then, a symbolic representation is produced by converting each video into sequences of dictionary elements. Finally, we train a classifier using this symbolic representation and geometric distribution within the new representation domain. The experiments are performed on a new thermal dataset for human re-identification, which includes various situations of human motion, poses and occlusion, and which is made publicly available for research purposes. The proposed approach has been tested on this dataset and its improvements over standard approaches have been demonstrated.

Multicamera Human Re-Identification based on Covariance Descriptor

Human re-identification is a crucial component of security and surveillance systems, smart environments and robots. In this paper a novel selective covariance-based method for human re-identification in video streams from multiple cameras is proposed. Our method, which includes human localization and human classification stages, is called selective covariance-based because before classifying the object using covariance descriptors (in this case the classes are the different people being re-identified) we extract (selection) specific regions, which are definitive for the class of objects we deal with (people). In our case, the region being extracted is the human head and shoulders. In the paper new feature functions for covariance region descriptors are developed and compared to basic feature functions, and a mask, filtering out the most of the background information from the region of interest, is proposed and evaluated. The use of the proposed feature functions and mask significantly improved the human classification performance (from 75% when using basic feature functions to 94.6% accuracy with the proposed method), while keeping computational complexity moderate.

Multi-shot human re-identification using a fast multi-scale video covariance descriptor

Multi-shot person re-identification in non-overlapping camera networks has become an important research area. In order to tackle this problem, a robust and adaptive person modeling against occlusion and uncontrolled changes is required. In this paper, a new Multi-Scale Video Covariance (MS-VC) unsupervised approach was proposed to efficiently describe human in motion and requires no labeled training data. The MS-VC approach is based on the computing of the features extracted from a new structured representation called Video Tree Structure (VIDTREST) of any video sequence and can efficiently describe behavioral biometrics and appearance of each human by combining spatio-temporal information in a fixed-size vector. The VIDTREST model captures moving regions of interest. In addition, it decreases the color weight which can discard background noise and resolve clothing similarity cases in the appearance models and other changes. Furthermore, a fast algorithm was suggested to decompose each sequence under VIDTREST, extract its multi-scale features and compute its covariance matrices in one pass. The proposed method was evaluated with CAVIAR and PRID datasets. Our experimental results outperform the recognition rates of the existing unsupervised approaches in-the-state-of-the-art.

An unsupervised meta-graph clustering based prototype-specific feature quantification for human re-identification in video surveillance

Human re-identification is an emerging research area in the field of visual surveillance. It refers to the task of associating the images of the persons captured by one camera (probe set) with the images captured by another camera (gallery set) at different locations in different time instances. The performance of these systems are often challenged by some factors—variation in articulated human pose and clothing, frequent occlusion with various objects, change in light illumination, and the cluttered background are to name a few. Besides, the ambiguity in recognition increases between individuals with similar appearance. In this paper, we present a novel framework for human re-identification that finds the correspondence image pair across non-overlapping camera views in the presence of the above challenging scenarios. The proposed framework handles the visual ambiguity having similar appearance by first segmenting the gallery instances into disjoint prototypes (groups), where each prototype represents the images with high commonality. Then, a weighing scheme is formulated that quantifies the selective and distinct information about the features concerning the level of contribution against each prototype. Finally, the prototype specific weights are utilized in the similarity measure and fused with the existing generic weighing to facilitates improvement in the re-identification. Exhaustive simulation on three benchmark datasets alongside the CMC (Cumulative Matching Characteristics) plot enumerate the efficacy of our proposed framework over the counterparts.

Fusion of appearance and motion-based sparse representations for multi-shot person re-identification

We present in this paper a multi-shot human re-identification system from video sequences based on interest points (IPs) matching. Our contribution is to take advantage of the complementary of person’s appearance and style of its movement that leads to a more robust description with respect to various complexity factors. The proposed contributions include person’s description and features matching. For person’s description, we propose to exploit a fusion strategy of two complementary features provided by appearance and motion description. We describe motion using spatiotemporal IPs, and use spatial IPs for describing the appearance. For feature matching, we use Sparse Representation (SR) as a local matching method between IPs. The fusion strategy is based on the weighted sum of matched IPs votes and then applying the rule of majority vote. This approach is evaluated on a large public dataset, PRID-2011. The experimental results show that our approach clearly outperforms current state-of-the-art.

Human Re-identification with Global and Local Siamese Convolution Neural Network

Human re-identification is an important task in surveillance system to determine whether the same human re-appears in multiple cameras with disjoint views. Mostly, appearance based approaches are used to perform human re-identification task because they are less constrained than biometric based approaches. Most of the research works apply hand-crafted feature extractors and then simple matching methods are used. However, designing a robust and stable feature requires expert knowledge and takes time to tune the features. In this paper, we propose a global and local structure of Siamese Convolution Neural Network which automatically extracts features from input images to perform human re-identification task. Besides, most of the current human re-identification task in single-shot approaches do not consider occlusion issue due to lack of tracking information. Therefore, we apply a decision fusion technique to combine global and local features for occlusion cases in single-shot approaches.

From the Lab to the Real World: Re-identification in an Airport Camera Network

Over the past ten years, human re-identification has received increased attention from the computer vision research community. However, for the most part, these research papers are divorced from the context of how such algorithms would be used in a real-world system. This paper describes the unique opportunity our group of academic researchers had to design and deploy a human re-identification system in a demanding real-world environment: a busy airport. The system had to be designed from the ground up, including robust modules for real-time human detection and tracking, a distributed, low-latency software architecture, and a front-end user interface designed for a specific scenario. None of these issues are typically addressed in re-identification research papers, but all are critical to an effective system that end users would actually be willing to adopt. We detail the challenges of the real-world airport environment, the computer vision algorithms underlying our human detection and re-identification algorithms, our robust software architecture, and the ground-truthing system required to provide the training and validation data for the algorithms. Our initial results show that despite the challenges and constraints of the airport environment, the proposed system achieves very good performance while operating in real time.

Human Re-identification with Global and Local Siamese Convolution Neural Network

Human Re-identification with Global and Local Siamese Convolution Neural Network