Non-overlapping Camera Network Research Articles

A Generative Approach to Person Reidentification.

Person Re-identification is the task of recognizing comparable subjects across a network of nonoverlapping cameras. This is typically achieved by extracting from the source image a vector of characteristic features of the specific person captured by the camera. Learning a good set of robust, invariant and discriminative features is a complex task, often leveraging contrastive learning. In this article, we explore a different approach, learning the representation of an individual as the conditioning information required to generate images of the specific person starting from random noise. In this way we decouple the identity of the individual from any other information relative to a specific instance (pose, background, etc.), allowing interesting transformations from one identity to another. As generative models, we use the recent diffusion models that have already proven their sensibility to conditioning in many different contexts. The results presented in this article serve as a proof-of-concept. While our current performance on common benchmarks is lower than state-of-the-art techniques, the approach is intriguing and rich of innovative insights, suggesting a wide range of potential improvements along various lines of investigation.

Joint learning with diverse knowledge for re-identification

Re-identification (re-ID) aims to search the target images of pedestrians or vehicles with the same identity in the non-overlapping camera network. Some factors, such as, cluttered background, different illumination and occlusion, which increases the difficulty of the re-identification task. To fulfill the re-ID task, this paper presents a re-ID method with diverse knowledge, which contains a multi-branch structure to extract various detailed features of samples from multiple perspectives. Furthermore, the attention mechanism is introduced to suppress the influence of background information. Moreover, to improve the discriminative ability, a joint learning strategy is presented to refine the network and align the output of multiple branches. To verify the effectiveness of the proposed method, corresponding experiments have been conducted on multiple public datasets and experimental results demonstrate that our method is effective and achieves the competitive results with existing methods on CUHK03 and Market1501.

Cross-camera Trajectories Help Person Retrieval in a Camera Network.

We are concerned with retrieving a query person from multiple videos captured by a non-overlapping camera network. Existing methods often rely on purely visual matching or consider temporal constraints but ignore the spatial information of the camera network. To address this issue, we propose a pedestrian retrieval framework based on cross-camera trajectory generation that integrates both temporal and spatial information. To obtain pedestrian trajectories, we propose a novel cross-camera spatio-temporal model that integrates pedestrians' walking habits and the path layout between cameras to form a joint probability distribution. Such a cross-camera spatio-temporal model can be specified using sparsely sampled pedestrian data. Based on the spatio-temporal model, cross-camera trajectories can be extracted by the conditional random field model and further optimised by restricted non-negative matrix factorization. Finally, a trajectory re-ranking technique is proposed to improve the pedestrian retrieval results. To verify the effectiveness of our method, we construct the first cross-camera pedestrian trajectory dataset, the Person Trajectory Dataset, in real surveillance scenarios. Extensive experiments verify the effectiveness and robustness of the proposed method.

Re-Identification in Urban Scenarios: A Review of Tools and Methods

With the widespread use of surveillance image cameras and enhanced awareness of public security, objects, and persons Re-Identification (ReID), the task of recognizing objects in non-overlapping camera networks has attracted particular attention in computer vision and pattern recognition communities. Given an image or video of an object-of-interest (query), object identification aims to identify the object from images or video feed taken from different cameras. After many years of great effort, object ReID remains a notably challenging task. The main reason is that an object’s appearance may dramatically change across camera views due to significant variations in illumination, poses or viewpoints, or even cluttered backgrounds. With the advent of Deep Neural Networks (DNN), there have been many proposals for different network architectures achieving high-performance levels. With the aim of identifying the most promising methods for ReID for future robust implementations, a review study is presented, mainly focusing on the person and multi-object ReID and auxiliary methods for image enhancement. Such methods are crucial for robust object ReID, while highlighting limitations of the identified methods. This is a very active field, evidenced by the dates of the publications found. However, most works use data from very different datasets and genres, which presents an obstacle to wide generalized DNN model training and usage. Although the model’s performance has achieved satisfactory results on particular datasets, a particular trend was observed in the use of 3D Convolutional Neural Networks (CNN), attention mechanisms to capture object-relevant features, and generative adversarial training to overcome data limitations. However, there is still room for improvement, namely in using images from urban scenarios among anonymized images to comply with public privacy legislation. The main challenges that remain in the ReID field, and prospects for future research directions towards ReID in dense urban scenarios, are also discussed.

Learning Deep RGBT Representations for Robust Person Re-identification

Person re-identification (Re-ID) is the scientific task of finding specific person images of a person in a non-overlapping camera networks, and has achieved many breakthroughs recently. However, it remains very challenging in adverse environmental conditions, especially in dark areas or at nighttime due to the imaging limitations of a single visible light source. To handle this problem, we propose a novel deep red green blue (RGB)-thermal (RGBT) representation learning framework for a single modality RGB person Re-ID. Due to the lack of thermal data in prevalent RGB Re-ID datasets, we propose to use the generative adversarial network to translate labeled RGB images of person to thermal infrared ones, trained on existing RGBT datasets. The labeled RGB images and the synthetic thermal images make up a labeled RGBT training set, and we propose a cross-modal attention network to learn effective RGBT representations for person Re-ID in day and night by leveraging the complementary advantages of RGB and thermal modalities. Extensive experiments on Market 1501, CUHK03 and DukeMTMC-reID datasets demonstrate the effectiveness of our method, which achieves state-of-the-art performance on all above person Re-ID datasets.

Face recognition and tracking for security surveillance

According to the National Crime Records Bureau, 63,407 children have gone missing in the year 2016, which makes almost 174 children go missing in India every day, out of which only 50% are ever found again. This brings up a need for an efficient solution to trace missing children. The proposed solution uses machine assistance during these search activities with face recognition technologies and can be used for essential development of applications which use CCTV footage across a camera network to identify the person lost. In our solution we use One Shot learning for face recognition to identify stranded people in places such as mass gatherings. The same technology can be used for identification of criminals across the city. The paper also talks about the tracking of people across a network of multiple non-overlapping cameras, with a feature of shifting the target tovehicle, if the target gets into one. The experimentation is performed using mobile cameras and thus, helps in monitoring actions of criminals and finding their hideout.

A unified framework for automated person re-indentification

Along with the strong development of camera networks, a video analysis system has been become more and more popular and has been applied in various practical applications. In this paper, we focus on person re-identification (person ReID) task that is a crucial step of video analysis systems. The purpose of person ReID is to associate multiple images of a given person when moving in a non-overlapping camera network. Many efforts have been made to person ReID. However, most of studies on person ReID only deal with well-alignment bounding boxes which are detected manually and considered as the perfect inputs for person ReID. In fact, when building a fully automated person ReID system the quality of the two previous steps that are person detection and tracking may have a strong effect on the person ReID performance. The contribution of this paper are two-folds. First, a unified framework for person ReID based on deep learning models is proposed. In this framework, the coupling of a deep neural network for person detection and a deep-learning-based tracking method is used. Besides, features extracted from an improved ResNet architecture are proposed for person representation to achieve a higher ReID accuracy. Second, our self-built dataset is introduced and employed for evaluation of all three steps in the fully automated person ReID framework.

A unified framework for automated person re-identification

Along with the strong development of camera networks, a video analysis system has been become more and more popular and has been applied in various practical applications. In this paper, we focus on person re-identification (person ReID) task that is a crucial step of video analysis systems. The purpose of person ReID is to associate multiple images of a given person when moving in a non-overlapping camera network. Many efforts have been made to person ReID. However, most of studies on person ReID only deal with well-alignment bounding boxes which are detected manually and considered as the perfect inputs for person ReID. In fact, when building a fully automated person ReID system the quality of the two previous steps that are person detection and tracking may have a strong effect on the person ReID performance. The contribution of this paper are two-folds. First, a unified framework for person ReID based on deep learning models is proposed. In this framework, the coupling of a deep neural network for person detection and a deep-learning-based tracking method is used. Besides, features extracted from an improved ResNet architecture are proposed for person representation to achieve a higher ReID accuracy. Second, our self-built dataset is introduced and employed for evaluation of all three steps in the fully automated person ReID framework.

Projection with Gaussian Kernel for Person Re-Identification

Person re-identification (ReID), the task of associating the detected images of a person as he/she moves in a non-overlapping camera network, is faced with different challenges including variations in the illumination, view-point and occlusion. To ensure good performance for person ReID, the state-of-the-art methods have leveraged different characteristics for person representation. As a result, a high-dimensional feature vector is extracted and used in the person matching step. However, each feature plays a specific role for distinguishing one person from the others. This paper proposes a method for person ReID wherein the correspondences between descriptors in high-dimensional space can be achieved via explicit feature selection and appropriate projection with a Gaussian kernel. The advantage of the proposed method is that it allows simultaneous matching of the descriptors while preserving the local geometry of the manifolds. Different experiments were conducted on both single-shot and multi-shot person ReID datasets. The experimental results demonstrates that the proposed method outperforms the state-of-the-art methods.

Training approach using the shallow model and hard triplet mining for person re‐identification

Multi-target tracking in a non-overlapping camera network is an active research field, and one of the important problems in it is the person re-identification problem. In this study, the authors propose an approach to improve the performance of the backbone model in the person re-identification. Their approach focuses on training a fusion model with a shallow model and making hard triplets with relationship matrices quickly and efficiently. The proposed approach is simple, but it improves the performance of the backbone. In addition, the hard triplet mining in their process is much faster than the conventional approach. Experimental evaluation shows that the proposed approach can improve the performances of the backbone model. The proposed approach improves rank-1 and mean average precision (mAP) performance by more than 12.54 and 15.44%, respectively, over the backbone models in the Market1501 and DukeMTMC-reID dataset. The approach also achieves competitive performances compared with state-of-the-art approaches.

MSMCT: Multi-State Multi-Camera Tracker

Visual tracking of multiple persons simultaneously is an important tool for group behaviour analysis. In this paper, we demonstrate that multi-target tracking in a network of non-overlapping cameras can be formulated in a framework, where the association among all given target hypotheses both within and between cameras is performed simultaneously. Our approach helps to overcome the fragility of multi-camera-based tracking, where the performance relies on the single-camera tracking results obtained at input level. In particular, we formulate an estimation of the target states as a multi-state graph optimization problem, in which the likelihood of each target hypothesis belonging to different identities is modeled. In addition, we learn the target-specific model to improve the similarity measure among targets based on the appearance cues. We also handle the occluded targets when there is no reliable evidence for the target’s presence and each target trajectory is expected to be fragmented into multiple tracks. An iterative procedure is proposed to solve the optimization problem, resulting in final trajectories that reveal the true states of the targets. The performance of the proposed approach has been extensively evaluated on challenging multi-camera non-overlapping tracking data sets, in which many difficulties, such as occlusion, viewpoint, and illumination variation, are present. The results of systematic experiments conducted on a large set of sequences show that the proposed approach outperforms several state-of-the-art trackers.

Person Re-Identification Based on SAD and Histogram in a Non-Overlapping Camera Network

In this paper, we present the conception and implementation of a system for person re-identification in a camera network, based on the appearance. This system aims to build an online database that contains the history of every person that enters the field of view of the cameras. This system is able to associate an identifier to each detected person, which keeps this identifier in the same camera and in other cameras even if he or she disappears and then appears again. Our system comprises a moving objects detection step that is implemented using the Mixture of Gaussians method and a proposed difference method, to improve the detection results. It also comprises a tracking step that is implemented using the sum of absolute differences algorithm. The re-identification stage is realized using three steps: the tracking for the temporal association, the histogram and the new smart exploitation of the interpolation technique for comparison. The global system was tested on a real data set collected by three cameras. The experimental results show that our approach gives very satisfactory results.

Video Person Re-Identification for Wide Area Tracking Based on Recurrent Neural Networks

In this paper, we propose a video-based person re-identification system for wide area tracking based on a recurrent neural network architecture. Given short video sequences of a person, generated by a tracking algorithm, our video re-identification algorithm links these tracklets in full trajectories across a network of non-overlapping cameras in an open-world scenario. In our system, features are first extracted from each frame using a convolutional neural network. Then, a recurrent layer combines information across time-steps. The features from all time-steps are finally combined using temporal pooling to give an overall appearance feature for the complete sequence. Our system is trained to perform re-identification using a Siamese network architecture. Experiments are conducted on the iLIDS-VID and PRID-2011 video re-identification data sets as well as in the DukeMTMC multi-camera tracking data set.

Cross-View Discriminative Feature Learning for Person Re-Identification

The viewpoint variability across a network of non-overlapping cameras is a challenging problem affecting person re-identification performance. In this paper, we investigate how to mitigate the cross-view ambiguity by learning highly discriminative deep features under the supervision of a novel loss function. The proposed objective is made up of two terms, the steering meta center term and the enhancing centers dispersion term, that steer the training process to mining effective intra-class and inter-class relationships in the feature domain of the identities. The effect of our loss supervision is to generate a more expanded feature space of compact classes where the overall level of the inter-identities' interference is reduced. Compared with the existing metric learning techniques, this approach has the advantage of achieving a better optimization because it jointly learns the embedding and the metric contextually. Our technique, by dismissing side-sources of performance gain, proves to enhance the CNN invariance to viewpoint without incurring increased training complexity (like in Siamese or triplet networks) and outperforms many related state-of-the-art techniques on Market-1501 and CUHK03.

Multi‐stage ranking approach for fast person re‐identification

One of the goals of person re-identification systems is to support video-surveillance operators and forensic investigators to find an individual of interest in videos acquired by a network of non-overlapping cameras. This is attained by sorting images of previously observed individuals for decreasing values of their similarity with a given probe individual. Existing appearance descriptors, together with their similarity measures, are mostly aimed at improving ranking quality. The authors address instead the issue of processing time, which is also relevant in practical applications involving interaction with human operators. They show how a trade-off between processing time and ranking quality, for any given descriptor, can be achieved through a multi-stage ranking approach inspired by multi-stage classification approaches, which they adapt to the re-identification ranking task. The authors analytically model the processing time of multi-stage system and discuss the corresponding accuracy, and derive from these results practical design guidelines. They then empirically evaluate their approach on three benchmark data sets and four state-of-the-art descriptors.

Equidistance constrained metric learning for person re-identification

Person re-identification (re-id), aiming to search a specific person among a non-overlapping camera network, has attracted plenty of interest in recent years. This task is highly challenging, especially when there exists only single image per person in the database. In this paper, we present an algorithm for learning a Mahalanobis distance for person re-identification. Our method has two distinctive features: (1) to obtain the best separability of the training data, we first minimize the intra-class distances to the most extent by forcing intra-class distances to be zero, and (2) to promote the generalization ability of the learned metric, we then maximize the minimum margin between different classes. Inspired by the simple geometric intuition that a regular simplex maximizes its minimum side length, provided the sum of all side length is fixed, our method, called EquiDistance constrained Metric Learning (EquiDML), applies least-square regression technique to map images of the same person to the same vertex of a regular simplex, and images of different persons to different vertices of a regular simplex. Consequently, under the learned metric, images of the same class are collapsed to a single point, while images of different classes are transformed to be equidistant. This simple motivation is further formulated as a convex optimization problem, solved by the projected gradient descent method and proved to be very effective in person re-identification task. Although it is fairly simple, our method outperforms the state-of-the-art methods on CUHK01, CUHK03, Market1501 and DukeMTMC-reID datasets, and achieves very competitive performance on the widely used VIPeR dataset.

Multi-shot human re-identification using a fast multi-scale video covariance descriptor

Multi-shot person re-identification in non-overlapping camera networks has become an important research area. In order to tackle this problem, a robust and adaptive person modeling against occlusion and uncontrolled changes is required. In this paper, a new Multi-Scale Video Covariance (MS-VC) unsupervised approach was proposed to efficiently describe human in motion and requires no labeled training data. The MS-VC approach is based on the computing of the features extracted from a new structured representation called Video Tree Structure (VIDTREST) of any video sequence and can efficiently describe behavioral biometrics and appearance of each human by combining spatio-temporal information in a fixed-size vector. The VIDTREST model captures moving regions of interest. In addition, it decreases the color weight which can discard background noise and resolve clothing similarity cases in the appearance models and other changes. Furthermore, a fast algorithm was suggested to decompose each sequence under VIDTREST, extract its multi-scale features and compute its covariance matrices in one pass. The proposed method was evaluated with CAVIAR and PRID datasets. Our experimental results outperform the recognition rates of the existing unsupervised approaches in-the-state-of-the-art.

Ring-push metric learning for person reidentification

Person reidentification (re-id) has been widely studied because of its extensive use in video surveillance and forensics applications. It aims to search a specific person among a nonoverlapping camera network, which is highly challenging due to large variations in the cluttered background, human pose, and camera viewpoint. We present a metric learning algorithm for learning a Mahalanobis distance for re-id. Generally speaking, there exist two forces in the conventional metric learning process, one pulling force that pulls points of the same class closer and the other pushing force that pushes points of different classes as far apart as possible. We argue that, when only a limited number of training data are given, forcing interclass distances to be as large as possible may drive the metric to overfit the uninformative part of the images, such as noises and backgrounds. To alleviate overfitting, we propose the ring-push metric learning algorithm. Different from other metric learning methods that only punish too small interclass distances, in the proposed method, both too small and too large inter-class distances are punished. By introducing the generalized logistic function as the loss, we formulate the ring-push metric learning as a convex optimization problem and utilize the projected gradient descent method to solve it. The experimental results on four public datasets demonstrate the effectiveness of the proposed algorithm.

Fully-automated person re-identification in multi-camera surveillance system with a robust kernel descriptor and effective shadow removal method

In this paper, a fully-automated person Re-ID (Re-identification) system is proposed for real scenarios of human tracking in non-overlapping camera network. The system includes two phases of human detection and Re-ID. The human ROIs (Regions of Interest) are extracted from human detection phase and then feature extraction is done on these ROIs in order to build human descriptor for Re-ID. Unlike other approaches which deal with manually-cropped human ROIs for person Re-ID, in this system, the person identity is determined based on the human ROIs extracted automatically by a combined method of human detection. Two main contributions are proposed on both phases of human detection and Re-ID in order to enhance the performance of person Re-ID system. First, an effective shadow removal method based on score fusion of density matching is proposed to get better human detection results. Second, a robust KDES (Kernel DEScriptor) is extracted from human ROI for person classification. Additionally, a new person Re-ID dataset is built in real surveillance scenarios from multiple cameras. The experiments on benchmark datasets and our own dataset show that the person Re-ID results using the proposed solutions outperform some of the state-of-the-art methods.

Structured learning of metric ensembles with application to person re-identification

Matching individuals across non-overlapping camera networks, known as person re-identification, is a fundamentally challenging problem due to the large visual appearance changes caused by variations of viewpoints, lighting, and occlusion. Approaches in literature can be categorized into two streams: The first stream is to develop reliable features against realistic conditions by combining several visual features in a pre-defined way; the second stream is to learn a metric from training data to ensure strong inter-class differences and intra-class similarities. However, seeking an optimal combination of visual features which is generic yet adaptive to different benchmarks is an unsolved problem, and metric learning models easily get over-fitted due to the scarcity of training data in person re-identification. In this paper, we propose two effective structured learning based approaches which explore the adaptive effects of visual features in recognizing persons in different benchmark data sets. Our framework is built on the basis of multiple low-level visual features with an optimal ensemble of their metrics. We formulate two optimization algorithms, CMC triplet and CMC top, which directly optimize evaluation measures commonly used in person re-identification, also known as the Cumulative Matching Characteristic (CMC) curve. The more standard CMC triplet formulation works on the triplet information by maximizing the relative distance between a matched pair and a mismatched pair in each triplet unit. The CMC top formulation, modeled on a structured learning of maximizing the correct identification among top candidates, is demonstrated to be more beneficial to person re-identification by directly optimizing an objective closer to the actual testing criteria. The combination of these factors leads to a person re-identification system which outperforms most existing algorithms. More importantly, we advance state-of-the-art results by improving the rank-1 recognition rates from 40% to 61% on the iLIDS benchmark, 16% to 22% on the PRID2011 benchmark, 43% to 50% on the VIPeR benchmark, 34% to 55% on the CUHK01 benchmark and 21% to 68% on the CUHK03 benchmark.