Non-overlapping Views Research Articles

Applying Ternion Stream DCNN for Real-Time Vehicle Re-Identification and Tracking across Multiple Non-Overlapping Cameras

The increase in security threats and a huge demand for smart transportation applications for vehicle identification and tracking with multiple non-overlapping cameras have gained a lot of attention. Moreover, extracting meaningful and semantic vehicle information has become an adventurous task, with frameworks deployed on different domains to scan features independently. Furthermore, approach identification and tracking processes have largely relied on one or two vehicle characteristics. They have managed to achieve a high detection quality rate and accuracy using Inception ResNet and pre-trained models but have had limitations on handling moving vehicle classes and were not suitable for real-time tracking. Additionally, the complexity and diverse characteristics of vehicles made the algorithms impossible to efficiently distinguish and match vehicle tracklets across non-overlapping cameras. Therefore, to disambiguate these features, we propose to implement a Ternion stream deep convolutional neural network (TSDCNN) over non-overlapping cameras and combine all key vehicle features such as shape, license plate number, and optical character recognition (OCR). Then jointly investigate the strategic analysis of visual vehicle information to find and identify vehicles in multiple non-overlapping views of algorithms. As a result, the proposed algorithm improved the recognition quality rate and recorded a remarkable overall performance, outperforming the current online state-of-the-art paradigm by 0.28% and 1.70%, respectively, on vehicle rear view (VRV) and Veri776 datasets.

V2ReID: Vision-Outlooker-Based Vehicle Re-Identification.

With the increase of large camera networks around us, it is becoming more difficult to manually identify vehicles. Computer vision enables us to automate this task. More specifically, vehicle re-identification (ReID) aims to identify cars in a camera network with non-overlapping views. Images captured of vehicles can undergo intense variations of appearance due to illumination, pose, or viewpoint. Furthermore, due to small inter-class similarities and large intra-class differences, feature learning is often enhanced with non-visual cues, such as the topology of camera networks and temporal information. These are, however, not always available or can be resource intensive for the model. Following the success of Transformer baselines in ReID, we propose for the first time an outlook-attention-based vehicle ReID framework using the Vision Outlooker as its backbone, which is able to encode finer-level features. We show that, without embedding any additional side information and using only the visual cues, we can achieve an 80.31% mAP and 97.13% R-1 on the VeRi-776 dataset. Besides documenting our research, this paper also aims to provide a comprehensive walkthrough of vehicle ReID. We aim to provide a starting point for individuals and organisations, as it is difficult to navigate through the myriad of complex research in this field.

3D Skeleton and Two Streams Approach to Person Re-identification Using Optimized Region Matching

Person re-identification (Re-ID) is a challenging and arduous task due to non-overlapping views, complex background, and uncontrollable occlusion in video surveillance. An existing method for capturing pedestrian local region information is to divide person regions into horizontal stripes, which may lead to invalid features and erroneous learning. To solve this problem, this paper proposes a 3D skeleton and a two-stream approach to person Re-ID. The first stream of the method uses the 3D skeleton for background filtering and region segmentation. The second stream uses Siamese net to extract the global descriptor. The features of the two streams are fused to preserve the integrity of the person. An optimized region matching method for metric learning is designed. Extensive comparing experiments were conducted with state-of-the-art Re-ID methods on the Market-1501, CUHK03, and DukeMTMC-reID datasets. Experimental results show that the proposed method outperforms the existing methods in recognition accuracy.

Re-ranking vehicle re-identification with orientation-guide query expansion

Vehicle re-identification, which aims to retrieve information regarding a vehicle from different cameras with non-overlapping views, has recently attracted extensive attention in the field of computer vision owing to the development of smart cities. This task can be regarded as a type of retrieval problem, where re-ranking is important for performance enhancement. In the vehicle re-identification ranking list, images whose orientations are dissimilar to that of the query image must preferably be optimized on priority. However, traditional methods are incompatible with such samples, resulting in unsatisfactory vehicle re-identification performances. Therefore, in this study, we propose a vehicle re-identification re-ranking method with orientation-guide query expansion to optimize the initial ranking list obtained by a re-identification model. In the proposed method, we first find the nearest neighbor image whose orientation is dissimilar to the queried image and then fuse the features of the query and neighbor images to obtain new features for information retrieval. Experiments are performed on two public data sets, VeRi-776 and VehicleID, and the effectiveness of the proposed method is confirmed.

A Survey on Generative Adversarial Network Based Person Re-Identification Method

In recent years, with the growing demand for public safety and the rapid development of intelligent surveillance networks, person re-identification (Re-ID) has become one of the hottest research topics in the field of computer vision, which aims at retrieving a person of interest through multiple cameras with non-overlapping views. First, the current classical person Re-ID datasets are introduced. Second, the latest GAN-based person Re-ID methods are focused on sorting out. These methods are classified into three categories: image-image style transfer, data enhancement and invariant feature learning according to the characteristics and application scenarios of GAN. And basic principles, advantages, and disadvantages are summarized for each category. Then, the states of the art of the classic algorithms mentioned are compared on the mainstream datasets. Finally, the present challenging problems of person Re-ID and directions for future research are discussed.

Multi-person multi-camera tracking for live stream videos based on improved motion model and matching cascade

Multiple live videos are streamed to the security room in the video surveillance system of a museum, shopping mall, plaza, etc. For public safety and crowd research, multi-person tracking is essential in these small-scale distributed multi-camera systems. This paper presents a multi-person multi-camera tracking framework for live stream videos. It is compatible with both overlapping and non-overlapping views. Since the tracking framework cannot consistently achieve real-time performance, live stream videos are probably sub-sampled. This sub-sampled issue will enlarge the error of every tracked person’s predicted bounding box at each frame. A time-based motion model studying the precise time intervals among sub-sampled frames is proposed. Besides, cameras have different orientations and exposures, resulting in false matches when tracked pedestrians cross the camera boundaries. When a tracked person is out of a camera’s view, its motion model will fail, and its tracking identity will probably switch when reentering the same camera. An improved multi-person matching cascade scheme is proposed to solve these problems. It can increase the accuracy of inter-camera person re-identification (Re-ID) by taking advantage of association priorities. Experiments are implemented with the EPFL dataset and live stream videos. Results show that the proposed method has robust performances in different situations.

Trends in Vehicle Re-Identification Past, Present, and Future: A Comprehensive Review

Vehicle Re-identification (re-id) over surveillance camera network with non-overlapping field of view is an exciting and challenging task in intelligent transportation systems (ITS). Due to its versatile applicability in metropolitan cities, it gained significant attention. Vehicle re-id matches targeted vehicle over non-overlapping views in multiple camera network. However, it becomes more difficult due to inter-class similarity, intra-class variability, viewpoint changes, and spatio-temporal uncertainty. In order to draw a detailed picture of vehicle re-id research, this paper gives a comprehensive description of the various vehicle re-id technologies, applicability, datasets, and a brief comparison of different methodologies. Our paper specifically focuses on vision-based vehicle re-id approaches, including vehicle appearance, license plate, and spatio-temporal characteristics. In addition, we explore the main challenges as well as a variety of applications in different domains. Lastly, a detailed comparison of current state-of-the-art methods performances over VeRi-776 and VehicleID datasets is summarized with future directions. We aim to facilitate future research by reviewing the work being done on vehicle re-id till to date.

Real-time human pose estimation on a smart walker using convolutional neural networks

Rehabilitation is important to improve quality of life for mobility-impaired patients. Smart walkers are a commonly used solution that should embed automatic and objective tools for data-driven human-in-the-loop control and monitoring. However, present solutions focus on extracting few specific metrics from dedicated sensors with no unified full-body approach. We investigate a general, real-time, full-body pose estimation framework based on two RGB+D camera streams with non-overlapping views mounted on a smart walker equipment used in rehabilitation. Human keypoint estimation is performed using a two-stage neural network framework. The 2D-Stage implements a detection module that locates body keypoints in the 2D image frames. The 3D-Stage implements a regression module that lifts and relates the detected keypoints in both cameras to the 3D space relative to the walker. Model predictions are low-pass filtered to improve temporal consistency. A custom acquisition method was used to obtain a dataset, with 14 healthy subjects, used for training and evaluating the proposed framework offline, which was then deployed on the real walker equipment. An overall keypoint detection error of 3.73 pixels for the 2D-Stage and 44.05 mm for the 3D-Stage were reported, with an inference time of 26.6 ms when deployed on the constrained hardware of the walker. We present a novel approach to patient monitoring and data-driven human-in-the-loop control in the context of smart walkers. It is able to extract a complete and compact body representation in real-time and from inexpensive sensors, serving as a common base for downstream metrics extraction solutions, and Human-Robot interaction applications. Despite promising results, more data should be collected on users with impairments, to assess its performance as a rehabilitation tool in real-world scenarios.

Implications of the Overlapping Degree Between Proximal Fibula and Tibia for Placing the Optimal Syndesmotic Screw: A Virtual Cadaveric Study.

To determine the optimal direction of the syndesmotic screw and to introduce a consistent landmark for practical application by analyzing three-dimensional (3D) modeling and virtual implantation. A total of 105 cadaveric lower legs (50 males and 55 females; average height, 160.6 ± 7.1cm) were used to reconstruct a 3D model by using the Mimics® software and the joint morphology was evaluated. Syndesmotic cylinders (Ø3.5mm/Length 100mm) were transversely placed in the proximal end of the incisura fibularis for simulating screw fixation. The tibial proximal cylinder, which was tangent to the posterior tibial condyles, was traced and the angle between the two cylinders was measured as the tibial torsion angle (TTA). After rotating the syndesmotic cylinder parallel to the ground, the overlapping degree between the proximal fibula and tibia was assessed as a radiologic indicator. Concerning tibial torsion, the TTA was an average of 36.7° (range, 17.2°-54.4°; SD, 8.78) When the syndesmotic cylinder was rotated to be parallel to the ground, the proximal fibula had nonlinear or linear overlap with the lateral border of the tibia, regardless of the joint morphology. In this non-overlapping view, three Weber's indices for normal fibular length could be better visualized than the mortise view. The syndesmotic cylinder in the proximal end of the incisura fibularis could be consistently placed parallel to the ground by internally rotating the tibia until there was a nonlinear or linear overlap between the proximal fibula and the tibia, regardless of the joint morphology.

Back-propagation of the Mahalanobis istance through a deep triplet learning model for person Re-Identification

The automatization of the Re-Identification of an individual across different video-surveillance cameras poses a significant challenge due to the presence of a vast number of potential candidates with a similar appearance. This task requires the learning of discriminative features from person images and a distance metric to properly compare them and decide whether they belong to the same person or not. Nevertheless, the fact of acquiring images of the same person from different, distant and non-overlapping views produces changes in illumination, perspective, background, resolution and scale between the person’s representations, resulting in appearance variations that hamper his/her re-identification. This article focuses the feature learning on automatically finding discriminative descriptors able to reflect the dissimilarities mainly due to the changes in actual people appearance, independently from the variations introduced by the acquisition point. With that purpose, such variations have been implicitly embedded by the Mahalanobis distance. This article presents a learning algorithm to jointly model features and the Mahalanobis distance through a Deep Neural Re-Identification model. The Mahalanobis distance learning has been implemented as a novel neural layer, forming part of a Triplet Learning model that has been evaluated over PRID2011 dataset, providing satisfactory results.

Parametric Tracking Across Multiple Cameras with Spatial Estimation

This paper presents a method of tracking a target based on the combination of parameters between colour blobs. Apart from tracking targets based on parameters between colour regions, the blob parameters can be used as a means for estimating the spatial location of the target relative to the location of the cameras. The blobs are obtained by segmenting the overall target into multiple colour regions. The colour regions are segmented using the EM method. After segmentation into different regions on different colour layers, parameters can be generated between colour regions of interest. In this instance, the colour regions of interest are the top and bottom colour regions. Parameters between colour regions of interest were the vector magnitude, vector angle and the value difference between colour regions. These parameters are used for tracking the target of interest across an array of cameras which in this instance are three cameras. The summarized results of tracking targets across three cameras have shown that parameters between colour regions have the capability to track targets across cameras. In addition to tracking targets across multiple cameras, the work presented in this paper also shows the capability of estimating spatial information based on tracking of targets as it moves across different camera view fields. The spatial information that can be estimated from an array of cameras with non-overlapping view fields is distance estimation of the target from camera and distance estimation between cameras.

Density-adaptive kernel based efficient reranking approaches for person reidentification

Person reidentification (ReID) refers to the task of verifying the identity of a pedestrian observed from nonoverlapping views in a surveillance camera network. It has recently been validated that reranking can achieve remarkable performance improvements in person ReID systems. However, current reranking approaches either require feedback from users or suffer from burdensome computational costs. In this paper, we propose to exploit a density-adaptive smooth kernel technique to achieve efficient and effective reranking. Specifically, we adopt a smooth kernel function to formulate the neighbor relationships among data samples with a density-adaptive parameter. Based on this new formulation, we present two simple yet effective reranking methods, termed inverse density-adaptive kernel based reranking (inv-DAKR) and bidirectional density-adaptive kernel based reranking (bi-DAKR), in which the local density information in the vicinity of each gallery sample is elegantly exploited. Moreover, we extend the proposed inv-DAKR and bi-DAKR methods to incorporate the available extra probe samples and demonstrate that when and why these extra probe samples are able to improve the local neighborhood and thus further refine the ranking results. Extensive experiments are conducted on six benchmark datasets, including: PRID450s, VIPeR, CUHK03, GRID, Market-1501 and Mars. The experimental results demonstrate that our proposals are effective and efficient.

A part-based attention network for person re-identification

Person re-identification (re-id) is the task of recognizing images of the same pedestrian captured by different cameras with non-overlapping views. Person re-id is a challenging task due to the existence of large view variations, such as spatial misalignment, background clutter and human poses change. In this paper, we handle these challenges from the following two aspects: utilizing attention mechanism to alleviate misalignment problem and exploiting the complementary effects of global-local features for more stable pedestrian descriptors. Specifically, we first present a part-based attention model consisting of a channel attention block and a spatial attention block to sequentially refine the convolutional descriptors of person body parts. The channel and spatial attention blocks weight the channels and positions of body-part feature maps to spot the informative channels and regions, respectively. Then global full-body and local body-part of the refined feature maps are pooled into global and local representations, which are jointly trained using identity classification loss. We conduct extensive experiments on four standard benchmark datasets including Market1501, CUHK03, DukeMTMC-reID, and CUHK01, and the experimental results demonstrate the effectiveness of the presented method.

Learning part-alignment feature for person re-identification with spatial-temporal-based re-ranking method

Person re-identification is to identify a target person in different cameras with non-overlapping views. It is a challenging task due to various viewpoints of persons, diversified illuminations, and complicated environments. In addition, body parts are usually misaligned because of the less precise bounding boxes, which play a significant role in person re-identification, so it is crucial to make them aligned for better performance. In this paper, we propose a network to learn powerful features combining global features and local-alignment features for person re-identification. For each body part, instead of fixed horizontal partition, a key points detection network is adopted to locate body parts that contain more precise and distinctive information. Besides, a novel re-ranking approach is proposed to refine the rough initial rank list by exploiting the spatial-temporal information. Unlike most existing re-ranking based methods fine-tuning the rough initial rank list only by k-nearest neighbors and their k-reverse-nearest neighbors, our method exploits spatial-temporal information which can be easily stored in the name of images, so it can be implemented in any baseline to improve the performance. Experiments on the GRID, Market-1501, and DukeMTMC-reID are conducted to prove the effectiveness of our method.

Improved person re-identification based on saliency and semantic parsing with deep neural network models

Given a video or an image of a person acquired from a camera, person re-identification is the process of retrieving all instances of the same person from videos or images taken from a different camera with non-overlapping view. This task has applications in various fields, such as surveillance, forensics, robotics, multimedia. In this paper, we present a novel framework, named Saliency-Semantic Parsing Re-Identification (SSP-ReID), for taking advantage of the capabilities of both clues: saliency and semantic parsing maps, to guide a backbone convolutional neural network (CNN) to learn complementary representations that improves the results over the original backbones. The insight of fusing multiple clues is based on specific scenarios in which one response is better than another, thus favoring the combination of them to increase performance. Due to its definition, our framework can be easily applied to a wide variety of networks and, in contrast to other competitive methods, our training process follows simple and standard protocols. We present extensive evaluation of our approach through five backbones and three benchmarks. Experimental results demonstrate the effectiveness of our person re-identification framework. In addition, we combine our framework with re-ranking techniques and compare it against state-of-the-art approaches, achieving competitive results.

Top distance regularized projection and dictionary learning for person re-identification

Person re-identification (PRID) is integral to many smart surveillance systems. However, owing to the visual ambiguities arising from the variability in viewing angles and illumination, and the presence of occlusions, PRID continues to present many challenges, especially when only a single image per view is available for each person. To overcome this problem, we propose a top distance regularized projection and dictionary learning (DL) model for PRID. The model incorporates both projection and DL to form a unified optimization framework to enhance the effectiveness of both these types of learning. Thus, the dictionary and projection matrix are jointly learned within this framework. In particular, the learned projection maps the coding coefficient into a discriminative space and minimizes the distance between the same persons across non-overlapping views such that the dictionary and projection can be discriminated. Moreover, we exploit listwise distances to capture all pairwise similarities. Based on this design, we derive a top distance regularization term to refine the solution space of the DL model such that the discriminative ability of the learned projection matrix and dictionary are further improved. Experiments on different challenging datasets demonstrate the effectiveness of our method and its superiority over a few current state-of-the-art approaches.

Learning multi-region features for vehicle re-identification with context-based ranking method

Vehicle re-identification is to identify a target vehicle in different cameras with non-overlapping views. It is a challenging task due to various viewpoints of vehicles, diversified illuminations and complicated environments. Most existing vehicle re-identification methods focus on learning global features, while neglecting the importance of local features. In this paper, we propose a Multi-Region Model (MRM) to learn powerful features for vehicle re-identification. In addition to extracting global region features, MRM also extracts features from a series of local regions. For each local region, instead of utilizing the rigid part to extract features directly, a Spatial Transformer Network (STN) based localization model is introduced to localize local regions which contain more distinctive visual cues. In order to further improve the performance of re-identification, we design a context-based ranking method which generates the ranking list by taking context and content into consideration to measure the similarity between neighbors. Experimental results clearly demonstrate that our method achieves excellent performance on both VehicleID dataset and VeRi-776 dataset.

Combining multilevel feature extraction and multi-loss learning for person re-identification

The goal of person re-identification (re-id) is to match images of the same person captured by multiple cameras with non-overlapping views. It is a challenging task due to the large spatial displacement and human pose change of person images across different views. Recently, the deep Convolutional Neural Network (CNN) has significantly improved the performance of person re-id. In this paper, we present a hybrid deep model that combines multilevel feature extraction and multi-loss learning for more robust pedestrian descriptors. The multi-loss function jointly optimizes the verification task that aims to verify if two images belong to same person, and the recognition task that aims to predict the identity of each image. Specifically, given two person images, we first apply a deep learning network, called Feature Aggregation Network (FAN), to extract their multilevel CNN features by fusing the information of different layers. For the verification task, a Recurrent Comparative Network (RCN) is presented to learn joint representation of paired CNN features. RCN determines whether two images depict the same person through focusing on discriminative regions and alternatively comparing their appearance. It is an algorithmic imitation of human decision-making process, in which a person repeatedly compares two objects before making decision about their similarity. For the recognition task, a parameter-free operation termed Global Average Pooling (GAP) is followed after each CNN feature to extract identity-related features. Extensive experiments are conducted on four datasets, including CUHK03, CUHK01, Market1501 and DukeMTMC, and the experimental results demonstrate the effectiveness of our presented method.

A Multi-Camera Rig with Non-Overlapping Views for Dynamic Six-Degree-of-Freedom Measurement.

Large-scale measurement plays an increasingly important role in intelligent manufacturing. However, existing instruments have problems with immersive experiences. In this paper, an immersive positioning and measuring method based on augmented reality is introduced. An inside-out vision measurement approach using a multi-camera rig with non-overlapping views is presented for dynamic six-degree-of-freedom measurement. By using active LED markers, a flexible and robust solution is delivered to deal with complex manufacturing sites. The space resection adjustment principle is addressed and measurement errors are simulated. The improved Nearest Neighbor method is employed for feature correspondence. The proposed tracking method is verified by experiments and results with good performance are obtained.

PaMM: Pose-Aware Multi-Shot Matching for Improving Person Re-Identification.

Person re-identification is the problem of recognizing people across different images or videos with non-overlapping views. Although a significant progress has been made in person re-identification over the last decade, it remains a challenging task because the appearances of people can seem extremely different across diverse camera viewpoints and person poses. In this paper, we propose a novel framework for person re-identification by analyzing camera viewpoints and person poses called pose-aware multi-shot matching. It robustly estimates individual poses and efficiently performs multi-shot matching based on the pose information. The experimental results obtained by using public person re-identification data sets show that the proposed methods outperform the current state-of-the-art methods, and are promising for accomplishing person re-identification under diverse viewpoints and pose variances.