Camera Viewpoint Changes Research Articles

CEE-Net: Complementary End-to-End Network for 3D Human Pose Generation and Estimation

The limited number of actors and actions in existing datasets make 3D pose estimators tend to overfit, which can be seen from the performance degradation of the algorithm on cross-datasets, especially for rare and complex poses. Although previous data augmentation works have increased the diversity of the training set, the changes in camera viewpoint and position play a dominant role in improving the accuracy of the estimator, while the generated 3D poses are limited and still heavily rely on the source dataset. In addition, these works do not consider the adaptability of the pose estimator to generated data, and complex poses will cause training collapse. In this paper, we propose the CEE-Net, a Complementary End-to-End Network for 3D human pose generation and estimation. The generator extremely expands the distribution of each joint-angle in the existing dataset and limits them to a reasonable range. By learning the correlations within and between the torso and limbs, the estimator can combine different body-parts more effectively and weaken the influence of specific joint-angle changes on the global pose, improving the generalization ability. Extensive ablation studies show that our pose generator greatly strengthens the joint-angle distribution, and our pose estimator can utilize these poses positively. Compared with the state-of-the-art methods, our method can achieve much better performance on various cross-datasets, rare and complex poses.

Internal and external spatial–temporal constraints for person reidentification

Spatial–temporal information is easy to achieve in a practical surveillance scene, but it is often neglected in most current person reidentification (ReID) methods. Employing spatial–temporal information as a constrain has been verified as beneficial for ReID. However, there is no effective modeling according to the pedestrian movement law. In this paper, we present a ReID framework with internal and external spatial–temporal constraints, termed as IESC-ReID. A novel residual spatial attention module is proposed to build a spatial–temporal constraint and increase the robustness to partial occlusions or camera viewpoint changes. A Laplace-based spatial–temporal constraint is also introduced to eliminate irrelevant gallery images, which are gathered by the internal learning network. IESC-ReID constrains the attention within the functioning range of the channel space, and utilizes additional spatial–temporal constrains to further constrain results. Intensive experiments show that these constraints consistently improve the performance. Extensive experimental results on numerous publicly available datasets show that the proposed method outperforms several state-of-the-art ReID algorithms. Our code is publicly available at https://github.com/jiaming-wang/IESC.

Simple and Effective: Spatial Rescaling for Person Reidentification.

Global average pooling (GAP) allows convolutional neural networks (CNNs) to localize discriminative information for recognition using only image-level labels. While GAP helps CNNs to attend to the most discriminative features of an object, e.g., head of a bird or one man's bag, it may suffer if that information is missing due to camera viewpoint changes and intraclass variations in some tasks. To circumvent this issue, we propose one new module to help CNNs to see more, namely, Spatial Rescaling (SpaRs) layer. It introduces spatial relations among the feature map activations back to the model, guiding the model to focus on a broad area in the feature map. With simple implementation, it can be inserted into CNNs of various architectures directly. SpaRs layer consistently improves the performance over the reidentification (re-ID) models. Besides, the new module based on different normalization methods also demonstrates the superiority of fine-grained and general image classification benchmarks. The visualization method shows the changes in activated regions when equipped with the SpaRs layer for better understanding. Our code is publicly available at https://github.com/HRanWang/Spatial-Re-Scaling.

Online Semantic Subspace Learning with Siamese Network for UAV Tracking

In urban environment monitoring, visual tracking on unmanned aerial vehicles (UAVs) can produce more applications owing to the inherent advantages, but it also brings new challenges for existing visual tracking approaches (such as complex background clutters, rotation, fast motion, small objects, and realtime issues due to camera motion and viewpoint changes). Based on the Siamese network, tracking can be conducted efficiently in recent UAV datasets. Unfortunately, the learned convolutional neural network (CNN) features are not discriminative when identifying the target from the background/clutter, In particular for the distractor, and cannot capture the appearance variations temporally. Additionally, occlusion and disappearance are also reasons for tracking failure. In this paper, a semantic subspace module is designed to be integrated into the Siamese network tracker to encode the local fine-grained details of the target for UAV tracking. More specifically, the target’s semantic subspace is learned online to adapt to the target in the temporal domain. Additionally, the pixel-wise response of the semantic subspace can be used to detect occlusion and disappearance of the target, and this enables reasonable updating to relieve model drifting. Substantial experiments conducted on challenging UAV benchmarks illustrate that the proposed method can obtain competitive results in both accuracy and efficiency when they are applied to UAV videos.

Single Modality-Based Event Detection Framework for Complex Videos

Event detection of rare and complex events in large video datasets or in unconstrained user-uploaded videos on internet is a challenging task. The presence of irregular camera movement, viewpoint changes, illumination variations and significant changes in the background make extremely difficult to capture underlying motion in videos. In addition, extraction of features using different modalities (single streams) may offer computational complexities and cause abstraction of confusing and irrelevant spatial and semantic features. To address this problem, we present a single stream (RGB only) based on feature of spatial and semantic features extracted by modified 3D Residual Convulsion Network. We combine the spatial and semantic features based on this assumption that difference between both types of features can discover the accurate and relevant features. Moreover, introduction of temporal encoding builds the relationship in consecutive video frames to explore discriminative long-term motion patterns. We conduct extensive experiments on prominent publically available datasets. The obtained results demonstrate the great power of our proposed model and improved accuracy compared with existing state-of-the-art methods.

Repetitive motion compensation for real time intraoperative video processing.

In this paper, we present a motion compensation algorithm dedicated to video processing during neurosurgery. After craniotomy, the brain surface undergoes a repetitive motion due to the cardiac pulsation. This motion as well as potential video camera motion prevent accurate video analysis. We propose a dedicated motion model where the brain deformation is described using a linear basis learned from a few initial frames of the video. As opposed to other works using linear basis for the flow, the camera motion is explicitly accounted in the transformation model. Despite the nonlinear nature of our model, all the motion parameters are robustly estimated all at once, using only one singular value decomposition (SVD), making our procedure computationally efficient. A Lagrangian specification of the flow field ensures the stability of the method. Experiments on in vivo data are presented to evaluate the capacity of the method to cope with occlusion or camera motion. The method we propose satisfies the intraoperative constraints: it is robust to surgical tools occlusions, it works in real time, and it is able to handle large camera viewpoint changes.

A Graph-Based Vehicle Proposal Location and Detection Algorithm

The majority of the existing appearance-based vehicle-detection systems make use of a sliding-window paradigm for vehicle-candidate regions location. In order to locate all vehicle regions with various sizes and shapes, a large number of search windows are generated by a sliding-window paradigm in most vehicle-detection systems. It is desirable to obtain fewer and more precisely located vehicle candidate regions for further detection. For this purpose, a novel graph-based algorithm is proposed to locate the vehicle proposal regions, which estimates the possibility of a vehicle contained in a bounding box. Experimental results on the public traffic analysis data set (KITTI) and PASCAL VOC 2007 show that the proposed region proposal approach leads to better performances compared with popular bottom-up region proposal methods. Moreover, the proposed vehicle-detection system is evaluated on the KITTI data set, which are determined to be satisfactory, even for the images containing vehicles that have undergone scale variations and camera viewpoint changes, as well as for images that were photographed with complex backgrounds.

Vehicle Detection by a Context-Aware Multichannel Feature Pyramid

The majority of the existing appearance-based vehicle-detection systems make use of local features for detection purposes, such as Haar-like, histograms of oriented gradients, scale-invariant feature transform, and so forth. However, these local features have limitations when dealing with illuminations, scale, shape variations, and complex background situations. It is desirable for a vehicle to have discriminative and robust features. For this purpose, a novel context-aware multichannel feature pyramid has been proposed in this paper. The main contribution of this paper is proposing two context-aware structural descriptors, termed as a context-aware difference sign transform feature and context-aware difference magnitude transform feature. An image has been tiled with a dense grid of the cells, and each cell is described by both local details and context-aware structural descriptors. The context-aware structural descriptors have the ability to capture the context-aware structural information of cells. The proposed context-aware multichannel feature pyramid is able to provide more effective features for vehicle detection. The results of the two public traffic analysis datasets show that the proposed approach leads to better performances when compared with the current state-of-the-art methods. Moreover, the experimental results are determined to be satisfactory, even for the images containing vehicles that have undergone scale variations and camera viewpoint changes, as well as for images that were photographed with complex backgrounds.

Automatic parsing of lane and road boundaries in challenging traffic scenes

Automatic detection of road boundaries in traffic surveillance imagery can greatly aid subsequent traffic analysis tasks, such as vehicle flow, erratic driving, and stranded vehicles. This paper develops an online technique for identifying the dominant road boundary in video sequences captured by traffic cameras under challenging environmental and lighting conditions, e.g., unlit highways captured at night. The proposed method works in real time of up to 20 frames/s and generates a ranked list of road regions that identify road and lane boundaries. Our method begins by segmenting each frame into a set of superpixels. An adaptive sampling step approximates superpixel contours to a collection of edge segments. Next, we show how online hierarchical clustering can be efficiently used to organize edges into clusters of colinearly similar sets. Promising clusters are paired with each other to form cluster pairs. Then we present and prove a statistical ranking measure that is used along with road-activity and perspective cues to find the dominant road boundaries. We evaluate the proposed approach on two real-world datasets to test our method under camera viewpoint changes and extreme environmental and lighting conditions. Results show that our method outperforms two state-of-the-art techniques in precision, recall, and runtime.

Video-Based Dynamic Stagger Measurement of Railway Overhead Power Lines Using Rotation-Invariant Feature Matching

In this paper we propose an effective method of assessing the reliability of railway overhead power lines by measuring the dynamic stagger of contact wires based on a video monitoring technique. Previously developed video monitoring methods may produce severe errors when applied to tilting trains due to changes in position and orientation of the pantograph. In particular, we propose to employ feature-based image matching techniques that are invariant to rotation and robust to changes in camera viewpoint. A pantograph tilting model is first developed from the video data acquired from an actual train based on the motion dynamics of the stagger behavior on moving train platform. We then evaluate the proposed method by comparing it with the conventional template matching in terms of tracking error. The experimental results confirm that the proposed method shows superior performance in all train traveling sequences, particularly over the pantograph tilting train motion segment.

Analysis of Using Metric Access Methods for Visual Search of Objects in Video Databases

This article presents an approach to object retrieval that searches for and localizes all the occurrences of an object in a video database, given a query image of the object. Our proposal is based on text-retrieval methods in which video key frames are represented by a dense set of viewpoint invariant region descriptors that enable recognition to proceed successfully despite changes in camera viewpoint, lighting, and partial occlusions. Vector quantizing these region descriptors provides a visual analogy of a word - a visual word. Those words are grouped into a visual vocabulary which is used to index all key frames from the video database. Efficient retrieval is then achieved by employing methods from statistical text retrieval, including inverted file systems, and text-document frequency weightings. Though works in the literature have only adopted a simple sequential scan during search, we investigate the use of different metric access methods (MAM): M-tree, Slim-tree, and D-index, in order to accelerate the processing of similarity queries. In addition, a ranking strategy based on the spatial layout of the regions (spatial consistency) is fully described and evaluated. Experimental results have shown that the adoption of MAMs not only has improved the search performance but also has reduced the influence of the vocabulary size over test results, which may improve the scalability of our proposal. Finally, the application of spatial consistency has produced a very significant improvement of the results.

Fusion of camera images and laser scans for wide baseline 3D scene alignment in urban environments

In this paper we address the problem of automatic laser scan registration in urban environments. This represents a challenging problem for two major reasons. First, two individual laser scans might be captured at significantly changed viewpoints (wide baseline) and have very little overlap. Second, man-made buildings usually contain many structures of similar appearances. This will result in considerable aliasing in the matching process. By sensor fusion of laser data with camera images, we propose a novel improvement to the existing 2D feature techniques to enable automatic 3D alignment between two widely separated scans. The key idea consists of extracting dominant planar structures from 3D point clouds and then utilizing the recovered 3D geometry to improve the performance of 2D image feature for wide baseline matching. The resulting feature descriptors become more robust to camera viewpoint changes after the procedure of viewpoint normalization. Moreover, the viewpoint normalized 2D features provide robust local feature information including patch scale and dominant orientation for effective repetitive structure matching in man-made environments. Comprehensive experimental evaluations with real data demonstrate the potential of the proposed method for automatic wide baseline 3D scan alignment in urban environments.

Improved feature extraction and matching in urban environments based on 3D viewpoint normalization

In this paper we present a novel approach for generating viewpoint invariant features from single images and demonstrate its application to robust matching over widely separated views in urban environments. Our approach exploits the fact that many man-made environments contain a large number of parallel linear features along several principal directions. We identify the projections of these parallel lines to recover a number of dominant scene planes and subsequently compute viewpoint invariant features within the rectified views of these planes. We present a set of comprehensive experiments to evaluate the performance of the proposed viewpoint invariant features. It is demonstrated that: (1) the resulting feature descriptors become more distinctive and more robust to camera viewpoint changes after the procedure of 3D viewpoint normalization; and (2) the features provide robust local feature information including patch scale and dominant orientation which can be effectively used to provide geometric constraints between views. Targeted at applications in urban environments, where many repetitive structures exist, we further propose an effective framework to use this novel feature for the challenging wide baseline matching tasks.