Viewpoint Estimation Research Articles

Exploiting View-Specific Appearance Similarities Across Classes for Zero-Shot Pose Prediction: A Metric Learning Approach

Viewpoint estimation, especially in case of multiple object classes, remains an important and challenging problem. First, objects under different views undergo extreme appearance variations, often making within-class variance larger than between-class variance. Second, obtaining precise ground truth for real-world images, necessary for training supervised viewpoint estimation models, is extremely difficult and time consuming. As a result, annotated data is often available only for a limited number of classes. Hence it is desirable to share viewpoint information across classes. Additional complexity arises from unaligned pose labels between classes, i.e. a side view of a car might look more like a frontal view of a toaster, than its side view. To address these problems, we propose a metric learning approach for joint class prediction and pose estimation. Our approach allows to circumvent the problem of viewpoint alignment across multiple classes, and does not require dense viewpoint labels. Moreover, we show, that the learned metric generalizes to new classes, for which the pose labels are not available, and therefore makes it possible to use only partially annotated training sets, relying on the intrinsic similarities in the viewpoint manifolds. We evaluate our approach on two challenging multi-class datasets, 3DObjects and PASCAL3D+.

Learning And-Or Models to Represent Context and Occlusion for Car Detection and Viewpoint Estimation.

This paper presents a method for learning an And-Or model to represent context and occlusion for car detection and viewpoint estimation. The learned And-Or model represents car-to-car context and occlusion configurations at three levels: (i) spatially-aligned cars, (ii) single car under different occlusion configurations, and (iii) a small number of parts. The And-Or model embeds a grammar for representing large structural and appearance variations in a reconfigurable hierarchy. The learning process consists of two stages in a weakly supervised way (i.e., only bounding boxes of single cars are annotated). Firstly, the structure of the And-Or model is learned with three components: (a) mining multi-car contextual patterns based on layouts of annotated single car bounding boxes, (b) mining occlusion configurations between single cars, and (c) learning different combinations of part visibility based on CAD simulations. The And-Or model is organized in a directed and acyclic graph which can be inferred by Dynamic Programming. Secondly, the model parameters (for appearance, deformation and bias) are jointly trained using Weak-Label Structural SVM. In experiments, we test our model on four car detection datasets - the KITTI dataset [1], the PASCAL VOC2007 car dataset [2], and two self-collected car datasets, namely the Street-Parking car dataset and the Parking-Lot car dataset, and three datasets for car viewpoint estimation - the PASCAL VOC2006 car dataset [2], the 3D car dataset [3], and the PASCAL3D+ car dataset [4]. Compared with state-of-the-art variants of deformable part-based models and other methods, our model achieves significant improvement consistently on the four detection datasets, and comparable performance on car viewpoint estimation.

Multi-View and 3D Deformable Part Models

As objects are inherently 3D, they have been modeled in 3D in the early days of computer vision. Due to the ambiguities arising from mapping 2D features to 3D models, 3D object representations have been neglected and 2D feature-based models are the predominant paradigm in object detection nowadays. While such models have achieved outstanding bounding box detection performance, they come with limited expressiveness, as they are clearly limited in their capability of reasoning about 3D shape or viewpoints. In this work, we bring the worlds of 3D and 2D object representations closer, by building an object detector which leverages the expressive power of 3D object representations while at the same time can be robustly matched to image evidence. To that end, we gradually extend the successful deformable part model [1] to include viewpoint information and part-level 3D geometry information, resulting in several different models with different level of expressiveness. We end up with a 3D object model, consisting of multiple object parts represented in 3D and a continuous appearance model. We experimentally verify that our models, while providing richer object hypotheses than the 2D object models, provide consistently better joint object localization and viewpoint estimation than the state-of-the-art multi-view and 3D object detectors on various benchmarks (KITTI [2] , 3D object classes [3] , Pascal3D+ [4] , Pascal VOC 2007 [5] , EPFL multi-view cars[6] ).

Fuzzy qualitative human model for viewpoint identification

The integration of advance human motion analysis techniques in low-cost video cameras has emerged for consumer applications, particularly in video surveillance systems. These smart and cheap devices provide the practical solutions for improving the public safety and homeland security with the capability of understanding the human behaviour automatically. In this sense, an intelligent video surveillance system should not be constrained on a person viewpoint, as in natural, a person is not restricted to perform an action from a fixed camera viewpoint. To achieve the objective, many state-of-the-art approaches require the information from multiple cameras in their processing. This is an impractical solution by considering its feasibility and computational complexity. First, it is very difficult to find an open space in real environment with perfect overlapping for multi-camera calibration. Secondly, the processing of information from multiple cameras is computational burden. With this, a surge of interest has sparked on single camera approach with notable work on the concept of view specific action recognition. However in their work, the viewpoints are assumed in a priori. In this paper, we extend it by proposing a viewpoint estimation framework where a novel human contour descriptor namely the fuzzy qualitative human contour is extracted from the fuzzy qualitative Poisson human model for viewpoint analysis. Clustering algorithms are used to learn and classify the viewpoints. In addition, our system is also integrated with the capability to classify front and rear views. Experimental results showed the reliability and effectiveness of our proposed viewpoint estimation framework by using the challenging IXMAS human action dataset.

Viewpoint-aware object detection and continuous pose estimation

We describe an approach to category-level detection and viewpoint estimation for rigid 3D objects from single 2D images. In contrast to many existing methods, we directly integrate 3D reasoning with an appearance-based voting architecture. Our method relies on a nonparametric representation of a joint distribution of shape and appearance of the object class. Our voting method employs a novel parameterization of joint detection and viewpoint hypothesis space, allowing efficient accumulation of evidence. We combine this with a re-scoring and refinement mechanism, using an ensemble of view-specific support vector machines. We evaluate the performance of our approach in detection and pose estimation of cars on a number of benchmark datasets. Finally we introduce the “Weizmann Cars ViewPoint” (WCVP) dataset, a benchmark for evaluating continuous pose estimation.

이동형 패럴랙스 배리어 모바일 3D 디스플레이를 위한 강인한 시청자 시역 위치 추정 알고리즘

본 논문에서는 이동형 패럴랙스 배리어 방식의 모바일 3D 디스플레이를 위해 급격한 조명 변화에도 강인한 시청자 시역위치 추정 알고리즘을 제안한다. 기존의 비올라-존스 검출기와 옵티컬-플로우를 선형 결합한 시역 위치 추정 알고리즘[1]은 급격하게 조명이 변하는 경우 잘못된 시역 위치를 추정하는 문제가 있음을 확인하였고, 특히 이러한 급격한 조명의 변화 문제는 모바일 환경에서 흔히 발생하는 환경조건이다. 그리고 어두운 공간에서 3D 디스플레이를 시청하는 경우 디스플레이 조명이 시청자의 얼굴에 비치기 때문에 조명 변화가 매우 크게 발생할 수 있다. 급격한 조명의 변화에 대한 문제점을 해결하기 위해 본 논문에서는 매 프레임마다 정확하게 옵티컬-플로우 얼굴 검출기의 정확도를 판단할 수 있는 방법을 새롭게 제안하고, 다양한 실험을 통해 그 효과를 검증한다. 조명의 변화등에 의해 옵티컬-플로우가 잘못된 추적 결과를 출력하는 경우, 기존의 두 알고리즘(비올라-존스 얼굴검출기+옵티컬-플로우 특징점 추적기)을 선별적으로 선택함으로써 강인한 얼굴검출 및 추적이 이루어지도록 하고, 다양한 실험 결과를 통해 그 효과를 검증한다. This paper presents a robust viewpoint estimation algorithm for Moving Parallax Barrier mobile 3D display in sudden illumination changes. We analyze the previous viewpoint estimation algorithm that consists of the Viola-Jones face detector and the feature tracking by the Optical-Flow. The sudden changes in illumination decreases the performance of the Optical-flow feature tracker. In order to solve the problem, we define a novel performance measure for the Optical-Flow tracker. The overall performance can be increased by the selective adoption of the Viola-Jones detector and the Optical-flow tracker depending on the performance measure. Various experimental results show the effectiveness of the proposed method.

이동형 패럴랙스 배리어 방식의 모바일 3D 디스플레이를 위한 시역계측기술에 관한 연구

In this paper, we present an effective viewpoint estimation algorithm for the Moving parallax barrier method of 3D display mobile device. Moving parallax barrier is designed to overcome the biggest problem, the limited view angle. To accomplish it, the position of the viewer's eyes or face should be estimated with strong stability and no latency. We focus on these requirements in the poor performance of mobile processors. We used a pre-processing algorithm in order to overcome the various illumination changes. And, we combined the conventional Viola-Jones face detection method and Optical-flow algorithm for robust and stable viewpoint estimation. Various computer simulations prove the effectiveness of the proposed method.

Tracking People on a Torus

We present a framework for monocular 3D kinematic pose tracking and viewpoint estimation of periodic and quasi-periodic human motions from an uncalibrated camera. The approach we introduce here is based on learning both the visual observation manifold and the kinematic manifold of the motion using a joint representation. We show that the visual manifold of the observed shape of a human performing a periodic motion, observed from different viewpoints, is topologically equivalent to a torus manifold. The approach we introduce here is based on the supervised learning of both the visual and kinematic manifolds. Instead of learning an embedding of the manifold, we learn the geometric deformation between an ideal manifold (conceptual equivalent topological structure) and a twisted version of the manifold (the data). Experimental results show accurate estimation of the 3D body posture and the viewpoint from a single uncalibrated camera.

Viewpoint Manifolds for Action Recognition

Action recognition from video is a problem that has many important applications to human motion analysis. In real-world settings, the viewpoint of the camera cannot always be fixed relative to the subject, so view-invariant action recognition methods are needed. Previous view-invariant methods use multiple cameras in both the training and testing phases of action recognition or require storing many examples of a single action from multiple viewpoints. In this paper, we present a framework for learning a compact representation of primitive actions (e.g., walk, punch, kick, sit) that can be used for video obtained from a single camera for simultaneous action recognition and viewpoint estimation. Using our method, which models the low-dimensional structure of these actions relative to viewpoint, we show recognition rates on a publicly available dataset previously only achieved using multiple simultaneous views.

多視点テニス画像からのプレーヤ視点画像の生成

We propose a new method for synthesizing player viewpoint images from multiple view tennis videos.Our method uses two key techniques : virtual-view synthesis and player's viewpoint estimation. In the former, we divide the object tennis scene into sub-regions, which are the background, tennis court ground, players, and ball. For each sub-region, a virtualviewpoint image is synthesized by considering the geometrical conditions of each region. Then the virtual viewpoint images of the sub-regions are merged into a virtual viewpoint image of the whole scene. In virtual viewpoint image generation, view interpolation, which is a way of synthesizing images at an intermediate viewpoint to two views of the real object, restricts the viewpoint to a position between the two views. To avoid this restriction, we propose our first key technique, whereby we are able to flexibly position the viewpoint. In our second technique, the viewpoint is computed using epipolar geometry from the center of gravity of a player. By applying the computed player's viewpoint to the former technique, we can synthesize player viewpoint images. Experimental results demonstrate that the proposed method can successfully provide a video of the tennis player's viewpoint from multiple-view video images.

Viewpoint estimation in three-dimensional images taken with perspective range sensors

We present a method for estimating the viewpoint from which a 3D image has been taken using a central-projection range sensor. We assume we have the 3D coordinates of the points, organized with a known topology, but considerable noise is present in the data. At points in the scene where there are surface discontinuities, we estimate step rays through a linear interpolation. The viewpoint is found as the point of minimum distance to the set of step rays. To cope with noise, we define an unbiased distance measure. The minimization of the sum of distances provides the viewpoint. We present results of several experiments carried out with 3D images of an old church.

画像技術における学習・適応・進化 モーメント特徴に基づくスクリーン上の注視点位置推定法

This paper proposes viewpoint estimation method intended to be used as the basic device to estimate user's intension in a visual environment. Our method is based on moment feature which represents non-local feature of image, such as the gravity center or the breadth, and has less computational cost and considerable robustness to noise. We also propose the form and use of marker suitable for our purpose. That is, putting an oval marker on the top of a user's head. Experimental result shows that our method has good performance.