Perspective Camera Model Research Articles

Robust Factorization Methods Using a Gaussian/Uniform Mixture Model

In this paper we address the problem of building a class of robust factorization algorithms that solve for the shape and motion parameters with both affine (weak perspective) and perspective camera models. We introduce a Gaussian/uniform mixture model and its associated EM algorithm. This allows us to address robust parameter estimation within a data clustering approach. We propose a robust technique that works with any affine factorization method and makes it robust to outliers. In addition, we show how such a framework can be further embedded into an iterative perspective factorization scheme. We carry out a large number of experiments to validate our algorithms and to compare them with existing ones. We also compare our approach with factorization methods that use M-estimators.

Multi-View AAM Fitting and Construction.

Active Appearance Models (AAMs) are generative, parametric models that have been successfully used in the past to model deformable objects such as human faces. The original AAMs formulation was 2D, but they have recently been extended to include a 3D shape model. A variety of single-view algorithms exist for fitting and constructing 3D AAMs but one area that has not been studied is multi-view algorithms. In this paper we present multi-view algorithms for both fitting and constructing 3D AAMs.Fitting an AAM to an image consists of minimizing the error between the input image and the closest model instance; i.e. solving a nonlinear optimization problem. In the first part of the paper we describe an algorithm for fitting a single AAM to multiple images, captured simultaneously by cameras with arbitrary locations, rotations, and response functions. This algorithm uses the scaled orthographic imaging model used by previous authors, and in the process of fitting computes, or calibrates, the scaled orthographic camera matrices. In the second part of the paper we describe an extension of this algorithm to calibrate weak perspective (or full perspective) camera models for each of the cameras. In essence, we use the human face as a (non-rigid) calibration grid. We demonstrate that the performance of this algorithm is roughly comparable to a standard algorithm using a calibration grid. In the third part of the paper, we show how camera calibration improves the performance of AAM fitting.A variety of non-rigid structure-from-motion algorithms, both single-view and multi-view, have been proposed that can be used to construct the corresponding 3D non-rigid shape models of a 2D AAM. In the final part of the paper, we show that constructing a 3D face model using non-rigid structure-from-motion suffers from the Bas-Relief ambiguity and may result in a "scaled" (stretched/compressed) model. We outline a robust non-rigid motion-stereo algorithm for calibrated multi-view 3D AAM construction and show how using calibrated multi-view motion-stereo can eliminate the Bas-Relief ambiguity and yield face models with higher 3D fidelity.

Structure and motion of nonrigid object under perspective projection

The paper focuses on the problem of structure and motion of nonrigid object from image sequence under perspective projection. Many previous methods on this problem utilize the extension technique of SVD factorization based on rank constraint to the tracking matrix, where the 3D shape of nonrigid object is expressed as a weighted combination of a set of shape bases. All these solutions are based on the assumption of Affine camera model. This assumption will become invalid and cause large reconstruction errors when the object is close to the camera. In this paper, we propose two algorithms, namely the linear recursive estimation and the nonlinear optimization, to extend these methods to general perspective camera model. Both algorithms are based on the shape and motion of weak perspective projection. The former one updates the solutions from weak perspective to perspective projection by refining the scalars corresponding to the projective depths recursively. The latter one is based on nonlinear optimization by minimizing the perspective reprojection residuals. Extensive experiments on simulated data and real image sequences are performed to validate the effectiveness of our new algorithms and noticeable improvements over the previous solutions are observed.

Human skeleton proportions from monocular data

This paper introduces a novel method for estimating the skeleton proportions of a human figure from monocular data. The proposed system will first automatically extract the key frames and recover the perspective camera model from the 2D data. The human skeleton proportions are then estimated from the key frames using the recovered camera model without posture re- construction. The proposed method is tested to be simple, fast and produce satisfactory results for the input data. The human model with estimated proportions can be used in future research involving human body modeling or human motion reconstruction.

Structure from motion with wide circular field of view cameras

This paper presents a method for fully automatic and robust estimation of two-view geometry, autocalibration, and 3D metric reconstruction from point correspondences in images taken by cameras with wide circular field of view. We focus on cameras which have more than 180 degrees field of view and for which the standard perspective camera model is not sufficient, e.g., the cameras equipped with circular fish-eye lenses Nikon FC-E8 (183 degrees), Sigma 8mm-f4-EX (180 degrees), or with curved conical mirrors. We assume a circular field of view and axially symmetric image projection to autocalibrate the cameras. Many wide field of view cameras can still be modeled by the central projection followed by a nonlinear image mapping. Examples are the above-mentioned fish-eye lenses and properly assembled catadioptric cameras with conical mirrors. We show that epipolar geometry of these cameras can be estimated from a small number of correspondences by solving a polynomial eigenvalue problem. This allows the use of efficient RANSAC robust estimation to find the image projection model, the epipolar geometry, and the selection of true point correspondences from tentative correspondences contaminated by mismatches. Real catadioptric cameras are often slightly noncentral. We show that the proposed autocalibration with approximate central models is usually good enough to get correct point correspondences which can be used with accurate noncentral models in a bundle adjustment to obtain accurate 3D scene reconstruction. Noncentral camera models are dealt with and results are shown for catadioptric cameras with parabolic and spherical mirrors.

Position estimation of a mobile robot using images of a moving target in intelligent space with distributed sensors

Latest advances in hardware technology and state-of-the-art of mobile robots and artificial intelligence research can be employed to develop autonomous and distributed monitoring systems. A mobile service robot requires the perception of its present position to co-exist with humans and support humans effectively in populated environments. To realize this, a robot needs to keep track of relevant changes in the environment. This paper proposes localization of a mobile robot using images recognized by distributed intelligent networked devices in intelligent space (ISpace) in order to achieve these goals. This scheme combines data from the observed position, using dead-reckoning sensors, and the estimated position, using images of moving objects, such as a walking human captured by a camera system, to determine the location of a mobile robot. The moving object is assumed to be a point-object and projected onto an image plane to form a geometrical constraint equation that provides position data of the object based on the kinematics of the ISpace. Using the a priori known path of a moving object and a perspective camera model, the geometric constraint equations that represent the relation between image frame coordinates for a moving object and the estimated robot's position are derived. The proposed method utilizes the error between the observed and estimated image coordinates to localize the mobile robot, and the Kalman filtering scheme is used for the estimation of the mobile robot location. The proposed approach is applied for a mobile robot in ISpace to show the reduction of uncertainty in determining the location of a mobile robot, and its performance is verified by computer simulation and experiment.

보행자의 영상정보를 이용한 인간추종 이동로봇의 위치 개선

The intelligent robots that will be needed in the near future are human-friendly robots that are able to coexist with humans and support humans effectively. To realize this, robots need to recognize their position and posture in known environment as well as unknown environment. Moreover, it is necessary for their localization to occur naturally. It is desirable for a robot to estimate of his position by solving uncertainty for mobile robot navigation, as one of the best important problems. In this paper, we describe a method for the localization of a mobile robot using image information of a moving object. This method combines the observed position from dead-reckoning sensors and the estimated position from the images captured by a fixed camera to localize a mobile robot. Using a priori known path of a moving object in the world coordinates and a perspective camera model, we derive the geometric constraint equations which represent the relation between image frame coordinates for a moving object and the estimated robot's position. Also, the control method is proposed to estimate position and direction between the walking human and the mobile robot, and the Kalman filter scheme is used for the estimation of the mobile robot localization. And its performance is verified by the computer simulation and the experiment.

이동물체의 기하학적 위치정보를 이용한 자율이동로봇의 위치추정

가까운 미래에 필요로 하게 될 지능형 로봇은 인간과 공존하고 효과적으로 인간을 도울 수 있는 인간 친화적인 로봇이다. 이러한 목적을 실현하기 위해서 무엇보다 로봇은 주어진 환경에서만 아니라 미지의 환경에서도 주행중인 자신의 위치와 자세를 인식할 수 있어야 한다. 더욱이, 자신의 위치가 자연스럽게 인식될 수 있어야 할 것이다. 그래서 이동로봇의 주행에서 발생되는 불확실을 해결함으로써 로봇의 위치를 추정할 수 있어야 할 것이다. 본 논문에서는 이동물체의 영상정보를 이용하여 이동로봇의 자기위치추정을 위한 방법을 제시하고 있다. 이것은 엔코더의 관측 위치정보와 이동로봇의 위치정보를 추정하기 위한 카메라영상에서의 추정된 위치정보를 결합하는 방법이다. 기준좌표계상에 이동물체의 사전 경로정보와 투영된 카메라 모델, 기하학적 구속식을 이용함으로써 이동물체에 대한 영상 좌표와 추정된 이동로봇의 위치 정보간의 관계를 표현 할 수 있다. 제시된 식은 추정된 위치에 근거하기 때문에 측정오차를 항상 가지게 된다. 제안된 방법은 관측되고 추정된 영상좌표간의 오차정보를 이용하여 이동로봇의 위치를 추정할 수 있었다. 이러한 추정을 위해서 칼만필터를 적용하였으며 제안된 방법의 성능은 시뮬레이션과 실험을 통하여 제시하고 있다. The intelligent robots that will be needed in the near future are human-friendly robots that are able to coexist with humans and support humans effectively. To realize this, robots need to recognize their position and posture in known environment as well as unknown environment. Moreover, it is necessary for their localization to occur naturally. It is desirable for a robot to estimate of his position by solving uncertainty for mobile robot navigation, as one of the best important problems. In this paper, we describe a method for the localization of a mobile robot using image information of a moving object. This method combines the observed position from dead-reckoning sensors and the estimated position from the images captured by a fixed camera to localize a mobile robot. Using the a priori known path of a moving object in the world coordinates and a perspective camera model, we derive the geometric constraint equations which represent the relation between image frame coordinates for a moving object and the estimated robot's position. Since the equations are based or the estimated position, the measurement error may exist all the time. The proposed method utilizes the error between the observed and estimated image coordinates to localize the mobile robot. The Kalman filter scheme is applied for this method. its performance is verified by the computer simulation and the experiment.

Localization of a mobile robot using the image of a moving object

This paper proposes a new approach for determining the location of a mobile robot using the image of a moving object. This scheme combines data from the observed position, using dead-reckoning sensors, and the estimated position, using images of moving objects captured by a fixed camera to determine the location of a mobile robot. Using the a priori known path of a moving object and a perspective camera model, the geometric constraint equations that represent the relation between image frame coordinates for a moving object and the estimated robot's position are derived. Since the equations are based on estimated position, measurement error may exist. However, the proposed method utilizes the error between the observed and estimated image coordinates to localize the mobile robot, and the Kalman filtering scheme is used for the estimation of the mobile robot location. The proposed approach is applied for a moving object on the wall to show the reduction of uncertainty in the determining of mobile robot location.

Volume visualization of multicolor laser confocal microscope data

The objective of this work is to give biologists the ability to visualize multicolor laser confocal microscope (MLCM) data sets. We present the application of volume ray casting techniques to create fast and accurate visualization of the spatial structures in MLCM data sets. We provide various volume pre-processing routines for noise removal, smoothing and edge enhancement. The tool has an interactive interface to assist the operator in classifying the volume using various transparency mappings. We have developed volume rendering using volumetric ray casting based upon a perspective camera model. We illustrate an innovative way to simultaneously visualize LCM data set from three lasers using voxel level merging. A method to generate key frame animation sequence of an arbitrary fly through path is also included in the visualization tool.

Robust stabilization of second-order image-based affine systems

In this paper, the problem of robust relative positioning between a 6-DOF robot camera and an object of interest is considered. Assuming weak perspective camera model and local linear approximation of visible object's surface, an image-based state space representation of robot camera–object interaction model is derived, based on the matrix of 2-D affine transformations. Dynamic extension of the visual model permits to estimate 3-D parameters directly as functions of state variables. The proposed nonlinear robust control law ensures asymptotic stability but at image singularities, assuming exact model and state measurements. In the presence of bounded uncertainties, under appropriate choice of control gains, ultimate boundedness of the state error is also formally proved. Simulation results validate the theoretical framework both in terms of system convergence and control robustness.

Pose Estimation using Point and Line Correspondences

The problem of a real-time pose estimation between a 3D scene and a single camera is a fundamental task in most 3D computer vision and robotics applications such as object tracking, visual servoing, and virtual reality. In this paper we present two fast methods for estimating the 3D pose using 2D to 3D point and line correspondences. The first method is based on the iterative use of a weak perspective camera model and forms a generalization of DeMenthon's method (1995) which consists of determining the pose from point correspondences. In this method the pose is iteratively improved with a weak perspective camera model and at convergence the computed pose corresponds to the perspective camera model. The second method is based on the iterative use of a paraperspective camera model which is a first order approximation of perspective. We describe in detail these two methods for both non-planar and planar objects. Experiments involving synthetic data as well as real range data indicate the feasibility and robustness of these two methods. We analyse the convergence of these methods and we conclude that the iterative paraperspective method has better convergence properties than the iterative weak perspective method. We also introduce a non-linear optimization method for solving the pose problem.

Finding the epipole from uncalibrated optical flow

This article presents a novel method for determining the location of the instantaneous epipole in a sequence of images acquired by an uncalibrated camera and containing a single, rigid motion (e.g. the camera moves in a static environment). The method uses the full perspective camera model and requires the estimation of the optical flow at a minimum of six image locations. The key observation is that the optical flow equations can be written in terms of the epipole in a strikingly simple form if the translational and rotational flow components are not separated as performed usually. The epipole location can then be obtained as the minimum of a least-square residual function associated to the computed optical flow. We report and discuss initial experiments on both synthetic and real data and illustrate possible developments of this method towards the use of uncalibrated optical flow for 3-D motion and structure reconstruction.

Iterative Pose Computation from Line Correspondences

This paper presents a method for estimating the position and orientation of a camera with respect to a known 3-D object from line correspondences. The main idea of the method is to estimate a pose with either a weak perspective or a paraperspective camera model and to improve this pose iteratively. At convergence the result is compatible with a perspective camera model. This iterative improvement of a linear (affine) camera model has already been used for points but has never been extended to lines. Known methods which compute pose from line correspondences deal with a set of nonlinear equations which are solved either in closed-form or using minimization techniques. These methods have to deal with multiple solutions. In contrast our method starts with a solution which is very close to the true solution and converges in very few iterations (typically three to five iterations). The rank analysis of the linear system to be solved at each iteration allows us to characterize geometric configurations which defeat the algorithm.

Factorization methods for structure from motion

In this article we present an overview of factorization methods for recovering structure and motion from image sequences. We distinguish these methods from general nonlinear algorithms primarily by their bilinear formulation in motion and shape parameters. The bilinear formulation makes possible powerful and efficient solution techniques including singular value decomposition. We show how factorization methods apply under various affine camera models and under the perspective camera model, and then we review factorization methods for various features including points, lines, directional point features and line segments. An extension to these methods enables them to segment and recover motion and shape for multiple independently moving objects. Finally we illustrate the generality of the factorization methods with two applications outside structure from motion.

Object Pose: The Link between Weak Perspective,Paraperspective, and Full Perspective

Recently, DeMenthon and Davis (1992, 1995) proposed a method for determining the pose of a 3-D object with respect to a camera from 3-D to 2-D point correspondences. The method consists of iteratively improving the pose computed with a weak perspective camera model to converge, at the limit, to a pose estimation computed with a perspective camera model. In this paper we give an algebraic derivation of DeMenthon and Davis‘ method and we show that it belongs to a larger class of methods where the perspective camera model is approximated either at zero order (weak perspective) or first order (paraperspective). We describe in detail an iterative paraperspective pose computation method for both non coplanar and coplanar object points. We analyse the convergence of these methods and we conclude that the iterative paraperspective method (proposed in this paper) has better convergence properties than the iterative weak perspective method. We introduce a simple way of taking into account the orthogonality constraint associated with the rotation matrix. We analyse the sensitivity to camera calibration errors and we define the optimal experimental setup with respect to imprecise camera calibration. We compare the results obtained with this method and with a non-linear optimization method.

Euclidean shape and motion from multiple perspective views by affine iterations

We describe a method for solving the Euclidean reconstruction problem with a perspective camera model by incrementally performing Euclidean reconstruction with either a weak or a paraperspective camera model. With respect to other methods that compute shape and motion from a sequence of images with a calibrated camera, this method converges in a few iterations, is computationally efficient, and solves for the sign (reversal) ambiguity. We give a detailed account of the method, analyze its convergence, and test it with both synthetic and real data.

An analytic solution for the perspective 4-point problem

The perspective n-point (P nP) problems is the problem of dinding the position and orientation of a camera with respect to a scene object from n correspondence points. In this paper we propose an analytic solution for the perspective 4-point problem. The solution is found by replacing the four points with a pencil of three lines and by exploring the geometric constraints available with the perspective camera model. We show how the P4P problem is cast into the problem of solving a biquadratic polynomial in one unknown. Although developed as part of an object recognition from a single view system [6], the solution migth well be used for hand-eye coordination, landmark-guided navigation, and for fast determination of exterior camera parameters in general.