Camera Motion Parameters Research Articles

Towards Unrestrained Depth Inference with Coherent Occlusion Filling

Traditional depth estimation methods typically exploit the effect of either the variations in internal parameters such as aperture and focus (as in depth from defocus), or variations in extrinsic parameters such as position and orientation of the camera (as in stereo). When operating off-the-shelf (OTS) cameras in a general setting, these parameters influence the depth of field (DOF) and field of view (FOV). While DOF mandates one to deal with defocus blur, a larger FOV necessitates camera motion during image acquisition. As a result, for unfettered operation of an OTS camera, it becomes inevitable to account for pixel motion as well as optical defocus blur in the captured images. We propose a depth estimation framework using calibrated images captured under general camera motion and lens parameter variations. Our formulation seeks to generalize the constrained areas of stereo and shape from defocus (SFD)/focus (SFF) by handling, in tandem, various effects such as focus variation, zoom, parallax and stereo occlusions, all under one roof. One of the associated challenges in such an unrestrained scenario is the problem of removing user-defined foreground occluders in the reference depth map and image (termed inpainting of depth and image). Inpainting is achieved by exploiting the cue from motion parallax to discover (in other images) the correspondence/color information missing in the reference image. Moreover, considering the fact that the observations could be differently blurred, it is important to ensure that the degree of defocus in the missing regions (in the reference image) is coherent with the local neighbours (defocus inpainting).

Penalizing Closest Point Sharing for Automatic Free Form Shape Registration

For accurate registration of overlapping free form shapes, different points in one shape must select different points in another as their most sensible correspondents. To reach this ideal state, in this paper we develop a novel algorithm to penalize those points in one shape that select the same closest point in another as their tentative correspondents. The novel algorithm then models the relative weight change over time of a tentative correspondence as the difference between the negative functions of the numbers of points in one shape that actually and ideally select the same closest point in another. Such modeling results in an optimal estimation of the weights of different tentative correspondences, in the sense of deterministic annealing, that lead the camera motion parameters to be estimated in the weighted least squares sense. The proposed algorithm is initialized using the pure translational motion derived from the centroids difference of the overlapping free form shapes being registered. Experimental results show that it outperforms three selected state-of-the-art algorithms on the whole for the accurate and robust registration of real overlapping free form shapes captured using two different laser scanners under typical imaging conditions.

Free form shape registration using the barrier method

This paper formalizes overlapping free form shape registration as a minimization problem, which minimizes a weighted sum of registration errors of tentative correspondences with the weights subject to boundary conditions. The tentative correspondences are established using the traditional closest point criterion. Then the powerful barrier method is employed to transform the constrained minimization problem to an unconstrained one. Setting the first order derivative of the unconstrained objective function to zero results in the weights being solved with a closed form solution. The weights are finally globally optimized using the deterministic annealing scheme. Outliers due to occlusion, appearance and disappearance of points in either free form shape are explicitly modelled using a constant. The camera motion parameters are updated in the weighted least squares sense. A comparative study based on both synthetic data and real images shows that the proposed algorithm is promising for the accurate and robust automatic registration of overlapping free form shapes.

Synthetic Video Generation for Evaluation of Sprite Generation

Sprite generation is the process of aligning, warping, and blending of pixels that belong to an object in a video. The evaluation of the correctness of a sprite is usually accomplished by a combination of objective and subjective evaluations. Availability of ground-truth image would help mere objective evaluation. In this paper, the authors present video generation from an image based on various camera motion parameters to be used as ground-truth for the sprite evaluation. This paper introduces a framework for evaluation of sprite generation algorithms. Experiments under the proposed framework were performed on the synthetic videos of different camera motion patterns to reveal the components of the sprite generation algorithm to be improved.

A Bayesian Reconstruction Method with Marginalized Uncertainty Model for Camera Motion in Microrotation Imaging

Reconstruction of a 3-D structure from multiple projection images requires prior knowledge of projection directions or camera motion parameters that describe the relative positions and orientations of 3-D structure with respect to the camera. These parameters can be estimated using, for instance, the conventional correlation alignment and feature-based methods. However, the alignment methods are not perfect, where the inaccuracy of the estimated motion parameters causes artifacts in the reconstruction. To overcome this problem, we propose a bayesian approach to reconstruct the object that takes the motion uncertainty distribution into account. Moreover, we consider the motion parameters as nuisance parameters and integrate them out from the posterior distribution, assuming a gaussian uncertainty model, which yields a statistical cost function to be minimized. The proposed method is applied in microrotation fluorescence imaging, where we aim at 3-D reconstruction of a rotating object from an image series, acquired by an optical microscope. The experiments with simulated and real microrotation datasets demonstrate that the proposed method provides visually and numerically better results than the traditional reconstruction methods, which ignore the uncertainty of the motion estimates.

Camera Motion-Based Analysis of User Generated Video

In this paper we propose a system for the analysis of user generated video (UGV). UGV often has a rich camera motion structure that is generated at the time the video is recorded by the person taking the video, i.e., the ?camera person.? We exploit this structure by defining a new concept known as camera view for temporal segmentation of UGV. The segmentation provides a video summary with unique properties that is useful in applications such as video annotation. Camera motion is also a powerful feature for identification of keyframes and regions of interest (ROIs) since it is an indicator of the camera person's interests in the scene and can also attract the viewers' attention. We propose a new location-based saliency map which is generated based on camera motion parameters. This map is combined with other saliency maps generated using features such as color contrast, object motion and face detection to determine the ROIs. In order to evaluate our methods we conducted several user studies. A subjective evaluation indicated that our system produces results that is consistent with viewers' preferences. We also examined the effect of camera motion on human visual attention through an eye tracking experiment. The results showed a high dependency between the distribution of fixation points of the viewers and the direction of camera movement which is consistent with our location-based saliency map.

Automatic Range Image Registration in the Markov Chain

In this paper, a novel entropy that can describe both long and short-tailed probability distributions of constituents of a thermodynamic system out of its thermodynamic limit is first derived from the Lyapunov function for a Markov chain. We then maximize this entropy for the estimation of the probabilities of possible correspondences established using the traditional closest point criterion between two overlapping range images. When we change our viewpoint to look carefully at the minimum solution to the probability estimate of the correspondences, the iterative range image registration process can also be modeled as a Markov chain in which lessons from past experience in estimating those probabilities are learned. To impose the two-way constraint, outliers are explicitly modeled due to the almost ubiquitous occurrence of occlusion, appearance, and disappearance of points in either image. The estimated probabilities of the correspondences are finally embedded into the powerful mean field annealing scheme for global optimization, leading the camera motion parameters to be estimated in the weighted least-squares sense. A comparative study using real images shows that the proposed algorithm usually outperforms the state-of-the-art ICP variants and the latest genetic algorithm for automatic overlapping range image registration.

Triplet-based sequential merging approach to omnidirectional camera motion reconstruction

A frame grouping and merging scheme of sub-sequences to estimate precise camera motion parameters over long omnidirectional sequences is addressed. In this triplet-based approach, the back-projection errors of corresponding points by the camera model for grouping subsequences are used. This process accounts for the relative reliability of the estimated camera model in field of view. Experimental results show that the proposed method solves the problem inherent in sequential algorithm: its susceptibility to drift over long sequences.

Replicator Dynamics in the Iterative Process for Accurate Range Image Matching

Iterative algorithms are often used for range image matching. In this paper, we treat the iterative process of range image matching as a live biological system: evolving from one generation to another. Whilst different generations of the population are regarded as range images captured at different viewpoints, the iterative process is simulated using time. The well-known replicator equations in theoretical biology are then adapted to estimate the probabilities of possible correspondences established using the traditional closest point criterion. To reduce the effect of image resolutions on the final results for efficient and robust overlapping range image matching, the relative fitness difference (rather than the absolute fitness difference) is employed in the replicator equations in order to model the probability change of possible correspondences being real over successive iterations. The fitness of a possible correspondence is defined as the negative of a power of its squared Euclidean distance. While the replicator dynamics penalize those individuals with low fitness, they are further penalised with a parameter, since distant points are often unlikely to represent their real replicators. While the replicator equations assume that all individuals are equally likely to meet each other and thus treat them equally, we penalise those individuals competing for the same points as their possible replicators. The estimated probabilities of possible correspondences being real are finally embedded into the powerful deterministic annealing scheme for global optimization, resulting in the camera motion parameters being estimated in the weighted least squares sense. A comparative study based on real range images with partial overlap has shown that the proposed algorithm is promising for automatic matching of overlapping range images.

Fast Camera Motion Estimation in MPEG Compressed Domain

In this paper, we propose a new algorithm for fast estimation of camera motion directly in MPEG compressed domain, which starts from an existing and commonly used camera motion model to characterize the relationship between corresponding points of the neighboring frames, and then exploits the MPEG motion estimation and compensation techniques to derive closed-form estimation of all the three camera motion parameters, including pan, tilt, and zoom. The proposed algorithm features in extremely low computing cost and high processing speed with 50% improvement over the representative existing algorithms. It is suitable for implementation on hardware, such as Pan-Tilt-Zoom (PTZ) video camera. Furthermore, the three separated processing units (pan, tilt, and zoom) can be integrated into one simplified Fast PTZ Processing Unit. Comparative experiments also reveal that the proposed algorithm achieves similar estimation precision for all the three camera motion parameters.

Robust Software for Computing Camera Motion Parameters

We revisit the method of Tsai, Huang, and Zhu for the computation of camera motion parameters in computer vision. We elucidate some spectral properties of the homography matrices that arise, which are rank-one perturbations of rotation matrices. We show how to correct for noise by finding the rank-one perturbation of a rotation closest to a give matrix. We illustrate some of the inaccuracies and computational failures that can arise when using the formulas given by Tsai, and we propose new formulas that avoid these pitfalls. A computational experiment shows that the new methods are indeed quite robust.

Camera Motion Estimation Through Planar Deformation Determination

In this paper, we propose a global method for estimating the motion of a camera which films a static scene. Our approach is direct, fast and robust, and deals with adjacent frames of a sequence. It is based on a quadratic approximation of the deformation between two images, in the case of a scene with constant depth in the camera coordinate system. This condition is very restrictive but we show that, provided translation and depth inverse variations are small enough, the error on optical flow involved by the approximation of depths by a constant is small. In this context, we propose a new model of camera motion which allows to separate the image deformation in a similarity and a “purely” projective application, due to change of optical axis direction. This model leads to a quadratic approximation of image deformation that we estimate with an M-estimator; we can immediately deduce camera motion parameters.

Real-time camera tracking for marker-less and unprepared augmented reality environments

For three-dimensional video-based augmented reality applications, accurate measurements of the 6DOF camera pose relative to the real world are required for proper registration of the virtual objects. This paper presents an accurate and robust system for real-time 6DOF camera pose tracking based on natural features in an arbitrary scene. Crucially, the calculation is based on pre-captured reference images. This prevents a gradual increase in the camera position error. Point features in the current image frame are first matched to two spatially separated reference images. This wide baseline correspondence problem is overcome by constructing (1) a global homography between current and previous image frame and (2) local affine transforms derived from known matches between previous frame and reference images. Chaining these two mappings constrains the search for potential matches in the reference images and allows the warping of corner intensity neighborhoods so that a viewpoint invariant similarity measure for assessing potential point matches can be defined. We then minimize deviations from the two-view and three-view constraints between the reference images and current frame as a function of the camera motion parameters to obtain an estimate of the current camera pose relative to the reference images. This calculation is stabilized using a recursive form of temporal regularization similar in spirit to the Kalman filter. We can track camera pose reliably over hundreds of image frames and realistically integrate three-dimensional virtual objects with only slight jitter. This paper also tries to simplify the above described algorithm and present a real-time, robust tracking system based on computing homographies. Homography can exactly describe the image motion between two frames when the camera motion is pure rotation, or it is viewing a planar scene. For outdoor registration applications, the system is robust under small translations as long as the majority of the scene contents are distant.

A mean field annealing approach to accurate free form shape matching

The SoftAssign algorithm is an elegant free form shape matching algorithm. While its objective function can be interpreted as consisting of three desired terms: minimising a weighted sum of matching errors of combinations of all the points in the two free form shapes to be matched, equalising their weights (probabilities) of being real ones and also maximising the overlapping area between the free form shapes to be matched, the last term has no effect on the optimisation of the parameters of interest due to normalisation. In this paper, we reformulate the last two terms using the inequality about the geometric and algebraic averages and the sum of the powers of these probabilities. For the sake of computational efficiency, instead of considering combinations of all the points in the overlapping free form shapes to be matched, we employ the traditional closest point criterion to establish possible correspondences between the two overlapping free form shapes to be matched. The saddle point solution of the resulting objective function no longer yields a closed form solution to the parameters of interest. For easy computation, we then adopt a pseudo-linearisation method to linearise the first order derivative of the objective function, leading the parameters of interest to be tracked and estimated with a closed form solution. The parameters of interest are finally optimised using the efficient deterministic annealing scheme with the camera motion parameters estimated using the quaternion method in the weighted least squares sense. A comparative study based on both synthetic data and real images with partial overlap has shown that the proposed algorithm is promising for the automatic matching of overlapping 3D free form shapes subject to a large range of motions.

Compressed domain video retrieval using object and global motion descriptors

Video content description has become an important task with the standardization effort of MPEG-7, which aims at easy and efficient access to visual information. In this paper we propose a system to extract object-based and global features from compressed MPEG video using the motion vector information for video retrieval. The reliability of the motion information is enhanced by a motion accumulation process. The global features like motion activity and camera motion parameters are extracted from the above enhanced motion information. The object features such as speed, area and trajectory are then obtained after the proposed object segmentation. The number of objects in a given video shot is determined by the proposed K-means clustering procedure. The object segmentation is done by applying EM algorithm.

Novel Approach to Determining Camera Motion Parameters

We discuss the expression of space rotation in terms of complex numbers and present explicit formulas for computing the direction of the rotation axis and angle. These formulas are applied to determining camera motion parameters from two perspective views. Our approach requires only 3 space points and their physical distances from the camera to determine motion parameters. We compare the proposed method with other methods and check it numerically.

The Number of Independent Kruppa Constraints from N Images

It is well known that without any priori knowledge on the scene, camera motion and camera intrinsic parameters, the only constraint between a pair of images is the so-called epipolar constraint, or equivalently its fundamental matrix. For each fundamental matrix, at most two independent constraints on the cameras’ intrinsic parameters are available via the Kruppa equations. Given N images, N(N−1)/2 fundamental matrices appear, and N(N−1) Kruppa constraints are available. However, to our knowledge, a formal proof of how many independent Kruppa constraints exist out of these N(N−1) ones is unavailable in the literature. In this paper, we prove that given N images captured by a pinhole camera with varying parameters and under general motion, the number of independent Kruppa constraints is (5N−9) (N > 2), and it is less than that of independent constraints from the absolute quadric by only one. This one-constraint-less property of the Kruppa equations is their inherent deficiency and is independent of camera motion. This deficiency is due to their failure of automatic enforcement of the rank-three-ness on the absolute quadric.

A robust real-time video stabilization algorithm

The acquisition of digital video usually suffers from undesirable camera jitters due to unstable camera motions. In this paper, we propose a robust real-time video stabilization algorithm that alleviates the undesirable jitter motions from the unstable video to produce a stabilized video. In the proposed algorithm, we first compute the sparse optical flow vectors between successive frames, followed by estimating the camera motion by fitting the computed optical flow vectors to a simplified affine motion model with a robust trimmed least squares method. Then the computed camera motion parameters are smoothed temporally to reduce the motion fluctuations by using a regularization method. Finally, we transform all frames in the video sequence based on the original and smoothed camera motions to obtain a stabilized video. Experimental results are given to demonstrate the stabilization performance and the efficiency of the proposed algorithm.

Camera Motion Parameter Estimation Technique using 2D Homography and LM Method based on Invariant Features

In this paper, we propose a method to estimate camera motion parameter based on invariant point features. Typically, feature information of image has drawbacks, it is variable to camera viewpoint, and therefore information quantity increases after time. The LM(Levenberg-Marquardt) method using nonlinear minimum square evaluation for camera extrinsic parameter estimation also has a weak point, which has different iteration number for approaching the minimal point according to the initial values and convergence time increases if the process run into a local minimum. In order to complement these shortfalls, we, first propose constructing feature models using invariant vector of geometry. Secondly, we propose a two-stage calculation method to improve accuracy and convergence by using homography and LM method. In the experiment, we compare and analyze the proposed method with existing method to demonstrate the superiority of the proposed algorithms.

Structure-from-motion using lines: Representation, triangulation, and bundle adjustment

We address the problem of camera motion and 3D structure reconstruction from line correspondences across multiple views, from initialization to final bundle adjustment. One of the main difficulties when dealing with line features is their algebraic representation. First, we consider the triangulation problem. Based on Plücker coordinates to represent the 3D lines, we propose a maximum likelihood algorithm, relying on linearizing the Plücker constraint and on a Plücker correction procedure, computing the closest Plücker coordinates to a given 6-vector. Second, we consider the bundle adjustment problem, which is essentially a nonlinear optimization process on camera motion and 3D line parameters. Previous overparameterizations of 3D lines induce gauge freedoms and/or internal consistency constraints. We propose the orthonormal representation, which allows handy nonlinear optimization of 3D lines using the minimum four parameters with an unconstrained optimization engine. We compare our algorithms to existing ones on simulated and real data. Results show that our triangulation algorithm outperforms standard linear and bias-corrected quasi-linear algorithms, and that bundle adjustment using our orthonormal representation yields results similar to the standard maximum likelihood trifocal tensor algorithm, while being usable for any number of views.