Motion Estimation Problem Research Articles

A hierarchy of cameras for 3D photography

The view-independent visualization of 3D scenes is most often based on rendering accurate 3D models or utilizes image-based rendering techniques. To compute the 3D structure of a scene from a moving vision sensor or to use image-based rendering approaches, we need to be able to estimate the motion of the sensor from the recorded image information with high accuracy, a problem that has been well-studied. In this work, we investigate the relationship between camera design and our ability to perform accurate 3D photography, by examining the influence of camera design on the estimation of the motion and structure of a scene from video data. By relating the differential structure of the time varying plenoptic function to different known and new camera designs, we can establish a hierarchy of cameras based upon the stability and complexity of the computations necessary to estimate structure and motion. At the low end of this hierarchy is the standard planar pinhole camera for which the structure from motion problem is non-linear and ill-posed. At the high end is a camera, which we call the full field of view polydioptric camera, for which the motion estimation problem can be solved independently of the depth of the scene which leads to fast and robust algorithms for 3D Photography. In between are multiple view cameras with a large field of view which we have built, as well as omni-directional sensors.

Analysis and comparison of two methods for the estimation of 3D motion parameters

This paper analyzes the problem of motion estimation from a sequence of stereo images. Two methods are considered and their differential and discrete approaches are compared. The differential approaches use differential optical flow whereas the discrete approaches use feature correspondences. Both methods are used to compute, first, the 3D velocity in the direction of the optical axis and, next, the complete rigid motion parameters. The uncertainty propagation models for both methods and approaches are derived. These models are analyzed in order to point out the critical variables for the methods. The methods were extensively tested using synthetic images as well as real images and conclusions are drawn from the results. Real images are used without any illumination control of the scene in order to study the behavior of the methods in strongly noisy environments with low resolution depth maps.

A statistical adaptive block-matching motion estimation

We address the problem of motion estimation (ME) in digital video sequences and propose a new fast, adaptive, and efficient block-matching algorithm. Higher quality and efficiency are achieved using a statistical model for the motion vectors. This model introduces adaptation in the search window, drastically reducing the number of positions where correlation-type computation is performed. The efficiency is further improved by progressively undersampling the macroblock. Patterns for undersampling are proposed to obtain the maximum benefit from single instruction multiple data (SIMD) instructions. In contrast with existing motion-estimation techniques, search strategy and subsampled patterns are closely linked. This shows that a good search strategy is much more important than blindly reducing the number of pixels considered for the matching pattern. We describe an implementation of the proposed matching strategy that exploits the very long instruction word (VLIW) and SIMD technology available in the new Itanium processor family. Results show that the proposed algorithm adapts easily to the evolution of the scene avoiding annoying quality drops that can be observed with other deterministic algorithms. The total number of operations required by the proposed method is inferior to those required by traditional approaches.

Motion and shape identification with vision and range

In this paper, we consider the problem of motion and shape estimation using a camera and a laser range finder. The object considered is a plane which is undergoing a Riccati motion. The camera observes features on the moving plane perspectively. The range-finder camera is capable of obtaining the range of the plane along a given "laser plane", which can either be kept fixed or can be altered in time. Finally, we assume that the identification is carried out as soon as the visual and range data is available, or after a suitable temporal integration. In each of these various cases, we derive to what extent the motion and shape parameters are identifiable and characterize the results as an orbit of a suitable group. The paper does not emphasize any specific choice of algorithm.

An analysis and efficient implementation of half-pel motion estimation

Subpixel motion compensation has become a main ingredient in many modern video compression standards due to its ability to provide accurate motion prediction. To obtain fine motion vector accuracy, video encoders incorporate a motion estimation process at the subpixel level. This process is generally limited to a small area around the integer pixel motion vector in order to keep complexity small. This paper proposes an efficient motion estimation method for video encoders to generate 8/spl times/8 half-pel motion vectors. By reusing the matching costs from a previous motion estimation task, the proposed method selectively ignores redundant candidate vectors that have already been searched, thereby reducing complexity without sacrificing search quality. An analysis of the redundancy is presented and simulation results are reported which show promising reductions in the number of 8/spl times/8 search point positions. Although this paper focuses on a specific problem of half-pel motion estimation for 8/spl times/8 blocks, the approach can be generalized for efficient motion estimation for other block sizes, pixel resolutions and search ranges.

Motion estimation in the frequency domain using fuzzy c-planes clustering

A recent work explicitly models the discontinuous motion estimation problem in the frequency domain where the motion parameters are estimated using a harmonic retrieval approach. The vertical and horizontal components of the motion are independently estimated from the locations of the peaks of respective periodogram analyses and they are paired to obtain the motion vectors using a procedure proposed. In this paper, we present a more efficient method that replaces the motion component pairing task and hence eliminates the problems of the pairing method described. The method described in this paper uses the fuzzy c-planes (FCP) clustering approach to fit planes to three-dimensional (3-D) frequency domain data obtained from the peaks of the periodograms. Experimental results are provided to demonstrate the effectiveness of the proposed method.

Epipolar geometry from profiles under circular motion

Addresses the problem of motion estimation from profiles (apparent contours) of an object rotating on a turntable in front of a single camera. A practical and accurate technique for solving this problem from profiles alone is developed. It is precise enough to reconstruct the shape of the object. No correspondences between points or lines are necessary. Symmetry of the surface of revolution swept out by the rotating object is exploited to obtain the image of the rotation axis and the homography relating epipolar lines in two views robustly and elegantly. These, together with geometric constraints for images of rotating objects, are used to obtain first the image of the horizon, which is the projection of the plane that contains the camera centers, and then the epipoles, thus fully determining the epipolar geometry of the image sequence. The estimation of this geometry by this sequential approach avoids many of the problems found in other algorithms. The search for the epipoles, by far the most critical step, is carried out as a simple 1D optimization. Parameter initialization is trivial and completely automatic at all stages. After the estimation of the epipolar geometry, the Euclidean motion is recovered using the fixed intrinsic parameters of the camera obtained either from a calibration grid or from self-calibration techniques. Finally, the spinning object is reconstructed from its profiles using the motion estimated in the previous stage. Results from real data are presented, demonstrating the efficiency and usefulness of the proposed methods.

Velocity tuned filters for motion estimation and segmentation in digital image sequences

This paper addresses the problem of motion estimation, selective reconstruction, and motion-based segmentation of objects undergoing rotational and translational motion and expansion, contraction, or shear type deformation. The goal for motion estimation and selective reconstruction is to identify the motion parameters of different moving objects in the scene and derive the location of the object with a specific motion parameter at each time instance. The information so obtained can be used in various autonomous tasks that have to be completed visually, for example, visual navigation of a mobile robot or tracking a moving part on a conveyor belt. An important objective of motion based segmentation is to simultaneously estimate the motion parameters of edges, on an image, together with their corresponding locations. Based upon position and velocity information of moving edges on the image, motion based clustering and segmentation can be readily performed. The algorithms that we have developed are based on 2 D + T filters that are tuned to a specific angular velocity and translational velocity. Additionally, we propose that the problem of estimating various deformational parameters can be reduced to that of computing the parameters of a translational motion by logsampling the image. Thus, one computes the deformational parameters and the magnitude and location of an edge undergoing deformation using translational velocity tuned filters. Additionally, the algorithm we propose can be applied equally well to both spinning and orbiting motion, unifying the treatment. A specific advantage of applying tuned filters in motion parameter estimation is that various different estimation problems can be decoupled, viz. rotational motion and translational motion parameters can be computed separately. The other important advantage is that they are not based on point correspondence between one image and the next. The algorithms we propose have been tested on both real image sequences and synthesized image sequences corrupted by noise, and are shown to be accurate and robust against noise and occlusion.

Underwater Video Mosaics as Visual Navigation Maps

This paper presents a set of algorithms for the creation of underwater mosaics and illustrates their use as visual maps for underwater vehicle navigation. First, we describe the automatic creation of video mosaics, which deals with the problem of image motion estimation in a robust and automatic way. The motion estimation is based on a initial matching of corresponding areas over pairs of images, followed by the use of a robust matching technique, which can cope with a high percentage of incorrect matches. Several motion models, established under the projective geometry framework, allow for the creation of high quality mosaics where no assumptions are made about the camera motion. Several tests were run on underwater image sequences, testifying to the good performance of the implemented matching and registration methods. Next, we deal with the issue of determining the 3D position and orientation of a vehicle from new views of a previously created mosaic. The problem of pose estimation is tackled, using the available information on the camera intrinsic parameters. This information ranges from the full knowledge to the case where they are estimated using a self-calibration technique based on the analysis of an image sequence captured under pure rotation. The performance of the 3D positioning algorithms is evaluated using images for which accurate ground truth is available.

Multiview video sequence analysis, compression, and virtual viewpoint synthesis

This paper considers the problem of structure and motion estimation in multiview teleconferencing-type sequences and its application for video-sequence compression and intermediate-view generation. First, we introduce a new approach for structure estimation from a stereo pair acquired by two parallel cameras. It is based on a 2-D mesh representation of both views of the imaged scene and a parametrization of the structure information by the disparity between corresponding nodes in the image pair. Next, we describe a novel image alignment approach which can convert images captured using nonparallel cameras to coplanar-like images. This approach greatly eases the computational burden incurred by the nonparallel camera geometry, where one must consider both horizontal and vertical disparities. Finally, we present a coder for multiview sequences, which exploits the proposed alignment and structure estimation algorithm. By extracting the foreground objects and estimating the disparity field between a selected view and a reference view the coder can compress the image pair very efficiently. In the meantime, by using the coded structure information, the decoder can generate virtual viewpoints between decoded views, which can be very helpful for telepresence applications.

Rigid 3-D motion estimation using neural networks and initially estimated 2-D motion data

This paper extends a known efficient technique for rigid three-dimensional (3-D) motion estimation so as to make it applicable to motion estimation problems occuring in image sequence coding applications. The known technique estimates 3-D motion using previously evaluated 3-D correspondence. However, in image sequence coding applications, 3-D correspondence is unknown and usually only two-dimensional (2-D) motion vectors are initially available. The novel neural network (NN) introduced in this paper uses initially estimated 2-D motion vectors to estimate 3-D rigid motion, and is therefore suitable for image sequence coding applications. Moreover, it is shown that the NN introduced in this paper performs extremely well even in cases where 3-D correspondence is known with accuracy. Experimental results are presented for the evaluation of the proposed scheme.

Polynomial search algorithms for motion estimation

This paper proposes a polynomial search (PS) algorithm and architecture to solve the motion estimation problem in video coding. Simulation results show that the proposed method is not only flexible, but also requires fewer computations to achieve the same mean absolute error results (for QCIF and sub-QCIF video) compared with the existing fast-search algorithms. Finally, a VLSI architecture is also developed to efficiently implement the PS algorithm.

Ground target tracking with variable structure IMM estimator

In this paper we present the design of a Variable Structure Interacting Multiple Model (VS-IMM) estimator for tracking groups of ground targets on constrained paths using Moving Target Indicator (MTI) reports obtained from an airborne sensor. The targets are moving along a highway, with varying obscuration due to changing terrain conditions. In addition, the roads can branch, merge or cross-the scenario represents target convoys along a realistic road network with junctions, changing terrains, etc. Some of the targets may also move in an open field. This constrained motion estimation problem is handled using an IMM estimator with varying mode sets depending on the topography, The number of models in the IMM estimator, their types and their parameters are modified adaptively, in real-time, based on the estimated position of the target and the corresponding road/visibility conditions. This topography-based variable structure mechanism eliminates the need for carrying all the possible models throughout the entire tracking period as in the standard IMM estimator, significantly improving performance and reducing computational load. Data association is handled using an assignment algorithm. The estimator is designed to handle a very large number of ground targets simultaneously. A simulated scenario consisting of over one hundred targets is used to illustrate the selection of design parameters and the operation of the tracker. Performance measures are presented to contrast the benefits of the VS-IMM estimator over the Kalman filter and the standard IMM estimator, The VS-IMM estimator is then combined with multidimensional assignment to gain time-depth. The additional benefit of using higher dimensional assignment algorithms for data association is also evaluated.

Linear Differential Algorithm for Motion Recovery: AGeometric Approach

The aim of this paper is to explore a linear geometric algorithm for recovering the three dimensional motion of a moving camera from image velocities. Generic similarities and differences between the discrete approach and the differential approach are clearly revealed through a parallel development of an analogous motion estimation theory previously explored in Vieville, T. and Faugeras, O.D. 1995. In Proceedings of Fifth International Conference on Computer Vision, pp. 750–756s Zhuang, X. and Haralick, R.M. 1984. In Proceedings of the First International Conference on Artificial Intelligence Applications, pp. 366–375. We present a precise characterization of the space of differential essential matrices, which gives rise to a novel eigenvalue-decomposition-based 3D velocity estimation algorithm from the optical flow measurements. This algorithm gives a unique solution to the motion estimation problem and serves as a differential counterpart of the well-known SVD-based 3D displacement estimation algorithm for the discrete case. Since the proposed algorithm only involves linear algebra techniques, it may be used to provide a fast initial guess for more sophisticated nonlinear algorithms (Ma et al., 1998c. Electronic Research Laboratory Memorandum, UC Berkeley, UCB/ERL(M98/37)). Extensive simulation results are presented for evaluating the performance of our algorithm in terms of bias and sensitivity of the estimates with respect to different noise levels in image velocity measurements.

Pel recursive motion estimation and compensation in subbands

In this correspondence, the problem of recursive motion estimation and compensation in image subbands is considered. A pel recursive algorithm is presented for this purpose and it is shown experimentally that the motion can be compensated almost as well as in the original fields. Based on this algorithm, a scalable and recursive video coding scheme is outlined which is compared successfully to a hybrid coding scheme based on block matching.

An efficient motion estimator with application to medical image registration

Image registration is a very important problem in computer vision and medical image processing. Numerous algorithms for registering single and multi-modal image data have been reported in these areas. Robustness as well as computational efficiency are prime factors of importance in image data registration. In this paper, a robust/reliable and efficient algorithm for estimating the transformation between two image data sets of a patient taken from the same modality over time is presented. Estimating the registration between two image data sets is formulated as a motion-estimation problem. We use a hierarchical optical flow motion model which allows for both global as well as local motion between the data sets. In this hierarchical motion model, we represent the flow field with a B -spline basis which implicitly incorporates smoothness constraints on the field. In computing the motion, we minimize the expectation of the squared differences energy function numerically via a modified Newton iteration scheme. The main idea in the modified Newton method is that we precompute the Hessian of the energy function at the optimum without explicitly knowing the optimum. This idea is used for both global and local motion estimation in the hierarchical motion model. We present examples of motion estimation on synthetic and real data (from a patient acquired during pre- and post-operative stages) and compare the performance of our algorithm with that of competing ones.

An efficient motion estimator with application to medical image registration

Image registration is a very important problem in computer vision and medical image processing. Numerous algorithms for registering single and multi-modal image data have been reported in these areas. Robustness as well as computational efficiency are prime factors of importance in image data registration. In this paper, a robust/reliable and efficient algorithm for estimating the transformation between two image data sets of a patient taken from the same modality over time is presented. Estimating the registration between two image data sets is formulated as a motion-estimation problem. We use a hierarchical optical flow motion model which allows for both global as well as local motion between the data sets. In this hierarchical motion model, we represent the flow field with a B-spline basis which implicitly incorporates smoothness constraints on the field. In computing the motion, we minimize the expectation of the squared differences energy function numerically via a modified Newton iteration scheme. The main idea in the modified Newton method is that we precompute the Hessian of the energy function at the optimum without explicitly knowing the optimum. This idea is used for both global and local motion estimation in the hierarchical motion model. We present examples of motion estimation on synthetic and real data (from a patient acquired during pre- and post-operative stages) and compare the performance of our algorithm with that of competing ones.

High Performance Hierarchical Block-based Motion Estimation for Real-Time Video Coding

The paper addresses the problem of block-based motion estimation (BBME) for video sequence coding, and a hierarchical strategy that sharply reduces the computation is proposed.Estimation is performed by computing a coarse motion field calculated at the highest level of a multiresolution pyramid, and successively refining it. The preliminary estimation is performed by a full-search fixed-size block matching, whose computational load is low due to the reduced scale.At the successive levels, the initial field is updated by both increasing the spatial density and refining the vectors. In the areas where the approximation is satisfactory, the spatial sampling is left unchanged and only the vectors are updated; in the other areas, a higher number of vectors are used.A novel strategy to perform vector propagation and adaptive correction jointly has been implemented, which allows both an efficient level-to-level update of the motion field and an effective recovery from wrong estimations.The advantages of the proposed method are two-fold: firstly, it produces a motion field whose density is locally adaptive, thus allowing the transmission of a smaller number of motion vectors. Moreover, it reduces the BBME task when a more detailed estimation would not produce a significant improvement, thus achieving a lower computational load that makes it particularly suited to the implementation of real-time video codecs.

Motion and structure from two perspective views: from essential parameters to Euclidean motion through the fundamental matrix

The standard approach to the motion and structure estimation problem consists of two stages: (1) using the eight-point algorithm to estimate the nine essential parameters defined up to a scale factor and (2) refining the motion estimation based on some statistically optimal criteria, which is a nonlinear estimation problem on a five-dimensional space. Unfortunately, the results obtained are often not satisfactory. The problem is that the second stage is very sensitive to the initial guess and that it is very difficult to obtain a precise initial estimate from the first stage. This is because one performs a projection of a set of quantities that are estimated in a space of eight dimensions (by neglecting the constraints on the essential parameters), a much higher dimension than that of the real space, which is five dimensional. A novel approach is proposed by the introduction of an intermediate stage, which consists in estimating a 3×3 matrix defined up to a scale factor by imposing the rank-2 constraint (the matrix has seven independent parameters and is known as the fundamental matrix). The idea is to project parameters estimated in a high-dimensional space gradually onto a slightly lower-dimensional space, namely, from eight dimensions to seven and finally to five. The proposed approach has been tested with synthetic and real data, and a considerable improvement has been observed. The conjecture is that the imposition of the constraints arising from projective geometry should be used as an intermediate step to obtain reliable three-dimensional Euclidean motion and structure estimation from multiple calibrated images. The software is available on the Internet.

Acoustic Motion Estimation in a Structured Environment for an Unmanned Underwater Vehicle

This paper addresses the problem of slow motion estimation and localization of an Unmanned Underwater Vehicle (UUV) in the proximity of underwater structures by means of range and bearing sonar measurements. A Kalman filter motion estimation technique based on dynamic model of vehicle motion in a partially known environment is discussed. Results of pool tests carried out with a high frequency pencil beam profiling sonar mounted on a prototype UUV are presented.