Non-rigid Motion Estimation Research Articles

Multi-View Stereo Reconstruction and Scene Flow Estimation with a Global Image-Based Matching Score

We present a new variational method for multi-view stereovision and non-rigid three-dimensional motion estimation from multiple video sequences. Our method minimizes the prediction error of the sha...

Curvature-based algorithms for nonrigid motion and correspondence estimation

We present a novel technique for utilizing the Gaussian curvature information in 3D nonrigid motion estimation in the absence of known correspondence. Differential-geometric constraints derived in the paper allow one to estimate parameters of the local affine motion model given the values of Gaussian curvature before and after motion. These constraints can be further combined with the previously known constraints based on the unit normals before and after motion. Our experiments demonstrate that the resulting hybrid algorithm is more accurate than each of its constituents and more accurate than the classical ICP algorithm. We also present a technique for curvilinear orthogonalization of quadratic Monge patches that is essential in our derivation and useful in other applications.

An integrated range-sensing, segmentation and registration framework for the characterization of intra-surgical brain deformations in image-guided surgery

Image-guided surgery (IGS) is a technique for localizing anatomical structures on the basis of volumetric image data and for determining the optimal surgical path to reach these structures, by the means of a localization device, or probe, whose position is tracked over time. The usefulness of this technology hinges on the accuracy of the transformation between the image volume and the space occupied by the patient anatomy and spanned by the probe. Unfortunately, in neurosurgery this transformation can be degraded by intra-surgical brain shift, which often measures more than 10 mm and can exceed 25 mm. We propose a method for characterizing brain shift that is based on non-rigid surface registration, and can be combined with a constitutively realistic finite element approach for volumetric displacement estimation. The proposed registration method integrates in a unified framework all of the stages required to estimate the movement of the cortical surface in the operating room: model-based segmentation of the pre-operative brain surface in magnetic resonance image data, range-sensing of the cortex in the OR, range–MR rigid transformation computation, and range-based non-rigid brain motion estimation. The brain segmentation technique is an adaptation of the surface evolution model. Its convergence to the brain boundary is the result of a speed term restricted to white and grey matter voxels made explicit by a classifier, and the final result is post-processed to yield a Closest Point Map of the brain surface in MR space. In turn, this Closest Point Map is used to produce the homologous pairs required to determine a highly efficient, 2D spline-based, Iterative Closest Point (ICP) non-rigid surface registration. The baseline for computing intra-operative brain displacement, as well as the initial starting point of the non-rigid ICP registration, is determined by a very good rigid range–MR transformation, produced by a simple procedure for relating the range coordinate system to that of the probe, and ultimately to that of the MR volume.

Rigid and non-rigid 3D motion estimation from multiview image sequences

Multiview image sequence processing has been the focus of considerable attention in recent literature. This paper presents an efficient technique for object-based rigid and non-rigid 3D motion estimation, applicable to problems occurring in multiview image sequence coding applications. More specifically, a neural network is formed for the estimation of the rigid 3D motion of each object in the scene, using initially estimated 2D motion vectors corresponding to each camera view. Non-linear error minimization techniques are adopted for neural network weight update. Furthermore, a novel technique is also proposed for the estimation of the local non-rigid deformations, based on the multiview camera geometry. Experimental results using both stereoscopic and trinocular camera setups illustrate and evaluate the proposed scheme.

FAP extraction using three-dimensional motion estimation

An integral part of the MPEG-4 standard is the definition of face animation parameters (FAPs). This paper presents a method for the determination of FAPs by using three dimensional (3-D) rigid and nonrigid motion of human facial features found from two-dimensional (2-D) image sequences. The proposed method assumes that a 3-D model has been fitted to the first frame of the sequence, tracks the motion of characteristic facial features, calculates the 3-D rigid and nonrigid motion of facial features, and through this, estimates the FAPs as defined by the MPEG-4 coding standard. The 2-D tracking process is based on a novel enhanced version of the algorithm proposed by Kanade, Lucas, and Tomasi (1994, 1991). The nonrigid motion estimation is achieved using the same tracking mechanism guided by the facial motion model implied by the MPEG-4 FAPs.

Recovery of global nonrigid motion: a model-based approach without point correspondences

We present a novel technique for the estimation of global nonrigid motion of an object's boundary without using point correspondences. A complete description of the motion of an object's boundary involves specifying a displacement vector at each point of the boundary. Such a description provides a large amount of information, which needs to be processed further for study of the global characteristics of the deformation. Nonrigid motion can be studied hierarchically in terms of global nonrigid motion and point-by-point local nonrigid motion. The technique presented gives a method for estimating a global affine or polynomial transformation between two object boundaries. The novelty of the technique lies in the fact that it does not use any point correspondences. Our method uses hyperquadric models to model the data and estimate the global deformation. We show that affine or polynomial transformation between two datasets can be recovered from the hyperquadric parameters. The usefulness of the technique is twofold. First, it paves the way for viewing nonrigid motion hierarchically in terms of global and local motion. Second, it can be used as a front end to other motion analysis techniques that assume small motion. For instance, most nonrigid motion analysis algorithms make some assumptions on the type of nonrigid motion (conformal motion, small motion, etc.) that are not always satisfied in practice. When the motion between two datasets is large, our algorithm can be used to estimate the affine transformation (which includes scale and shear) or a polynomial transformation between the two datasets, which can then be used to warp the first dataset closer to the second so as to satisfy the small-motion assumption. We present experimental results with real and synthetic two- and three-dimensional data.

Dynamic modeling of butterfly subdivision surfaces

The authors develop integrated techniques that unify physics based modeling with geometric subdivision methodology and present a scheme for dynamic manipulation of the smooth limit surface generated by the (modified) butterfly scheme using physics based "force" tools. This procedure based surface model obtained through butterfly subdivision does not have a closed form analytic formulation (unlike other well known spline based models), and hence poses challenging problems to incorporate mass and damping distributions, internal deformation energy, forces, and other physical quantities required to develop a physics based model. Our primary contributions to computer graphics and geometric modeling include: (1) a new hierarchical formulation for locally parameterizing the butterfly subdivision surface over its initial control polyhedron, (2) formulation of dynamic butterfly subdivision surface as a set of novel finite elements, and (3) approximation of this new type of finite elements by a collection of existing finite elements subject to implicit geometric constraints. Our new physics based model can be sculpted directly by applying synthesized forces and its equilibrium is characterized by the minimum of a deformation energy subject to the imposed constraints. We demonstrate that this novel dynamic framework not only provides a direct and natural means of manipulating geometric shapes, but also facilitates hierarchical shape and nonrigid motion estimation from large range and volumetric data sets using very few degrees of freedom (control vertices that define the initial polyhedron).

Volumetric Deformation Analysis Using Mechanics-Based Data Fusion: Applications in Cardiac Motion Recovery

Non-rigid motion estimation from image sequences is essential in analyzing and understanding the dynamic behavior of physical objects. One important example is the dense field motion analysis of the cardiac wall, which could potentially help to better understand the physiological processes associated with heart disease and to provide improvement in patient diagnosis and treatment. In this paper, we present a new method of estimating volumetric deformation by integrating intrinsic instantaneous velocity data with geometrical token displacement information, based upon continuum mechanics principles. This object-dependent approach allows the incorporation of physically meaningful constraints into the ill-posed motion recovery problem, and the integration of the two disparate but complementary data sources overcomes some of the limitations of the single-image-source-based motion estimation approaches.

Shape and nonrigid motion estimation through physics-based synthesis

A physics-based framework for 3-D shape and nonrigid motion estimation for real-time computer vision systems is presented. The framework features dynamic models that incorporate the mechanical principles of rigid and nonrigid bodies into conventional geometric primitives. Through the efficient numerical simulation of Lagrange equations of motion, the models can synthesize physically correct behaviors in response to applied forces and imposed constraints. Applying continuous Kalman filtering theory, a recursive shape and motion estimator that employs the Lagrange equations as a system model is developed. The system model continually synthesizes nonrigid motion in response to generalized forces that arise from the inconsistency between the incoming observations and the estimated model state. The observation forces also account formally for instantaneous uncertainties and incomplete information. A Riccati procedure updates a covariance matrix that transforms the forces in accordance with the system dynamics and prior observation history. Experiments involving model fitting and tracking of articulated and flexible objects from noisy 3-D data are described. >

Nonrigid object motion and deformation estimation from three‐dimensional data

AbstractThis article presents a model‐based approach for nonrigid object motion and deformation analysis from 3D data. The modeling primitives used in this research are the superquadrics, which have already been proven useful in describing a variety of natural and man‐made objects. This model‐based approach is not only in accordance with the human visual perception process but also able to decouple the large and unstructured nonrigid motion estimation system into simple and well structured subsystems. We develop a recursive algorithm for estimating global motion and object shape, which effectively incorporates a priori knowledge of the object into the estimation procedure and obtains a good estimate of the global motion and object shape even if the given 3D points are distributed bias. After compensating for the global motion of the object a tensor model of local deformation is introduced and a spherical harmonic surface‐fitting algorithm is described such that the localized deformations of the object surface can be characterized. The local deformations of the object are then estimated using tensor‐description‐based analysis and parametrized by the directions and magnitudes of the extreme deformations in a localized surface element. To illustrate the potential of this model‐based approach for nonrigid motion analysis, a real data example is presented using the proposed approach. This example involves estimating the left ventricle motion and deformations from a time sequence of 3D coordinates of coronary artery bifurcation points. The estimation results show the success of the model‐based approach even when the given bifurcation points are distributed only on half of the left ventricle surface.

The estimation of geometry and motion of a surface from image sequences by means of linearization of a parametric model

A method is given to estimate the geometry and motion of a moving body surface from image sequences. To this aim a parametric model of the surface is used, in order to reformulate the problem to one of parameter estimation. After linearization of the model standard linear estimation methods can be used to estimate the parameters. The main contribution of this paper is that a method is provided to perform the linearization without specifying the model. Therefore structure from motion estimation and nonrigid body motion estimation can be performed regardless of the model.