Sequence Of Range Images Research Articles

移動ロボットによる環境認識のための実時間距離画像位置合わせ

移動ロボットによる環境認識のための実時間距離画像位置合わせ

3D nonrigid motion analysis under small deformations

Recovery of motion parameters and point correspondences is a fundamental problem in computer vision. Although a great deal of research has been done solving rigid motion, nonrigid motion analysis has only recently been addressed, and is gaining interest due to its wide range of applications. This thesis introduces a novel method for estimating motion parameters and point correspondences between 2D curves and 3D surfaces under small nonrigid deformations. We use curvature to track point correspondences and motion parameters between 2D curves under small deformations. Curvature is invariant to the type of parameterization used, and to rigid transformations. These properties help us to deal with purely nonrigid motion, as curvature is not affected by translation/rotation of a given point. The motion parameter in this analysis is a point function, called the displacement function. In this work, it was defined as a polynomial function of a point on the curve under motion. Simulations are performed for illustrating accuracy of the algorithm's estimation by generating deformations on different curves. Real data experiments are also performed, using MR grid data, breast contours from mammogram images and contours of the left ventricle of heart. Three approaches are presented in this thesis for tracking point correspondences, and estimating motion parameters in 3D nonrigid motion. Unlike the 2D case, the motion parameter used here is a vector point function (not scalar), also called displacement function. It it represents both direction and magnitude of deformation at a given point. The first approach involves the use of changes in the discriminant for nonrigid motion analysis. Discriminant is defined as the infinitesimal area of surface at a given point. The second approach involves the use of changes in the unit normal at any given point on a surface under deformation. The third approach uses the challenges in Gaussian curvature along with the changes in unit normal direction. In all these approaches, the displacement function can be any point function representing the deformation involved. In our work, we have chosen a polynomial vector function to approximate the deformation. Simulations are performed for each approach by generating nonrigid motion on an ellipsoidal data set to illustrate performance and accuracy of the derived algorithms. Then, the algorithm is tested on a sequence of facial range images. A total of 16 sets of images were used in this experiment, where eight persons have generated two expressions each. We demonstrate the correct correspondence recovery by tracking points on the face during each facial expression. The correspondences estimation is also evaluated by comparing the algorithm's output against manual tracking of feature points by different users. In addition, nonrigid motion segmentation is successfully performed on these images. The motion parameters generated by the algorithm have also been used to demonstrate the abnormality in cardiac motion from CT images.

Investigating Anytime Algorithms for Future Distance Warning Systems

Most road-accidents are caused by a driver not paying attention to the traffic situation. This problem can be solved by assistance systems which warn the driver of potential dangers. Recent progress in sensor technology makes range sensors very attractive for automotive applications, but up to now little research has been done into the interpretation of range image sequences in outdoor environments. In this article an overall approach to the analysis of range image sequences for use in future distance warning systems is presented. A method for obstacle detection is described using radial slope evaluation and 3D connected component analysis. For obstacle tracking a new approach based on an anytime algorithm is proposed. This algorithm has an interpretation of the input data, possibly sub-optimal, ready whenever it is stopped and improves the interpretation with additional computing time. The different approaches proposed in this paper have been tested on a data set of 11 range image sequences consisting of 595 frames in total. Experimental results reveal that a reliable interpretation can be obtained in real time.

Eigen-Texture method: appearance compression and synthesis based on a 3D model

Image-based and model-based methods are two representative rendering methods for generating virtual images of objects from their real images. However, both methods still have several drawbacks when we attempt to apply them to mixed reality where we integrate virtual images with real background images. To overcome these difficulties, we propose a new method, which we refer to as the Eigen-Texture method. The proposed method samples appearances of a real object under various illumination and viewing conditions, and compresses them in the 2D coordinate system defined on the 3D model surface generated from a sequence of range images. The Eigen-Texture method is an example of a view-dependent texturing approach which combines the advantages of image-based and model-based approaches. No reflectance analysis of the object surface is needed, while an accurate 3D geometric model facilitates integration with other scenes. The paper describes the method and reports on its implementation.

Nonrigid motion analysis based on dynamic refinement of finite element models

We propose new algorithms for accurate nonrigid motion tracking. Given an initial model representing general knowledge of the object, a set of sparse correspondences, and incomplete or missing information about geometry or material properties, we can recover dense motion vectors using finite element models. The method is based on the iterative analysis of the differences between the actual and predicted behaviors. Unknown parameters are recovered using an iterative descent search for the best nonlinear finite element model that approximates nonrigid motion of the given object. During this search process, we not only estimate material properties, but also infer dense point correspondences from our initial set of sparse correspondences. Thus, during tracking, the model is refined which, in turn, improves tracking quality. Experimental results demonstrate the success of the proposed algorithm. Our work demonstrates the possibility of accurate quantitative analysis of nonrigid motion in range image sequences with objects consisting of multiple materials and 3D volumes.

Model-based force-driven nonrigid motion recovery from sequences of range images without point correspondences

In this article we propose a new method for accurate nonrigid motion analysis when point correspondence data is not available. Nonlinear finite element models are constructed by integrating range data and prior knowledge about an object's properties. The motion sequence is recovered given an initial alignment of the model with the first frame of the sequence. The main idea of the method is to find the forces that are responsible for the motion or shape deformation of the given object. The task is broken into subtasks of finding the forces for each frame. Both absolute values and directions of these forces are taken into consideration and iteratively varied not only for each frame, but also between the frames. Experimental results demonstrate the success of the proposed algorithm. The method is applied to man-made elastic materials and human hand modeling. It allows for recovery of single and multiple forces using restricted (elastic-articulated) and completely unrestricted (elastic) models. Our work demonstrates the possibility of accurate nonrigid motion analysis and force recovery from range image sequences containing nonrigid objects and large motion without interframe point correspondences.

Real-time tracking of free object based on measurement and synthesis of range image sequence

Measurement and recognition of moving objects by a visual system is crucial to realization of robots that can handle a changing environment flexibly. In this article, we propose a system that tracks 6-DOF motion of a solid object of any shape in real time using a range image sequence. We compare range images measured by the Silicon Range Finder high-speed range image sensor with those synthesized from a computer model, and we assert that motion can be estimated independently of the form of objects. Furthermore, we propose a method to generate range images from a model very rapidly with CG rendering hardware. We also describe a process to model the shape of objects simultaneously with tracking, and a method to keep the object within the measurement region by rotating the measurement direction of the sensors. We performed experiments to verify the effectiveness of our method and found that all processes were performed at a speed of 7 frames per second and that the measurement errors were less than 1% of the scope of measurement. © 1999 Scripta Technica, Syst Comp Jpn, 30(5): 56–64, 1999

Detecting 3-D motion field from range image sequences

An algorithm for computing three-dimensional (3-D) velocity field and motion parameters from a range image sequence is presented. It is based on a new integral 3-D rigid motion invariant feature-the trace of a 3x3 "feature matrix" related to the moment of inertia tensor. This trace can be computed at every point of a moving surface and provides a quantitative measure of the local shape of the surface. Based on the feature's conservation along the trajectory of every moving point, a 3-D Flow Constraint Equation is formulated and solved for the velocity field. The robustness of the feature in presence of noise and discontinuity is analyzed.

距離動画像を用いた頭部ジェスチャの認識

Gesture recognition using images is recently studied as a technology of human friendly man-machine interface. In this paper, recognition of 3D gesture by using range images is discussed. Two methods for recognizing head gestures from a sequence of range images are proposed. One uses differential range images for identifying ‘Yes’ motion, and the other detects and tracks nose position for identifying ‘Yes’ and ‘No’ motion. Experiments to recognize real person's gestures using a range sensor system are performed to show the effectiveness of the presented methods.

Obstacle Detection in Range Image Sequences Using Radial Slope

The present paper proposes an approach to obstacle detection in range image sequences of traffic scenes using radial slope. Slope is a very useful feature for determining the traversability of an environment and can be computed efficiently in radial direction. Combined with 3D sensory information, a robust detection of obstacles in real-time is possible. Once an obstacle is detected, it can be tracked over time by means of matching along the time axis. Results of our approach are shown for real as well as simulated range image sequences.

Efficient Nonlinear Finite Element Modeling of Nonrigid Objects via Optimization of Mesh Models

In this paper we propose a new general framework for the application ofthe nonlinear finite element method(FEM) to nonrigid motion analysis. We construct the models by integrating image data and prior knowledge, using well-established techniques from computer vision, structural mechanics, and computer-aided design (CAD). These techniques guide the process of optimization of mesh models.Linear FEM proved to be a successful physically based modeling tool in solving limited types of nonrigid motion problems. However, linear FEM cannot handle nonlinear materials or large deformations. Application of nonlinear FEM to nonrigid motion analysis has been restricted by difficulties with high computational complexity and noise sensitivity.We tackle the problems associated with nonlinear FEM by changing the parametric description of the object to allow easy automatic control of the model, using physically motivated analysis of the possible displacements to address the worst effects of the noise, applying mesh control strategies, and utilizing multiscale methods. The combination of these methods represents a new systematic approach to a class of nonrigid motion applications for which sufficiently precise and flexible FEM models can be built.The results from the skin elasticity experiments demonstrate the success of the proposed method. The model allows us to objectively detect the differences in elasticity between normal and abnormal skin. Our work demonstrates the possibility of accurate computation of point correspondences and force recovery from range image sequences containing nonrigid objects and large motion.

Reflectance Analysis for 3D Computer Graphics Model Generation

For synthesizing realistic images of a real three dimensional object, reflectance properties of the object surface, as well as the object shape, need to be measured. This paper describes one approach to create a three dimensional object model with physically correct reflectance properties by observing a real object. The approach consists of three steps. First, a sequence of range images and color images is measured by rotating a real object on a rotary table with fixed viewing and illumination directions. Then, the object shape is obtained as a collection of triangular patches by merging multiple range images. Second, by using the recovered object shape, color pixel intensities of the color image sequence are mapped to the object surface and separated into the diffuse and specular reflection components. Finally, the separated reflection components are used to estimate parameters of the Lambertian reflection model and a simplified Torrance–Sparrow reflection model. We have successfully tested our approach using images of a real object. Synthesized images of the object under arbitrary illumination conditions are shown in this paper.

Surface Correspondence and Motion Computation from a Pair of Range Images

The estimation of the motion transformation of a moving object from a sequence of images is of prime interest in computer vision. In this paper, the issues in estimating the motion parameters from a pair of range images are addressed. The motion estimation task, in the domain of range image sequences, has two components: (1) extract the surfaces and establish the correspondence of the surfaces over the frames in the sequence of range images, and (2) compute the motion transformation using these surface correspondences. A novel procedure based on a hypergraph representation is presented for finding surface correspondence. Two scenes are modeled as hypergraphs and the hyperedges are matched using a subgraph isomorphism algorithm. The hierarchical representation of hypergraphs not only reduces the search space significantly but also facilitates the encoding of the topological and geometrical information used to direct the search procedure. Results obtained from real range image pairs show that the algorithm is robust and performs well in presence of occlusions and incorrect segmentations. Motion transformation between image frames is computed using the planar and the quadric surface pairings. A least-squares minimization procedure is formulated that estimates the best motion transform, subject to the constraints of rigid motion. For the case of linear feature pairings, the motion computation becomes tractable because the rotation and the translation computations become independent of each other. However, for quadric surfaces this is not true. The equation to be minimized is highly nonlinear and the uniqueness of solution cannot be guaranteed. The solution obtained computes the motion by extracting unique linear features from the quadric surfaces and using them to compute the motion transformation. The main contribution of the work is a surface-based framework for motion estimation from a sequence of range images. The primary issues of correspondence and motion computation are formulated and solved in terms of the surface descriptions.

Three-dimensional object tracking using a Fourier transform of sine-coded range images

An efficient method for tracking the attitude and translation of an object from a sequence of dense range images is demonstrated. The range data are sine coded and then Fourier transformed. By this process, planar surfaces in the image produce distinctive peaks in the Fourier transform spectrum. The position of a peak provides a direct measure of the normal of the plane. Tracking the positions of these peaks enables tracking the orientation of a known object. After computing and undoing the rotation, the translation is obtained by measuring the 3-D centroid of the segmented planar surfaces. All computations are closed form. The Fourier transform method is also used for segmentation. The integrative nature of the Fourier transform is very effective in attenuating the effect of noise and outliers. Results of tracking an object in a simulated range image sequence show high accuracy [0.2 deg in orientation and 0.2% root-mean-square (rms) error in translation with respect to the width of the object]. The system runs presently at 3 to 5 frames/s. The method promises real-time (video rate) performance with the addition of accelerator hardware for computing the Fourier transform. The approach is well suited for dynamic robotic vision applications.

Principal component analysis with missing data and its application to polyhedral object modeling

Observation-based object modeling often requires integration of shape descriptions from different views. To overcome the problems of errors and their accumulation, we have developed a weighted least-squares (WLS) approach which simultaneously recovers object shape and transformation among different views without recovering interframe motion. We show that object modeling from a range image sequence is a problem of principal component analysis with missing data (PCAMD), which can be generalized as a WLS minimization problem. An efficient algorithm is devised. After we have segmented planar surface regions in each view and tracked them over the image sequence, we construct a normal measurement matrix of surface normals, and a distance measurement matrix of normal distances to the origin for all visible regions over the whole sequence of views, respectively. These two matrices, which have many missing elements due to noise, occlusion, and mismatching, enable us to formulate multiple view merging as a combination of two WLS problems. A two-step algorithm is presented. After surface equations are extracted, spatial connectivity among the surfaces is established to enable the polyhedral object model to be constructed. Experiments using synthetic data and real range images show that our approach is robust against noise and mismatching and generates accurate polyhedral object models. >

3-D Motion Analysis from Range Image Sequences

3-D Motion Analysis from Range Image Sequences

Vision-based obstacle detection and grouping for helicopter guidance

Electro-optical sensors can be used to compute range to objects in the flight path of a helicopter. The computation is based on the optical flow/motion at different points in the image. The motion algorithms provide a sparse set of ranges to discrete features in the image sequence as a function of azimuth and elevation. For obstacle avoidance guidance and display purposes, these discrete set of ranges, varying from a few hundreds to several thousands, need to be grouped into sets which correspond to objects in the real world. This paper presents a new method for object segmentation based on clustering the sparse range information provided by motion algorithms together with the spatial relation provided by the static image. The range values are initially grouped into clusters based on depth. Subsequently, the clusters are modified by using the K-means algorithm in the inertial horizontal plane and the minimum spanning tree algorithms in the image plane. The object grouping allows interpolation within a group and enables the creation of dense range maps. Researchers in robotics have used densely scanned sequence of laser range images to build three-dimensional representation of the outside world. Thus, modeling techniques developed for dense range images can be extended to sparse range images. The paper presents object segmentation results for a sequence of flight images.

Analysis of Intensity and Range Image Sequences Using Adaptive-Size Meshes

This paper presents a new technique for reconstructing and analyzing sequential images using adaptive-size physically based models. In this method, the mesh size increases or decreases dynamically during reconstruction to locate nodes near surface areas of interest (such as high curvature points) and to optimize the fitting error. Image sequences of range and intensity data are used to demonstrate the power of the technique for shape estimation. Since the consecutive frames are often similar, the sequential image can be efficiently reconstructed by the adaptive-size meshes. In addition, a priori information about nonrigidity can be included so that surface model deforms to fit moving data points while preserving some basic nonrigid constraints (e.g., isometry or conformality). Several intensity and range image sequences are reconstructed efficiently using the adaptive-size meshes. The accuracy of the model is estimated in order to demonstrate the performance of this algorithm. Implementation of the proposed algorithm with and without isometric/conformal constraints is presented. The tracking of corresponding nodes using adaptive-size meshes on the face image sequences is also presented. Performance and accuracy of derived algorithms are demonstrated on simulated data of deforming ellipsoidal and bending planar shapes. Then the algorithm is applied to real range data for bending paper and to volumetric temporal left ventricular data.

Direct estimation of range flow on deformable shape from a video rate range camera

A method of estimating range flow (space displacement vector field) on nonrigid as well as rigid objects from a sequence of range images is described. This method can directly estimate the deformable motion parameters by solving a system of linear equations that are obtained from substituting a linear transformation of nonrigid objects expressed by the Jacobian matrix into motion constraints based on an extension of the conventional scheme used in intensity image sequences. The range flow is directly computed by substituting these estimated motion parameters into the linear transformation. The algorithm is supported by experimental estimations of range flow on a sheet of paper, a piece of cloth, human skin, and a rubber balloon being inflated, using real range image sequences acquired by a video rate range camera.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Dynamic integration of height maps into a 3D world representation from range image sequences

Integration of 21/2D sketches obtained at different observation stations into a consistent world (or object) representation is one of the central issues in computer vision and robotics. The resolution and accuracy of 21/2D sketches may be different from one view point to another, and inconsistent data between different observations may occur. This article presents an approach to building a spatiotemporal representation of dynamic scenes including moving objects from a sequence of range images taken by a moving observer. A range image is transformed into a height-map representation, which is segmented into the ground plane and objects on it. In order to capture the resolution and accuracy of the range information and the consistency of the height information between different height maps, we define a reliability measure of the height information for each bucket on the height map. Using this reliability, the system finds the correspondences of both static and moving objects between different observations, and successively refines the height information and its reliability with newly acquired data, dealing with inconsistent data. Final representation of the integrated height map consists of the time stamp of the last observation, region labels of static and moving objects and their spatiotemporal properties such as height information, its reliability, and the velocities of both the observer and independently moving objects. We applied the method to road scenes physically simulated by landscape toy models and show the experimental results.