Multiple Stereo Research Articles

Improved motion stereo matching based on a modified dynamic programming

A new method for computing precise depth map estimates of 3D shape of a moving object is proposed. 3D shape recovery in motion stereo is formulated as a matching optimization problem of multiple stereo images. The proposed method is a heuristic modification of dynamic programming applied to two-dimensional optimization problem. 3D shape recovery using real motion stereo images demonstrates a good performance of the algorithm in terms of reconstruction accuracy.

Practical Structure and Motion from Stereo When Motion is Unconstrained

This paper describes a system which robustly estimates motion, and the 3D structure of a rigid environment, as a stereo vision platform moves through it. The system can cope with any camera motion, and any scene structure and is successful even in the presence of large jumps in camera position between the capture of successive image pairs, and when point matching is ambiguous. The system was developed to provide robust obstacle avoidance for a partially sighted person. The process described attempts to maximise use of the abundant information present in a stereo sequence. Key features include the use of multiple stereo match hypotheses, efficient motion computation from three images, and the use of this motion to ensure reliable matching, and to eliminate multiple stereo matches. Points are reconstructed in 3D space and tracked in a static coordinate frame with a Kalman Filter. This results in good 3D scene reconstructions. Structure which is impossible to match with certainty is absent, rather than being incorrectly reconstructed. As a result, the system is appropriate for obstacle detection. The results of processing some indoor and outdoor scenes, are given in the paper, and practical issues are highlighted throughout.

Generation of multiviewpoint video from stereoscopic video

This paper proposes a new technique for generating multiviewpoint video from a two-view stereo video sequence. The two-view stereo video can be easily obtained by using inexpensive two-view stereo video capture devices. For each image pair in the two-view stereoscopic video signals, our system first estimates the corresponding points of each pixel based on an epipolar constraint. A smoothing algorithm is used to smooth the estimation result, and then generate the disparity maps for each image pair. Then the proposed system can generate multiple perspective stereo video by interpolating or extrapolating the original views based on the generated disparity maps. Compared to the traditional method of capturing the multiple perspective video directly, our method can significantly reduce both the difficulties of video capturing and processing as well as the amount of video data.

ステレオ画像からの3次元立体の復元の一手法 (第1報)

ステレオ画像からの3次元立体の復元の一手法 (第1報)

Stereo and Specular Reflection

The problem of accurate depth estimation using stereo in the presence of specular reflection is addressed. Specular reflection, a fundamental and ubiquitous reflection mechanism, is viewpoint dependent and can cause large intensity differences at corresponding points, resulting in significant depth errors. We analyze the physics of specular reflection and the geometry of stereopsis which lead to a relationship between stereo vergence, surface roughness, and the likelihood of a correct match. Given a lower bound on surface roughness, an optimal binocular stereo configuration can be determined which maximizes precision in depth estimation despite specular reflection. However, surface roughness is difficult to estimate in unstructured environments. Therefore, trinocular configurations, independent of surface roughness are determined such that at each scene point visible to all sensors, at least one stereo pair can produce correct depth. We have developed a simple algorithm to reconstruct depth from the multiple stereo pairs.

Accuracy assessment of stereo-extracted data from airborne SAR images

The paper presents a method and results of stereo extraction of planimetric and altimetric features from digital airborne SAR data on a low cost PC based stereo-workstation (DVP). A challenging study area was chosen in a mountainous area of Costa Rica. Accuracies of 24 m, 39 m and 40m have been achieved for roads, creeks and digital elevation models respectively. Better contrasts in the feature definition is a key point to improve planimetric positioning accuracies. If roads and creeks accuracies are almost independent of the location in the stereo-model, larger intersection angles enable a better digital elevation model accuracy (around 20 per cent). Surprisingly, the type of the relief does not affect the results, except for very shallow angles. The multiple stereo capability of RADARSAT will be a valuable source to ensure precise results of this research.

Stereo matching algorithm based on modified wavelet decomposition process

Multiresolutional representation such as pyramidal structures is useful for stereo matching as coarse-to-fine strategy. However, conventional pyramidal structures using Gaussian or Laplacian filters lose much information due to their low-pass filtering characteristics and also cannot obtain any spatial orientation selectivity. The adoption of wavelet transform can remedy these problems, but at the time of image translation, it changes wavelet coefficients. In this paper, a pyramid using modified wavelet decomposition process is proposed to have translation invariance. The image transformed by the proposed method is converted into appropriate multiple features without loss of information. Since the importance of each feature is determined heuristically in the multiple feature-based stereo matching method, it is very difficult to fuse them adequately. In the proposed algorithm, the weight of each feature, that is, the relative importance of each feature, is decided from the similarity between the intensities in the local region of each left and right wavelet channels. Since the window size used for the decision of weight and disparity values greatly influences the processed result, the window is adaptively determined from the disparities estimated in the coarse resolution and low-varying channel of fine resolution. The window size must be large enough to obtain signal-to-noise ratio, but not too large as to induce the effects of projective distortion. Also, a new relaxation algorithm which can reduce false matches without blurring the disparity edge is proposed. By integrating adaptive weight, variable window selection method, and relaxation process, an accurate and stable disparity map is obtained. Experimental results for various images show that the proposed algorithm has good performance even if the image has the unfavorable conditions.

Recursive resolving algorithm for multiple stereo and motion matches

In this paper, a recursive algorithm is proposed to resolve the multiple stereo and motion matches with stereo motion sequence. An extended Kalman filter is applied to filter out some of the noise and to estimate recursively both three-dimensional (3D) relative motion and the depth of objects moving in 3D space. It is assumed that the objects move independently of each other, and have a small enough size to represent them as point tokens. We derive the conditions under which ambiguity in 3D reconstruction occurs, and discuss how to escape the pitfalls with the help of motion information. Virtual objects are generated from the ambiguous multiple stereo matches, and tracked through stereo motion sequences. When any stereo match is not observed within the search range predicted by the virtually generated token, the token becomes resolved to be a false match. The innovation in the Kalman filter is used to select the best one from among the tokens sharing an observation in a sequential manner. Simulation is carried out to show the capability of resolving multiple stereo and motion matches for a synthetic stereo sequence. Experimental results with a real stereo image sequence are presented to show the effectiveness of the algorithm.

From Multiple Stereo Views to Multiple 3-D Surfaces

We present a framework for 3-dimensional surface reconstruction that can be used to model fully 3-D scenes from an arbitrary number of stereo views taken from vastly different viewpoints. This is a key step toward producing 3-D world-descriptions of complex scenes using stereo and is a very challenging problem: real-world scenes tend to contain many 3-D objects, they do not usually conform to the 2-1/2-D assumption made by traditional algorithms, and one cannot take it for granted that the computed 3-D points can easily be clustered into separate groups. Furthermore, stereo data is often incomplete and sometimes erroneous, which makes the problem even more difficult. By combining a particle-based representation, robust fitting, and optimization of an image-based objective function, we have been able to reconstruct surfaces without any a priori knowledge of their topology and in spite of the noisiness of the stereo data. Our current implementation goes through three steps—initializing a set of particles from the input 3-D data, optimizing their location, and finally grouping them into global surfaces. Using several complex scenes containing multiple objects, we demonstrate the competence of this method and its ability to merge information and thus to go beyond what can be done with conventional stereo alone.

Three-dimensional stereo by photometric ratios

We present a novel robust methodology for corresponding a dense set of points on an object surface from photometric values for three-dimensional stereo computation of depth. The methodology utilizes multiple stereo pairs of images, with each stereo pair being taken of the identical scene but under different illumination. With just two stereo pairs of images taken under two different illumination conditions, respectively, a stereo pair of ratio images can be produced, one for the ratio of left-hand images and one for the ratio of right-hand images. We demonstrate how the photometric ratios composing these images can be used for accurate correspondence of object points. Object points having the same photometric ratio with respect to two different illumination conditions constitute a well-defined equivalence class of physical constraints defined by local surface orientation relative to illumination conditions. We formally show that for diffuse reflection the photometric ratio is invariant to varying camera characteristics, surface albedo, and viewpoint and that therefore the same photometric ratio in both images of a stereo pair implies the same equivalence class of physical constraints. The correspondence of photometric ratios along epipolar lines in a stereo pair of images under different illumination conditions is therefore a robust correspondence of equivalent physical constraints, and the determination of depth from stereo can be performed without explicit knowledge of what these corresponding physical constraints actually are. Whereas illumination planning is required, our photometric-based stereo methodology does not require knowledge of illumination conditions in the actual computation of three-dimensional depth and is applicable to perspective views. This technique extends the stereo determination of three-dimensional depth to smooth featureless surfaces without the use of precisely calibrated lighting. We demonstrate experimental depth maps from a dense set of points on smooth objects of known ground-truth shape, determined to within 1% depth accuracy.

A multiple-baseline stereo

A stereo matching method that uses multiple stereo pairs with various baselines generated by a lateral displacement of a camera to obtain precise distance estimates without suffering from ambiguity is presented. Matching is performed simply by computing the sum of squared-difference (SSD) values. The SSD functions for individual stereo pairs are represented with respect to the inverse distance and are then added to produce the sum of SSDs. This resulting function is called the SSSD-in-inverse-distance. It is shown that the SSSD-in-inverse-distance function exhibits a unique and clear minimum at the correct matching position, even when the underlying intensity patterns of the scene include ambiguities or repetitive patterns. The authors first define a stereo algorithm based on the SSSD-in-inverse-distance and present a mathematical analysis to show how the algorithm can remove ambiguity and increase precision. Experimental results with real stereo images are presented to demonstrate the effectiveness of the algorithm. >

3-D OBJECT CLASSIFICATION: APPLICATION OF A CONSTRUCTIVE ALGORITHM

A system for the classification of real 3-D objects is presented. Ten objects are presented in arbitrary orientation (and position, within limits). The perception of an object is achieved by the use of multiple stereo pairs of images taken from different view positions. Classification of the spectrum of distances between edge-points perceived on an object is achieved using a constructive algorithm. Convergence to zero errors on the set of training examples is guaranteed. The generalization capability is tested on a set of 10–15 novel presentations of each object.

Geometrical Modeling from Multiple Stereo Views

The components of the Sheffield Artificial Intelligence Vision Research Unit (AIVRU) three-dimensional (3D) vision sys tem, which currently supports model-based object recognition and location, are described. Its potential for robotics applica tions is demonstrated by its guidance of a Universal Machine Intelligence robot arm in a pick-and-place task. The system comprises (1) the recovery of a sparse depth map using edge- based, passive stereo triangulation; (2) the grouping, descrip tion, and segmentation of edge segments to recover a 3D representation of the scene geometry in terms of straight lines and circular arcs; (3) the statistical combination of 3D de scriptions for object model creation from multiple stereo views and the propagation of constraints for within-view re finement ; and (4) the matching of 3D wireframe object models to 3D scene descriptions in order to recover an initial estimate of their position and orientation.

Optimal Combination and Constraints for Geometrical Sensor Data

A formalism is described for the statistical combination of geometrical information from multiple sensors. This is illus trated by its applications to stereo vision, including the com bination of multiple stereo views to increase the accuracy of a wire frame model and the consistent imposition of geometri cal constraints on such a model.

TINA: a 3D vision system for pick and place

The paper describes the Sheffield AIVRU 3D vision system for robotics. The system currently supports model-based object recognition and location; its potential for robotics applications is demonstrated by its guidance of a UMI robot arm in a pick-and-place task. The system comprises: recovery of a sparse depth map using edgebased passive stereo triangulation; grouping, description and segmentation of edge segments to recover a 3D description of the scene geometry in terms of straight lines and circular arcs; statistical combination of 3D descriptions for the purpose of object model creation from multiple stereo views, and the propagation of constraints for within-view refinement; and matching 3D wireframe models to 3D scene descriptions to recover an initial estimate of their position and orientation.

Acquiring 3-D spatial data of a real object

A method of acquiring 3-D information from a real object via a stereometric system is presented. Three-dimensional (3-D) data of an object are acquired by (1) camera calibration; (2) stereo matching; (3) multiple stereo views covering the whole object; (4) geometrical computations to determine the 3-D coordinates for each sample point of the object. The analysis and the experimental results indicate that the method implemented is capable of measuring the spatial data of a real object with satisfactory accuracy.