Guided Stereo to Improve Depth Resolution of a Small Baseline Stereo Camera Using an Image Sequence

Abstract

Using calibrated synchronised stereo cameras significantly simplifies multi-image 3D reconstruction. This is because they produce point clouds for each frame pair, which reduces multi-image 3D reconstruction to a relatively simple process of pose estimation followed by point cloud merging. There are several synchronized stereo cameras available on the market for this purpose, however a key problem is that they often come as fixed baseline units. This is a problem since the baseline that determines the range and resolution of the acquired 3D. This work deals with the fairly common scenario of trying to acquire a 3D reconstruction from a sequence of images, when the baseline of our camera is too small. Given such a sequence, in many cases it is possible to match each image with another in the sequence that has a more appropriate baseline. However is there still value in having calibrated stereo pairs then? Clearly not using the calibrated stereo pairs reduces the problem to a monocular 3D reconstruction problem, which is more complex with known issues such as scale ambiguity. This work attempts to solve the problem by proposing a guided stereo strategy that refines the coarse depth estimates from calibrated narrow stereo pairs with frames that are further away. Our experimental results are promising, since they show that this problem is solvable provided there are appropriate frames in the sequence to supplement the depth estimates from the original narrow stereo pairs.