Multibaseline stereo in the presence of specular reflections

Abstract

We address the problem of accurate depth estimation using multibaseline stereo in the presence of specular reflections. Specular reflections can cause the intensity and color of corresponding points to change dramatically according to different viewpoints, thus producing severe matching errors for various stereo algorithms. In this paper we propose a new method to deal with this problem by treating specular reflections as occlusions. Our idea is to first detect specular pixels by computing the uncertainty of depth estimates. Then we combine the use of flexible windows and an adaptively selected subset of images to avoid these specular areas in all the multibaseline stereo images. Even though specularities may exist in the reference image, accurate depth is nevertheless estimated for all pixels. Experiments show that our consideration of specular reflections leads to improved stereo results.