Shape from Specular Reflection and Optical Flow

Abstract

Inferring scene geometry from a sequence of camera images is one of the central problems in computer vision. While the overwhelming majority of related research focuses on diffuse surface models, there are cases when this is not a viable assumption: in many industrial applications, one has to deal with metal or coated surfaces exhibiting a strong specular behavior. We propose a novel and generalized constrained gradient descent method to determine the shape of a purely specular object from the reflection of a calibrated scene and additional data required to find a unique solution. This data is exemplarily provided by optical flow measurements obtained by small scale motion of the specular object, with camera and scene remaining stationary. We present a non-approximative general forward model to predict the optical flow of specular surfaces, covering rigid body motion as well as elastic deformation, and allowing for a characterization of problematic points. We demonstrate the applicability of our method by numerical experiments on synthetic and real data.