Iterative shape recovery from multiple images

Abstract

In general, shape from shading (SFS) involves the solution of an under-determined system, so it is difficult to always obtain a correct and unique solution. Since only one input image is used, in order to recover the shape of the object as completely as possible, the image has to be taken with careful light source placement in order to illuminate most of the object. Shape from photometric stereo avoids the under-determined problem by using three input images; however, the shape can only be recovered in the areas that are illuminated in all three images. In real life, when a larger sequence of images is available, it is possible to solve these problems. In this paper, we present a method which recovers shape from a sequence of images taken with different illumination directions. The sequence can be of any length (at least three images), and given in any order, as long as the light source directions are not coplanar. The process can be viewed as cascading shape from photometric stereo, which is formulated in the framework of the linear Kalman filter in order to iteratively recover and refine the shape and the surface albedo. The algorithm can be repeated in multiple cycles over the same sequence of images, which results in an improvement of the recovered shape and albedo. It can also be initiated at any point, stopped at any point, and continued whenever new images arrive. By allowing a longer sequence of input images, the algorithm provides enough information to incrementally recover most of a scene with shadow areas.