Separating reflection components in images under multispectral and multidirectional light sources

Abstract

The appearance of an object depends on the color as well as the direction of a light source illuminating the object. The progress of LEDs enables us to capture the images of an object under multispectral and multidirectional light sources. Separating diffuse and specular reflection components in those images is important for preprocessing of various computer vision techniques such as photometric stereo, material editing, and relighting. In this paper, we propose a robust method for separating reflection components in a set of images of an object taken under multispectral and multidirectional light sources. We consider the set of images as the 3D data whose axes are the pixel, the light source color, and the light source direction, and then show the inherent structures of the 3D data: the rank 2 structure derived from the dichromatic reflection model, the rank 3 structure derived from the Lambert model, and the sparseness of specular reflection components. Based on those structures, our proposed method separates diffuse and specular reflection components by combining sparse NMF and SVD with missing data. We conducted a number of experiments by using both synthetic and real images, and show that our method works better than some of the state-of-the-art techniques.