Dynamic photometric stereo method using multi-tap CMOS image sensor

Abstract

Photometric stereo enables the estimation of surface normals from images that were captured using different known lighting directions. The classical photometric stereo method requires at least three images to determine the normals of a given scene. This method therefore cannot be applied to a dynamic scene, because it is assumed that the scene should remain static while the required images are captured. We present a dynamic photometric stereo method to estimate the surface normals in a dynamic scene. We use a multi-tap complementary metal-oxide-semiconductor (CMOS) image sensor to capture the input images for the photometric stereo method. The image sensor can divide the electrons from the photodiode of a single pixel into different taps of exposures, and can therefore capture multiple images under different lighting conditions with almost the same timing. We implemented a prototype camera that was synchronized with a lighting system, and subsequently realized photometric stereo of a dynamic scene.