Pro-Cam SSfM: projector–camera system for structure and spectral reflectance from motion

Abstract

Image-based reconstruction of an object’s 3D shape having the wavelength-by-wavelength spectral reflectance property enables higher-fidelity object 3D modeling compared with typical RGB-based modeling. In this paper, we propose a novel projector–camera system for practical and low-cost acquisition of a dense object 3D model with the spectral reflectance property. Different from existing spectral 3D data acquisition systems that use a dedicated multispectral camera or light, we use a standard RGB camera and an off-the-shelf projector as active illumination for both the 3D reconstruction and the spectral reflectance estimation. We first propose a calibration-free multi-view structured-light method to reconstruct the 3D points while estimating the intrinsic parameters and the poses of both the camera and the projector, which are alternately moved around the object during our image acquisition procedure. We then exploit the projector for multispectral imaging and estimate the spectral reflectance of each 3D point based on a novel spectral reflectance estimation method considering the geometric relationship between the reconstructed 3D points and the estimated projector positions. Experimental results on both synthetic and real data demonstrate that our system can precisely acquire a dense spectral 3D model using off-the-shelf devices.