Real-time 3D surface modeling for image based relighting

Abstract

This paper proposes to obtain 3D modeling of object surface and to create a relit image in real-time. The proposed algorithm uses a single camera and synchronized stereo lights to capture the light field images by turning on and off the lights alternatively. The light field images from the controlled lighting approximate the reflectance models on object surfaces. The surface normal directions of object surface are modeled by the dichromatic reflectance model and Phong's shading geometry. The estimated normal map of object surface improves the rendering quality in image based relighting system. The proposed system is efficiently implemented in GPU-based programming, so it operates in real-time including 3D modeling and rendering. The proposed system is expected to be useful for video conference system, where the users can be immersed in video teleconference as if they were in a virtual environment.