Single Image Based Illumination Estimation for Lighting Virtual Object in Real Scene

Abstract

Rendering virtual objects into real scenes with real illumination can greatly increase the realism of virtual objects and the consistency between the virtual and the real. The main challenge lies in illumination estimation from a single image. This article proposes a novel method of single image based illumination estimation for lighting virtual object in real scene. Only a single image, without any knowledge of the 3D geometry or reflectance, is needed, which greatly increases the applicability of the method. We first estimate coarse scene geometry and intrinsic components including shading image and reflectance image. Then the sparse radiance map of the scene is inferred based on the scene geometry and intrinsic components. Finally, the virtual objects are illuminated by the estimated sparse radiance map. Some experimental results show that this method can convincingly light virtual objects into a single real image, without any pre-recorded 3D geometry and reflectance, illumination acquisition equipments or imaging information of the image.