Spatially-Varying Diffuse Reflectance Capture Using Irradiance Map Rendering for Image-Based Modeling Applications

Abstract

Image-based 3D modelling using Structure-form-Motion (SfM) has matured significantly over the last decade. Standard SfM methods create the object's texture from the appearance of the physical object at the time of acquisition. We propose a method for acquiring the diffuse per-point reflectance of the modelled object, as part of the image acquisition work flow, only adding one extra captured image and an irradiance rendering step, making it easy for anyone to digitize physical objects to create 3D content for AR/VR using only consumer grade hardware. Current state of the art of spatially varying reflectance capture requires either large, expensive, and purpose built setups or are optimization based approaches, whereas the proposed approach is model based. This paper proposes adding a render of irradiance with modelled camera and light source, using off the shelf hardware for image capture. The key element is taking two images at each imaging location: one with just the ambient illumination conditions, and one where the light from an on-camera flash is included. It is demonstrated how to get the ambient illumination to cancel out, and by assuming Lambertian materials, render the irradiance corresponding to the flash-only image, enabling computation of spatially varying diffuse reflectance rather than appearance. Qualitative results demonstrate the added realism of the modelled objects when used as assets in renders under varying illumination conditions, including limited outdoor scenarios. Quantitative tests demonstrate that the reflectance can be estimated correctly to within a few percent even in cases with severe un-even ambient illumination.