Shape from Multi-Bounce Light Paths

Abstract

Classical shape estimation techniques often have simplified but unrealistic assumptions, such as light interacting with the scene with only single-bounce light paths and the scene being Lambertian. However, a real scene often interacts with light in significantly more complex ways; multi-bounce light paths are ubiquitous and real-world materials are typically non-Lambertian. In such cases, shape estimation is not justchallenging, but also beyond the reach of commonly used techniques. We propose a shape estimation framework that deals with multi-bounce light paths. The proposed framework uses light paths, as opposed to images, as the primitive forshape estimation. The core of our idea is that we can trace the optical journey of a collection of photons as light paths with multiple bounces, where each bounce is aninstance of a light-object interaction. These interactions can often be explained by simple physical laws that govern how properties of light rays change; for example, amirror simply changes the orientation of light rays while preserving their radiance; a diffuse wall scatters light in all directions and changes the light ray’s radiance. Shapeestimation problem now reduces to identifying underlying physical phenomena related to each bounce in the light path. Our proposed shape recovery framework is particularly effective for scenes that interact with light in complex ways. We explore three such scenarios. First, we characterizehow information pertaining to the shape of a transparent object is encoded in the deflection of light rays and use it to recover the shape of transparent objects.Second, we characterize how the path length of two-bounce light paths are sufficient for concave shape recovery. Finally, we show a specialized scenario where the objectof interest can only be imaged through multi-bounce light paths. In all three scenarios, we show that by using physical properties related to each bounce of light paths, we canfind constraints on the geometry of the scene, this leads to accurate shape estimation algorithms. The techniques developed in this thesis are applicable to a wide range ofresearch fields. The image formation discussed in this thesis shares a lot of similarities in other fields, such as medical imaging, acoustic imaging, and wifi localization.We hope this thesis can inspire more researchers to deal with multi-bounce effects indifferent research fields.