Physics-based learning: Adaptive structured light for active stereo depth estimation

Abstract

Active stereo systems based on structured light are widely used for 3D vision in various applications, which project specially designed patterns onto object surfaces to encode each position in space for accurate 3D measurements. However, existing approaches use pre-determined patterns that are isolated from the scene properties (object reflectivity, distance, ambient light, inter-reflection), equipment (projector, camera), and reconstruction. Meanwhile, the parameters of the structured light are determined through empirical analysis or several experiments. In this paper, we propose a novel structured light design approach, named Physics-Based Learning Adaptive Structured Light (PBL-ASL), which directly learns the optimal structured light patterns from the scene. To this end, (1) we design a decoder with a non-sinusoidal error suppression module for PBL-ASL, which can accurately estimate the disparity during structured light optimization; (2) we propose a physics-based learning algorithm consisting of a self-supervised objective function and a differentiable imaging model, which computes the disparity error and back-propagates the gradient to the encoding vector to optimize the structured light. Our experiments demonstrate that PBL-ASL can significantly improve the depth estimation accuracy of active stereo systems over several state-of-art methods.