Flexible Rectification of a Speckle Projection System for Depth Sensing

Abstract

Rectification is a critical process for speckle projection systems to ensure reliable image matching in producing a depth map. However, few studies have discussed it up to the present. In this article, we present a flexible but robust approach to address this issue. It mainly consists of two steps: calibration and rectification. The calibration step takes one or several pairs of images of the projected speckle pattern on a plane to estimate a unit vector indicating the direction of the line where emitter center lies on. The rectification step virtually rotates the camera based on this vector to obtain rectified images. Given camera intrinsic parameters, this approach only needs a minimum configuration consisting of two freely captured speckle images of a planar surface to complete system rectification, which makes it particularly suited for the case with no access to the internal projected speckle pattern. Elaborate experiments on real speckle images and synthetic images from different camera poses are conducted, demonstrating the effectiveness and robustness of the proposed approach. Furthermore, we also demonstrate that when the internal pattern is recovered from the rectified images, it reaches an accuracy comparable with the ones from the best state-of-the-art methods in performing a stereo matching, but with a much simpler operation to obtain.