The VLSI implementation of a high-resolution depth-sensing SoC based on active structured light

Abstract

This paper presents the full VLSI implementation of a new high-resolution depth-sensing system on a chip (SoC) based on active infrared structured light, which estimates the 3D scene depth by matching randomized speckle patterns, akin to the Microsoft Kinect. We present a module to enhance the consistency of speckle patterns for robust matching, and hence improve the range of depth estimation. We present a simple and efficient hardware structure for a block-matching-based disparity estimation algorithm, which facilitates rapid generation of disparity maps in real time. For depth estimation from disparity maps, we propose a hardware friendly solution by producing a lookup table from a curve fitted to calculate and calibrate the depth value in a single step, which does not need to explicitly calibrate the parameters of the imaging sensors, such as the length of the baseline. We have implemented these ideas in an end-to-end SoC using FPGA. Compared with the Kinect, our depth-sensing SoC has wider effective ranging limit (0.6---4.5 m), and has an uppermost $$1280\times 1024$$1280×1024 processing capacity with 60 Hz frame frequency. Its depth resolution is 1 mm@0.82 m. Our system is superior to Kinect in terms of operating range, processing frame rate, and resolution.