FPGA Design and Implementation of Kinect-Like Depth Sensing

Abstract

The Kinect depth sensor has become a versatile tool in different research fields due to its low cost and ease of use, yet its implementation still remains a black box to a large extent. Moreover, the accuracy and speed of Kinect are limited for certain applications. To boost the performance of Kinect while preserving its advantages, we have proposed a depth acquisition method that uses speckle patterns generated by commodity laser emitters as in Kinect, but the speckle density is modulated to allow higher accuracy depth measurement. In this paper, we present the field-programmable gate array (FPGA) design and implementation for the above Kinect-like depth sensing. To achieve a high-speed performance on a normal FPGA (Altera Stratix III EP3SL150), we elaborate the hardware architecture to realize a full-pipeline framework for the two main modules: 1) pixel-wise phase generation and 2) normalized cross-correlation-based phase unwrapping. Experiments show a reconstruction speed of 572 fps at a resolution of $640 \times 480$ with improved depth accuracy over Kinect and that an even higher resolution is supported. The previous pattern design and the FPGA design in this paper jointly demonstrate the feasibility of next-generation fast, accurate, and low-cost depth sensors.