Lite-Stereo: A Resource-Efficient Hardware Accelerator for Real-Time High-Quality Stereo Estimation Using Binary Neural Network

Abstract

Stereo estimation plays a key role in many autonomous systems, such as robotics and self-driving cars. Recent work on StereoEngine, an FPGA-based accelerator for deep neural network (DNN)-based stereo estimation, has been demonstrated as a promising solution to achieve both real-time and high accuracy performance for depth sensing. However, this solution still suffers from over-utilizing the hardware resource of FPGAs. In this article, we present Lite-Stereo, a resource-efficient DNN-based stereo vision accelerator to improve the hardware efficiency for StereoEngine running on a resource-constrained FPGA. To achieve this, we design a set of optimized hardware architectures for resource-demanding bottleneck modules. In order to balance the gap between the processing speed and resource efficiency, the process elements in binary neural network modules are shared within and across modules. In addition, we provide reusing strategies on path aggregation and neighbor calculation to improve the resource efficiency of the semi-global matching module. Evaluation results demonstrate that Lite-Stereo reduces the hardware cost of ALUTs and RAM bits by 60% and 29%, respectively, without compromising the accuracy and energy efficiency compared with StereoEngine.