Real-Time Dense Stereo Matching With ELAS on FPGA-Accelerated Embedded Devices

Abstract

For many applications in low-power real-time robotics, stereo cameras are the sensors of choice for depth perception as they are typically cheaper and more versatile than their active counterparts. Their biggest drawback, however, is that they do not directly sense depth maps; instead, these must be estimated through data-intensive processes. Therefore, appropriate algorithm selection plays an important role in achieving the desired performance characteristics. Motivated by applications in space and mobile robotics, we implement and evaluate a field programmable gate arrays (FPGA) accelerated adaptation of the efficient large-scale stereo (ELAS) algorithm. Despite offering one of the best tradeoffs between efficiency and accuracy, ELAS has only been shown to run at 1.5-3 fps on a high-end CPU. Our system preserves all intriguing properties of the original algorithm, such as the slanted plane priors, but can achieve a frame rate of 47 fps whilst consuming under 4 W of power. Unlike previous FPGA-based designs, we take advantage of both components on the CPU/FPGA system-on-chip to showcase the strategy necessary to accelerate more complex and computationally diverse algorithms for such low power, real-time systems.