The Development and Implementation of a Real-Time Depth Image Capturing System Using SoC FPGA

Abstract

It requires a great deal of computation to compute real-time depth information through imaging technology. It would not meet the needs of real-time computing if software computations only performed by ARM embedded processor collocated with Embedded Linux operating system. This paper uses SoC FPGA from Altera Cyclone V series for the real-time Stereo Vision Capturing System. Through the integration of hardware and software, hardware implementations would be conducted by algorithms of heavy computation via VHDL / Verilog, and applications with large elastic demand would be implemented by software SoC, that will help shorten the time of system development and will be beneficial to the efficiency of the implementation. In this paper, depth information of images would be calculated mainly through Sum of Absolute Difference algorithm. Two USB cameras set up in the Embedded Linux operating system, developing Video Capturing Software by using V4L2 API, the image data would be transferred to the FPGA memory through AXI Bridges, and the depth image would be obtained by computing with SAD algorithm, then the processed depth information would be synthesized and output to VGA to display. Meanwhile, information would be sent back to the HPS Memory for applications in Embedded Linux to read and use. When the real-time video is at a resolution of 640x480, the operating frequency of FPGA is 50MHz, and the time to process an image is up to 140 fps by using Stereo matching algorithm.