Variant center-symmetric census transform for real-time stereo vision architecture on chip

Abstract

Stereo vision is one of the most crucial operations in many computer vision applications, and stereo matching is its most important step. In recent years, stereo matching has developed in the direction of increasing accuracy and higher computational speed, and the real-time stereo matching algorithm based on hardware architecture has been increasingly emphasized due to its applicability in embedded systems. The most frequently used method on FPGA is the census transform (CT) because of its simple structure and easy parallelization and the high quality of the generated disparity maps, but CT has the drawbacks of mismatches in some regions and dependence on a central pixel. In this paper, an improved CT-based semi-global stereo matching algorithm with pipeline and parallel operation based on FPGA is proposed in order to increase matching accuracy in specific regions while satisfying real-time constraints. In the matching cost step, pixels are divided into two parts, with the two methods calculated to generate the bit-vector feature. A four-path semi-global matching algorithm with twice aggregate is proposed in the cost aggregation stage. The left–right check, mismatching point filling, and the median filter to enhance the final disparity map are also required. The novel algorithm is evaluated on the Middleburry benchmark and implemented on Xilinx ZYNQ-7000 SoCs, which results in a throughput of 640 × 480/60 fps, with 64 disparity levels at 100 MHz. Compared with the related work, we improve the average accuracy by 1.61%, making our approach suitable for real-time embedded systems.