A 1920 $\times$ 1080 25-Frames/s 2.4-TOPS/W Low-Power 6-D Vision Processor for Unified Optical Flow and Stereo Depth With Semi-Global Matching

Abstract

This paper presents a unified 6-D vision processor that enables dense real-time 3-D depth and 3-D motion perception at full-high-definition (1920 × 1080, FHD) resolution. The proposed design implements a neighbor-guided semi-global matching (NG-SGM) algorithm to unify the stereo depth and optical flow matching problem and to reduce computation by 98% compared with the original SGM. We introduce a new custom-designed, high-bandwidth coalescing crossbar circuit that automatically coalesces redundant memory accesses to mitigate the highly irregular memory accesses observed in NG-SGM. The proposed 6-D vision processor also maximizes on-chip memory reuse by using 64 on-chip rotating image buffers that cover a wide optical flow and depth disparity search range of 176 pixels per dimension. The processor implements massive parallel processing with 576 compute units that are deeply pipelined with a dependency-resolving skewed-diagonal scan to hide the dynamic and variable dependency in the pipeline. The fabricated processor performs dense NG-SGM at 25 frames/s for optical flow or 30 frames/s for stereo depth at FHD resolution while consuming only 760 mW in 28-nm CMOS.