A Real-Time High-Quality Complete System for Depth Image-Based Rendering on FPGA

Abstract

Depth image-based rendering (DIBR) techniques have recently drawn more attention in various 3D applications. In this paper, a real-time high-quality DIBR system that consists of disparity estimation and view synthesis is proposed. For disparity estimation, a local approach that focuses on depth discontinuities and disparity smoothness is presented to improve the disparity accuracy. For view synthesis, a method that contains view interpolation and extrapolation is proposed to render high-quality virtual views. Moreover, the system is designed with an optimized parallelism scheme to achieve a high throughput, and can be scaled up easily. It is implemented on an Altera Stratix IV FPGA at a processing speed of 45 frames per second for 1080p resolution. Evaluated on selected image sets of the Middlebury benchmark, the average error rate of the disparity maps is 6.02%; the average peak signal to noise ratio and structural similarity values of the virtual views are 30.07 dB and 0.9303, respectively. The experimental results indicate that the proposed DIBR system has the top-performing processing speed and its accuracy performance is among the best of state-of-the-art hardware implementations.