A bundled-optimization model of multiview dense depth map synthesis for dynamic scene reconstruction

Abstract

Depth map is the basic requirement for all three-dimensional (3D) applications, but facing sensor noises, low frame-rate and low resolution in the procedure of data acquisition, especially in multiview cases. These problems bring obstacles to high quality 3D applications. Among the existing approaches, depth propagation is one of the promise approaches, and it can be utilized in temporal or spatial manner. However, propagation based algorithms process one aspect of the mentioned problems to pursuit local optimal solution. Actually, the process chain of depth map is from capture to application, and the optimization should be coupled instead of mutually independent. In this paper, we proposed a bundled-optimization scheme to process the thorough chain from capture to multiview dense depth map generation for the 3D applications. In this scheme, sensor noises in the captured low-resolution depth map are first detected and removed through a frequency-counting based non-linear filter. The filter refrains from the noise amplification in the procedure of depth map up-sampling. Low-pass blurring effect around high frequency areas is the by-product in up-sampling, and it is hard to detect in the depth map. We therefore propose a Blocklet based depth map optimization method for this blurring effect, and the accuracy of the high resolution depth map is then improved. Temporal depth propagation is then utilized on the optimized depth maps through the optical flow field rectified by temporal and spatial constrains. After that, a multi-set graph cut model is proposed to synthesize the multiview dense depth map. The experimental results indicate that our scheme can achieve at least 13.2575% PSNR gains when comparing to the benchmark depth map synthesis methods, and suggest the effectiveness of the proposed bundled-optimization method.