Depth map reconstruction and rectification through coding parameters for mobile 3D video system

Abstract

Depth maps in mobile asymmetric 3D video system usually have the same video-rate as corresponding color video but lower resolution. In this system, depth maps will be re-scaled to the same resolution as the color video at the receiver side after decoding. In this case, blurring effect will appear after direct up-sampling process, causing artifacts in 3D scene reconstruction. In this paper, we propose a method for depth map reconstruction and rectification for the asymmetric coding scheme to avoid the blurring effect by using coding parameters in bit-streams. In the first step, we utilize an edge-oriented interpolation algorithm to improve the accuracy around edge-related regions for the up-sampled depth map. A weight model is built by edge and structural similarities, where edge similarity is a measurement between depth maps and their corresponding texture images, and structural similarity is for different depth maps. Based on the weight model, the up-sampling coefficients can be selected adaptively according to the edge orientation. After that, a block-based histogram rectification method is proposed to remove blurring artifacts caused by loop filter in decoder and up-sampling. A mapping function is built for the cumulative histogram of the up-sampled depth map and its original depth map to correct the error pixels in the up-sampled depth map. The block information of the histogram rectification method is extracted from the bit-stream at the decoder. The experimental results show that the proposed depth map reconstruction method has gains on both subjective and objective performances compared with existing methods. For subjective performance, our method can suppresses image blurring and preserves sharp edges in depth maps. As for objective performance, the proposed method achieves a maximum 2.93dB and an average 1.60dB PSNR gain compared the state-of-the-art methods.