New Depth Coding Techniques With Utilization of Corresponding Video

Abstract

With the development of 3D display and interactive multimedia systems, new 3D video applications, such as 3DTV and Free Viewpoint Video, are attracting significant interests. In order to enable these new applications, new data formats including captured 2D video sequences and corresponding depth maps have been proposed. Compared to conventional video frames, depth maps have very different characteristics. First, they typically consist of homogeneous areas partitioned by sharp edges representing depth discontinuities, while the sharp discontinuities play very important roles in view rendering. Second, there exists structure similarity between depth map and corresponding video, in which the edges in depth exhibit quite similar behaviors as the edges in the corresponding video. In conventional video coding techniques with transforms followed by quantization, there usually exist large artifacts along sharp edges and it costs significant more bits to represent the edges with higher accuracy. In order to suppress the coding artifacts while preserving edges, and to better compress depth information, we propose in this paper two new depth compression techniques: Trilateral Filter and Sparse Dyadic Mode. Both techniques utilize the structure similarity between depth and corresponding video while focusing on different aspects in depth compression. As a new in-loop filter, Trilateral Filter is designed to filter depth with coding artifacts based on the proximity of pixel positions, the similarity among depth samples, as well as the similarity among the collocated pixels in the video frame. While Sparse Dyadic Mode is used as an intra mode to reconstruct depth map with sparse representations of depth blocks and effective reference of edge information from video frames. With these two new coding tools, we can achieve up to about 1.5 dB gain on rendering quality as compared to depth sequences coded using MVC under the same coding rate.