No-reference depth quality assessment for texture-plus-depth images

Abstract

In 3D video (3DV) and free-viewpoint video (FVV), it is vitally important to detect the errors and assess depth quality. However, since ground-truth depth maps are often unattainable, assessing depth quality without reference becomes an imperative task for many applications. This research considers the texture-plus-depth format in 3DV and FVV, and focuses on the misalignment error at depth discontinuities. A matching algorithm between depth and texture edges is proposed to determine corresponding matching pairs and to identify serious mismatches. The matching procedure is based on both spatial distances and direction similarities between texture and depth edges, and the algorithm is performed between edge segments, instead of edge pixels, in order to improve the robustness of matching. Furthermore, an adaptive algorithm is designed to divide depth edges into segments with different lengths based on the curvature of the edges. After the matching is completed, misalignments between matching pairs are used to generate a no-reference assessment metric. Experimental results demonstrate that the proposed matching scheme is able to achieve accurate matches between depth and texture edges. More importantly, it has been shown that the correlations between the proposed metric and the widely accepted full-reference metric are greater than 0.9, making this no-reference depth quality assessment scheme suitable for contemporary 3DV and FVV applications.