Temporal pixel classification and smoothing for higher depth video compression performance

Abstract

Multi-view view plus depth (MVD) is an effective scheme for representing 3D scenes. However, weak temporal correlation of depth video, caused by inaccuracy depth estimation and capture, seriously decreases its compression ratio and deteriorates the video quality of rendered virtual views. This paper proposes a depth video preprocessing algorithm to obtain high depth video compression performance. Firstly, depth video and its associated texture video in common temporal-spatial domain (CTSD) are transformed into coordinate transform domain (CTD). Secondly, depth pixels are classified with the aid of the associated texture pixels. Then, the luminance of each pixel in the same class is smoothened by using low-pass filter. Finally, the smoothened depth video in CTD is inversely transformed to CTSD. Experimental results show that the bit rate is saved by the depth video preprocessing algorithm, ranging from 3.15% to 37.94%, while the proposed preprocessing algorithm keeps the rendering quality of virtual view images1.