Gradual view refresh in depth-enhanced multiview video

Abstract

Depth-enhanced multiview video, such as the multiview video plus depth (MVD) format, can be used to provide displaying-time view adjustment capability through depth-image-based rendering (DIBR) and additional compression improvement compared to the Multiview Video Coding standard. In this paper a gradual view refresh (GVR) method is presented to code random access points and provide fast startup in streaming for MVD bitstreams. When decoding is started from a GVR point, a subset of the views can be accurately decoded, while the remaining views can be approximately reconstructed using DIBR. Perfect reconstruction of all views can be reached at a subsequent random access point. The GVR coding was found to be effective with up to 10% bitrate reduction for sequences with static camera arrangement. The use of GVR for fast startup in video streaming was found to be clearly superior to transmitting a MVD bitstream conventionally from rate-distortion point of view. It has been found in earlier studies that there seems to be a delay from stimulus onset until depth is fully perceived, hence giving a reason to believe that accurate reconstruction of all views might not be necessary immediately after starting decoding. Furthermore, it was observed in this paper that the objective picture quality reduction during GVR was only moderate, verifying the applicability of the presented GVR method.