Epipolar Geometry-Based Side Information Creation for Multiview Wyner–Ziv Video Coding

Abstract

The side information (SI) quality significantly influences the rate-distortion (RD) performance of both monoview and multiview Wyner-Ziv (WZ) video coding. Efficient multiview WZ video coding schemes typically exploit both temporal and inter-view correlation during the SI creation process targeting to achieve high SI quality and, thus, high RD performance. In this context, the overall SI creation process typically involves fusing a temporally created SI with an inter-view created SI. Consequently, the final SI quality does not only depend on the temporal and inter-view SI creation techniques but also on the fusion process. Thus, the main objective of this paper is to propose an efficient SI creation solution for multiview WZ video coding by simultaneously tackling the two main issues aforementioned with the following technical novelty: 1) a disparity-based view synthesis technique accounting for the scene geometry to create high quality inter-view SI and 2) a time-view driven fusion technique efficiently selecting, on a pixel basis, the temporal or inter-view SI depending on which one is estimated to be closer the original WZ data. Experimental results show considerable RD performance improvements, notably gains up to 2 dB or more, regarding the best performing state-of-the-art multiview WZ video coding solutions available.