A Novel Rate Control Method for Free-viewpoint Video in MV-HEVC

Abstract

In free-viewpoint video streaming, the available camera views and any virtual viewpoint between the camera views can be chosen by users. Since the amount of data that has to be stored or transmitted increases proportionally with the number of cameras, efficient compression of views is crucial. The Multiview extension of the High Efficiency Video Coding standard, called MV-HEVC, uses the well-known disparity-compensated prediction and advanced techniques for inter-view and inter-component prediction in order to reduce the high bandwidth requirement. But, even with the compression offered by MV-HEVC, this huge amount of data are currently beyond the capabilities of most mass consumers. In addition, the quality perception in free-viewpoint videos is different from regular videos. In free-viewpoint videos, the user should be able to see the scene from numerous different directions to cover the entire view. Hence, the higher coverage of Field of View (FOV) is more important than the quality of each view, the latter an important factor in regular videos. In this paper, we present a novel method based on MV-HEVC for rate assignment in free-viewpoint videos to overcome the discussed challenges. We use the distance between the various view directions to assign the proper rate to each view in order to provide a broader coverage of FOV for viewers. Simulation results with the Ballet and Break-dancer video sequences indicate that our proposed approach achieves a 27.06 reduction in bandwidth and 1.17 improvement in video quality in the BjOntegaard-Bitrate and BjOntegaard-PSNR scales, respectively, compared to standard MV-HEVC using the $\lambda$ domain rate control algorithm.