Light field image compression using Versatile Video Coding

Abstract

Light field cameras (or plenoptic cameras) made it possible to not only capture information about the light intensity in a scene, but also the light direction traveling in space. The resulting light field images added two extra spatial dimensions to conventional 2D images by capturing a scene from different angles. Light field technology is expected to deliver the needed six-degrees-of-freedom in virtual reality, and it is a cornerstone technology in holography. Due to the vast amount of data a light field needs to capture for representing high-resolution images, the need of new coding solutions has been considered. While coding standards aiming at plenoptic images are something to be expected in the near future, video coding standards are one of the most efficient solutions to compress light fields. There are countless scanning topologies for restructuring a light field image into a 2D video, and similarly, numerous prediction structures to be used for inter-prediction. In this paper, we provide a comparison of light field image coding performance using the three latest generations of video compression standards. Moreover, we assess a set of coding structures aiming for optimal compression rates as well as random access capabilities. Results show the necessary trade-off between bandwidth and potential random access between light field views at decoding level. Bitrate savings up to 25% were achieved at certain cases where the proposed prediction structure is used over a common scheme used in the literature. The findings of this overview provide a useful benchmark for future development of light field image coding solutions using video coding standards.