Light-field compression using a pair of steps and depth estimation.

Abstract

Advanced handheld plenoptic cameras are being rapidly developed to capture information about light fields (LFs) from the 3D world. Rich LF data can be used to develop dense sub-aperture images (SAIs) that can provide a more immersive experience for users. Unlike conventional 2D images, 4D SAIs contain both the positional and directional information of light rays; the practical applications of handheld plenoptic cameras are limited by the huge volume of data required to capture this information. Therefore, an efficient LF compression method is vital for further application of the cameras. To this end, the pair of steps and depth estimation (PoS&DE) method is proposed in this paper, and the multiview video and depth (MVD) coding structure is used to relieve the LF coding burden. More specifically, a precise depth-estimation approach is presented for SAIs based on the cost function, and an SAI-guided depth optimization algorithm is designed to refine the initial depth map based on pixel variation tendency. Meanwhile, to reduce running time, intermediate SAI synthesis quality and coding bitrates, including the key SAIs selected and cost-computation steps, are set via extensive statistical experiments. In this way, only a limited number of optimally selected SAIs and their corresponding depth maps must be encoded. The experimental results demonstrate that our proposed LF compression solution using PoS&DE can obtain a satisfied coding performance.