View synthesis-based light field image compression using a generative adversarial network

Abstract

Light field (LF) imaging has generated considerable interest owing to its ability to capture both spatial and angular information of light rays simultaneously. However, the extremely large volume of data associated with LF imaging poses challenges to both data storage and transmission. This study addresses this issue by proposing a view synthesis-based LF image compression method using a generative adversarial network (GAN). The primary basis of compression relies on the fact that adjacent sub-aperture images (SAIs) are highly correlated. Accordingly, only sparsely sampled SAIs are transmitted and the others are reconstructed at the decoder side. The proposed sparse SAI sampling method enhances the quality of reconstructed SAIs by considering a fair trade-off between the number of SAIs available for use as priors in the synthesis process and SAI redundancy. The quality of reconstructed SAIs is further enhanced by a GAN-based SAI synthesis method, where the synthesis procedure is broken into disparity estimation and un-sampled SAI estimation components, and the adversarial nature of the jointly trained generative and discriminative networks results in a more accurate generative model. Furthermore, more texture details can be preserved in the synthesized SAIs by adopting a loss function in the GAN model based on perceptual quality. Extensive experimental results demonstrate the superiority of the proposed method relative to several other state-of-the-art compression methods in terms of standard quality metrics and the perceptual quality of the synthetic SAIs at the decoder side.