Energy-driven reference selection for hierarchical light field compression

Abstract

The vast amount of light field data taken by plenoptic camera poses a great challenge for compression. Researchers attempt to design various sub-aperture image (SAI) prediction algorithms to reduce the redundancy inside the light field. However, they ignore the fact that it may limit the performance of light field compression when selecting the reference SAIs without regard to the scene content. In this paper, we propose a novel reference SAIs selection algorithm for a hierarchical compression structure from the perspective of light field energy maximization. Specifically, we firstly apply a low-rank model to the feature domain of light field, in which the features are extracted by the difference of Gaussian. In this way, a few SAIs containing sufficient light field energy are selected as the references, and the light field hierarchical structure is accordingly constructed by these selected SAIs and the remainder SAIs. Then, for better prediction of remainder SAIs in the hierarchical structure, we propose a compressive sensing-based weight determination algorithm, which computes the weight parameters for each selected reference SAI in pursuit of minimizing the prediction errors. Experimental results demonstrate that the hierarchical light field compression structure with the proposed reference SAIs selection algorithm can achieve better compression performance, as well as superior refocusing and depth-of-field extending capabilities compared to state-of-the-art methods.