Light Field Image Super-Resolution Based on Multilevel Structures

Abstract

Light field (LF) images suffer from low spatial resolution due to the trade-off between angular and spatial resolutions. Thus, spatial super-resolution (SR) of LF images is an essential task to obtain high-quality LF images. However, the existing SR networks still have limitations, since they exploit only single-level features to use sub-pixel information in LF images. In this paper, we proposed a light field super-resolution (LFSR) network to effectively improve the spatial resolution of light field images. The proposed network takes one target image and its 8-neighboring images for references. We construct multi-level structures for the proposed network to effectively estimate and mix sub-pixel information in reference images. The proposed network is composed of a feature extractor, a feature warping module, a feature mixing module, and a upscaling module. The feature extractor provides multi-level features for SR and offsets to the feature warping module to obtain aligned features for multiple reference images. The feature mixing module mixes multiple aligned features based on the similarity between the target and reference images to obtain multi-level mixed features. Finally, the upscaling module generates a high-resolution residual image using the multi-level mixed features. Experimental results demonstrate the proposed network outperforms the state-of-the-art methods on various light field datasets. The pre-trained model and source codes are available at https://github.com/Hwa-Jong/LF_MLS.