Light field image super-resolution using a Content-Aware Spatial–Angular Interaction network

Abstract

Light field images record a series of viewpoints of a scene and thus have many attractive applications. However, the trade-off between angular and spatial resolution in the imaging process makes light field image super-resolution necessary. In this paper, we propose a Content-Aware Spatial–Angular Interaction (dubbed CASAI) network for light field image super-resolution. The gradient branch of CASAI makes full use of the context information of the low-resolution gradient map and the multi-level features of the super-resolution branch to generate the high-resolution gradient map, which enables the awareness of structure, texture and detail, and provides effective prior knowledge for the super-resolution process. The super-resolution branch of CASAI generates high-quality super-resolution images by making full use of intra-view (i.e., spatial) and inter-view (i.e., angular) information through a spatial–angular adaptive interaction block and using the high-resolution gradient prior as guidance. The spatial–angular adaptive interaction block enables the awareness of the different importance of spatial features and angular features, so as to better integrate intra-view and inter-view information to improve the performance of light field image super-resolution. The experimental results indicate that when the upsampling factor is 4, our method outperforms our baseline (LF-InterNet) with an average PSNR increase of 0.48 dB, while also achieving the best SSIM. Visualization results demonstrate the advantage of our method in simultaneously generating natural SR images and restoring structures.