Two-stage convolutional neural network for light field super-resolution

Abstract

In this paper, we investigate a convolutional neural network (CNN) approach for light field (LF) super-resolution (SR). We are motivated by the assumption that image priors can be embedded into CNN, and both external and internal correlations are important in LFSR. The LF images are indeed natural images except for its angular resolution, so the external correlations help to super-resolve a single image from a collection of general images, whilst the internal correlations are essential to enhance a single view in LF with the details in the other views. Accordingly, we propose a two-stage CNN, where the two stages exploit the external and internal correlations, respectively. Moreover, to improve the generalization ability of the second-stage CNN for inter-view SR, we propose to align different views at patch level to compensate for the disparity that is essential to LFSR, thus the second stage is termed multi-patch fusion CNN. Experimental results demonstrate the superior performance of our two-stage CNN compared with the state-of-the-art CNN-based SR methods.