Multiple Magnification Spatial Super-Resolution Network for Light Field Images Based on EPI Solid

Abstract

The Light field (LF) imaging technology can obtain the spatial information and angular information of light simultaneously. Since, the sensor captures multiple images from different angular directions, the spatial resolution of each view image is limited. LF image super-resolution (SR) methods based on deep-learning have been proposed to solve this problem. However, correlation of LF image at two angular dimensions have not been fully exploited in algorithms that based on epipolar plane images (EPIs). In this paper, we propose a joint LF image SR network by combining a multiple magnification single image SR (SISR) network and an inpainting network for EPI solids. The EPI solids, which contain information of both angular dimensions, are extracted from the output of SISR network and used as input data of the inpainting network. The inpainting network uses channel attention mechanism in feature extraction module to extracting line and gradient feature contained by EPI solids and correlations between EPI slices of EPI sold for restoring the geometric continuity of different views. Experimental results show that the proposed multiple magnification spatial LF image SR network which is trained by dataset with mixed magnifications has better performance than other approaches that are specially trained at each magnification.