A Coarse-to-Fine Convolutional Neural Network for Light Field Angular Super-Resolution

Abstract

AbstractDensely-sampled light field (LF) images are drawing increasing attention for their wide applications, such as 3D reconstruction, virtual reality, and depth estimation. However, due to the hardware restriction, it is usually challenging and costly to capture them. In this paper, we propose a coarse-to-fine convolutional neural network (CNN) for LF angular super-resolution (SR), which aims at generating densely-sampled LF images from sparse observations. Our method contains two stages, i.e., coarse-grained novel views synthesis and fine-grained view refinement. Specifically, our method first extracts the multi-scale correspondence in the sparse views and generates coarse novel views. Then we propose a structural consistency enhancement module to regularize them for LF parallax structure preservation. Experimental results on both real-world and synthetic datasets demonstrate that our method achieves state-of-the-art performance. Furthermore, we show the promising application of the reconstructed LF images by our method on the depth estimation task.KeywordsLight fieldAngular super-resolutionView synthesis