Optical flow-assisted multi-level fusion network for Light Field image angular reconstruction

Abstract

Light Field (LF) imaging can record both the intensities and directions of light rays in a single exposure, which has received extensive attentions. However, the limited angular resolution becomes the primary bottleneck for the wide-spread applications of LF imaging. To this end, this paper proposes a novel optical flow-assisted multi-level fusion network for LF angular reconstruction. In our method, we propose to infer the multi-angular optical flows to explore long-range dependency of LF sub-aperture images (SAIs) for high-quality angular reconstruction. By aligning the SAIs in multi-angular directions, the geometric consistency of reconstructed dense LF can be preserved. Moreover, a multi-level fusion framework for LF angular reconstruction is introduced, which consists of two stages, namely texture-optical flow feature fusion and parallax structure-information fusion. The former firstly extracts the texture and optical flow features from the reconstructed coarse LF and then fuses these two features by using the proposed texture-optical flow fusion-block. The latter further blends the LF parallax structure information with the fused texture and optical flow features using the proposed parallax structure-information fusion network. Comprehensive experiments on both real-world and synthetic LF scenes demonstrate the superiority of the proposed method for reconstructing high-quality dense LF. Moreover, practical application on depth estimation also validates that our method can recover more texture details, particularly for some occlusion regions.