Multi-Angular Epipolar Geometry Based Light Field Angular Reconstruction Network

Abstract

Densely-sampled light field (LF) image is drawing increased attention for its wide applications in 3D reconstruction, digital refocusing, depth estimation, and virtual/augmented reality, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">et al.</i> In order to reconstruct a densely-sampled LF with high angular resolution, many computational methods have been proposed. However, most existing methods consider LF angular reconstruction based on neighboring views, local epipolar plane images (EPIs) or EPI volume information, which overlook the rich LF angular information and fail to restore more texture details, especially for occlusion regions. In order to mitigate this problem, we introduce a multi-angular epipolar geometry (MA-EG) structure for LF angular reconstruction. The MA-EG structure contains multi-angular directional LF geometry information and can provide multi-angular geometry characteristics in LF reconstruction process, which benefits in recovering more texture details. Based on the MA-EG structure, we further put forward a multi-angular LF angular reconstruction network (MALFRNet) to learn a mapping from sparse LF to a densely-sampled LF. The proposed MALFRNet adopts a multi-stream framework, which can fully explore rich LF angular information and implicitly learn LF angular consistency and spatial geometry information from LF MA-EG. Comprehensive experiments on real-world and synthetic LF scenes demonstrate that the proposed MALFRNet can recover more texture details and achieve a better reconstruction quality. Moreover, ablation studies and LF depth estimation applications also illustrate the advantages of using more angular information in LF angular reconstruction through the proposed MALFRNet.