Learning to simultaneously enhance field of view and dynamic range for light field imaging

Abstract

Light field (LF) imaging, which simultaneously captures the intensity and direction information of light rays, enabling many vision applications, has received widespread attention. However, limited by the optical structure of the LF camera, the acquired LF images usually suffer from narrow field of view (FOV) and low dynamic range. To address these problems, this paper proposes an unsupervised wide-FOV high dynamic range (HDR) LF imaging method, which can effectively reconstruct a wide-FOV HDR LF image from a set of source LF images captured from different perspectives and simultaneously with different exposures. Specifically, the proposed method first exploits tensor decomposition to obtain a compact representation of high-dimensional LF image, so as to enable a computationally efficient 2D neural network for LF registration. Subsequently, an exposure restoration network is constructed to recover the multi-exposure information of the registered non-overlapping regions, which is then linearly fused with the previous registered results to generate the stitched wide-FOV multi-exposure LF images. Finally, an HDR LF blending network with two ingenious unsupervised losses is designed to blend the stitching results to generate the desired wide-FOV HDR LF image. Experimental results show that the proposed method achieves superior performance compared with the state-of-the-art methods in both qualitative and quantitative evaluation. Moreover, a series of ablation studies effectively validate the performance of each module in the proposed method.