IRLF-SRNet: A super-resolution network based on local–global feature enhance-refine for camera-array based infrared light field images

Abstract

Super resolution (SR) is essential for the camera-array based infrared light field (IRLF) images. However, the existing SR methods are mainly designed for visible LF images and are unsuitable for IRLF. This is because the sub-aperture images (SAIs) in IRLF hold blurred edges and pixel-level shifts. Artifacts and unbalanced SR performance may exist. In this paper, we propose a novel SR network, named IRLF-SRnet, to mitigate the above problems. It includes two designs: (1) local–global feature enhance-refine strategy, and (2) multi-branch cross-view correlation modeling block. The former uses global features from all SAIs to enhance the local feature from one SAI, which enhances the edges in each SAI implicitly. Then, it refines the global feature with the enhanced local feature, which further refines the edges. The latter first divides and sends the features of SAIs into multiple branches, and then models the cross-view correlation among features in each branch by bidirectional convolutional long-short term memory (BiConvLSTM), so that be aware of the pixel-level shifts. Qualitative and quantitative results show that our method outperforms state-of-the-art methods, with more balanced SR performance and fewer artifacts.