Cooperative Light-Field Image Super-Resolution Based on Multi-Modality Embedding and Fusion With Frequency Attention

Abstract

Light field (LF) imaging is an advanced visual perception system, which can record the intensity and direction information of light rays and provide multi-viewpoint images from a single capture. However, there is a trade-off between spatial and angular resolutions due to the restricted sensor size, which limits the wide applications of LF cameras. To address this problem, we propose a cooperative network to super-resolve LF sub-aperture images based on the multi-modality fusion. Specifically, in order to fully explore the LF information, we adopt various modalities and extract corresponding features to emphasise diverse LF characteristics. Then, we design a multi-scale fusion module to effectively integrate global and local LF features and apply frequency-aware attention mechanism to adaptively reinforce fused features. Extensive experiments demonstrate the superiority of our method on both qualitative and quantitative evaluations, with competitive execution efficiency.