Abstract
Light field (LF) cameras can record scenes from multiple perspectives, and thus introduce beneficial angular information for image super-resolution (SR). However, it is challenging to incorporate angular information due to disparities among LF images. In this paper, we propose a deformable convolution network (i.e., LF-DFnet) to handle the disparity problem for LF image SR. Specifically, we design an angular deformable alignment module (ADAM) for feature-level alignment. Based on ADAM, we further propose a collect-and-distribute approach to perform bidirectional alignment between the center-view feature and each side-view feature. Using our approach, angular information can be well incorporated and encoded into features of each view, which benefits the SR reconstruction of all LF images. Moreover, we develop a baseline-adjustable LF dataset to evaluate SR performance under different disparity variations. Experiments on both public and our self-developed datasets have demonstrated the superiority of our method. Our LF-DFnet can generate high-resolution images with more faithful details and achieve state-of-the-art reconstruction accuracy. Besides, our LF-DFnet is more robust to disparity variations, which has not been well addressed in literature.
Highlights
A LTHOUGH light field (LF) cameras enable many attractive functions such as post-capture image editing [1]–[3], depth sensing [4]–[9], saliency detection [10]–[14], and de-occlusion [15]–[17], the resolution of a sub-aperture image (SAI) is much lower than that of the total sensors
It is worth noting that, the PSNR and SSIM improvements of our LF-DFnet are very significant on the STFgantry dataset for 2×SR
Scenes in the STFgantry dataset are captured by a moving camera mounted on a gantry, and have relatively large baselines and significant disparity variations
Summary
A LTHOUGH light field (LF) cameras enable many attractive functions such as post-capture image editing [1]–[3], depth sensing [4]–[9], saliency detection [10]–[14], and de-occlusion [15]–[17], the resolution of a sub-aperture image (SAI) is much lower than that of the total sensors. The low spatial resolution problem hinders the development of LF imaging [18]. Since high-resolution (HR) images are required in various LF applications, it is necessary to reconstruct. Manuscript received April 11, 2020; revised September 2, 2020 and November 1, 2020; accepted November 25, 2020. Date of publication December 8, 2020; date of current version December 14, 2020. The associate editor coordinating the review of this manuscript and approving it for publication was Prof.
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callMore From: IEEE transactions on image processing : a publication of the IEEE Signal Processing Society
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
