Cross-Scale Reference-Based Light Field Super-Resolution

Abstract

Light fields suffer from a fundamental resolution tradeoff between the angular and the spatial domain. In this paper, we present a novel cross-scale light field super-resolution approach (up to $\text{8}\times$ resolution gap) to super-resolve low-resolution (LR) light field images that are arranged around a high-resolution (HR) reference image. To bridge the enormous resolution gap between the cross-scale inputs, we introduce an intermediate view denoted as single image super-resolution (SISR) image, i.e., super-resolving LR input via single image based super-resolution scheme, which owns identical resolution as HR image yet lacks high-frequency details that SISR scheme cannot recover under such significant resolution gap. By treating the intermediate SISR image as the low-frequency part of our desired HR image, the remaining issue of recovering high-frequency components can be effectively solved by the proposed high-frequency compensation super-resolution (HCSR) method. Essentially, HCSR works by transferring as much as possible the high-frequency details from the HR reference view to the LR light field image views. Moreover, to solve the nontrivial warping problem that induced by the significant resolution gaps between the cross-scale inputs, we compute multiple disparity maps from the reference image to all the LR light field images, followed by a blending strategy to fuse for a refined disparity map; finally, a high-quality super-resolved light field can be obtained. The superiority of our proposed HCSR method is validated on extensive datasets including synthetic, real-world and challenging microscope scenes.