Super-resolution image de-fencing using a nonlocal nonconvex prior.

Abstract

Today's smartphones/phablets/tablets are equipped with cameras that have enabled people to capture their favorite moments. However, images or videos taken at public places using inexpensive low-resolution cameras are often degraded by the presence of occlusions such as fences/barricades. In order to reconstruct a fence-free high-resolution image, we use a video of a scene captured by panning a hand-held camera and model the effects of various degradations. Initially, we obtain the spatial locations of fences by semantic segmentation and subsequently estimate the subpixel motion between the degraded low-resolution frames. The unknown high-resolution de-fenced image is modeled as a nonlocal discontinuity-adaptive Markov random field (NL-DAMRF) and its maximum a posteriori estimate is obtained by minimizing an appropriate objective function. We propose a nonlocal extension of DAMRF prior to preserve high-frequency information in the reconstruction process. Specifically, we use the graduated nonconvexity algorithm to minimize the proposed nonconvex energy function. Experiments with both synthetic and real-world data demonstrate the efficacy of the proposed algorithm.