Multi-frame example-based super-resolution using locally directional self-similarity

Abstract

This paper presents a multi-frame superresolution approach to reconstruct a high-resolution image from several low-resolution video frames. The proposed algorithm consists of three steps: i) definition of a local search region for the optimal patch using motion vectors, ii) adaptive selection of the optimum patch based on lowresolution image degradation model, and iii) combination of the optimum patch and reconstructed image. As a result, the proposed algorithm can remove interpolation artifacts using directionally adaptive patch selection based on the lowresolution image degradation model. Moreover, superresolved images without distortion between consecutive frames can be generated. The proposed method provides a significantly improved super-resolution performance over existing methods in the sense of both subjective and objective measures including peak-to-peak signal-to-noise ratio (PSNR), structural similarity measure (SSIM), and naturalness image quality evaluator (NIQE). The proposed multi-frame super-resolution algorithm is designed for realtime video processing hardware by reducing the search region for optimal patches, and suitable for consumer imaging devices including ultra-high-definition (UHD) digital televisions, surveillance systems, and medical imaging systems for image restoration and enhancement.