Noise-robust video super-resolution using an adaptive spatial-temporal filter

Abstract

In this paper, we introduce a new interpolation-based super-resolution scheme for super-resolving a low-resolution video that contains large-scale local motions and/or heavy noise. Our scheme leverages an efficient space-time descriptor to adapt the interpolation kernel to the video's spatial and temporal structures. Nevertheless, in the presence of large-scale local motions, the kernel suffers from tracking the motions incorrectly, leading to inaccurate temporal averaging. To address this problem, prior to computing the interpolation kernel, a mobile-neighborhood strategy that can identify the appropriate neighborhoods in adjacent frames is applied to neutralize the large-scale motions. Furthermore, we incorporate an adaptive sharpening technique into the kernel computation to remove the background noise and enhance the fine details simultaneously. Extensive experimental results on real-world videos show that the proposed method outperforms certain other state-of-the-art video super-resolution algorithms both visually and quantitatively, particularly in the presence of large-scale motions and/or heavy noise.