Video super-resolution via nonlocal deformable alignment and frame recursive progressive fusion network

Abstract

Video super-resolution (VSR) is a process of high-resolution reconstruction of low-resolution video. To address the problems of the previous VSR methods with poor temporal consistency and unsatisfactory SR results, we proposed a nonlocal deformable alignment and frame recursive progressive fusion (RPF) network combining sliding window and recursive methods, which uses nonlocal operations to align sequential frame features and later applies recursion to temporally model the hidden information and alignment features of the previous moment, thus improving temporal consistency. The RPF unit is used to fully fuse the hidden information with the currently aligned features, acquiring more supporting information to be obtained from adjacent frames, resulting in better SR results. The results were evaluated on the three public VSR datasets of Vid4, udm10, and Vimeo-90K, and the experimental results show that the proposed method can achieve state-of-the-art performance on VSR task.