Frame Attention Recurrent Back-Projection Network for Accurate Video Super-Resolution

Abstract

In this study, we aim to recover high-resolution (HR) images from multiple low-resolution images using a multi-frame super-resolution (SR) or video SR to increase the resolution of surveillance camera images. The challenge of the multi-frame method is the high-precision alignment between frames, and many techniques have been proposed recently to solve it using convolutional networks, e.g., in the state-of-the-art method (i.e., recurrent back-projection network), the residuals between frames are calculated as subtle changes in the frames and back-projected onto the input image to achieve HR, including alignment. However, since the method uniformly concatenates multiple HR feature maps generated between each frame, it is impossible to focus on extracting important features for HR. Therefore, this study proposes a frame attention recurrent back-projection network for accurate video SR by emphasizing the essential frame features.