Video Super-resolution Reconstruction Using Deep and Shallow Convolutional Neural Networks

Abstract

So far, convolutional neural networks have proven successful in computer vision fields. In this article, we build a video super-resolution model using deep and shallow convolutional neural networks (VRDS). Continuous video frames are used as input of VRDS after motion compensation, and VRDS provides video frames recovered with super-resolution as output. The shallow CNN restores the fundamental components of HR images, while deep CNN predicts high frequency texture and contour details. To facilitate network training, we jointly train our deep CNN and the relatively shallow CNN as the entirety, resulting in faster convergence and better performance. In deep CNN, the multi-scale receptive field pattern is used for HR recovery, and it can combine the short-distance and long-distance contextual message in the meantime to restore HR images more accurately. Our networks adopt loss fusion, optimized MSE and SSIM to provide high quality visual perception. In the end, the experimental results prove that our method is better than the existing methods in the publicly available datasets.