Deep learning methods in real-time image super-resolution: a survey

Abstract

Super-resolution is generally defined as a process to obtain high-resolution images form inputs of low-resolution observations, which has attracted quantity of attention from researchers of image-processing community. In this paper, we aim to analyze, compare, and contrast technical problems, methods, and the performance of super-resolution research, especially real-time super-resolution methods based on deep learning structures. Specifically, we first summarize fundamental problems, perform algorithm categorization, and analyze possible application scenarios that should be considered. Since increasing attention has been drawn in utilizing convolutional neural networks (CNN) or generative adversarial networks (GAN) to predict high-frequency details lost in low- resolution images, we provide a general overview on background technologies and pay special attention to super-resolution methods built on deep learning architectures for real-time super-resolution, which not only produce desirable reconstruction results, but also enlarge possible application scenarios of super resolution to systems like cell phones, drones, and embedding systems. Afterwards, benchmark datasets with descriptions are enumerated, and performance of most representative super-resolution approaches is provided to offer a fair and comparative view on performance of current approaches. Finally, we conclude the paper and suggest ways to improve usage of deep learning methods on real-time image super-resolution.