A Data Enhancement Method toward Real-World Single Image Super-Resolution

Abstract

Introducing convolutional neural networks (CNNs) into single image super-resolution (SISR) has achieved remarkable performance in experimental scenarios over the past decades. However, maintaining constant mapping complexity, most existing CNN-based SISR approaches make it challenging to reconstruct high-quality images in real scenarios because of considerable variation between authentic degradation and simulated degradation. In this paper, to improve SISR models’ capabilities in the real world, we propose a data enhancement method, consisting of frequency domain data augmentation (FDA) and dynamical frequency loss (DFL). By introducing high-resolution images’ frequency information, the FDA enhances the quality and quantity of data, regularizing the model to learn “how” to appropriately apply super-resolution on different frequency components of a given image. In addition, to further strengthen the regularization effect, we design DFL to dynamically assign the loss trade-off hyperparameters of distinct frequency components, guiding the model to allocate computational overhead reasonably. Considering quantitative analysis and visual quality in combination, extensive experiments prove that the proposed enhancement method significantly improves SR models’ capabilities in real scenarios without adding any computational cost to the evaluation procedure.