Image Degradation Modeling for Real-World Super Resolution via Conditional Normalizing Flow

Abstract

In recent years, deep-learning-based super-resolution (SR) methods have obtained impressive performance gains on synthetic clean datasets, but they fail to perform well in real-world scenarios due to insufficient real-world training data. To tackle this issue, we propose a conditional-normalizing-flow-based method named IDFlow for image degradation modeling that aims to generate various degraded low-resolution (LR) images for real-world SR model training. IDFlow takes image degradation modeling as a problem of learning a conditional probability distribution of LR images given the high-resolution (HR) ones, and learns the distribution from existing real-world SR datasets. It first decomposes the image degradation modeling into blur degradation modeling and real-world noise modeling. It then utilizes two multi-scale invertible networks to model these two steps, respectively. Before applied into real-world SR, IDFlow is first trained supervisedly on two real-world datasets RealSR and DPED with negative log-likelihood (NLL) loss. It is then used to generate a large number of HR-LR image pairs from an arbitrary HR image dataset for SR model training. Extensive experiments on IDFlow with RealSR and DPED are conducted, including evaluations on image degradation stochasticity, degradation modeling, and real-world super resolution. Two known SR models are trained with IDFlow and named as IDFlow-SR and IDFlow-GAN. Testing results on the RealSR and DPED testing dataset show that not only can IDFlow generate realistic degraded images close to real-world images, but it is also beneficial to real-world SR performance improvement. IDFlow-SR achieves 4× SR performance gains of 0.91 dB and 0.161 in terms of image quality assessment metrics PSNR and LPIPS. Moreover, IDFlow-GAN can super-resolve real-world images in the DPED testing dataset with richer textures and maintain clearer patterns without visible noises when compared with state-of-the-art SR methods. Quantitative and qualitative experimental results well demonstrate the effectiveness of the proposed IDFlow.