Deep Image Prior for Super Resolution of Noisy Image

Sujy Han,Tae Bok Lee,Yong Seok Heo

doi:10.3390/electronics10162014

Abstract

Single image super-resolution task aims to reconstruct a high-resolution image from a low-resolution image. Recently, it has been shown that by using deep image prior (DIP), a single neural network is sufficient to capture low-level image statistics using only a single image without data-driven training such that it can be used for various image restoration problems. However, super-resolution tasks are difficult to perform with DIP when the target image is noisy. The super-resolved image becomes noisy because the reconstruction loss of DIP does not consider the noise in the target image. Furthermore, when the target image contains noise, the optimization process of DIP becomes unstable and sensitive to noise. In this paper, we propose a noise-robust and stable framework based on DIP. To this end, we propose a noise-estimation method using the generative adversarial network (GAN) and self-supervision loss (SSL). We show that a generator of DIP can learn the distribution of noise in the target image with the proposed framework. Moreover, we argue that the optimization process of DIP is stabilized when the proposed self-supervision loss is incorporated. The experiments show that the proposed method quantitatively and qualitatively outperforms existing single image super-resolution methods for noisy images.

Highlights

Single image super-resolution (SISR) aims to generate a high-resolution (HR) image from a low-resolution (LR) image
Unlike most deep learning models that are trained on large-scale datasets, Ulyanov et al [1] recently proposed a deep image prior (DIP) that utilizes a deep neural network (DNN) as a strong prior for image restoration by using only a single image
The results of DIP show that the DNN is useful for capturing meaningful low-level image statistics

Summary

Introduction

Single image super-resolution (SISR) aims to generate a high-resolution (HR) image from a low-resolution (LR) image. Unlike most deep learning models that are trained on large-scale datasets, Ulyanov et al [1] recently proposed a deep image prior (DIP) that utilizes a deep neural network (DNN) as a strong prior for image restoration by using only a single image. The results of DIP show that the DNN is useful for capturing meaningful low-level image statistics. With the success of DIP [1], it has been utilized in several ways due to its usefulness for a variety of purposes. DIP has significance in the applications where collecting largescale of datasets is difficult and expensive, such as hyperspectral image processing [2,3]

Objectives

Results

Conclusion