Image Reconstruction with Multiscale Interest Points Based on a Conditional Generative Adversarial Network

Abstract

A new image reconstruction (IR) algorithm from multiscale interest points in the discrete wavelet transform (DWT) domain was proposed based on a modified conditional generative adversarial network (CGAN). The proposed IR-DWT-CGAN model generally integrated a DWT module, an interest point extraction module, an inverse DWT module, and a CGAN. First, the image was transformed using the DWT to provide multi-resolution wavelet analysis. Then, the multiscale maxima points were treated as interest points and extracted in the DWT domain. The generator was a U-net structure to reconstruct the original image from a very coarse version of the image obtained from the inverse DWT of the interest points. The discriminator network was a fully convolutional network, which was used to distinguish the restored image from the real one. The experimental results on three public datasets showed that the proposed IR-DWT-CGAN model had an average increase of 2.9% in the mean structural similarity, an average decrease of 39.6% in the relative dimensionless global error in synthesis, and an average decrease of 48% in the root-mean-square error compared with several other state-of-the-art methods. Therefore, the proposed IR-DWT-CGAN model is feasible and effective for image reconstruction with multiscale interest points.