The Frequency Discrepancy Between Real and Generated Images

Abstract

Despite the success of Generative Adversarial Networks (GANs), little work has focused on the discrepancy between real and generated images in frequency domain. In this work, we provide a systematic analysis on this topic. We first demonstrate the general existence of the frequency discrepancy and further perform extensive experiments both on datasets with various frequency distributions and models with different upsampling methods to reveal the sources of the discrepancy. Experimental results show that: resize-convolution is not a perfect alternative to deconvolution, and natural images and unnatural images should be treated separately during training. Based on these studies, we provide some novel solutions to reduce the discrepancy. Finally, we further show the effectiveness of our solutions on Variational Auto Encoders (VAEs). We hope that the community should pay equal attention to the performance of generative models both in spatial and frequency domain.