ProtoGAN: Towards high diversity and fidelity image synthesis under limited data

Abstract

Recent years have witnessed tremendous progress with regard to few-shot learning methods, especially for classification and segmentation tasks. However, the challenging few-shot image generation has garnered less attention. Training GANs with limited data is a formidable task since discriminator overfitting and memorizing degrade synthesis quality. Existing approaches endeavor to expand the training set with data augmentation techniques. In this paper, we propose ProtoGAN, a GAN that incorporates the metric-learning-based prototype mechanism into adversarial learning. Concretely, we align the prototypes and features of real and generated images in the embedding space, facilitating the fidelity of the generated images. We include a variance term in the objective function in order to promote the synthesis diversity of the generative network. As an intuitive yet effective method, ProtoGAN is easy to implement and is complementary to existing methods. We perform extensive experiments on nineteen datasets spanning a variety of domains and resolutions. Both quantitative and qualitative results demonstrate the effectiveness of our proposed method for high-quality image synthesis under limited data. Our code is available at https://github.com/kobeshegu/ProtoGAN.