DFSGAN: Introducing editable and representative attributes for few-shot image generation

Abstract

Training generative adversarial networks (GANs) usually requires large-scale data and massive computation resources. The performance of GANs plummets when given limited data due to the discriminator overfitting, thus providing meaningless feedback to the generator during the adversarial training. Existing few-shot GANs are primarily concerned with transferring knowledge from models that have been pre-trained on large-scale datasets or using data augmentation to expand the training sets. However, previous methods consistently take latent codes sampled from a single distribution as the generator’s input. We contend that more complicated latent codes can provide the generator with more editable attributes. In this paper, we propose DFSGAN for few-shot image generation, which takes dynamic Gaussian mixture (DGM) latent codes as the generator’s input. Our DFSGAN can select the Gaussian components of the latent codes quantitatively. We also design two techniques to strengthen the representative ability of intermediate features of the generating process to improve the fidelity and maintain the content and layout information of the synthesized images. Our DGM and intermediate representation enhancement techniques complement each other and improve synthesis quality. We conduct extensive experiments on 15 few-shot datasets with different resolutions spanning from art paintings to realistic photos. Qualitative and quantitative results demonstrate the superiority and effectiveness of our model.