DGattGAN: Cooperative Up-Sampling Based Dual Generator Attentional GAN on Text-to-Image Synthesis

Abstract

Text-to-image synthesis task aims at generating images consistent with input text descriptions and is well developed by the Generative Adversarial Network (GAN). Although GAN based image generation approaches have achieved promising results, synthesizing quality is sometimes unsatisfied due to discursive generation of background and object. In this article, we propose a cooperative up-sampling based Dual Generator attentional GAN (DGattGAN) to generate high-quality images from text description. To achieve this, two generators with individual generation purpose are established to decouple object and background generation. In particular, we introduce a cooperative up-sampling mechanism to build cooperation between object and background generators during training. This strategy is potentially very useful as any dual generator architecture in GAN models can benefit from this mechanism. Furthermore, we propose an asymmetric information feeding scheme to distinguish two synthesis tasks, such that each generator only synthesizes based on semantic information they accept. Taking advantage of effective dual generator, the attention mechanism we incorporated on object generator could devote to fine-grained details generation on actual targeted objects. Experiments on Caltech-UCSD Bird (CUB) and Oxford-102 datasets suggest that generated images by the proposed model are more realistic and consistent with input text, and DGattGAN is competent compared to state-of-the-art methods according to Inception Score (IS) and R-precision metrics. Our codes are available at: https://github.com/ecfish/DGattGAN .