Abstract
Text-to-image synthesis task aims at generating images consistent with input text descriptions and is well developed by the Generative Adversarial Network (GAN). Although GAN based image generation approaches have achieved promising results, synthesizing quality is sometimes unsatisfied due to discursive generation of background and object. In this article, we propose a cooperative up-sampling based Dual Generator attentional GAN (DGattGAN) to generate high-quality images from text description. To achieve this, two generators with individual generation purpose are established to decouple object and background generation. In particular, we introduce a cooperative up-sampling mechanism to build cooperation between object and background generators during training. This strategy is potentially very useful as any dual generator architecture in GAN models can benefit from this mechanism. Furthermore, we propose an asymmetric information feeding scheme to distinguish two synthesis tasks, such that each generator only synthesizes based on semantic information they accept. Taking advantage of effective dual generator, the attention mechanism we incorporated on object generator could devote to fine-grained details generation on actual targeted objects. Experiments on Caltech-UCSD Bird (CUB) and Oxford-102 datasets suggest that generated images by the proposed model are more realistic and consistent with input text, and DGattGAN is competent compared to state-of-the-art methods according to Inception Score (IS) and R-precision metrics. Our codes are available at: https://github.com/ecfish/DGattGAN .
Highlights
In recent years, text-to-image synthesis has drawn much interest and rapidly expand the area of computer vision
We found from this figure, there are still some images generated by DGattGAN shows unsatisfied object shapes, general quality verifies that DGattGAN has achieved great improvement in synthesizing more vivid objects
We have proposed a novel dual generator attentional Generative Adversarial Network (GAN) based on cooperative up-sampling scheme for text-to-image synthesis
Summary
Text-to-image synthesis has drawn much interest and rapidly expand the area of computer vision. Conventional GAN architecture generating based on input noises is less contributing to match the text information Another image generation model Conditional GAN (cGAN) [4] is proposed, which almost all later text-to-image models are built based on this condition restrained architecture [1], [2], [5]. Lacking crucial fine-grained information is discovered as main problem hindering qualified image generation by StackGAN and StackGAN++ Another text-to-image synthesis model AttnGAN [2] is proposed which aims at synthesizing more realistic and fine-grained images based on attentional word-level feature fusion. Data spaces aren’t fully decoupled, which gains some extents of improvement by depicting object shape suggested by LR-GAN [8] Inspired by these models, FineGAN [9] establishes a new unsupervised hierarchical image synthesis method with fine-grained details emphasized. Two generators inserted still lead to some problems in synthesizing, the overall quality of realistic and detailed object generation had been promoted
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.