Text-to-Face Generation via Multi-Modal Attention Memory Network with Fine-Grained Feedback

Abstract

Over the past few years, a lot of works have explored on Text-to-Image (TTI), which aims to generate images from text descriptions. Compared with Text-to-Image (TTI), Text-to-Face (TTF) is a subtopic with greater challenges but meaningful, and less of the related studies. The text description of the face is more abstract and complex, which leads to poor quality generated face images and a low level of semantic consistency between text description and face images. As a solution to this problem, we propose a Multi-Modal Attention Memory Network with Fine-Grained Feedback. The proposed method uses a Multi-Modal Attention Memory Network to refine the face image features continuously and to improve the quality of the resulting face image. And the word-level discriminator is designed to establish the correlation between words and facial attributes, providing the generator with fine-grained training feedback. In order to verify the effectiveness of our method, we have performed a verification on the Multi-Modal CelebA-HQ dataset, and the experimental results confirm its effectiveness.