Text Guided Image Using Machine Learning

Abstract

bstract: Given a corrupted image, image inpainting aims to complete the image and outputs a plausible result. When we complete the missing region, we always borrow the information from a known area, which is aimless and causes unsatisfactory results. In our daily life, some other information is often used for corrupted images, such as text descriptions. Therefore, we introduce the use of text information to guide image inpainting. To fullfill this task, We introduce an inpainting model named TG Net (Text-Guided Inpainting Network). We provide a text-image gated feature fusion module to fuse text feature and image feature deeply. A mask attention module is proposed to enhance the consistency of known areas and the repaired area. Extensive quantitative and qualitative experiments on three public datasets with captions demonstrate the effectiveness of our method. The goal of our paper is to semantically edit parts of an image matching a given text that describes desired attributes (e.g., texture, colour, and background), while preserving other contents that are irrelevant to the text. To achieve this, we propose a novel generative adversarial network (Mani GAN), which contains two key components: text-image affine combination module (ACM) and detail correction module (DCM). The ACM selects image regions relevant to the given text and then correlates the regions with corresponding semantic words for effective manipulation. Meanwhile, it encodes original image features to help reconstruct text-irrelevant contents. The DCM rectifies mismatched attributes and completes missing contents of the synthetic image. Finally, we suggest a new metric for evaluating image manipulation results, in terms of both the generation of new attributes and the reconstruction of text-irrelevant contents. Extensive experiments on the CUB and COCO datasets demonstrate the superior performance of the proposed method. Code is available at this https URL.