Person image generation with attention-based injection network

Abstract

Person image generation becomes a challenging problem due to the content ambiguity and style inconsistency. In this paper, we propose a novel Attention-based Injection Network (AIN) to address this issue. Instead of directly learning the relationship between the source and target image, we decompose the process into two accessible modules, namely Semantic-guided Attention Network (SAN) and Pose-guided Attention Network (PAN). SAN is proposed to capture the semantic information which can embed the human attributes into the latent space via the semantic layout. PAN enables a natural re-coupling of the pose and appearance, which can selectively integrate features to complete the human pose transformation. Additionally, a semantic layout loss is proposed to focus on the semantic content similarity between the source and generated images. Compared with other methods, our networks can enforce the local textures and styles consistency between the source and generated image. Experiments show that superior both qualitative and quantitative results are obtained on Market-1501 and DeepFashion datasets. On the basis of AIN, our network can further achieve the data augmentation for person re-identification (Re-ID) with dramatically improving the person Re-ID accuracy.