Appearance and Pose-Conditioned Human photograph generation using Deformable GANs

Abstract

In this study, we address the challenge of generating images of individuals based on pose and appearance data. Specifically, we take an image, xa, of an individual and a target pose, P (xb), extracted from another image, xb. We then generate a new image of the same individual in the target pose, P (xb), while preserving the visual details from xa. To manage pixel-to-pixel misalignments caused by pose differences between P (xa) and P (xb), we incorporate deformable skip connections in our Generative Adversarial Network’s generator. Additionally, we propose a nearest-neighbour loss as an alternative to the standard L1 and L2 losses to match the texture of the generated image with the target image. Our approach demonstrates competitive quantitative and qualitative results using standard datasets and protocols recently proposed for this task. We also carry out a comprehensive evaluation using off-the-shelf person-identification (Re-id) systems trained with person-generation-based augmented data, a key application for this task. Our experiments reveal that our Deformable GANs can significantly boost Re-id accuracy, surpassing data-augmentation techniques specifically trained using Re-identification losses. index Terms—Conditional GAN, Image Generation, Deformable Objects, Human Pose.