Pose Guided Person Image Generation Based on Pose Skeleton Sequence and 3D Convolution

Abstract

This paper proposes a novel approach for pose transfer, which aims at transferring the pose of a given person to a target pose. Unlike previous works directly simulating the target pose, we emphasize the pose geometric constraint in our approach to tackle this task. To capture the geometric constraints among the condition pose and target pose, we generate a sequence of different intermediate pose skeleton. Moreover, we introduced the 3D convolution into the generator of our Generative Adversarial Network (GAN) in order to learn spatiotemporal features from the generated pose skeleton sequence. The final target pose will be estimated and refined by a pose transfer block and a series of residual blocks based on the spatiotemporal features. Compared with the existing works, our generated person pose images achieved better performance in terms of Inception Score and Structure Similarity. Extensive experiments on dataset Market-1501 demonstrate the effectiveness of the pose skeleton sequence on pose transfer.