A Novel Human Image Sequence Synthesis Method by Pose-Shape-Content Inference

Abstract

In online clothing sales, static model images only describe specific clothing statuses towards consumers. Without increasing shooting costs, it is a subject to display clothing dynamically by synthesizing a continuous image sequence between static images. This paper proposes a novel human image sequence synthesis method by pose-shape-content inference. In the condition of two reference poses, the pose is interpolated in the pose manifold controlled by a linear parameter. The interpolated pose is transferred into the end shape by AdaIN and the attention mechanism to infer target shape. Then the content in the reference image is transferred into this target shape. In the content transfer, the visual features of the human body cluster and clothing cluster are extracted, respectively. And the Sobel gradient is adopted to extract clothing texture variation. In the feature inferring, the multiscale feature-level optical flow warps source features, and style code infusion infers new region content without source features. Extensive experiments demonstrate that our method is superior in inferring clear layouts and transferring reasonable content compared to the pose transfer baselines. Moreover, our method has been verified to apply in parsing-guided image inference and dynamic display based on the pose sequence. The code will be available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/NerdFNY/psc</uri>