Automatic 3D virtual fitting system based on skeleton driving

Abstract

To facilitate the input and improve the practicality of the existing virtual fitting systems, we propose a fully automatic 3D virtual fitting system to fit garment onto human models with various shapes and poses using a 2D full-body image and a 3D garment model. The proposed method constructs the 3D human model from the input 2D full-body image of the user by adopting the SMPLify method. To automatically position garment models onto human models with arbitrary postures, we present a 3D mesh segmentation method based on the discrete Reeb graph to accurately segment the different parts of a garment model, and a skeleton driving method based on mean curvature flow, which automatically adjusts the posture of the garment model according to the skeleton structural difference between the human model and the garment model. In addition, for the purpose of obtaining a more natural dress effect, we further adopt interpenetration removal and physical simulation for the deformed garment model. Compared to existing automatic 3D virtual fitting systems, the experimental results, we obtained based on the Leeds Sports Pose dataset, reveal that the proposed virtual fitting system is stable and effective.