A computer-vision based framework for virtual 3D garment reconstruction

Abstract

AbstractExisting 3D garment reconstruction methods are difficult to implement for online fashion design and e-commerce or special applications. This paper proposes a novel computer-vision framework for 3D garment digital reconstruction, which aims to reconstruct high-quality and realistic virtual 3D garments with fabric mechanic properties for 3D virtual try-on. The new segmentation scheme is proposed to separate the 3D garment point clouds from background points, which is suitable for 3D human shapes and is adaptive for different 3D garment models in different scenes. The new Statistical Outlier Removal algorithm and the learning-based method PointCleanNet are combined to remove noise and outliers in 3D garment point clouds, which provides high-fidelity and high-quality 3D garment point clouds. The 3D garment meshes are then reconstructed from their corresponding point clouds with a modified rolling ball algorithm. Finally, the meshes are improved and converted into physics-based virtual try-on 3D garments with fabric mechanic properties added, which enables the assessment of different body shapes with varied sizes for the same reconstructed 3D garment. Comparison experiments demonstrate that our framework achieves high-quality and realistic 3D garment reconstruction and accurate 3D virtual try-on from 2D garment images. We also demonstrate the proposed framework on a large range of various garments to show this approach has a great potential for garment future technology, such as online garment shopping, garment design and manufacturing.