A progressive distillation network for practical image-based virtual try-on

Abstract

The image-based virtual try-on technology aims to match in-store clothing to person's image wearing clothes. Previous methods require a large amount of input information for each try-on result, which is not practical, and slight deviations in input information can cause a large number of artifacts in the try-on results. In recent years, researchers have paid attention to parsing-free virtual try-on methods, and a groundbreaking work used knowledge distillation techniques to eliminate the redundant input. This method uses the parsing-based virtual try-on model as the supervision information to train a student network model, which can generate try-on results without extra input. However, due to the huge knowledge gap between the teacher-student networks, direct distillation makes it difficult for the student network to fully simulate the teacher network. To solve this problem, we propose a progressive distillation scheme for image-based virtual try-on called PD-VTON, using an assistant network to alleviate this huge knowledge gap. Our method can generate more realistic and reasonable try-on results without requiring extra body parsing information or body pose information. Specifically, unlike existing distillation-based parsing-free virtual try-on methods, we adopt an assistant-supported progressive distillation network to alleviate the insufficient learning caused by the large knowledge gap and design an Adaptive Choose Teacher (ACT) module to optimize the distillation. Moreover, we introduce a novel cross attention-based stitching structure when generating try-on images, aiming to better constrain the alignment between the high-level semantic features of person image and warped clothing image, and use a transformer-assisted generator to generate the results. Finally, extensive experimental evaluations demonstrate the unique advantages of our method.