BeautyGAN+: mixed-supervised makeup transfer learning algorithm based on new PMT dataset

Abstract

As one of the important ways to change the appearance of face image, makeup transfer has received more and more attention in recent years. Makeup transfer networks can translate the makeup style of a reference image to any other non-makeup one while preserving face identity, helping people find the most suitable makeup for them and get the beautified image. BeautyGAN, a makeup transfer network, has successfully demonstrated the results of unsupervised makeup transfer. However, unsupervised learning strategies lack supervised labels that can provide makeup details, leading to problems such as background discoloring, artifacts on edges and illumination disturbance in the experimental results. Constraint of BeautyGAN with makeup and non-makeup pairwise images can solve this problem, but academia lacks such fine-labeled paired dataset. To solve this problem, we propose BeautyGAN+, which adds supervised loss to BeautyGAN to train, and build up a new makeup dataset, the PMT dataset, that consists of makeup and non-makeup pairwise images. Finally, we find that combining unsupervised (migration) and supervised (details) loss can get better results through experiments. The experimental results and user study (14 participants) indicate that BeautyGAN+ significantly improves the results of makeup transfer, solving the problem of lack of paired dataset and various deviations in the transfer results.