A Pose and Style-Invariant Reenactment Technique for Artistic Portraits Using GAN

Abstract

Face reenactment techniques change the facial expression of a character in a photograph. This technique is very attractive in that it helps in the creation of new content from existing content. Many researchers have been developing face reenactment techniques, including generative adversarial network-based techniques using action unit vectors. However, face reenactment techniques for artwork are still insufficient. Unlike photographs, artwork includes a variety of poses and styles. To expand the existing techniques into the artwork domain, we propose the following technique. First, we use a rotation module to produce robustly qualitive results even in various poses. This rotation makes source portraits with excessively rotated poses frontal, creating a state in which face reenactment techniques are easy to apply. In addition, we use style loss and attention map to maintain the style of the artwork. To evaluate the proposed technique, we objectively and subjectively compare the results of existing techniques with those of our technique. Our metrics include preservation of identity and facial expression, suppression of artifacts, and conservation of artistic style.