SC-GAN: Subspace Clustering based GAN for Automatic Expression Manipulation

Abstract

In recent years, the topics of facial attribute manipulation and decomposition have gained great popularity in computer vision and human computer interaction. Even though such methods have been preliminarily employed in some photo beautification applications, it still remains challenging due to the highly-versatile facial attributes and their drastic appearance changes subject to variation of deformation, illumination, pose, etc. The prevailing problems are especially severe when we are faced with group photos involving many faces. To overcome such critical limitations and discover more meaningful visual attributes and their possible decompositions, we develop a subspace clustering based generative adversarial network (SC-GAN) in this paper. Our SC-GAN can simultaneously decompose multiple subspaces and generate diverse samples correspondingly, thus the training of the generative models could be more effectively guided by facial attribute and its decomposition and manipulation in a natural and meaningful fashion. Our SC-GAN incorporates the SIFT K-means cluster, which could split the holistic semantic facial space into different subspaces without supervision, and help the new GAN generate more convincing results within specific subspaces. Extensive experiments and comprehensive evaluations confirm that, our method can greatly reduce the unexpected influences caused by portrait diversities and outperform the state-of-the-art facial attribute manipulation approaches.11https://github.com/buaaswf/SC-GAN/