Automatic Generation of High-Resolution Facial Expression Images with End-to-End Models Using Pix2Pix and Super-Resolution Convolutional Neural Network

Abstract

Recently, the means to see human face images have increased owing to the spread of smartphones and social networking services. Therefore, research on facial image generation, such as facial expression transformation, has been actively conducted. Especially, in the field of face images, the generation of face images using facial expression transformation has already been realized using generative adversarial networks (i.e., pix2pix). However, in the conventional models, only low-resolution images can be generated owing to limited computational resources, and the generated images are blur or aliasing. To solve this problem, we improved the resolution of generated images by training the Pix2Pix and super-resolution convolutional neural network methods as one model end-to-end instead of training them separately. Using the peak signal-to-noise ratio as an evaluation index, image quality was improved by 0.391 dB compared with the conventional model.