Generating Users’ Desired Face Image Using the Conditional Generative Adversarial Network and Relevance Feedback

Caie Xu,Xiaoyang Mao,Masahiro Toyoura,Ying Tang,Jiayi Xu

doi:10.1109/access.2019.2956574

Caie Xu, Xiaoyang Mao + Show 3 more

Open Access

https://doi.org/10.1109/access.2019.2956574

Copy DOI

Journal: IEEE Access	Publication Date: Jan 1, 2019
Citations: 5	License type: CC BY 4.0

Affiliation: University of Yamanashi, Hangzhou Dianzi University

Abstract

In this study, we propose a novel method for generating an image of the target face by using the generative adversarial network (GAN) and relevance feedback. Combining GAN with relevance feedback compensates for the lack of user intervention in GAN and the low image quality in traditional methods. The feature points of face images, namely, the landmarks, are used as the conditional information for GAN to control the detailed features of the generated face image. An optimum-path forest classifier is applied to categorize the relevance of training images based on the user's feedback so that the user can quickly retrieve the training images that are most similar to the target face. The retrieved training images are then used to create a new landmark for synthesizing the target face image. The experimental results showed that users can generate images of their desired faces, including the faces in their memory, with a reasonable number of iterations, therefore demonstrating the potential of applying the proposed method for forensic purposes, such as creating the face images of criminals based on the memories of witnesses or victims.

Highlights

With the rapid development of deep learning technology, various research areas and applications, such as computer vision, robotics, big data analysis, and pilotless automobiles, have achieved major advancements
In 2015, Jon Gauthier et al proposed the use of conditional generative adversarial network (CGAN) for convolutional face generation [3], which added to CGAN the capability of generating face images with specific attributes, such as race, age, and emotion, by varying the conditional information
Our experiment showed that the proposed method can generate a result similar to the target face in the user’s memory, and it demonstrated its potential for forensic purpose, such as assisting the police to create the face image of a suspect based on the feedback of victims or witnesses

Summary

Introduction

With the rapid development of deep learning technology, various research areas and applications, such as computer vision, robotics, big data analysis, and pilotless automobiles, have achieved major advancements. The emergence of the generative adversarial network (GAN), which is a type of neural network architecture for the generative model first proposed by Goodfellow et al in 2014 [1], brought about a major breakthrough in the field of face image generation. To gain some control over the generated results, Mehdi and Simon proposed the conditional generative adversarial network (CGAN) in the same year, which allows inputting a condition to the model in addition to the noise [2]. This model set a solid foundation for the emergence of various variants of GAN. Grigory et al proposed the AgeGAN [4] for automatically simulating face aging based on CGAN; AgeGAN emphasizes the preservation of the original person’s identity in the aged version of his/her

Methods

Findings

Discussion

Conclusion