Geometry-Aware GAN for Face Attribute Transfer

Abstract

In this paper, the geometry-aware GAN, referred to as GAGAN, is proposed to address the issue of face attribute transfer with unpaired data. The source and target images are not aligned and come from different individuals. The key idea is to deform the source image according to the geometry features and generate a high-resolution image with the desired attribute. To address the problem of the unpaired training samples, the CycleGAN architecture is applied to form an attribute adding and removing cycle, where the bilateral mappings between the source and target domains are learned. The geometry flow and occlusion mask are learned by the warping sub-network to capture the geometric variation between the two domains. In the attribute adding process, the spatial transformer network (STN) warps the source face into the desired pose and shape according to the flow, and the transfer sub-network hallucinates new components on the warped image. In the attribute removing process, the recover sub-network and the STN reverts the sample back to the source domain. Experiments on the benchmarks CELEBA and CELEBA-HQ datasets demonstrate the advantages of our method compared to the baselines, in terms of both quantitative and qualitative evaluation.