Frequency‐structure‐aware modelling for unsupervised zero‐shot cross‐domain heterogeneous face translation

Abstract

AbstractUnsupervised heterogeneous face translation requires obtaining heterogeneous images with the same identities at training time, limiting the use in unconstrained real‐world scenarios. Taking a step further towards unconstrained heterogeneous face translation, the authors explore unsupervised zero‐shot heterogeneous face translation for the first time, which is expected to synthesize images that resemble the style of target images and whose identities in the source domain have been preserved but never seen in the target domain during training. Essentially, asymmetry between heterogeneous faces under the zero‐shot setting further exacerbates the distortion and blurring of the translated images. The authors therefore propose a novel frequency‐structure‐guided regularization, which can jointly encourage to capture detailed textures and maintain identity consistency. Through extensive experimental validation and comparisons to several baseline methods on benchmark datasets, the authors verify the effectiveness of the proposed framework.