Towards semantically continuous unpaired image-to-image translation via margin adaptive contrastive learning and wavelet transform

Abstract

Unpaired image-to-image translation aims to preserve the semantics of the input image while mimicking the style of target domains without paired data. However, existing methods often suffer from semantic distortions if the source and target domains have large mismatched semantics distributions. To address semantic distortions in translation outputs without paired supervision, we propose a Margin Adaptive Contrastive Learning Network (MACL-Net) that drives contrastive learning as a local semantic descriptor while using a pre-trained Vision Transformer (ViT) as a global semantic descriptor to learn domain-invariant features in the translation process. Specifically, we design a novel margin adaptive contrastive loss to enforce intra-class compactness and inter-class discrepancy. Besides, to better retain the semantic structure of the translated image and improve its fidelity, we use Discrete Wavelet Transform (DWT) to supplement the low-frequency and high-frequency information of the input image into the generator, and effectively fuse the feedforward features and inversed frequency information through a novel normalization scheme, Feature-Frequency Transformation Normalization (FFTN). In terms of experimental results, MACL-Net effectively reduces semantic distortions and generates translation outputs that outperform state-of-the-art techniques both quantitatively and qualitatively.