Intrinsic-style distribution matching for arbitrary style transfer

Abstract

Although arbitrary style transfer has been a hot topic in computer vision, most existing methods that directly align style and content features frequently result in unnatural effects in the generated images, such as over-stylization and style leakage. In this paper, we introduce a novel progressive Intrinsic-style Distribution Matching (ISDM) approach which initially aligns the intrinsic style distribution of both style and content images and then integrates it with the image’s content component. This novel approach can effectively alleviate the issue of over-stylization while preserving the content structure, particularly in the case of high contrast and vivid colors. To further enhance the performance of our method, we propose a learnable multi-level style modulation module to assist the network in aligning the intrinsic style distributions. Moreover, two contrastive objectives are proposed to improve the ability of the encoder to extract more distinct and representative intrinsic content and intrinsic style features. Extensive experimental results showcase that our approach can preserve more content details than state-of-the-art methods. It can also generate more natural images, especially when the style image has high contrast and vivid colors.