Multi-modal image synthesis combining content-style adaptive normalization and attentive normalization

Abstract

We propose a novel unsupervised image translation model following an end-to-end manner,which incorporates Content-Style Adaptive Normalization(CSAN) and Attentive Normalization(AN). First of all, a new attentive normalization is applied for the first time in the style transfer task, which is an improvement and supplement to the traditional instance normalization, it helps to guide the model to pay more attention to the key areas in image translation, while ignoring the secondary areas. Secondly, our proposed CSAN function absorbs not only information of style codes, but also that of content codes. Compared with Adaptive Instance Normalization(AdaIN), CSAN is more favorable to retain content information of input images. In addition, CSAN can help the attention mechanism to flexibly control the amount of change in texture and shape of input images. Finally, a series of comparative experiments and qualitative and quantitative evaluations on the challenging datasets prove that the proposed model is superior and more advanced than State-Of-The-Art(SOTA) in terms of visual quality, diversity,semantic integrity, and style reflection of generated images.