LPAdaIN: Light Progressive Attention Adaptive Instance Normalization Model for Style Transfer

Abstract

To improve the generation quality of image style transfer, this paper proposes a light progressive attention adaptive instance normalization (LPAdaIN) model that combines the adaptive instance normalization (AdaIN) layer and the convolutional block attention module (CBAM). In the construction of the model structure, first, a lightweight autoencoder is built to reduce the information loss in the encoding process by reducing the number of network layers and to alleviate the distortion of the stylized image structure. Second, each AdaIN layer is progressively applied after the three relu layers in the encoder to obtain the fine-grained stylized feature maps. Third, the CBAM is added between the last AdaIN layer and the decoder, ensuring that the main objects in the stylized image are clearly visible. In the model optimization, a reconstruction loss is designed to improve the decoder’s ability to decode stylized images with more precise constraints and refine the structure of the stylized images. Compared with five classical style transfer models, the LPAdaIN is visually shown to more finely apply the texture of the style image to the content image, in order to obtain a stylized image, in which the main objects are clearly visible and the structure can be maintained. In terms of quantitative metrics, the LPAdaIN achieved good results in running speed and structural similarity.