GAN with opposition-based blocks and channel self-attention mechanism for image synthesis

Abstract

Recently, image synthesis has always been a research hotspot in the field of deep learning. Generally, the methods based on generative adversarial networks (GANs) directly feed the semantic layout as input to obtain the photorealistic images for image synthesis. However, these methods based on GANs have not achieved satisfactory reconstructed results in quality. One of the main reasons is that the normalization layers in these methods will cause the loss of the semantic information. Another of the main reason is that the information contained in the semantic layout is sparse. In order to solve the above problems, GAN with opposition-based blocks and channel self-attention mechanism (OCGAN) is proposed. In OCGAN, the opposition-based learning method and the proposed adaptive normalization method are used to design the opposition-based blocks (OB Blks). The proposed channel self-attention mechanism (CSAM) is employed to give different focus to each channel of the semantic layout. The generator of OCGAN uses the opposition-based blocks and the channel self-attention mechanism to maintain and capture the important details from the semantic layouts. Experiments on several challenging datasets demonstrate the advantages of our method over existing approaches, regarding both visual quality and the representative evaluating criteria.