Trilateral GAN with Channel Attention Residual for Semantic Image Synthesis

Abstract

In this paper, a novel method for synthesizing photo-realistic images from semantic label maps using GANs is proposed, which is an ideal and challenging task in computer vision and image synthesis. Due to the sparsity of the information contained in semantic label maps, it is difficult for some existing methods to achieve satisfactory synthesis effect. This paper proposes a trilateral generative adversarial network to support multi-directional transmission between images of different resolutions, called TrilateralGAN. Compared with the traditional single-directional transmission, the design of our TrilateralGAN network can better retain the information in the original image to avoid loss of details. In addition, we further propose a new channel attention residual as the main part of the TrilateralGAN network. This part can enhance the retained information to varying degrees, which can make the image synthesized by TrilateralGAN have clearer edges and richer details. The experimental results on Cityscapes and ADE20K datasets demonstrate the advantage of TrilateralGAN over the state-of-the-art approaches, regarding both visual quality and the representative evaluating criteria.