Generative Adversarial Networks with Bi-directional Normalization for Semantic Image Synthesis

Abstract

Semantic image synthesis aims at translating semantic label maps to photo-realistic images. However, most of previous methods easily generate blurred regions and artifacts, and the quality of these images is far from realistic. There are two unresolved problems existing: first, these methods directly feed the semantic label as input to the deep network, through convolution operation to produce the normalization parameters γ and β, we find that the semantic labels are different from real scene images, they are not able to provide detailed structural information, making it difficult to synthesize local details and structures; second, there are no bi-directional information flow between the semantic labels and the real scene images, this leads to inefficiently utilize the semantic information and maintain semantic constrains to preserve the semantic information in the process of semantic image synthesis. We propose Bi-directional Normalization (BDN) in our generative adversarial networks to solve these problems, which allows semantic label information and real scene image feature representation to be effectively utilized by a bi-directional way for generating high quality images. Extensive experiments on several challenging datasets demonstrate significantly better than that results of existing methods in both visual fidelity and quantitative metrics.