Multi streams with dynamic balancing-based Conditional Generative Adversarial Network for paired image generation

Abstract

Computer vision society experienced the birth of new CNN architecture known as Generative Adversarial Networks (GANs), which can generate fake images similar to real ones. The widespread use of GANs leads the image-to-image translation strategy dealing with more diverse tasks that were treated using traditional CNNs, such as medical analysis and semantic segmentation. In this paper, we propose a generic GAN referred to as Multi Streams with Dynamic Balancing-based Conditional Generative Adversarial Network (MSDB-CGAN). The MSDB-CGAN serves more challenging applications, that require multi input images such as binocular depth estimation, efficiently through its dedicated input streams and automatic skip connections. Moreover, the proposed GAN analyzes the inputs according to the target image, then assigns dynamic weights to the input streams. To validate the proposed MSDB-CGAN, we targeted four challenging tasks: binocular depth estimation, human-pose translation, middle frame interpolation, and future frame prediction. These applications present different inputs requirements and configurations. The reported quantitative and qualitative comparisons prove that the MSDB-CGAN significantly outperforms the existing GANs as well as traditional CNN-based architectures.