DuCaGAN: Unified Dual Capsule Generative Adversarial Network for Unsupervised Image-to-Image Translation

Guifang Shao,Tundong Liu,Meng Huang,Liduan Li,Fengqiang Gao

doi:10.1109/access.2020.3007266

Guifang Shao, Tundong Liu + Show 3 more

Open Access

https://doi.org/10.1109/access.2020.3007266

Copy DOI

Abstract

With the appearance of Generative Adversarial Network (GAN), image-to-image translation based on a new unified framework has attracted growing interests. As a new technique, it can generate synthesizing images for various requirements in both computer vision and image processing. However, the cycle consistent structure adopted in some common models, such as cycle generative adversarial network (CycleGAN), is usually unable to learn more abundant image features. In this work, we developed a novel model based on GAN, named as dual capsule generative adversarial network (DuCaGAN), by utilizing the distinctive characteristic of view angle invariance and rotation equivariance in capsule network. Firstly, two capsule networks were introduced into the traditional CycleGAN model as discriminators to form our proposed model with six agents. To improve the feature capturing performance, we modified the full objective by combining the margin loss and the original adversarial loss. Furthermore, the Routing Algorithm in the capsule network was optimized by changing its compression function. Finally, experimental results on conventional visual tasks with paired and unpaired datasets demonstrated the superiority and effectiveness of the proposed approach compared to both deep convolutional generative adversarial network (DCGAN) and CycleGAN methods. More importantly, the proposed DuCaGAN was applied for the first time to augment the surface defect data from the real industrial field, and exhibited better performance than those methods available.

Highlights

Image-to-image translation, mapping an image from one domain to another, can resolve many problems in computer vision and image processing, such as texture synthesis, image super-resolution, image segmentation, style transfer, season transfer, and data augmentation [1]
This model based on the existing framework of CycleGAN [12]. optimizes the loss function and network structure Two capsule networks were introduced as the discriminators in our model to learn more detailed features, such as geometric features
EXPERIMENT SETTING To evaluate the efficiency of the proposed dual capsule generative adversarial network (DuCaGAN), experiments on various tasks are conducted

Summary

Introduction

Image-to-image translation, mapping an image from one domain to another, can resolve many problems in computer vision and image processing, such as texture synthesis, image super-resolution, image segmentation, style transfer, season transfer, and data augmentation [1]. Surface defect data from the real industrial field can be formalized as the image translation problem [4], [5]. Due to the multiple sub-phases and various devices used in a complete industrial process, there may exist different surface defects containing limited feature information on one product. These surface defects on the product appear occasionally and result in the rare defect samples. Insufficient sample sizes and sample class imbalances have become an urgent problem to be solved for the defect data in the real industrial process [4], [5].

Methods

Results

Conclusion