High-quality matched transfer generation adversarial network for synthetic cross-material surface defect images

Abstract

Generating Adversarial Network based on style transfer is an effective method to expand sample data. Nonetheless, an urgent issue that demands resolution is the fusion of cross-material defects and backgrounds to generate high-quality defect samples. In this paper, we propose the High-quality Matching Transfer Generative Adversarial Network (HMTGAN), an innovative framework. This network is based on the principles of CycleGAN and StyleGAN, specifically designed for advanced image generation and migration tasks. The network model incorporates multidimensional latent variables within the input image. It employs a unique method of latent space random sampling to effectively combine source defects and background materials. Additionally, a novel loss function is designed, leveraging the straight-through gradient with discrete random variables back-propagation, to effectively compare the binary defects between the source and synthetic images. Following this process, some realistic-looking defect samples are generated. We obtained a KID score of 120.76 and a FID score of 0.16, along with a Class IOU score of 0.11 on the crack defect dataset, by guiding the image generation through training style transformations. We conducted defect migration experiments on various textured surfaces to explore crack generation and migration. Extensive experiments on aluminum plate defect and crack datasets show that our method achieves state-of-the-art performance for image generation.