Region- and Strength-Controllable GAN for Defect Generation and Segmentation in Industrial Images

Abstract

Deep learning for computer vision has achieved remarkable results based on massive, diverse, and well-annotated training sets. However, it is difficult to collect defect datasets that cover all possible features, especially for small, weak defects. Therefore, in this article, a defect image generation method with controllable defect regions and strength is proposed. Regarded as image inpainting that uses a generative adversarial network, generated defect regions are controlled by using defect masks. Moreover, the defect direction vector is constructed in the latent variable space based on the feature continuity between defects and nondefects to control the defect strength, which enables a one-to-many correspondence between defect masks and images. Moreover, a defect attention loss is also designed to force the generation model to focus on the defect regions. Experimentally, our method yields generated images of better quality and diversity and thus significantly improves defect segmentation performance (an intersection over union of 63.20% and 61.86% on the Kolektor surface-defect and the metal hook defect datasets, respectively), especially for small, weak defects.