Erasing-inpainting-based data augmentation using denoising diffusion probabilistic models with limited samples for generalized surface defect inspection

Abstract

Surface defect inspection aims to identify defective regions in product surface images to ensure product quality. Existing deep learning methods have developed rapidly on surface defect inspection. However, their excellent performances rely on a large number of training samples, which are hard to acquire in practical industrial scenarios due to the continuous improvements of the production line. To solve this issue, we propose an erasing-inpainting-based data augmentation method using a denoising diffusion probabilistic model (DDPM) with limited samples for generalized surface defect inspection. Our method is based on the idea that a defect image is difficult to recover to its previous state after undergoing a large-scale erasure operation, thus diverse defect images can be generated using different settings in the inpainting model. Specifically, we first train a DDPM model using limited defective images. Then, we erase large-scale parts of an input image to obtain a degraded image and restore the erased areas using the trained DDPM. Finally, the repaired images are further used for updating the DDPM. The main advantage of our method is to generate diverse images by only being trained using limited training samples. On the one hand, our method fundamentally avoids the dimension inconsistency between the sampled noise and the generated image by sampling from a two-dimensional noise map with the same resolution as the output image based on DDPM. On the other hand, the proposed erasing-inpainting operation promotes the recombination of the real features from the training set and the learned features from the trained DDPM to fully use the limited defective samples and the easily obtainable defect-free samples. Extensive experiments demonstrated the effectiveness and advantagesof our model on data augmentation for generalized surface defect inspection.