Prior-guided GAN-based interactive airplane engine damage image augmentation method

Abstract

Deep learning-based methods have achieved remarkable success in object detection, but this success requires the availability of a large number of training images. Collecting sufficient training images is difficult in detecting damages of airplane engines. Directly augmenting images by rotation, flipping, and random cropping cannot further improve the generalization ability of existing deep models. We propose an interactive augmentation method for airplane engine damage images using a prior-guided GAN to augment training images. Our method can generate many types of damages on arbitrary image regions according to the strokes of users. The proposed model consists of a prior network and a GAN. The Prior network generates a shape prior vector, which is used to encode the information of user strokes. The GAN takes the shape prior vector and random noise vectors to generate candidate damages. Final damages are pasted on the given positions of background images with an improved Poisson fusion. We compare the proposed method with traditional data augmentation methods by training airplane engine damage detectors with state-of-the-art object detectors, namely, Mask R-CNN, SSD, and YOLO v5. Experimental results show that training with images generated by our proposed data augmentation method achieves a better detection performance than that by traditional data augmentation methods.