Generative adversarial networks for extrapolation of corrosion in automobile images

Abstract

Deep learning has shown success in several applications involving pattern recognition, expert systems, and scientific discovery. However, existing methods struggle with industrial applications, which are often challenged by non-ideal datasets. In many cases, the datasets are small, poorly labeled, noisy, or have unbalanced class distribution, or any combination of such problems. In this contribution, we propose a generative adversarial network (GAN) strategy that is able to circumvent limitations imposed by very small datasets. As case study, we use the extrapolation of corrosion in automobiles and feed our deep learning framework with only few dozen images, as opposed to the thousands to million images commonly found in many computer vision problems. In order to handle such a reduced dataset, we use one GAN for the rust level and one for the rust texture. The rust level GAN is conditional on random samples from the dataset and uses an additive random noise in the latent space to add variability to the generated rust level maps. The rust texture GAN adds the shades of brown to the outputs of the rust level GAN. To improve the robustness of our approach, the rust level GAN is a conditional GAN with random noise added in the latent space of the generator; while the rust texture GAN uses supervised loss functions coming from the observed data set as well as an unsupervised loss function coming from the rust level GAN. In addition, given the very reduced size of our dataset, it is unfeasible to break down the data into training, validation, and test sets. We overcome this limitation by using the discrepancy between the generated and target distributions of the rust level and texture intensities as a way to monitor convergence of training. The resulting models are able to ingest an image with a car having no corrosion and generates an image of this car with parts exhibiting varying degrees of corrosion (from mild, to moderate, to severe). The source codes and links to the data can be found in the following GitHub repository https://github.com/PML-UCF/rusty_fender_gan.