Selective generative adversarial network for raindrop removal from a single image

Abstract

The removal of raindrops from a single image is still challenging because of the diversity and density of raindrops existing in the rainy image. Moreover, the colors of raindrops are constantly changing with the background which also makes the raindrops cannot be well removed by using the current methods. In this paper, we tackle these limitations by combining the raindrops shape features with the background structure features to guide the network to accurately remove raindrops. Specifically, we propose a selective skip connection GAN (SSCGAN) combining the selective skip connection and self-attention mechanism to restoring the clean image from a raindrop degraded one. Our main idea is selectively transmitting the information of raindrops to the decoder through Gated Recurrent Units (GRU) to better generate a clean image. During the training, the selective skip connection model (SSCM) extract raindrops binary mask from the rainy image and eliminate the interference of background noise. Simultaneously, we use self-attention blocks (SABs) to make the generator network pay more attention to global structure features of the rainy image and conversely correct the raindrops binary mask. Experiments show that our method has better performance than previous methods.