Attentive Generative Adversarial Network with Dual Encoder-Decoder for Shadow Removal

Abstract

Shadow removal is a fundamental task that aims at restoring dark areas in an image where the light source is blocked by an opaque object, to improve the visibility of shadowed areas. Existing shadow removal methods have developed for decades and yielded many promising results, but most of them are poor at maintaining consistency between shadowed regions and shadow-free regions, resulting in obvious artifacts in restored areas. In this paper, we propose a two-stage (i.e., shadow detection and shadow removal) method based on the Generative Adversarial Network (GAN) to remove shadows. In the shadow detection stage, a Recurrent Neural Network (RNN) is trained to obtain the attention map of shadowed areas. Then the attention map is injected into both generator and discriminator to guide the shadow removal stage. The generator is a dual encoder-decoder that processes the shadowed regions and shadow-free regions separately to reduce inconsistency. The whole network is trained with a spatial variant reconstruction loss along with the GAN loss to make the recovered images more natural. In addition, a novel feature-level perceptual loss is proposed to ensure enhanced images more similar to ground truths. Quantitative metrics like PSNR and SSIM on the ISTD dataset demonstrate that our method outperforms other compared methods. In the meantime, the qualitative comparison shows our approach can effectively avoid artifacts in the restored shadowed areas while keeping structural consistency between shadowed regions and shadow-free regions.