Domain adaptation for complex shadow removal with shadow transformer network

Abstract

Shadows observed in practical scenes have complex shapes, and removing them is a very challenging task in computer vision. In particular, previous studies on complex shadow removal have limitations in that they can be learned only on paired datasets. Taking the issues into consideration, we present a new domain-adaptive shadow removal framework. The proposed approach includes domain adaptation, detection, and removal stages. The shadow-preserving domain translator in the first stage compensates for the lack of real data through domain transformation of the synthetic data. In the second stage, efficient shadow detection is performed through the domain adaptive mean teacher network. Last, a novel attention network removes complex shadows using detected shadows as a query, effectively removing complex shadows. The feasibility and effectiveness of the proposed framework are validated through the newly collected Grand Theft Auto-Road Shadow dataset. The proposed method outperforms existing methods for quantitative and qualitative metrics related to shadow detection and removal.