MGRLN-Net: Mask-Guided Residual Learning Network for Joint Single-Image Shadow Detection and Removal

Abstract

AbstractAlthough significant progress has been made in single-image shadow detection or single-image shadow removal, only few works consider these two problems together. However, the two problems are complementary and can benefit from each other. In this work, we propose a Mask-Guided Residual Learning Network (MGRLN-Net) that jointly estimates shadow mask and shadow-free image. In particular, MGRLN-Net first generates a shadow mask, then utilizes a feature reassembling module to align the features from the shadow detection module to the shadow removal module. Finally, we leverage the learned shadow mask as guidance to generate a shadow-free image. We formulate shadow removal as a masked residual learning problem of the original shadow image. In this way, the learned shadow mask is used as guidance to produce better transitions in penumbra regions. Extensive experiments on ISTD, ISTD+, and SRD benchmark datasets demonstrate that our method outperforms current state-of-the-art approaches on both shadow detection and shadow removal tasks. Our code is available at https://github.com/LeipingJie/MGRLN-Net.KeywordsShadow detection and removalMulti-task learningMasked residual learning