RMLANet: Random Multi-Level Attention Network for Shadow Detection and Removal

Abstract

This paper addresses the problem of shadow detection and shadow removal from a single image. Despite awareness of utilizing both local and global contexts, previous works only aggregate features level by level in a coarse-to-fine manner. To overcome this problem, we present RMLANet, a novel Random Multi-Level Attention Network. To be specific, we first design a shuffled multi-level feature aggregation module to fuse the multi-level features and the guiding features using the self-attention mechanism. Nevertheless, the computational complexity of dense self-attention is unaffordable when processing high-resolution inputs. We argue that dense attention between any pixel pair is unnecessary due to the local consistency in images. Then we further propose a sparse attention mechanism to reduce the number of attention pairs, which greatly reduces the computational complexity. Through extensive experiments on four shadow detection and three shadow removal benchmark datasets, our proposed RMLANet achieves superior performance over current state-of-the-art approaches for both shadow detection and shadow removal. Codes are publicly available at https://github.com/LeipingJie/RMLANet.