Shadow-aware decomposed transformer network for shadow detection and removal

Abstract

Shadow detection and removal are important yet challenging computer vision tasks. Existing methods simultaneously contend with the brightness and color information of input image while treating different regions of input images equally. We argue that these operations fail to effectively extract the relationship between shadow and non-shadow regions. To relieve these problems, this paper proposes a shadow-aware decomposed transformer network for shadow detection and removal. The network decomposes the input image into brightness and color maps using its bright channel, which are concatenated with the original image as the input data, enabling the network to pay balanced attention to the brightness and color information when calculating the relationship between regions. Additionally, given that the correlation matrix of the transformer measures the relative dependency between regions, the proposed shadow-aware transformer block can extract the relationship between shadow and non-shadow regions more effectively by retaining the specific elements of the correlation matrix. We conduct extensive experiments on three shadow detection benchmark datasets and two shadow removal benchmark datasets. Experimental results show that the proposed method performs favorably against state-of-the-art methods. Codes have been made available at https://github.com/XIAOWANG914/SADT.