Dual-Branch Network for Cloud and Cloud Shadow Segmentation

Abstract

Cloud and cloud shadow segmentation is one of the most important issues in remote sensing image processing. Most of the remote sensing images are very complicated. In this work, a dual-branch model composed of Transformer and convolution network is proposed to extract semantic and spatial detail information of the image respectively to solve the problems of false detection and missed detection. To improve the model’s feature extraction, a Mutual Guidance Module is introduced so that the Transformer Branch and the Convolution Branch can guide each other for feature mining. Finally, in view of the problem of rough segmentation boundary, this work uses different features extracted by the Transformer Branch and the Convolution Branch for decoding, and repairs the rough segmentation boundary in the decoding part to make the segmentation boundary clearer. Experimental results on the Landsat-8, Sentinel-2 data, the public dataset HRC_WHU of Wuhan University and the public dataset SPARCS demonstrate the effectiveness of our method and its superiority to existing state-of-the-art Cloud and Cloud Shadow Segmentation approaches.