Lightweight U-Net for cloud detection of visible and thermal infrared remote sensing images

Jiaqiang Zhang,Tingliang Hu,Pengcheng Sun,Xiaoyan Li,Xiaofeng Su,Liyuan Li,Fansheng Chen

doi:10.1007/s11082-020-02500-8

Abstract

Accurate and rapid cloud detection is exceedingly significant for improving the downlink efficiency of on-orbit data, especially for the microsatellites with limited power and computational ability. However, the inference speed and large model limit the potential of on-orbit implementation of deep-learning-based cloud detection method. In view of the above problems, this paper proposes a lightweight network based on depthwise separable convolutions to reduce the size of model and computational cost of pixel-wise cloud detection methods. The network achieves lightweight end-to-end cloud detection through extracting feature maps from the images to generate the mask with the obtained maps. For the visible and thermal infrared bands of the Landsat 8 cloud cover assessment validation dataset, the experimental results show that the pixel accuracy of the proposed method for cloud detection is higher than 90%, the inference speed is about 5 times faster than that of U-Net, and the model parameters and floating-point operations are reduced to 12.4% and 12.8% of U-Net, respectively.

Highlights

According to the results of the International Satellite Cloud Climatology Project (ISCCP), clouds cover two-thirds of the land surface on earth (Rossow and Schiffer 1991), and high cloud coverage will reduce the accuracy and application of remote sensing data, resultingJiaqiang Zhang and Xiaoyan Li have contributed to this work.Extended author information available on the last page of the article1 3 Vol.:(0123456789)397 Page 2 of 14 in changes in the texture and spectral information of remote sensing images and affecting the radiation correction, geometric calibration and distortion correction (Li et al 2019)
The visible bands 4, 3, and 2 of the Landsat 8 Cloud Cover Assignment (L8 CCA) dataset (Foga et al 2017) are used as RGB channels to merge true color images. 72 images from Landsat 8 CCA were selected as the data set and processed into 28,800 patches with size of 224 × 224, 24,000 of which were used as the training set, and 2400 of which were used as the validation set. 6 images of 4480 × 4480 size were used as test set
Depthwise separable convolutions is applied to compress the model of U-Net and accelerate the inference speed of the neural network

Summary

Introduction

According to the results of the International Satellite Cloud Climatology Project (ISCCP), clouds cover two-thirds of the land surface on earth (Rossow and Schiffer 1991), and high cloud coverage will reduce the accuracy and application of remote sensing data, resulting. Deep learning networks combined with superpixels can obtain a good accuracy for cloud detection, the methods mentioned above are difficult to be implemented on-orbit and cannot complete pixel-wise cloud detection. The cloud detection methods mentioned above can complete pixel-wise end-to-end cloud detection with a good result, the size and the computational cost of the networks cannot meet the demands of small satellites with limited computing power and power consumption. We propose a lightweight network based on U-Net to reduce the size of model and computational cost of the pixel-wise cloud detection method and improve the portability of the cloud detection method on embedded platforms. The proposed method improves the pixel accuracy of the visible bands on the Landsat 8 cloud cover assessment (L8 CCA) validation dataset, and greatly reduces the size of the network, computing power requirements and inference time. Lightweight U-Net for cloud detection of visible and thermal

U‐Net based on depthwise separable convolutions

Metrics

Experimental images and parameter setting

Experimental results and analysis

Conclusion