Seeing Through Clouds in Satellite Images

Abstract

This paper presents a neural network-based solution to recover pixels occluded by clouds in satellite images. We leverage radio frequency (RF) signals in the ultra/super-high frequency band that penetrate clouds to help reconstruct the occluded regions in multispectral images. We introduce the first multi-modal multi-temporal method for cloud removal. Our model uses publicly available satellite observations and produces daily cloud-free images. Experimental results show that our system outperforms several baselines on multiple metrics. We also demonstrate use cases of our system in digital agriculture, flood monitoring, and wildfire detection.