Improving spatial variation of ground-level PM2.5 prediction with contrastive learning from satellite imagery

Abstract

Convolutional Neural Networks (CNNs) are a promising technique to predict highly localized fine particulate matter (i.e., PM2.5 levels) based on high-resolution satellite imagery. Unfortunately, CNNs typically require large amounts of supervised data to perform well, whereas this application generally has lots of unsupervised data (all satellite imagery) and relatively sparse supervised data (measurements from ground sensors). Previous work used transfer learning from another visual task to initialize the CNN weights; however, we hypothesize that standard transfer learning strategies would bias the CNN to focus on irrelevant details of the image for our applications. Instead, we develop a novel framework called Spatiotemporal Contrastive Learning (SCL) to pre-train the CNN. We test both regular contrastive learning and SCL on predicting PM2.5 levels from satellite images in two different cities, Delhi and Beijing, and compare to CNNs with parameters initialized randomly and by transfer learning. Our results show that regular contrastive learning and our SCL frameworks both manage to better capture spatial variation of ground-level PM2.5 concentrations compared to traditional initialization schemes, and that this performance gap increases as the number of ground sensors decreases, implying that the approach will be even more valuable in cities with fewer ground sensors. Our work demonstrates that contrastive learning is a powerful pre-training technique to build better spatial maps of PM2.5, and can be broadly applied in related situations.