Improvement in crop mapping from satellite image time series by effectively supervising deep neural networks

Abstract

Deep learning methods have achieved promising results in crop mapping using satellite image time series. A challenge still remains on how to better learn discriminative feature representations to detect crop types when the model is applied to unseen data. To address this challenge and reveal the importance of proper supervision of deep neural networks in improving performance, we propose to supervise intermediate layers of a designed 3D Fully Convolutional Neural Network (FCN) by employing two middle supervision methods: Cross-entropy loss Middle Supervision (CE-MidS) and a novel middle supervision method, namely Supervised Contrastive loss Middle Supervision (SupCon-MidS). This method pulls together features belonging to the same class in embedding space, while pushing apart features from different classes. We demonstrate that SupCon-MidS enhances feature discrimination and clustering throughout the network, thereby improving the network performance. In addition, we employ two output supervision methods, namely F1 loss and Intersection Over Union (IOU) loss. Our experiments on identifying corn, soybean, and the class Other from Landsat image time series in the U.S. corn belt show that the best set-up of our method, namely IOU+SupCon-MidS, is able to outperform the state-of-the-art methods by mIOU scores of 3.5% and 0.5% on average when testing its accuracy across a different year (local test) and different regions (spatial test), respectively. Further, adding SupCon-MidS to the output supervision methods improves mIOU scores by 1.2% and 7.6% on average in local and spatial tests, respectively. We conclude that proper supervision of deep neural networks plays a significant role in improving crop mapping performance. The code and data are available at: https://github.com/Sina-Mohammadi/CropSupervision.