A new attention-based CNN approach for crop mapping using time series Sentinel-2 images

Abstract

Accurate crop mapping is of great importance for agricultural applications, and deep learning methods have been applied on multi-temporal remotely sensed images to classify crops. However, due to the geographic heterogeneity, the spectral profiles of the same crop can vary spatially, and thus using the spectral features alone can limit the model performance in mapping crops in large scales. Moreover, it is a challenge for traditional deep learning models to accurately capture the important information from a large number of features. To address these issues, in this study, we developed a novel attention-based convolutional neural network (CNN) approach (Geo-CBAM-CNN) for crop classification using time series Sentinel-2 images. Specifically, geographic information of crops was first integrated into an advanced attention module, Convolutional Block Attention Module (CBAM) to form a Geo-CBAM module which can help mitigate the impacts of geographic heterogeneity and restrain unnecessary information. Then, the developed Geo-CBAM module was embedded into a CNN model to boost the model’s attention both spectrally and spatially. The proposed Geo-CBAM-CNN model was validated on four main crops over six counties with different geographic environments in the U.S. Also, it was compared to three other state-of-the-art machine learning approaches, including CBAM-CNN, CNN and Random Forest (RF). The results showed that the proposed model achieved the best performance, reaching 97.82% overall accuracy, 96.82% Kappa coefficient and 96.96% Macro-average F1 score. Moreover, the developed Geo-CBAM-CNN model showed strong spatial adaptability, indicating its superior performance in large scale applications. Furthermore, by visualizing the structure of the Geo-CBAM-CNN, we found that the model automatically allocated different weights to the features, and generally, the red-edge features in the middle of the year obtained more attention.