Evaluation of Five Deep Learning Models for Crop Type Mapping Using Sentinel-2 Time Series Images with Missing Information

Hongwei Zhao,Sibo Duan,Liang Sun,Jia Liu,Louis Reymondin

doi:10.3390/rs13142790

Abstract

Accurate crop type maps play an important role in food security due to their widespread applicability. Optical time series data (TSD) have proven to be significant for crop type mapping. However, filling in missing information due to clouds in optical imagery is always needed, which will increase the workload and the risk of error transmission, especially for imagery with high spatial resolution. The development of optical imagery with high temporal and spatial resolution and the emergence of deep learning algorithms provide solutions to this problem. Although the one-dimensional convolutional neural network (1D CNN), long short-term memory (LSTM), and gate recurrent unit (GRU) models have been used to classify crop types in previous studies, their ability to identify crop types using optical TSD with missing information needs to be further explored due to their different mechanisms for handling invalid values in TSD. In this research, we designed two groups of experiments to explore the performances and characteristics of the 1D CNN, LSTM, GRU, LSTM-CNN, and GRU-CNN models for crop type mapping using unfilled Sentinel-2 (Sentinel-2) TSD and to discover the differences between unfilled and filled Sentinel-2 TSD based on the same algorithm. A case study was conducted in Hengshui City, China, of which 70.3% is farmland. The results showed that the 1D CNN, LSTM-CNN, and GRU-CNN models achieved acceptable classification accuracies (above 85%) using unfilled TSD, even though the total missing rate of the sample values was 43.5%; these accuracies were higher and more stable than those obtained using filled TSD. Furthermore, the models recalled more samples on crop types with small parcels when using unfilled TSD. Although LSTM and GRU models did not attain accuracies as high as the other three models using unfilled TSD, their results were almost close to those with filled TSD. This research showed that crop types could be identified by deep learning features in Sentinel-2 dense time series images with missing information due to clouds or cloud shadows randomly, which avoided spending a lot of time on missing information reconstruction.

Highlights

Inspired by Che et al [23], a Mask layer to overcome the problem of missing values in time series was adopted, which made the pixels covered by the cloud not participate in the calculation, and the networks were labelled as Mask long short-term memory (LSTM) recurrent neural network (RNN)
The missing elements of samples in the time series data (TSD) were filled in using time series linear interpolation and Savitzky–Golay smoothing
The overall accuracy (OA) of these models in different groups were close. These results indicated that the five models could deep learn features of different crop types from Sentinel-2 dense TSD with missing information, even though the missing rate of Sentinel-2 TSD of all of the samples was 43.5%

Summary

Introduction

Accurate crop type information plays an important role in food security due to its widespread applicability, such as in yield estimates, crop rotation, and agricultural disaster assessment [1,2]. Optical time series data (TSD) have been proven to be efficient for crop type mapping, because the phenological evolution of each crop produces a unique temporal profile of reflectance [3]. Filling in missing information due to clouds in optical imagery is always needed [4,5,6,7]. Many different methods of missing information reconstruction have been developed [8,9,10], the majority of the high-precision methods are time-consuming and need a significant amount of computing resources [11], especially 4.0/).

Methods

Results

Discussion

Conclusion