Improving crop type mapping by integrating LSTM with temporal random masking and pixel-set spatial information

Crop type classification is critical for crop production estimation and optimal water allocation. Crop type data are challenging to generate if crop reference data are lacking, especially for target years with reference data missed in collection. Is it possible to transfer a trained crop type classification model to retrace the historical spatial distribution of crop types? Taking the Hetao Irrigation District (HID) in China as the study area, this study first designed a 10 m crop type classification framework based on the Google Earth Engine (GEE) for crop type mapping in the current season. Then, its interannual transferability to accurately retrace historical crop distributions was tested. The framework used Sentinel-1/2 data as the satellite data source, combined percentile, and monthly composite approaches to generate classification metrics and employed a random forest classifier with 300 trees for crop classification. Based on the proposed framework, this study first developed a 10 m crop type map of the HID for 2020 with an overall accuracy (OA) of 0.89 and then obtained a 10 m crop type map of the HID for 2019 with an OA of 0.92 by transferring the trained model for 2020 without crop reference samples. The results indicated that the designed framework could effectively identify HID crop types and have good transferability to obtain historical crop type data with acceptable accuracy. Our results found that SWIR1, Green, and Red Edge2 were the top three reflectance bands for crop classification. The land surface water index (LSWI), normalized difference water index (NDWI), and enhanced vegetation index (EVI) were the top three vegetation indices for crop classification. April to August was the most suitable time window for crop type classification in the HID. Sentinel-1 information played a positive role in the interannual transfer of the trained model, increasing the OA from 90.73% with Sentinel 2 alone to 91.58% with Sentinel-1 and Sentinel-2 together.

Crop type mapping at high resolution is crucial for various purposes related to agriculture and food security, including the monitoring of crop yields, evaluating the potential effects of natural disasters on agricultural production, analyzing the potential impacts of climate change on agriculture, etc. However, accurately mapping crop types and ranges on large spatial scales remains a challenge. For the accurate mapping of crop types at the regional scale, this paper proposed a crop type mapping method based on the combination of multiple single-temporal feature images and time-series feature images derived from Sentinel-1 (SAR) and Sentinel-2 (optical) satellite imagery on the Google Earth Engine (GEE) platform. Firstly, crop type classification was performed separately using multiple single-temporal feature images and the time-series feature image. Secondly, with the help of information entropy, this study proposed a pixel-scale crop type classification accuracy evaluation metric, i.e., the CA-score, which was used to conduct a vote on the classification results of multiple single-temporal images and the time-series feature image to obtain the final crop type map. A comparative analysis showed that the proposed classification method had excellent performance and that it can achieve accurate mapping of multiple crop types at a 10 m resolution for large spatial scales. The overall accuracy (OA) and the kappa coefficient (KC) were 84.15% and 0.80, respectively. Compared with the classification results that were based on the time-series feature image, the OA was improved by 3.37%, and the KC was improved by 0.03. In addition, the CA-score proposed in this study can effectively reflect the accuracy of crop identification and can serve as a pixel-scale classification accuracy evaluation metric, providing a more comprehensive visual interpretation of the classification accuracy. The proposed method and metrics have the potential to be applied to the mapping of larger study areas with more complex land cover types using remote sensing.

High-resolution crop type mapping is of importance for site-specific agricultural management and food security in smallholder farming regions, but is challenging due to limited data availability and the need for image-based algorithms. In this paper, we developed an efficient object- and pixel-based mapping algorithm to generate a 10 m resolution crop type map over large spatial domains by integrating time series optical images (Sentinel-2) and synthetic aperture radar (SAR) images (Sentinel-1) using the Google Earth Engine (GEE) platform. The results showed that the proposed method was reliable for crop type mapping in the study area with an overall accuracy (OA) of 93.22% and a kappa coefficient (KC) of 0.89. Through experiments, we also found that the monthly median values of the vertical transmit/vertical receive (VV) and vertical transmit/horizontal receive (VH) bands were insensitive to crop type mapping itself, but adding this information to supplement the optical images improved the classification accuracy, with an OA increase of 0.09–2.98%. Adding the slope of vegetation index change (VIslope) at the critical period to crop type classification was obviously better than that of relative change ratio of vegetation index (VIratio), both of which could make an OA improvement of 2.58%. These findings not only highlighted the potential of the VIslope and VIratio indices during the critical period for crop type mapping in small plots, but suggested that SAR images could be included to supplement optical images for crop type classification.

Appropriate crop type mapping to monitor and control land management is very important in developing countries. It can be very useful where digital cadaster maps are not available or usage of Remote Sensing (RS) data is not utilized in the process of monitoring and inventory. The main goal of the present research is to compare and assess the importance of optical RS data in crop type classification using medium and high spatial resolution RS imagery in 2018. With this goal, Landsat 8 (L8) and Sentinel-2 (S2) data were acquired over the Tashkent Province between the crop growth period of May and October. In addition, this period is the only possible time for having cloud-free satellite images. The following four indices “Normalized Difference Vegetation Index” (NDVI), “Enhanced Vegetation Index” (EVI), and “Normalized Difference Water Index” (NDWI1 and NDWI2) were calculated using blue, red, near-infrared, shortwave infrared 1, and shortwave infrared 2 bands. Support-Vector-Machine (SVM) and Random Forest (RF) classification methods were used to generate the main crop type maps. As a result, the Overall Accuracy (OA) of all indices was above 84% and the highest OA of 92% was achieved together with EVI-NDVI and the RF method of L8 sensor data. The highest Kappa Accuracy (KA) was found with the RF method of L8 data when EVI (KA of 88%) and EVI-NDVI (KA of 87%) indices were used. A comparison of the classified crop type area with Official State Statistics (OSS) data about sown crops area demonstrated that the smallest absolute weighted average (WA) value difference (0.2 thousand ha) was obtained using EVI-NDVI with RF method and NDVI with SVM method of L8 sensor data. For S2-sensor data, the smallest absolute value difference result (0.1 thousand ha) was obtained using EVI with RF method and 0.4 thousand ha using NDVI with SVM method. Therefore, it can be concluded that the results demonstrate new opportunities in the joint use of Landsat and Sentinel data in the future to capture high temporal resolution during the vegetation growth period for crop type mapping. We believe that the joint use of S2 and L8 data enables the separation of crop types and increases the classification accuracy.

Timely and accurate crop type mapping is a critical prerequisite for the estimation of water availability and environmental carrying capacity. This research proposed a method to integrate time series Sentinel-1 (S1) and Sentinel-2 (S2) data for crop type mapping over oasis agricultural areas through a case study in Northwest China. Previous studies using synthetic aperture radar (SAR) data alone often yield quite limited accuracy in crop type identification due to speckles. To improve the quality of SAR features, we adopted a statistically homogeneous pixel (SHP) distributed scatterer interferometry (DSI) algorithm, originally proposed in the interferometric SAR (InSAR) community for distributed scatters (DSs) extraction, to identify statistically homogeneous pixel subsets (SHPs). On the basis of this algorithm, the SAR backscatter intensity was de-speckled, and the bias of coherence was mitigated. In addition to backscatter intensity, several InSAR products were extracted for crop type classification, including the interferometric coherence, master versus slave intensity ratio, and amplitude dispersion derived from SAR data. To explore the role of red-edge wavelengths in oasis crop type discrimination, we derived 11 red-edge indices and three red-edge bands from Sentinel-2 images, together with the conventional optical features, to serve as input features for classification. To deal with the high dimension of combined SAR and optical features, an automated feature selection method, i.e., recursive feature increment, was developed to obtain the optimal combination of S1 and S2 features to achieve the highest mapping accuracy. Using a random forest classifier, a distribution map of five major crop types was produced with an overall accuracy of 83.22% and kappa coefficient of 0.77. The contribution of SAR and optical features were investigated. SAR intensity in VH polarization was proved to be most important for crop type identification among all the microwave and optical features employed in this study. Some of the InSAR products, i.e., the amplitude dispersion, master versus slave intensity ratio, and coherence, were found to be beneficial for oasis crop type mapping. It was proved the inclusion of red-edge wavelengths improved the overall accuracy (OA) of crop type mapping by 1.84% compared with only using conventional optical features. In comparison, it was demonstrated that the synergistic use of time series Sentinel-1 and Sentinel-2 data achieved the best performance in the oasis crop type discrimination.

Knowledge about crop type distribution is valuable information for effective management of agricultural productivity, food security estimation, and natural resources protection. Algorithms for automatic crop type detection have great potential to positively influence these aspects as well as speed up the process of crop type mapping in larger areas. In the presented study, we used 14 Sentinel-2 images to calculate 12 widely used spectral vegetation indices. Further, to evaluate the effect of reduced dimensionality on the accuracy of crop type mapping, we utilized principal component analysis (PCA). For this purpose, random forest (RF)-supervised classifications were tested for each index separately, as well as for the combinations of various indices and the four initial PCA components. Additionally, for each RF classification feature importance was assessed, which enabled identification of the most relevant period of the year for the differentiation of crop types. We used 34.6% of the ground truth field data to train the classifier and calculate various accuracy measures such as the overall accuracy (OA) or Kappa index. The study showed a high effectiveness of the Modified Chlorophyll Absorption in Reflectance Index (MCARI) (OA = 86%, Kappa = 0.81), Normalized Difference Index 45 (NDI45) (OA = 85%, Kappa = 0.81), and Weighted Difference Vegetation Index (WDVI) (OA = 85%, Kappa = 0.80) in crop type mapping. However, utilization of all of them together did not increase the classification accuracy (OA = 78%, Kappa = 0.72). Additionally, the application of the initial three components of PCA allowed us to achieve an OA of 78% and Kappa of 0.72, which was unfortunately lower than the single-index classification (e.g., based on only NDVI45). This shows that dimensionality reductions did not increase the classification accuracy. Moreover, feature importance from RF indicated that images captured from June and July are the most relevant for differentiating crop types. This shows that this period of the year is crucial to effectively differentiate crop types and should be undeniably used in crop type mapping.

Mapping crop types and land cover in smallholder farming systems in sub-Saharan Africa remains a challenge due to data costs, high cloud cover, and poor temporal resolution of satellite data. With improvement in satellite technology and image processing techniques, there is a potential for integrating data from sensors with different spectral characteristics and temporal resolutions to effectively map crop types and land cover. In our Malawi study area, it is common that there are no cloud-free images available for the entire crop growth season. The goal of this experiment is to produce detailed crop type and land cover maps in agricultural landscapes using the Sentinel-1 (S-1) radar data, Sentinel-2 (S-2) optical data, S-2 and PlanetScope data fusion, and S-1 C2 matrix and S-1 H/α polarimetric decomposition. We evaluated the ability to combine these data to map crop types and land cover in two smallholder farming locations. The random forest algorithm, trained with crop and land cover type data collected in the field, complemented with samples digitized from Google Earth Pro and DigitalGlobe, was used for the classification experiments. The results show that the S-2 and PlanetScope fused image + S-1 covariance (C2) matrix + H/α polarimetric decomposition (an entropy-based decomposition method) fusion outperformed all other image combinations, producing higher overall accuracies (OAs) (>85%) and Kappa coefficients (>0.80). These OAs represent a 13.53% and 11.7% improvement on the Sentinel-2-only (OAs < 80%) experiment for Thimalala and Edundu, respectively. The experiment also provided accurate insights into the distribution of crop and land cover types in the area. The findings suggest that in cloud-dense and resource-poor locations, fusing high temporal resolution radar data with available optical data presents an opportunity for operational mapping of crop types and land cover to support food security and environmental management decision-making.

Mapping of crop types in 1989, 1999, 2009 and 2019 to assess major land cover trends of the Udon Thani Province, Thailand

Developments in space-based hyperspectral sensors, advanced remote sensing, and machine learning can help crop yield measurement, modelling, prediction, and crop monitoring for loss prevention and global food security. However, precise and continuous spectral signatures, important for large-area crop growth monitoring and early prediction of yield production with cutting-edge algorithms, can be only provided via hyperspectral imaging. Therefore, this article used new-generation Deutsches Zentrum für Luft- und Raumfahrt Earth Sensing Imaging Spectrometer (DESIS) images to classify the main crop types (hybrid corn, soybean, sunflower, and winter wheat) in Mezőhegyes (southeastern Hungary). A Wavelet-attention convolutional neural network (WA-CNN), random forest and support vector machine (SVM) algorithms were utilized to automatically map the crops over the agricultural lands. The best accuracy was achieved with the WA-CNN, a feature-based deep learning algorithm and a combination of two images with overall accuracy (OA) value of 97.89% and the user's accuracy producer's accuracy was from 97% to 99%. To obtain this, first, factor analysis was introduced to decrease the size of the hyperspectral image data cube. A wavelet transform was applied to extract important features and combined with the spectral attention mechanism CNN to gain higher accuracy in mapping crop types. Followed by SVM algorithm reported OA of 87.79%, with the producer's and user's accuracies of its classes ranging from 79.62% to 96.48% and from 79.63% to 95.73%, respectively. These results demonstrate the potentiality of DESIS data to observe the growth of different crop types and predict the harvest volume, which is crucial for farmers, smallholders, and decision-makers.

A timely and accurate distribution of crop types and cropping patterns provides a crucial reference for the management of agriculture and food security. However, accurately mapping crop types and cropping patterns in subtropical hilly and mountainous areas often face challenges such as mixed pixels resulted from fragmented patches and difficulty in obtaining optical satellites due to a frequently cloudy and rainy climate. Here we propose a crop type and cropping pattern mapping framework in subtropical hilly and mountainous areas, considering multiple sources of satellites (i.e., Landsat 8/9, Sentinel-2, and Sentinel-1 images and GF 1/2/7). To develop this framework, six types of variables from multi-sources data were applied in a random forest classifier to map major summer crop types (singe-cropped rice and double-cropped rice) and winter crop types (rapeseed). Multi-scale segmentation methods were applied to improve the boundaries of the classified results. The results show the following: (1) Each type of satellite data has at least one variable selected as an important feature for both winter and summer crop type classification. Apart from the endmember variables, the other five extracted variable types are selected by the RF classifier for both winter and summer crop classifications. (2) SAR data can capture the key information of summer crops when optical data is limited, and the addition of SAR data can significantly improve the accuracy as to summer crop types. (3) The overall accuracy (OA) of both summer and winter crop type mapping exceeded 95%, with clear and relatively accurate cropland boundaries. Area evaluation showed a small bias in terms of the classified area of rapeseed, single-cropped rice, and double-cropped rice from statistical records. (4) Further visual examination of the spatial distribution showed a better performance of the classified crop types compared to three existing products. The results suggest that the proposed method has great potential in accurately mapping crop types in a complex subtropical planting environment.

Accurate and rapid crop type mapping is critical for agricultural sustainability. The growing trend of cloud-based geospatial platforms provides rapid processing tools and cloud storage for remote sensing data. In particular, a variety of remote sensing applications have made use of publicly accessible data from the Sentinel missions of the European Space Agency (ESA). However, few studies have employed these data to evaluate the effectiveness of Sentinel-1, and Sentinel-2 spectral bands and Machine Learning (ML) techniques in challenging highly heterogeneous and fragmented agricultural landscapes using the Google Earth Engine (GEE) cloud computing platform. This work aims to map, accurately and early, the crop types in a highly heterogeneous and fragmented agricultural region of the Tadla Irrigated Perimeter (TIP) as a case study using the high spatiotemporal resolution of Sentinel-1, Sentinel-2, and a Random Forest (RF) classifier implemented on GEE. More specifically, five experiments were performed to assess the optical band reflectance values, vegetation indices, and SAR backscattering coefficients on the accuracy of crop classification. Besides, two scenarios were used to assess the monthly temporal windows on classification accuracy. The findings of this study show that the fusion of Sentinel-1 and Sentinel-2 data can accurately produce the early crop mapping of the studied area with an Overall Accuracy (OA) reaching 95.02%. The scenarios prove that the monthly time series perform better in terms of classification accuracy than single monthly windows images. Red-edge and shortwave infrared bands can improve the accuracy of crop classification by 1.72% when compared to only using traditional bands (i.e., visible and near-infrared bands). The inclusion of two common vegetation indices (The Normalized Vegetation Index (NDVI), the Enhanced Vegetation Index (EVI)) and Sentinel-1 backscattering coefficients to the crop classification enhanced the overall classification accuracy by 0.02% and 2.94%, respectively, compared to using the Sentinel-2 reflectance bands alone. The monthly windows analysis indicated that the improvement in the accuracy of crop classification is the greatest when the March images are accessible, with an OA higher than 80%.

Sentinel SAR-optical fusion for crop type mapping using deep learning and Google Earth Engine

How much does multi-temporal Sentinel-2 data improve crop type classification?

Accurate crop type maps play an important role in food security due to their widespread applicability. Optical time series data (TSD) have proven to be significant for crop type mapping. However, filling in missing information due to clouds in optical imagery is always needed, which will increase the workload and the risk of error transmission, especially for imagery with high spatial resolution. The development of optical imagery with high temporal and spatial resolution and the emergence of deep learning algorithms provide solutions to this problem. Although the one-dimensional convolutional neural network (1D CNN), long short-term memory (LSTM), and gate recurrent unit (GRU) models have been used to classify crop types in previous studies, their ability to identify crop types using optical TSD with missing information needs to be further explored due to their different mechanisms for handling invalid values in TSD. In this research, we designed two groups of experiments to explore the performances and characteristics of the 1D CNN, LSTM, GRU, LSTM-CNN, and GRU-CNN models for crop type mapping using unfilled Sentinel-2 (Sentinel-2) TSD and to discover the differences between unfilled and filled Sentinel-2 TSD based on the same algorithm. A case study was conducted in Hengshui City, China, of which 70.3% is farmland. The results showed that the 1D CNN, LSTM-CNN, and GRU-CNN models achieved acceptable classification accuracies (above 85%) using unfilled TSD, even though the total missing rate of the sample values was 43.5%; these accuracies were higher and more stable than those obtained using filled TSD. Furthermore, the models recalled more samples on crop types with small parcels when using unfilled TSD. Although LSTM and GRU models did not attain accuracies as high as the other three models using unfilled TSD, their results were almost close to those with filled TSD. This research showed that crop types could be identified by deep learning features in Sentinel-2 dense time series images with missing information due to clouds or cloud shadows randomly, which avoided spending a lot of time on missing information reconstruction.

Multi-source remote sensing-derived information on crops contributes significantly to agricultural monitoring, assessment, and management. In Africa, some challenges (i.e., small-scale farming practices associated with diverse crop types and agricultural system complexity, and cloud coverage during the growing season) can imped agricultural monitoring using multi-source remote sensing. The combination of optical remote sensing and synthetic aperture radar (SAR) data has emerged as an opportune strategy for improving the precision and reliability of crop type mapping and monitoring. This work aims to conduct an extensive review of the challenges of agricultural monitoring and mapping in Africa in great detail as well as the current research progress of agricultural monitoring based on optical and Radar satellites. In this context optical data may provide high spatial resolution and detailed spectral information, which allows for the differentiation of different crop types based on their spectral signatures. However, synthetic aperture radar (SAR) satellites can provide important contributions given the ability of this technology to penetrate cloud cover, particularly in African tropical regions, as opposed to optical data. This review explores various combination techniques employed to integrate optical and SAR data for crop type classification and their applicability and limitations in the context of African countries. Furthermore, challenges are discussed in this review as well as and the limitations associated with optical and SAR data combination, such as the data availability, sensor compatibility, and the need for accurate ground truth data for model training and validation. This study also highlights the potential of advanced modelling (i.e., machine learning algorithms, such as support vector machines, random forests, and convolutional neural networks) in improving the accuracy and automation of crop type classification using combined data. Finally, this review concludes with future research directions and recommendations for utilizing optical and SAR data combination techniques in crop type classification for African agricultural systems. Furthermore, it emphasizes the importance of developing robust and scalable classification models that can accommodate the diversity of crop types, farming practices, and environmental conditions prevalent in Africa. Through the utilization of combined remote sensing technologies, informed decisions can be made to support sustainable agricultural practices, strengthen nutritional security, and contribute to the socioeconomic development of the continent.