Reinforced deep learning approach for analyzing spaceborne-derived crop phenology

Abstract

Vegetation Index (VI) curves capture the crop-specific phenological events from multi-temporal Earth observation images. Although advanced machine learning classifiers are the existing state-of-the-art, most of them assume the samples to be independent and identically distributed, which is not true for VI curve classification. This research proposed the consideration of spatial non-stationarity for VI curve-based crop classification. The proposed approach does not assume the samples to be independent and identically distributed. Different state-of-the-art approaches attempted to solve this problem by introducing architectural variations and considering the neighborhood patches. However, the inherent nature of the problem demanded an online approach. In the current research, this problem is formulated in a reinforcement learning framework where the generalizability of the predictor is improved using a feedback system. Different approaches for feedback were investigated in this study. The predictor was composed of a graph convolutional framework where the VI curves were transformed into a graph representation, incorporating the spatial, spectral, and temporal context. Graph convolutional operations were then used to learn the embedded representations and assign labels to the unlabeled points based on the labeled ones in the proximity. Further, a semi-supervised strategy was proposed where some initial feedback was employed to improve the generalizability of the model. Additionally, neighborhood-based feedback was also considered. The experiments using the VI curves collected over three farms in Israel covering wheat, potato, and barley crops illustrated the need for accounting the spatial non-stationarity in VI curve classification. The proposed dynamic feedback system and the graph-based strategy to consider spatial heterogeneity significantly improved the accuracy of the proposed framework. Additionally, the proposed reinforcement learning strategy ensures the dynamic refinement of model weights to facilitate the real-time applications even when the target data varies significantly. Although the proposed approach is generic, the end-to-end framework, as adopted in this research, is suitable for aerial or agricultural datasets. Hence, further fine tuning and remodeling may be required for non-agriculture or non-spatial datasets. However, the dynamic weight update may be adopted for any applications irrespective of the datasets.