Deep feature learning and latent space encoding for crop phenology analysis

Abstract

The high spatial, spectral, and temporal resolutions of the Vegetation and Environment monitoring New Micro-Satellite (VENµS) satellite data facilitate field-level phenological analysis of crops. This study proposes deep learning (DL) based approaches to resolve the issues prevalent in crop phenology-based fingerprint estimation at field-level using VENµS satellite data. An encoder-decoder-based framework, called piece-wise kernel encoding network (PKNet), is proposed for missing data imputation of the vegetation index (VI) curves derived from time-series image data. PKNet adopts interpolation-based convolution, dynamic time wrapping (DTW) based layer formulation, and imputation-specific constraints for optimal smoothing of the irregularly sampled VI curves. Besides, PKNet learns kernel parameters dynamically. A variational encoding framework called a dynamic-projection-based generalization network (DPGNet), is proposed to generalize the pixel-level VI curves to synthesize a representative VI curve for a given field. DPGNet is more effective than the use of multiple moments as it is resilient to outliers and learns normally distributed latent space with a small number of samples. The current research also proposes a classifier, called dynamic time wrapping based capsule network (DTCapsNet), which learns a discriminative latent space and accurately models the VI curve features. The DTCapsNet considers the time-series nature of the input using DTW-based convolution layers. The feature characterization improves generalizability and gives good results, even with a limited number of training samples. Experiments using the ground truth information and satellite images, acquired over two farms in Israel, illustrate that the proposed frameworks give better results than the commonly-used existing approaches.