Identifying sunflower lodging based on image fusion and deep semantic segmentation with UAV remote sensing imaging

Abstract

Sunflower lodging is a common agricultural disorder taking place in the middle and late sunflower growth periods. This disorder reduces the sunflower seed yield, damages the seed quality, and hence usually causes great losses in both crop quantity and quality. Sunflower lodging is mainly caused by extreme and destructive weather events, which have been recently occurring more frequently. This is why it is highly crucial to develop methods for fast and accurate identification of sunflower lodging. In this work, an efficient method for sunflower lodging identification is proposed based on image fusion and deep semantic segmentation of remote sensing images obtained from an unmanned aerial vehicle (UAV). First, the resolution of low-resolution multispectral images was enhanced through matching their features with those of high-resolution visible-range images. Then, for effective lodging assessment, high-quality multispectral images with rich spectral information and high spatial resolution were obtained through fusing the visible-range images and the enhanced multispectral ones. Subsequently, in order to refine the identification outcomes, a variant of the segmentation network (SegNet) deep architecture was developed for semantic segmentation. This variant has skip connections, separable convolution, and a conditional random field. Experimental evaluation shows that the fusion-based approaches clearly outperform the no-fusion ones in terms of the lodging identification accuracy for all compared architectures including support vector machine (SVM), fully convolutional network (FCN), SegNet, and the proposed SegNet variant. Meanwhile, the deep semantic segmentation methods consistently outperform the classical SVM one with hand-crafted features. As well, the improved SegNet method outperformed all of the compared methods and achieved the best accuracies of 84.4% and 89.8% without and with image fusion, respectively, on one test. The corresponding accuracies on another test set were 76.6% and 83.3%, respectively. Moreover, the proposed method can also identify the sunflower lodging and non-lodging patterns and separate them from the background. These capabilities are highly beneficial for lodging hazard assessment and sunflower harvest survey. Overall, the proposed method effectively exploited UAV remote sensing image data with fusion and deep semantic segmentation modules in order to provide a useful reference for sunflower lodging assessment and mapping.