Robust detection of headland boundary in paddy fields from continuous RGB-D images using hybrid deep neural networks

Abstract

Accurate and robust headland boundary detection in the field is crucial for formulating turning strategies for agro-machinery. The headland area is challenging to be detected because of its similar appearance to the farmland, coupled with complex field conditions such as high weed pressure and discontinuous headland edges. In this study, a machine vision-based method for headland boundary detection was proposed, which introduced depth information in addition to RGB images to improve its detection accuracy. A deep learning-based network combining convolutional neural network (CNN) and recurrent neural network (RNN) was constructed for headland semantic segmentation. An interactive attention module (IAM) was proposed to fuse complementary information in RGB-D images adaptively. A time series information processing module (TPM) composed of a set of bidirectional convolution long short-term memories (ConvLSTMs) was used to extract interrelated information from consecutive images. Image preprocessing techniques and a proposed distance-based boundary point clustering algorithm were applied to the headland segmentation mask to obtain the boundary line on the working side of the agro-machinery. Ablation study results confirmed the effectiveness of IAM and TPM in improving the headland segmentation performance. The network presented achieved excellent headland segmentation performance, with a mean intersection over union (mIoU) of up to 95.7%. The mean deviation of boundary line extraction on 256*144 resolution images was 3.57 pixels. The detection rate reached 25.8 frames per second (FPS), which could provide real-time reference lines for agro-machinery turning path planning.