Navigation path recognition between rows of fruit trees based on semantic segmentation

Abstract

The navigation path recognition has been recognized as one of the most important subtasks of intelligent agricultural equipment in orchard operations. However, there are still some challenges in recognizing navigation paths between rows of fruit trees, including the accuracy, real-time performance, generalization of deep learning models. The Fast-Unet model was proposed by pruning and optimization based on Unet for recognizing navigation paths between rows of fruit trees, which inherited encoding–decoding structure and multi-layer feature sensing capability. The number of convolutional kernels used to extract features in the Fast-Unet was reduced to one-fourth of that in Unet to improve inference speed. To address the blurring of the boundary of the recognized object due to the reduction in the number of convolutional kernels, the atrous spatial pyramid pooling (ASPP) module was used in the encoding part to extract the multiscale information to improve the recognition accuracy. The navigation path edges determined by Fast-Unet and Canny operators, navigation lines and yaw angles were generated by the least square method.. In this study, the Fast-Unet model was first trained on the peach dataset, and then the trained model was transferred to the small dataset of oranges and kiwifruits for navigation path recognition to verify the generalization. The Mean Intersection over Union (MIOU) of the Fast-Unet for peaches, oranges and kiwifruits navigation path extraction accuracy were 0.977, 0.987 and 0.956, respectively. The mean difference between the predicted yaw angle of peaches, oranges and kiwifruits navigation paths and the labelled were 0.397°, 0.102° and 0.239°, respectively. In terms of real-time performance, the inference speed was 48.8 frames per second (FPS) to process the RGB image data on a single-core CPU. The inference speed of the Fast-Unet model was 1.59 times higher than that of Unet. Through transfer learning, the Fast-Unet model can realize real-time recognition of navigation paths for peaches, oranges and kiwifruits. These results can provide technical and theoretical support to the development of navigation equipment and visual slip prediction for intelligent agricultural machinery in the orchard.