Automatic branch detection of jujube trees based on 3D reconstruction for dormant pruning using the deep learning-based method

Abstract

Pruning is a time-consuming and labor-intensive practice for managing of dormant jujube orchards, in which dormant pruning is still mainly dependent on manual operation. Automated pruning using a robotic platform could be a better solution to overcome the skilled labor shortage and increased labor costs. With the development of dwarf and dense planting of jujube trees, the tree architectures are suitable for robotic pruning. This study concentrated on the detection and identification of pruning branches, which was a critical step for automating pruning operations. The method in this study mainly consisted of three steps: first, present a vision system based on two synchronous consumer-level RGB-D cameras to obtain the high-quality 3D point cloud in filed; second, propose the reconstruction pipeline for the desired three-dimensional model (basically a complete 3D model) using only two perspectives; and third, automatically segment the trunks and branches based on deep learning method (SPGNet), and then apply the DBSCAN clustering algorithm for estimating branch counts of jujube trees. For the reconstruction of jujube tree, registration errors II (0°-180°) were greater than errors I (0°-55°), and registration errors I (0°-55°) were less than 1 mm under different light conditions. Experimental results demonstrated that trunks and branches were segmented successfully with class accuracies of 0.93 and 0.84, and another metric intersection-over-union (IoU) was 0.85 and 0.76, respectively; the coefficient of determination (R2) values analyzed between the ground-truth and cluster results were 0.83, 0.88, and 0.89 in sunny, cloudy, and night conditions, respectively. These results showed that the proposed method for detecting branches could be utilized to generate substantial information, such as the diameter length and diameter of the branch, which was critical for the automated dormant pruning of jujube trees in the future orchard field.