Branch Identification and Junction Points Location for Apple Trees Based on Deep Learning

Abstract

Branch identification is key to the robotic pruning system for apple trees. High identification accuracy and the positioning of junction points between branch and trunk are important prerequisites for pruning with a robotic arm. Recently, with the development of deep learning, Transformer has been gradually applied to the field of computer vision and achieved good results. However, the effect of branch identification based on Transformer has not been verified so far. Taking Swin-T and Resnet50 as a backbone, this study detected and segmented the trunk, primary branch and support of apple trees on the basis of Mask R-CNN and Cascade Mask R-CNN. The results show that, when Intersection over Union (IoU) is 0.5, the bbox mAP and segm mAP of Cascade Mask R-CNN Swin-T are the highest, which are 0.943 and 0.940; as for the each category identification, Cascade Mask R-CNN Swin-T shows no significant difference with the other three algorithms in trunk and primary branch; when the identified object is a support, the bbox AP and segm AP of Cascade Mask R-CNN Swin-T is significantly higher than that of other algorithms, which are 0.879 and 0.893. Next, Cascade Mask R-CNN SW-T is combined with Zhang & Suen to obtain the junction point. Compared with the direct application of Zhang & Suen algorithm, the skeleton obtained by this method is advantaged by trunk diameter information, and its shape and junction points position are closer to the actual apple trees. This model and method can be applied to follow-up research and offer a new solution to the robotic pruning system for apple trees.