Phenotyping of individual apple tree in modern orchard with novel smartphone-based heterogeneous binocular vision and YOLOv5s

Abstract

Phenotyping plays a significant role in the breeding of apple tree. However, existing researches mainly relied on instruments, such as LiDAR, RGB-D camera or UAV (unmanned aerial vehicle) embedded with depth sensor, etc., which requires additional costs for users and also inconvenient. Therefore, a novel method of smartphone-based heterogeneous binocular vision was developed to fulfill low-cost automated phenotyping for apple tree. In this study, a pair of cameras on multi-camera smartphone was selected to obtain heterogeneous binocular camera. After that, a so-called virtual focal method was developed to generate standard binocular images from heterogeneous binocular images of smartphone. A well-known YOLOv5s object detection model was trained on a four-class dataset to detect fruits, grafts, trunks and whole trees. Then, the model was simplified to fit the deployment on smartphone. Finally, five phenotypes (trunk diameter, ground diameter, tree height, fruit vertical diameter, and fruit horizontal diameter) of individual apple tree were obtained by pinhole camera model and standard binocular vision. After evaluation of phenotyping manually and by smartphone, our method shows MAPE (mean average percentage error) ranging from 6.00 % to 13.73 % for the five phenotypes. Compared with the existing studies, our method has reached a close or even better phenotyping accuracy with only a smartphone. As more and more smartphones have multi-camera, our method is probably the lowest cost phenotyping method for most of the potential users. Results indicated that the approach could be utilized to phenotyping of apple tree.