Remote estimation of grafted apple tree trunk diameter in modern orchard with RGB and point cloud based on SOLOv2

Abstract

Apple tree phenotyping can reflect individual development of single apple tree, which mainly involves tree height, crown width, and diameter of apple tree trunk (DATT). This study aimed to estimate diameter of grafted apple tree trunk, whose target position of diameter estimation is about 10 cm above grafting position. An estimated DATT approach of combining red–greenblue-depth (RGB-D) sensor with SOLOv2 was proposed. Firstly, Kinect V2 was employed to obtain original RGB images and point clouds of the grafted apple trees simultaneously. There were 120 and 60 RGB images and corresponding point clouds randomly collected from two modern apple orchards. Secondly, SOLOv2 deep learning model was selected and trained to instance segment grafting position from RGB image for determining it automatically. Then, corresponding exact position of the grafting position in point cloud was mapped by coordinate transformation of its pixel coordinates, which was obtained by trained SOLOv2 model. Finally, DATT was estimated by calculating the difference between maximum and minimum Y coordinates of points selected by distance thresholds in X, Y, and Z directions near the target position, which were 0.10 m, 0.035 m, and 0.20 m, respectively. Results showed that average precision and average recall of the trained SOLOv2 model for instant segmenting the grafting position were 0.811 and 0.830, respectively. Mean absolute error, mean absolute percentage error, and root mean square error of the proposed method were 3.01 mm, 5.86%, and 3.79 mm, respectively. It illustrates that the proposed method can estimate DATT and thus contribute to automatic apple tree phenotyping.