Multi-class detection of kiwifruit flower and its distribution identification in orchard based on YOLOv5l and Euclidean distance

Abstract

Asynchrony of kiwifruit flowering time results in different flower phenological stages in canopy at the same time. Pollination quality of flowers is influenced by their phenological stages, while their distributions determine fruit distributions and influence kiwifruit quality and yield. Thus, it’s necessary to find suitable flowers to be pollinated based on flower phenology and its distribution. However, influences of flower phenology and flower distribution were not considered in most previous studies about robotic pollination of kiwifruit, where pollination of all open flowers was indiscriminate. Therefore, a method was proposed for multi-class detection of kiwifruit flower and its distribution identification in orchard, which was based on You Only Look Once version 5 large (YOLOv5l) and Euclidean distance. According to kiwifruit flower phenology, kiwifruit flowers were classified into 10 classes to find suitable flowers for pollination, while flower cluster and branch junction were divided into 4 classes for obtaining flower distributions. All classes were manually labeled by rectangular bounding boxes for training and testing. Considering high detection accuracy requirements with small model size, YOLOv5l was applied to do transfer learning for multi-class detection of kiwifruit flower. Then, pixels coordinate of multi-class objects and their corresponding Euclidean distances could be gained. Finally, flower distributions in canopy were obtained by matching method. Total mean Average Precision (mAP) was 91.60 % in YOLOv5l, while the mAP of multi-class flower (10 classes) was 93.23 %, which was 5.70 % higher than that of the other 4 classes. Matching accuracy (MA) of flowers matching flower clusters was up to 97.60 %. Moreover, MA of flower cluster matching branch junction (96.20 %) and total MA (93.30 %) increased by 1.20 % and 1.00 % based on improved matching method, respectively. Total processing speed of multi-class flower detection and its distribution identification was 112.46 ms per image including 15.50 ms for image detection by YOLOv5l. Results showed that multi-class kiwifruit flowers and relative flower distributions could be fast and accurately obtained for further selecting suitable flowers for robotic pollination.