Instance Segmentation of Lotus Pods and Stalks in Unstructured Planting Environment Based on Improved YOLOv5

Abstract

Accurate segmentation of lotus pods and stalks with pose variability is a prerequisite for realizing the robotic harvesting of lotus pods. However, the complex growth environment of lotus pods causes great difficulties in conducting the above task. In this study, an instance segmentation model, LPSS-YOLOv5, for lotus pods and stalks based on the latest YOLOv5 v7.0 instance segmentation model was proposed. The CBAM attention mechanism was integrated into the network to improve the model’s feature extraction ability. The scale distribution of the multi-scale feature layer was adjusted, a 160 × 160 small-scale detection layer was added, and the original 20 × 20 large-scale detection layer was removed, which improved the model’s segmentation accuracy for small-scale lotus stalks and reduced the model size. On the medium-large scale test set, LPSS-YOLOv5 achieved a mask mAP0.5 of 99.3% for all classes. On the small-scale test set, the mAP0.5 for all classes and AP0.5 for stalks were 88.8% and 83.3%, which were 2.6% and 5.0% higher than the baseline, respectively. Compared with the mainstream Mask R-CNN and YOLACT models, LPSS-YOLOv5 showed a much higher segmentation accuracy, speed, and smaller size. The 2D and 3D localization tests verified that LPSS-YOLOv5 could effectively support the picking point localization and the pod–stalk affiliation confirmation.