YOLO5-spear: A robust and real-time spear tips locator by improving image augmentation and lightweight network for selective harvesting robot of white asparagus

Abstract

The characteristics of underground cultivation have posed a challenge to the development of white asparagus selective harvesting robot. Because the ridge surface is mixed with soil particles and has a complex background including variable soil moisture and illumination, detecting and locating spear tips rapidly and accurately is a key difficulty. To address this problem, image augmentation is applied firstly to extract spear tip patches from the harvesting area image in actual scenarios to form a multi-scale combined image, and then processed with proposed resampling-based image transformation, such as illumination, rotation, mirror, motion blur, and shadow. Secondly, a model referred to as YOLO5-Spear is proposed to detect spear tips by replacing C3 and Conv of YOLO5 with LC3 (Light C3) and DWConv (Depthwise-separable Convolution) and by adding the SE (Squeeze-and-Excitation) module to improve both the detection speed and accuracy. Finally, the model is deployed on embedded devices as a spear tips locator for a selective harvesting robot. The results showed that YOLO5-Spear achieved 97.8% at AP0.5, 2.4% higher than YOLO5. Moreover, its parameters, computation, model size, and detection time were reduced by 51.3%, 33.7%, 50.3%, and 18.2%, respectively. Further, the average inference time on Jetson Nano decreased to 63 ms, which meets the requirement for real-time performance of robotic harvesting. Compared with YOLO4-scaled, YOLO5-Spear increased accuracy by a maximum of 31.4%, was nearly 5 times faster, and reduced the model size by 94.9%. Localisation accuracy in different scenarios offers directions to optimise robot design and planting patterns to reduce the complexity.