TDPPL-Net: A Lightweight Real-Time Tomato Detection and Picking Point Localization Model for Harvesting Robots

Abstract

Existing target detection models are large and have multiple network parameters, which can severely slow down the detection speed when deployed on small, low-cost GPU-free Industrial Personal Computers (IPC). As a result, this study proposes a lightweight real-time tomato detection and point-picking integrated network model based on YOLOv5 (TDPPL-Net). Firstly, the algorithm replaces the YOLOv5 backbone with a four-group lightweight downsampling model consisting of Ghost Conv and Ghost Bottleneck to reduce the model size, while adding the attention mechanism SimAM module to improve detection accuracy after each scale’s feature map. Secondly, the Spatial Pyramid Pooling-Fast(SPPF) network structure is used and the convolutional layers in the (Feature Pyramid Network and Path Aggregation Network)FPN+PAN structure are replaced with a depth-separable convolution to reduce the computational effort. Finally, the center of the bounding box is used as the picking point, and the corresponding depth information is obtained in combination with the Intel RealSense D435 camera, which is converted into 3D coordinates under the robot arm coordinate system after hand-eye calibration. The experimental results show that TDPPL-Net reduces the number of parameters by 59.84% compared with the original YOLOv5, the model volume is only 40% of the original, the mAP is 93.36, and the real-time detection speed on the IPC without GPU acceleration is 31.41 FPS, which is 170.31% higher than YOLOv5. The TDPPL-Net increases detection speed on low-performance equipment without compromising detection accuracy. It can detect and locate tomato picking points in real-time in the complex natural environment, which can meet the working requirements of harvesting robots.