A real-time table grape detection method based on improved YOLOv4-tiny network in complex background

Abstract

Accurate identification of table grapes is crucial to the harvesting process of the grape picking robot. This paper proposes an efficient grape detection model, YOLO-Grape, to solve the problem of unrecognition or decreased recognition accuracy caused by the complicated growth environment, shadows of branches and leaves, and overlapping grapes. To improve the network recognition accuracy, a down-sampling fusion structure is integrated into the network, and the Mish activation function is used. Meanwhile, an attention mechanism is added to the network, and the non-maximum suppression (NMS) function is replaced with the soft non-maximum suppression (Soft-NMS) function, thereby reducing the missing of predicted boxes due to overlapping grapes. Besides, the depthwise separable convolution is introduced to improve the detection speed of the network. In addition, transfer learning is used in the training process to improve the detection accuracy and generalization ability of the model. On the test data set of 700 grape images, the experimental results show that YOLO-Grape achieves an F1-score of 90.47%, a mAP of 91.08% and a detection speed of 81 frames per second. Compared with Faster-RCNN(Resnet50), SSD300, YOLOv4, and YOLOv4-tiny, the mAP of the YOLO-Grape model is increased by 1.67%, 2.28%, 0.84%, and 6.69%, respectively. The average recognition speeds of the YOLO-Grape model were 31.15, 3.38 and 6.45 times of Faster-RCNN(Resnet50), SSD300, and YOLOv4 respectively. Through four sets of comparative experiments, it is found that the proposed YOLO-Grape model achieves high recognition accuracy for occluded grapes, meeting the requirements of grape picking robots for real-time detection of multiple varieties of table grapes in complex situations. • Attention mechanism and soft-NMS function added to YOLO-grape network. • The YOLO-grape model is able to accurately identify six varieties of grapes. • YOLO-grape identifies obscured grapes with a mAP of 89.93%. • The detection speed of the YOLO-Grape model is 81 frames per second (1050Ti).