Real-time, highly accurate robotic grasp detection utilizing transfer learning for robots manipulating fragile fruits with widely variable sizes and shapes

Abstract

The automatic picking, sorting, and packaging of fruits require robots to accurately detect the grasping position of fruits. However, accurate detection of grasping positions is challenging due to the diversity of fruit shapes and sizes. At present, research objects of grasping detection are mainly daily necessities and office items, and few studies on fruit-grasping detection are available. To solve these problems, four end-to-end detection models were designed based on three convolutional neural network architectures: Xception, MobileNetV3, and DenseNet. In addition, considering the large amount of data required for deep learning, data augmentation and transfer learning techniques were applied to improve model accuracy and generalization performance. The most widely applied evaluation criteria were used to evaluate the models, and the accuracy of the four models ranged within 83.86%–93.64%. All the models were capable of rapid real-time detection. To verify the robustness, the models were tested under different evaluation thresholds, and the results showed that the models performed well under higher evaluation criteria. Additionally, a dataset containing 4400 images of 11 common fruits was established due to the current lack of data for fruit grasp detection.