Design, integration, and evaluation of a robotic peach packaging system based on deep learning

Abstract

Fruit packaging is one of the most time-consuming and labor-intensive tasks during postharvest commercialization. With the aging of the global population, it is necessary to apply robot to replace the manual manipulation. However, robotic packaging for fragile fruit is more complicated and difficult, compared with other postharvest processes, such as quality detection and grading. In this study, for better positioning accuracy and grasping robustness, we developed a prototype for peach packaging robot based on deep learning. First, the dataset for peach object detection was built, and YOLO v5 models with different width and depth were trained in an end-to-end manner on this dataset. Considering the requirements on both accuracy and real-time performance in fruit postharvest processing, YOLO v5 - S was finally adopted as the peach detection model for robotic manipulation, which can achieve mAP-50 = 0.996 on validation set and runs at 142.86 fps on RTX 3060. Next, the “Eye-on-Base” hand eye calibration method was used to solve the coordinate transformation matrix from the camera coordinate system to the robot base coordinate system. The results on landmark positioning experiment showed that the positioning error along X and Y axes was 4.87 mm and 5.00 mm on average, respectively. The positioning error alone Z direction was 18.47 mm on average, which was caused mainly by the depth perception error of the RGB-D camera. In the grasping experiment, the influence of the depth perception accuracy was explored, the grasping success rate for small, medium, and large size of peaches was 100 %, 97 %, and 97 %, respectively. Besides, the entire pipeline proposed in this study took 252.81 ms on average for depth perception, object detection, coordinate transformation, and grasping path planning. Finally, the early stage bruise of the peaches was also evaluated through SFDI technology. In general, this research provided a feasible and reliable scheme for the fruit packaging robot, which is potential to be deployed in postharvest commercialization.