3D pose estimation of green pepper fruit for automated harvesting

Abstract

This paper presents a novel pose estimation algorithm for stem position detection in Japanese green pepper automatic harvesting. When the available visual cues do not provide sufficient information to the harvesting robot, information about the pose of the fruit in space is necessary for accurate stem position detection. In the proposed method the orientation of a fruit in space is obtained by fitting a model to surface points of the fruit. These surface points are acquired using a Lidar type laser range finder, and the point matching is performed using a coherent point drift algorithm with two model transformation methods, rigid and affine. The performance of the proposed method was evaluated both under laboratory conditions and in a greenhouse. In the laboratory test, the mean total error for the affine transformation was less than 25mm in 42 of 49 positions, less than 20mm in 28 of 49 positions and less than 15mm in 19 of 49 positions. For the rigid transformation, the same error was less than 25mm in 39 of 49 positions, less than 20mm in 31 of 49 positions and less than 15mm in 11 of 49 positions. The total error of the affine transformation was found to be proportional to the inclination angle, as the mean error was 11mm, 15mm, and 23mm for inclination angles of 15, 30 and 45 degrees, respectively. No relationship was found between the mean total error and the inclination angle for the rigid transformation, as the calculated mean total error was 20mm, 18mm, and 20mm for inclination angles of 15, 30 and 45 degrees, respectively. In the greenhouse test, the stem was calculated to be within the cutting range for 81 of 107 instances for affine transformation and for 66 of 107 for rigid transformation. These results suggest that the proposed method is suitable for stem position detection in the automatic harvesting of green pepper, and could be adjusted for use with other fruits and vegetables.