Pose estimation and adaptable grasp configuration with point cloud registration and geometry understanding for fruit grasp planning

Abstract

Picking and placing fruits is a fundamental capability and common operation for agricultural robots in harvesting, sorting and packing tasks. Secure grasping fruits without potential slip and damage is challenging due to the shape varying, bruise sensitizing, as well as hardness changing during fruit maturation. In this study, a state-of-art point cloud processing based pre-grasp planning method was proposed to complete accurate 6D pose estimation and adaptable grasp configuration with point cloud registration and geometry understanding algorithm for grasp planning. Fruits 6D pose was estimated with SAC-IA ICP algorithm by matching Kinect-based online target point cloud obtained from 3D reconstruction with laser scanner based offline template. Simultaneously, fruits geometry understanding is completed with RANSAC algorithm, and the shape of typical fruits were classified into sphere, cylinder and cone. To pre-define grasp configuration for all kinds of shapes, fruits mesh models are also created with greedy projection algorithm for rapid collision detection and contact determination in GraspIt. Finally, the grasp information can be retrieved and transformed to match the position and orientation of the object by combining the results of 6D pose and geometry understanding. Compared with deep learning based pose estimation algorithm, large training data set for each novel, unknown object is not required in our method. The performance of the proposed system and algorithm has been evaluated on orange, cucumber, pineapple, kiwifruit and other agricultural products. The result indicates that our method is reliable with high efficiency and accuracy, and the 6D pose estimation along with adaptable grasp configuration results can provide a theoretical basis for grasp planning.