Using color and 3D geometry features to segment fruit point cloud and improve fruit recognition accuracy

Abstract

Fruit image segmentation is an essential step to distinguish fruits from the background. In order to improve the fruits recognition accuracy for harvesting robots in three-dimensional (3D) space, a method with the fusion of color and 3D geometry features for fruit point cloud segmentation was proposed in this study. The local descriptor was applied to obtain the candidate regions, and the global descriptor was used to obtain the final segmented results. Firstly, the hue, saturation, value (HSV) color features and normal orientation features of pixels were fused to obtain the preliminary segmentation results. Then, the pre-processed color image and depth image were converted to a point cloud, and it was clustered into multiple regions by the Euclidean clustering algorithm. Finally, we utilized the viewpoint feature histogram (VFH) of each point cloud cluster to remove the remaining non-fruit regions. The experiments showed that the segmentation accuracy of the proposed method was 98.99%, and the precision was 80.09%, which are both superior to the traditional color segmentation methods. In addition, a fruit detection method based on shape analysis showed that it is more effective in improving fruit recognition rate and reducing false detection rate than the color segmentation methods.