Location and model reconstruction algorithm for overlapped and sheltered spherical fruits based on geometry

Abstract

For fruit picking robot, it is an essential prerequisite for achieving fruit picking using machine vision technology to accurately identify the fruits growing in the natural environment. This article presents a vision system of fruit picking robot to perform fruit location and three-dimensional model reconstruction. Firstly, combining the features of color and shape of fruit to reconstruct the actual contour of overlapped and sheltered fruits. Secondly, the least square method was used to reconstruct the three-dimensional model of each fruit according to the spatial coordinates corresponding to image location. Finally, fruit picking experiments in the laboratory environment are used to verify the feasibility of the proposed vision system. Three parameters including Segmentation Error, Intersection Over Union, and False Negative Rate are used to evaluate the performance of the algorithm. The average Segmentation Error, Intersection Over Union, and False Negative Rate of the fruit location algorithm based on geometry were 6.36%, 87.9%, and 6.72%, respectively. The experimental results showed that the average computation time of the algorithm is 3.2 s and the reconstructed three-dimensional model matched the size and position of fruits in the actual scene. The research results can be applied to the vision system of fruit picking robot.