Three-dimensional location methods for the vision system of strawberry-harvesting robots: development and comparison

Abstract

For most fruit-harvesting robots, an essential task of the machine vision system is to provide the manipulator with an accurate three-dimensional location of the target. However, the accuracy of this location can be affected by various factors. This study aimed to develop seven location methods, to investigate their effectiveness, as well as the influences of different camera modes and camera types, and, ultimately, to ascertain which was the optimal method. These methods utilized the pixels of the detected targets in each image, the corresponding depth values, as well as the locations of the 2D bounding boxes extracted from the detection results. These location methods differed in the way that they obtained the position of the 3D bounding box, and in their use of point clustering or colour thresholding. The images were collected via two types of 3D camera, patterned structured light and time-of-flight. Comparative analysis showed that methods using the 2D bounding box and the selected depth value to calculate the 3D bounding box were faster (0.2–8.4 ms compared to 151.9–325.2 ms) and performed better than the 3D clustering methods. In addition, four modes of the structured light camera were tested and compared. The results showed that the high-accuracy mode had fewer noise points but a lower location rate (89.2–89.9%), while the high-density mode created more noise points but a higher location rate (98.9%). Evaluations also indicated that the data from the time-of-flight camera better represented the 3D shape (26.3% more accurate along the camera’s depth direction). Therefore, time-of-flight camera was considered better for the applications that required more accurate 3D shape. This paper, thus, provided references in the selection of location methods, cameras and corresponding modes for related work.