Precise geometry and pose measurement of in-hand objects with simple features using a multi-camera system

Abstract

In precision pick-and-place tasks, robotic grasp errors can cause the failure of subsequent placement steps. Therefore, in-hand object pose measurement has attracted the attention of the intelligent manufacturing industry. However, existing algorithms have limited accuracy and insufficient practicality. Moreover, they often assume that the actual object geometry is identical to the CAD model. To alleviate the problems in terms of accuracy, practicality, and uncritical assumptions, this paper proposes an in-hand measurement solution based on multi-view reconstruction for objects with simple features. In this concept, multiple monocular cameras are deployed at the robot end to calculate the object’s actual geometry and 6D pose by reconstructing its key geometric features. An in-hand pose compensation strategy and a robot terminal system integrated with manipulation and measurement are developed. To verify the solution’s effectiveness, several experiments were carried out. First, grasp errors caused by multiple sources are intuitively demonstrated in a simulation environment. The in-hand measurement is validated to compensate for grasp errors through comparison experiments. Then, robots can rearrange various irregularly piled objects in real-world experiments based on the developed terminal system. Results demonstrate that maximum remaining position errors are within 2 mm with maximum angle errors of less than 1°. The maximum error compensation rate reaches 95%, which is superior to existing in-hand measurement algorithms. It can be observed that the proposed solution is valid for multiple objects. Geometry measurement can help robots calculate the in-hand object pose and identify the object’s category and availability. Therefore, the proposed measurement solution seems accurate and universal for robot applications in smart manufacturing industries, enhancing the accuracy and flexibility of robot manipulation.