Precise grabbing of overlapping objects system based on end-to-end deep neural network

Abstract

In recent years, robotic arm technology is in dire need of reform because of the remarkable advances in artificial intelligence and computer vision. The traditional robotic arm techniques, e.g., template matching algorithm and iterative closest point algorithm, suffer from the low precision issue, especially when the target objects overlap with each other, resulting in inaccurate estimation of overlapping objects. This paper proposes a precise grabbing of overlapping objects system based on an end-to-end deep neural network. The successful grabbing is realized in the case of overlapping objects. First, the datasets needed for network training were established, utilizing structured light to obtain the point cloud information of the arbitrarily placed target objects. Furthermore, we collect the corresponding postures as data labels via the teaching device of the robotic arm, and train the network models using the datasets and labels. Finally, we can predict the postures of the target objects in real time and transmit the results to a robotic arm to complete the grabbing work. The experiment results indicate that the proposed grabbing system can grab small irregular objects accurately, only using the point cloud information, estimating the posture of multiple target objects in the scene simultaneously, and estimating the posture of overlapping small objects in the scene.