A Novel Camera Fusion Method Based on Switching Scheme and Occlusion-Aware Object Detection for Real-Time Robotic Grasping

Abstract

Real-time vision-based robotic grasping is challenging in clutter. In such scene, the target object should be perceived accurately, where it may be occluded and misrecognized by many distractors including irrelevant objects and the robotic arm. In addition, the limited field of view (FOV) of camera makes it prone for objects to get out of the camera view. We develop a novel camera fusion method of pose estimation based on switching scheme for real-time robotic grasping under hybrid eye-in-hand (EIH)/eye-to-hand (ETH) configurations. The objects are locked based on occlusion-aware object detection to apply switching function for single pose estimation or multiple vision fusion. This method improves the accuracy of pose estimation and robustness of dynamic grasping under occlusion. Experimental results on pose estimation and real-time robotic grasping in clutter verify the effectiveness of the proposed method.