Vision-guided Robot System for Picking Objects by Casting Shadows

Abstract

We present a complete vision-guided robot system for model-based three-dimensional (3D) pose estimation and picking of singulated 3D objects. Our system employs a novel vision sensor consisting of a video camera surrounded by eight flashes (light emitting diodes). By capturing images under different flashes and observing the shadows, depth edges or silhouettes in the scene are obtained. The silhouettes are segmented into different objects and each silhouette is matched across a database of object silhouettes in different poses to find the coarse 3D pose. The database is pre-computed using a computer-aided design (CAD) model of the object. The pose is refined using a fully projective formulation of Lowe’s model-based pose estimation algorithm. The estimated pose is transferred to a robot coordinate system utilizing the hand—eye and camera calibration parameters, which allows the robot to pick the object. Our system outperforms conventional systems using two-dimensional sensors with intensity-based features as well as 3D sensors. We handle complex ambient illumination conditions, challenging specular backgrounds, diffuse as well as specular objects, and texture-less objects, on which traditional systems usually fail. Our vision sensor is capable of computing depth edges in real time and is low cost. Our approach is simple and fast for practical implementation. We present real experimental results using our custom designed sensor mounted on a robot arm to demonstrate the effectiveness of our technique.