Generating quality grasp rectangle using Pix2Pix GAN for intelligent robot grasping

Abstract

Intelligent robot grasping is a very challenging task due to its inherent complexity and non-availability of sufficient labeled data. Since making suitable labeled data available for effective training for any deep learning-based model including deep reinforcement learning is so crucial for successful grasp learning, in this paper we propose to solve the problem of generating grasping poses/rectangles using a Pix2Pix Generative Adversarial Network (Pix2Pix GAN), which takes an image of an object as input and produces the grasping rectangle tagged with the object as output. Here, we have proposed an end-to-end grasping rectangle generating methodology and embedding it to an appropriate place of an object to be grasped. We have developed two modules to obtain an optimal grasping rectangle. With the help of the first module, the pose (position and orientation) of the generated grasping rectangle is extracted from the output of Pix2Pix GAN, and then, the extracted grasp pose is translated to the centroid of the object, since here we hypothesize that like the human way of grasping of regular shaped objects, the center of mass and centroids are the best places for stable grasping. For other irregular shaped objects, we allow the generated grasping rectangles as it is to be fed to the robot for grasp execution. The accuracy has significantly improved for generating the grasping rectangle with limited number of Cornell Grasping Dataset augmented by our proposed approach to the extent of 87.79%. Rigorous experiments with the Anukul/Baxter robot, which has 7 degrees of freedom, causing redundancy have been performed. At the grasp execution level, we propose to solve the inverse kinematics problems for such robots using Numerical Inverse-Pose solution together with Resolve-Rate control which proves to be more computationally efficient due to the common sharing of the Jacobian matrix. Experiments show that our proposed generative model-based approach gives the promising results in terms of executing successful grasps for seen as well as unseen objects (refer to demonstrative video).