A 3-Stage Machine Learning-Based Novel Object Grasping Methodology

Jacques Janse Van Vuuren,Ibrahim Al-Bahadly,Liqiong Tang,Khalid Mahmood Arif

doi:10.1109/access.2020.2987341

Jacques Janse Van Vuuren, Ibrahim Al-Bahadly + Show 2 more

Open Access

https://doi.org/10.1109/access.2020.2987341

Copy DOI

Abstract

The automatic grasping of objects previously unseen by a robotic system is a difficult task-of which there is currently no robust solution. The research presented in this article improves upon previous works that employ depth data and learning techniques to generate and select from a pool of hypothesised grasps by focusing on the pruning and selection process. In this work, a vision-based, sampling methodology that generates candidate grasps through a convolutional neural network is proposed. Each candidate grasp is assessed using scores derived from the candidate itself and other related input modalities-such as the centre of gravity of the object. The final selection is determined by a learning algorithm. To overcome human bias, objective measures of grasp performance are established that comprehensively measure the error introduced by the grasp trial itself. The proposed metrics are empirically demonstrated to quantify grasp quality, offer useful criteria for network training and provide better descriptive power than traditional measures of grasp outcome. Experimentation showed that the proposed methodology can generate a meaningful, final grasp within 1.3 seconds. Trials quantitatively demonstrate a small-object-in-isolation performance of 99%. For unknown objects, this equates to a 10% improvement relative to other similar methodologies. Testing also showed that grasp performance was improved by 5% when implementing the proposed metrics-compared to the baseline.

Highlights

Autonomous novel object grasping and handling is a wide-ranging, high-impact field with many implications, especially within domestic and industrial application
The relationship between these variables and a specific grasping strategy, robotic hardware and a gripper is not always clear. Research in this field has been active for decades, yielding a colourful range of promising avenues—especially with the recent interest from well-known and well-resourced research institutions, such as Google and the Massachusetts Institute of Technology (MIT)
This paper presents a grasping methodology based on machine learning that aims to improve on research related to novel object detection, grasping and handling with autonomous robotic systems—further closing the gap between manual and fully automated production

Summary

Introduction

Autonomous novel object grasping and handling is a wide-ranging, high-impact field with many implications, especially within domestic and industrial application. Some instances where automatic grasping has been studied include an automated checkout robot [1], garbage sorting [2], cloth manipulation [3], bed making [4], dishwasher unloading [5], automated cooking [6], [7], service robotics [8]–[11], general household-related grasping [12]–[14], clutter clearing [15], [16] and stowing, picking and packing for warehouse automation [17]–[20]—which has gained significant traction since the 2017 Amazon Robotics Challenge [21]. The relationship between these variables and a specific grasping strategy, robotic hardware and a gripper is not always clear. Research in this field has been active for decades, yielding a colourful range of promising avenues—especially with the recent interest from well-known and well-resourced research institutions, such as Google and the Massachusetts Institute of Technology (MIT).

Objectives

Methods

Results