Recognition and Localization of Sacks for Autonomous Container Unloading by Fitting Superquadrics in Noisy, Partial Views from a Low-cost RGBD Sensor

Abstract

A significant amount of cargo worldwide is transported in sacks and bags e.g. wheat, rice, coffee and cacao beans, etc. Despite being very strenuous and the health risks involved, the handling of sacks in logistics is predominantly done through manual labor. Hence, the automation of tasks such as cargo unloading from shipping containers is of high importance. However, it faces many challenges due to the unstructured nature of packaging. One of the prerequisites for creating autonomous systems for handling bags or sacks is a robust perception component. In this work, we present a perception pipeline to recognize and localize sacks with a low-cost sensor in unstructured settings with partial views. The backbone of our perception strategy is based on two main contributions presented in this work. First, we introduce a fast convexity test between neighboring patches, which is a part of a two-level segmentation leading to a robust detection of object candidates. Second, we formulate a numerically stable form of superquadric fitting, which allows for an extension of the feasible region of the corresponding optimization problem. Both of the contributions are of interest for applications using superquadrics for representing curved object parts and hence extend beyond the specific scenario of sack/bag recognition and localization presented here. The perception modules introduced in this work are embedded into a newly designed robotic platform capable of manipulating 70 kg sacks - a standard weight when transporting coffee and cocao beans. Moreover, the robot is fully integrated in a coffee storage warehouse. Therefore we substantiate our approach with experiments in a real-world scenario of autonomous unloading of coffee cargo delivered in a shipping container.