Autonomous active recognition and unfolding of clothes using random decision forests and probabilistic planning

Abstract

We present a novel approach to the problem of autonomously recognizing and unfolding articles of clothing using a dual manipulator. The problem consists of grasping an article from a random point, recognizing it and then bringing it into an unfolded state. We propose a data-driven method for clothes recognition from depth images using Random Decision Forests. We also propose a method for unfolding an article of clothing after estimating and grasping two key-points, using Hough forests. Both methods are implemented into a POMDP framework allowing the robot to interact optimally with the garments, taking into account uncertainty in the recognition and point estimation process. This active recognition and unfolding makes our system very robust to noisy observations. Our methods were tested on regular-sized clothes using a dual- arm manipulator and an Xtion depth sensor. We achieved 100% accuracy in active recognition and 93.3% unfolding success rate, while our system operates faster compared to the state of the art. I. INTRODUCTION Robots doing the housework have recently attracted the attention of scientists. Our interest is focused in the task of folding clothes and particularly in the first part of the proce- dure, which is the unfolding of an article of clothing. Starting from a crumbled initial configuration, we want to recognize the article and then bring it into an unfolded state so that it is ready for folding. One of the key challenges in clothes perception and manipulation is handling the variabilities in geometry and appearance. These variabilities are due to the large number of different configurations of a garment, self- occlusions and the wide range of cloth textures and colors. Research on clothes perception and manipulation started in the middle 90s (1), presenting some first clothes recogni- tion techniques with the help of a dual manipulator. Later, research has been made in garment modelling and feature extraction (2) (3), while only recently scientists were able to completely fold an article of clothing starting from a crum- pled initial configuration (4) (5) (2). The main limitations in the state-of-the-art are a) slow performance and b) difficulty to generalize to a variety of shapes and materials. This stems mainly from the model-driven approaches used and associated simplifying assumptions made. To address these limitations we propose a data-driven approach for clothes recognition and unfolding. We first recognize the type of the article from raw depth data using Random Forests. Based on the recognition result, a pair of key-points are identified such that the article will naturally unfold when held by these two points (Fig. 1). Point estimation is based on Hough Forests, a random forest framework with Hough voting. An active manipulation (perception-action) approach based on POMDPs is also proposed that accounts for uncertainty in (a) Random Initial Configura- tion, grasping lowest point (b) Recognizing garment, then grasping 1 st estimated point