A Reinforcement Learning Method for Rearranging Scattered Irregular Objects Inside a Crate

Abstract

Arranging objects from a random and scattered distribution into an integral part has many applications, including bin packing, logistics, and other industrial fields. Measurement noises, manipulation uncertainties, models of irregular objects, and rich contacts bring considerable challenges to the improvements of the overall performance, such as seamlessness of the final pattern and task efficiency. In this paper, we propose an end-to-end reinforcement learning strategy that generates a series of pushing movements for scattered irregular objects inside a crate. An abstracted and sparse reward function is proposed to evaluate the pushing performance, and Proximal Policy Optimization (PPO) learning method that simultaneously trains a Convolutional Neural Network (CNN) and a fully connected actor network is developed for end-to-end decision making. The proposed method is evaluated in both simulation and real-world scenarios. The results show that the proposed method can arrange the scattered objects tightly to fit into each other in an efficient and flexible way, and can be transferred to the real world with unseen objects.