Integrated learning framework for multistep pick-place-arrange of arbitrarily shaped objects in a narrow crate

Abstract

To maximize the space occupied by unknown objects in a narrow crate and handle uncertain situations, we propose a novel human-like system-level learning framework: sequential infiltration synthesis of combined learning (SISCL). This framework leverages the synergies between picking by grasping, three-dimensional modeling, preferable placement, and seamless arranging by pushing in a systematic perspective. Within SISCL, a closed-loop system organizes different designing methods including reinforcement learning, knowledge induction, and analytic methods. This integration trains the system to autonomously package unknown objects and accommodate for uncertainties in measurements or executions, combining efficiency and flexibility. Initially trained in simulations, the SISCL framework underwent comprehensive evaluation in the laboratory. Furthermore, an initial validation was conducted on an extraterrestrial test ground. The results demonstrate the adaptability of the proposed framework to different shapes, materials, and stiffness of objects, as well as the capacity for uncertainties, thereby considerably reducing the need for precision in each sub-procedure in a human-like manner, facilitating seamless operations and efficient utilization of limited space.