A Novel Simulation-Reality Closed-Loop Learning Framework for Autonomous Robot Skill Learning

Abstract

In recent years, data-driven learning methods have been widely studied for autonomous robot skill learning. However, these methods rely on large amounts of robot–environment interaction data for training, which largely prevents them from being applied to real-world robots. To address this problem, this article proposes a novel simulation-reality closed-loop learning framework for autonomous robot skill learning that can improve data efficiency, enhance policy stability, and achieve effective policy simulation-to-reality (sim2real) transfer. First, a hybrid control model combining the asymmetric deep deterministic policy gradients (Asym-DDPGs) model and the forward prediction control (FPC) model is proposed to learn vision-based manipulation policies in simulations, which can decompose complex tasks to improve learning efficiency. Second, a novel pixel-level domain adaptation method named Position-CycleGAN is designed to translate real images to simulated images while also preserving the task-related information. The policy trained in simulations can be directly migrated into real robots in a reverse reality-to-simulation manner using the Position-CycleGAN model. The experimental results validate the effectiveness of the proposed framework. This work provides an efficient and feasible path for achieving autonomous skill learning.