GeneWorker: An end-to-end robotic reinforcement learning approach with collaborative generator and worker networks

Abstract

Reinforcement learning aided by the skill conception exhibits potent capabilities in guiding autonomous agents toward acquiring meaningful behaviors. However, in the current landscape of reinforcement learning, a skill is often merely a rudimentary abstraction of a sequence of primitive actions, serving as a component of the input to policy networks with fixed network parameters. This rigid methodology presents obstacles when attempting to integrate with burgeoning techniques such as meta-learning and large language models. To address this issue, we introduce a unique neural skill representation that abstracts the activation of neurons in each neural layer. Based on this, a novel end-to-end robotic reinforcement learning algorithm is proposed, in which two sub-networks, i.e., generator and worker networks, implement collaborative inferences via neural skills. Specifically, the generator produces a series of multi-spatial neural skills, providing efficient guidance for subsequent decision-making; by integrating these skills, the worker can determine its own network weights and biases to cope with various environmental conditions. Therefore, actions can be sampled with flexibly changeable network parameters through the collaboration between generator and worker networks. The experiments demonstrate that GeneWorker can achieve a mean success rate of over 90.67% on continuous robotic tasks and outperforms previous state-of-the-art methods by a minimum of 54% on the pick-and-place task.