Markovian policy network for efficient robot learning

Abstract

Listen

Translate article

What poses a challenge for robot learning is the fact that considerable training samples and time are required to find an optimal policy in the high-dimensional robot state space. To mitigate this problem of robot learning, there is an incentive in incorporating appropriate prior knowledge. In this paper, a novel framework named Markovian Policy Network (MPN) is presented to take full advantage of the structural prior knowledge for robots to achieve efficient robot learning. Specifically, we first make efforts to reveal the Markov Property of the robot, which characterizes the intrinsic prior knowledge for robotic physical structure. Next, the proposed framework is constructed by employing a modified graph neural network to incorporate the Markov Property of the robot into policy networks. By exploiting the structural prior knowledge of robots in our framework, the high-dimensional original state space could be decomposed into several low-dimensional state spaces with conditional independence to realize substantial reductions in the dimensionality of state space. Empirical results on diverse robotic systems including the challenging dexterous manipulation task, demonstrate not only the effectiveness but the robustness of the proposed MPN for robotic skill learning, delivering advancement substantially over traditional policy learning methods.