A Reinforcement Learning Strategy for the Swing-Up of the Double Pendulum on a Cart

Abstract

The effective control design of a dynamical system traditionally relies on a high level of system understanding, usually expressed in terms of an exact physical model. In contrast to this, reinforcement learning adopts a data-driven approach and constructs an optimal control strategy by interacting with the underlying system. To keep the wear of real-world systems as low as possible, the learning process should be short. In our research, we used the state-of-the-art reinforcement learning method PILCO to design a feedback control strategy for the swing-up of the double pendulum on a cart with remarkably few test iterations at the test bench. PILCO stands for “probabilistic inference for learning control” and requires only few expert knowledge for learning. To achieve the swing-up of a double pendulum on a cart to its upper unstable equilibrium position, we introduce additional state restrictions to PILCO, so that the limited cart distance can be taken into account. Thanks to these measures, we were able to learn the swing up at the real test bench for the first time and in only 27 learning iterations.