Control of the Cart-Pole System: Model-based vs. Model-free Learning

Abstract

In this paper, we study the control design problem for a cart-pole system without the prior knowledge of its physical parameters. The control task involves both swing-up and balancing. Two control methods are compared: 1) model-free reinforcement learning; 2) system-identification based control design. The former uses a popular deep deterministic policy gradient (DDPG) algorithm, whereas the latter uses a model-based design together with a system identification method for parameter estimation. The results show that system-identification based control design is far superior than reinforcement learning in terms of training, computation complexity and control performance, but requiring skills for system modelling, parameter estimation and control design.