Model-Based Reinforcement Learning Control of Electrohydraulic Position Servo Systems

Abstract

Even though the unprecedented success of AlphaGo Zero demonstrated reinforcement learning as a feasible complex problem solver, the research on reinforcement learning control of hydraulic systems is still void. We are motivated by the challenges presented in hydraulic systems to develop a new model-based reinforcement learning controller that achieves high-accuracy tracking at performance level and with asymptotic stability guarantees at system level. In this article, the proposed design consists of two frameworks: A recursive robust integral of the sign of the error (RISE) control approach to providing closed-loop system stability framework, and a reinforcement learning approach with actor-critic structure to dealing with the unknown dynamics or more specifically, the actor neural network is used to reduce the high feedback gain of the recursive RISE control approach by compensating the unknown dynamic while the critic neural network is integrated to improve the control performance by evaluating the filtered tracking error. A theoretical guarantee for the stability of the overall dynamic system is provided by using Lyapunov stability theory. Simulation and experimental results are provided to demonstrate improved control performance of the proposed controller.