Reinforcement Learning-Based Adaptive Optimal Control for Partially Unknown Systems Using Differentiator

Abstract

An adaptive optimal controller is designed by solving the infinite-horizon optimal control issue based on reinforcement learning (RL) technique for partially unknown systems. Since the solution to Hamilton-Jacobi-Bellman equation includes the drift dynamics, a first-order robust exact differentiator (RED) is designed to provide an approximation for the unknown drift dynamics considering the known input dynamics. To obtain the approximation of the optimal control policy and value function, an actor-critic neural network (NN) structure is built. A synchronous update algorithm based on the first-order RED and the RL technique for the two NNs. By employing Lyapunov theorem, the convergence and stability are proved for the proposed control method. Eventually, to show the performance of the proposed controller, both linear and nonlinear simulation examples are given, repectively.