Optimized tracking control based on reinforcement learning for a class of high-order unknown nonlinear dynamic systems

Abstract

The article is to develop an optimized tracking control using fuzzy logic system (FLS)-based reinforcement learning (RL) for a class of unknown nonlinear dynamic single-input–single-output (SISO) system under canonical form. From mathematical viewpoint, optimal nonlinear control depends on the solution of Hamilton–Jacobi-Bellman (HJB) equation, but finding the equation’s analytic solution is almost impossible because of the strong nonlinearity. For addressing the problem in this work, the RL approximation strategy is employed by on-line iterating both critic and actor FLSs, where critic FLS aims for evaluating control performance and making feedback to actor, and actor FLS aims for executing the modified control behavior. Different with the previous RL-based optimization approach, the proposed approach derives RL training laws from negative gradient of a simple positive function rather than the square of HJB equation’s approximation. As a result, the control algorithm can be significantly simplified. Furthermore, it can avoid two general conditions, persistence excitation and known dynamic, required in most RL optimal control methods. Finally, the proposed optimized control is demonstrated from both aspects of theory proof and computation simulation.