Reinforcement learning-based optimised control for a class of second-order nonlinear dynamic systems

Abstract

This paper presents an optimised tracking control scheme based on reinforcement learning (RL) for a class of second-order nonlinear systems with unknown dynamics. Different from the first-order dynamic system control, the second-order case is required to synchronously steer two variables of position and velocity, hence it makes this optimised control more challenging to accomplish. To achieve the optimised control, first, neural network (NN) is employed to approximate the solution of Hamilton–Jacobi–Bellman (HJB), and then an RL is performed by constructing both critic and actor based on the NN approximation. Since the RL training laws are derived from the negative gradient of a simple positive function generated in accordance with the partial derivative of HJB equation, it can make the control algorithm significantly simple to compare with the existing optimal control methods. Meanwhile, it can also release the condition of persistence excitation and compensate for the nonlinear uncertainty. Finally, the proposed adaptive control method can guarantee the desired results that are demonstrated by theorem, proof and simulation.