Dynamic compensator-based near-optimal control for unknown nonaffine systems via integral reinforcement learning

Abstract

In this paper, a dynamic compensator-based near-optimal control approach for unknown nonaffine nonlinear systems is developed by using integral reinforcement learning. Since system dynamics is unknown, it is difficult to obtain the optimal control policy via neuro-dynamic programming. To address this problem, a general dynamic compensator is introduced as the virtual control input to augment the unknown nonaffine nonlinear system as a partially unknown affine system. For the augmented system, a novel quadratic value function is designed with the system states, the actual control input and the virtual control input. The optimal control of the augmented system can be regarded as the near-optimal control for the original system since the novel optimal value function is an upper bound of the original optimal value function. In order to avoid the identification of system dynamics, the integral reinforcement learning framework is utilized to derive the optimal control based on the solution of Hamilton–Jacobi–Bellman equation via the critic-only structure. Meanwhile, the weight learning rule of the critic neural network is presented with the experience replay technique to relax the persistence of excitation condition. Moreover, the uniform ultimate boundedness of weight estimation errors and the stability of the closed-loop system are guaranteed by using the Lyapunov’s direct method. Finally, simulation results of two examples demonstrate the effectiveness of the developed dynamic compensator-based near-optimal control method.