Adjustable iterative Q-learning for advanced neural tracking control with stability guarantee

Abstract

In this article, an accelerated Q-learning algorithm with evolving control is established to solve the optimal tracking control problem. First, an accelerated Q-learning scheme is constructed with an advanced Q-function. By utilizing the advanced Q-function, calculating of the feedforward control input can be avoided and the terminal tracking error can be eliminated. Then, by introducing the relaxation factor, the convergence rate of the iterative Q-function sequence is accelerated significantly, which is a potential way to diminish the computational burden. Furthermore, the convergence, positive definiteness, and stability conditions of the accelerated Q-learning algorithm are analyzed with some preconditions of the relaxation factor. Thus, the developed algorithm can achieve evolving control. Finally, the fantastic performance of the developed algorithm with critic network implementation is verified through two simulation examples.