Incremental model based online heuristic dynamic programming for nonlinear adaptive tracking control with partial observability

Abstract

Heuristic dynamic programming is a class of reinforcement learning, which has been introduced to aerospace engineering to solve nonlinear, optimal adaptive control problems. However, it requires an off-line learning stage to train a global system model to represent the system dynamics. This paper uses an incremental model in heuristic dynamic programming to improve the online learning ability, which is incremental model based heuristic dynamic programming. The trait of the online identification of the incremental model makes this method an option for fault-tolerant control and partially observable control problems. This study, therefore, also extends this method to deal with partial observability. The presented method has been validated on two different online tracking problems: missile fault-tolerant control with full-state measurements and also spacecraft attitude control disturbed with liquid sloshing under partially observable conditions. The results reveal that the proposed method outperforms the conventional heuristic dynamic programming method in fault-tolerant control tasks, deals with partial observability, and is robust to internal uncertainties and external disturbances.