Optimal control of nonaffine nonlinear discrete-time systems using kernel-based adaptive dynamic programming

Abstract

In this paper, we investigate the optimal control problem of unknown nonaffine nonlinear discrete-time (DT) systems by kernel-based dual heuristic programming (KDHP). First, under the framework of Markov decision process (MDP), we build the optimal control model of unknown nonaffine nonlinear DT systems. Second, in order to improve the computational efficiency of the kernel machine, approximate linear dependence (ALD) analysis is used to design a novel kernel-based ADP algorithm (KDHP). By using KDHP, the optimal control for unknown nonaffine nonlinear DT systems is obtained. Compared to traditional ADP, the KDHP algorithm not only increases the generalization capability but also improve representation learning efficiency in optimal control. Finally, an illustrated example is provided to demonstrate the effectiveness of the KDHP algorithm. Finally, by integrating kernel methods into the critic learning of ADP algorithms, a novel structure of ADP with sparse kernel machine is presented.