Novel generalized policy iteration for efficient evolving control of nonlinear systems

Abstract

In this article, we construct a novel generalized policy iteration framework to address optimal regulation problems for discrete-time nonlinear systems in a more efficient way. Relevant properties are investigated for the framework, including monotonicity and convergence of the iterative value function sequence as well as the admissibility of the iterative control policy. Additionally, an innovative approach is developed to seek an initial admissible control policy for the framework with an adjustable searching speed. Based on these, an evolving control algorithm is presented with stability guarantee. This algorithm employs iterative control policies for system control during the computation of the optimal control policy, as opposed to waiting for the generation of the optimal control policy before implementing control. Eventually, two simulation experiments are conducted with real-world physical backgrounds, in order to illustrate the performance of the proposed strategy.