A Cost Function Based Single Network Adaptive Critic architecture for optimal control synthesis for a class of nonlinear systems

Abstract

Approximate dynamic programming implemented with an Adaptive Critic (AC) neural network (NN) structure has evolved as a powerful alternative technique that eliminates the need for excessive computations and storage requirements in solving optimal control problems. A typical AC structure consists of two interacting NNs. In this paper, a new architecture, called the “Cost Function Based Single Network Adaptive Critic (J-SNAC)” is presented. This approach is applicable to a wide class of nonlinear systems where the optimal control (stationary) equation can be explicitly expressed in terms of the state and cost variables. Selection of this terminology is guided by the fact that it eliminates the use of one NN (namely the action network) that is part of a typical dual network AC. In order to demonstrate the benefits and the control synthesis technique in using the J-SNAC, two problems have been solved with the AC and the J-SNAC approaches. Results are presented that show savings of about 50% of the computational costs by J-SNAC while having the same accuracy levels of the dual network structure in solving for optimal control. Convergence of the J-SNAC iterations is discussed as well as the reduction of the iterative process to the familiar algebraic Ricatti equation in the case of linear systems.