Biologically inspired scheme for continuous-time approximate dynamic programming

Abstract

Control system theory has been based on certain well understood and accepted techniques, such as transfer function-based methods, adaptive control, robust control, non-linear systems theory, state-space methods, etc. However, recently, the hypothesis that methods of modelling and analysis of life science systems hold out the power of significantly improving the performance of man-made control systems has gained increased interest, while becoming more and more a fact. For instance, in the last few decades many successful results were obtained by combining the potential of artificial neural networks (ANNs) with classical control structures. It has been shown that certain types of ANN can extend the capabilities of adaptive controllers by making them applicable for more complex non-linear systems, and at the same time greatly improving the system performance. In this paper, we will present a biologically inspired structure that will learn the optimal state feedback controller for a linear system, while at the same time performing continuous-time online control for the system at hand. Being based on a reinforcement learning technique, the optimal controller will be obtained while only making use of the input-to-state system dynamics. Mathematically speaking, the solution of the algebraic Riccati equation underlying the control problem will be obtained without making use of any knowledge of the system internal dynamics.