Most recent development of dynamic programming techniques and their application to optimal systems design

Abstract

Dynamic programming theory will solve, in principle, many of the problems connected with optimal systems synthesis, but the number of complex problems actually solved with this method is not large, because the calculations become more and more difficult the more complicated the problem becomes. Introduction of ‘sufficient coordinates’, on which the risk function depends, eases the situation, the ‘sufficient coordinates’ forming the space in which the Bellman equation is considered. In the paper a non-trivial example illustrates the effectiveness of introducing sufficient coordinates. In the example the sufficient coordinates are a combination of a posteriori probabilities and the dynamic variable of the controlled plant. A practical result of the introduction of sufficient coordinates is hat the optimal control breaks down in two units, one producing he sufficient coordinates, the other providing optimal control.