A Reinforcement Learning Approach to Intelligent Goal Coordination of Two-Level Large-Scale Control Systems

Abstract

Two principles for coordination of large-scale systems, namely Interaction Prediction Principle and Interaction Balance Principle were postulated by Mesarovic et al. [1], [2] to provide guidance in synthesizing structures for multi-level or hierarchical control of largescale systems and obtain the optimal solution. Hierarchical structures are feasible structures which reduce the complexity of large-scale control systems and improve the solution through decomposition, coordination and parallel processing [3]-[6]. In two-level hierarchical approaches, the overall system is first decomposed into several interactive subsystems, at the first level, where the optimization problem is redefined for each one of them. The interactions between these sub-systems, at the first level, and the coordinator, at the second level, called the coordination parameters, are used so that the overall solution is obtained. In compare to centralized approaches, where the whole problem is considered for the solution at once, the computational efforts in hierarchical approaches are based on subproblems, having smaller order, requiring less computational time, in addition to the coordination strategy. The Goal Coordination based on Interaction Balance Principle approach of Mesarovic et al. has already been applied to large-scale systems and the results are reported in [3][5]. In applying the Interaction Balance Principle, the supremal controller modifies the infimal (i.e. first-level) performance functions, compares the interface inputs (interactions) demanded by the infimal controllers and those which actually occur, then provides new performance modifications whenever the error is observed as being outside the acceptable bounds. A brief description of the Goal Coordination and Interaction Balance Principle is presented in the following section. Although a more detailed discussion of this principle can be found in [1] ,[2] and also, voluminous literature on large-scale systems theories and applications including survey articles, textbooks and monographs can be found in [6]-[12]. Based on Interaction Balance Principle, a new goal coordination scheme, as a foundation for intelligent coordination of large-scale systems is postulated in this chapter. The approach is formulated in an intelligent manner such that it provides the update of the coordination