Reinforcement Learning-Based Composite Optimal Operational Control of Industrial Systems With Multiple Unit Devices

Abstract

This article investigates the optimal operational control (OOC) problem for a class of industrial systems consisting of multiple unit devices with fast dynamics and an unknown operational process with slow dynamics. First, the OOC problem is formulated as a noncascade optimal control problem of two-time-scale systems with a novel performance function. Second, using singular perturbation theory, a decentralized composite control scheme is proposed by decomposing the original optimal problem into reduced-order fast and slow subsystem problems. Then, in the framework of reinforcement learning, an online controller design method for the slow subsystem is proposed by using the online measurement, and an offline controller design for the fast subsystem is proposed by using the unit device models. The obtained decentralized composite optimal controller achieves both the desired operational index tracking and disturbance rejection without requiring the dynamics of the operational process. Different from the existing cascade design methods, the proposed approach regulates the unit devices and operational process simultaneously, as well as overcomes the potential high dimensionality and ill-conditioned numerical issues. Finally, a mixed separation thickening process and a numerical example are given to illustrate the presented results.