Enhanced reinforcement learning in two-layer economic model predictive control for operation optimization in dynamic environment

Abstract

Economic model predictive control (EMPC) has attracted an abundance of interest in both academic and industrial communities in recent years because it is able to increase the economic profits of dynamic systems. For greater computational efficiency, two-layer EMPC schemes are applied in some complex process control. However, the performance of two-layer EMPC is significantly influenced by the accuracy of the chosen process model. Reinforcement learning (RL) has been studied as a model-free strategy of model-based control approaches, but its safety and stability remain a concern. In order to estimate the model parameters of nonlinear dynamic systems in real time, this work introduces a unique scheme for merging two-layer EMPC and RL. In this scheme, the two-layer EMPC technique maintains closed-loop stability and recursive feasibility while operating the closed-loop dynamic system optimally. And the RL agent continually compares the observed states to the predictions made by the EMPC and modifies the time-varying parameters as necessary. The usability of the proposed scheme is shown on a chemical process of ethylene oxide production in dynamic environment. This work enables online and continuous control, optimization, and model correction, and makes process production optimization more feasible and profitable.