Iterative Learning Control (ILC) Guided Reinforcement Learning Control (RLC) Scheme for Batch Processes

Abstract

Iterative learning control (ILC) is a kind of effective learning control scheme which is mainly designed to solve the problems in controlling a batch or repetitive process. Although the control performances of ILC systems can improved from batch to batch, it still strongly depends on the repeatability of the process and control target. Reinforcement learning (RL) is another learning based optimization algorithm which can be applied to many complicated decision-making scenarios. Data-driven based RL algorithms have good robustness due to the generalization of the policy neural network, however, it is low-data efficiency in network training. In this paper, for batch process control we propose a new reinforcement learning control (RLC) scheme which is guided by classical iterative leaning control. On the one hand, this RLC scheme has capability to optimize the policy network faster than RL algorithm without guidance, on the other hand, the generalization of deep policy network improves the robustness of the control system. Based on the numerical simulations, the effectiveness of the proposed control scheme is demonstrated by comparing with the conventional reinforcement learning algorithm and the P-type iterative learning control scheme. This paper provides a new way for the application of reinforcement learning algorithm to batch process control.