강화학습 기반 화학 공정 제어 성능 향상을 위한 보상 함수 시뮬레이션 연구

Abstract

Process control is vital for operating chemical process safely and efficiently while maintaining product quality. Process control has been implemented by utilizing traditional techniques such as PID (Proportional-Integrate-Derivative) control and MPC (Model Predictive Control). However, these approaches have disadvantages when the PID tuning rules do not provide optimal tuning parameters for various operating scenarios or when the model fails to match the actual plant. RL (Reinforcement Learning) is one of the most prominent data-driven techniques for addressing such issues and has gained popularity in recent years. While RL has been extensively studied and succeeds in controlling chemical processes, the reward function has a significant impact on its performance. Therefore, it is essential to specify the reward function in order to efficiently control the process. In this study, we propose a reward function to enhance the performance of RL and compare the control performances of MPC and RLs with four distinct reward functions. The controllers track the setpoint of the product in the Van de Vusse process and are evaluated based on the deviation between the setpoint and output. RL with the proposed reward function outperformed the other controllers. Its performance was 12% greater than that of MPC and 6-30% greater than other RL controllers. In chemical processes, the control performance of RL is enhanced by incorporating the time term and positive reward in the reward function, thus outperforming conventional control approaches.