A practical Reinforcement Learning implementation approach for continuous process control

Abstract

Industrial process control using model-based technologies is well established. These technologies are typically non-adaptive and so have limitations. Reinforcement Learning (RL) provides a model-free adaptive alternative. RL is a type of machine learning (ML) where models or data sets of the environment are not necessary before learning can start. It generates data, by exploring the environment and then learn the behavior from it. Though RL has been successfully applied for learning and playing various games such as Go, Chess, Atari; its application to continuous process control problems is not trivial. There is a need for online RL implementation to be safe, fast learning and explainable when applied to industrial control problems. Rather than adding to the extensive research on augmenting existing RL algorithms, the paper presents a unique systematic method of formulating the RL problem incorporating domain-specific knowledge about process constraints and objectives, resulting in reduced dimensionality, along with modifications to the exploration process, applicable to any model free RL algorithm supporting continuous states and actions, to enhance safety, speed and explainability of online RL implementation without requiring a simulation model. The approach is successfully implemented on two multivariable processes: a simulated distillation column and a temperature control lab setup using the Deep Deterministic Policy Gradient (DDPG) algorithm. The work demonstrates that the presented method is applicable to multivariable, noisy, non-linear processes with disturbances. It will further the potential of introducing the advances in Artificial Intelligence and Machine Learning algorithms for intelligent process control capable of enabling autonomous operation in the process industry.