A Reinforcement Learning Approach to Online Scheduling of Single-Stage Batch Chemical Production Processes

Abstract

Abstract The field of Reinforcement Learning (RL) has received a lot of attention for decision-making under uncertainty. Lately, much of this focus has been on the application of RL for combinatorial optimisation. Recent work has showcased the use of RL on a single-stage continuous chemical production scheduling problem. This work highlighted the potential of RL for optimal decision-making under uncertainty in the paradigm of (bio)chemical production scheduling. However, this novel approach is yet to be tested in the context of parallel unit operations and batch processing systems. In this work, we outline a framework for the use of RL to handle single-stage parallel, batch production. In particular, we incorporate elements such as uncertainties in the model data, limited batch size, sequencing constraints, and uncertainties in processing times and product demand, which make for a substantially harder problem. To handle the presence of precedence or succession constraints, by taking inspiration from approaches such as generalised disjunctive programming, we propose a novel methodology that identifies transformations of the control set available to the RL at each control interaction. Given that production typically operates under standard operating procedures, such transformations can be identified by logic. The efficacy of policy synthesis via evolutionary RL methods is benchmarked against mixed integer programming. The results of this study provide further support for the use of RL in online scheduling.