Reinforcement Learning for Process Time Optimization in an Assembly Process Utilizing an Industry 4.0 Demonstration Cell

Abstract

The process time of a production process is an important result of planning in supply networks, which in turn is a defining parameter, significant for further organizational decisions. Optimizing it requires extensive knowledge of the underlying processes and parameters involved. It is imperative to reduce process time while also ensuring the quality of the products to stay competitive in an ever-evolving environment. This paper demonstrates a solution for a reinforcement learning (RL) application to optimize the process time of an assembly case. Using an actual industry 4.0 demonstration cell as a hands-on, model-free simulation environment, an RL Agent interacts with an OPC UA interface to gather machine sensor data and control the machine drives. Using Q-learning, an online off-policy algorithm, with a discretized action space we achieve self-optimization of the assembly case by decreasing process time while simultaneously ensuring that the quality of the products stays within tolerable parameters. Our findings demonstrate the usefulness of RL applications in process control, in this case optimizing machine parameters. As an addition, we deduce design guidelines from this model and its implementation to help reduce possible sources of error while implementing similar approaches for industrial applications.