Towards live decision-making for service-based production: Integrated process planning and scheduling with Digital Twins and Deep-Q-Learning

Abstract

Listen

Translate article

Production flow is becoming increasingly complex since manufacturers must react quickly to changing markets demands and diverse customer requirements. In order to ensure production efficiency, it is essential to have an adequate scheduling system capable of managing diverse process flows and handling unforseen changes. In this paper, we present an approach leveraging Digital Twins (DTs) and Deep-Q-Learning to perform integrated process planning and scheduling for service-based production. DTs of production assets provide live information about their physical entities for our approach to perform live decision-making based on the current operation conditions. We use Deep-Q-Learning which is a deep Reinforcement Learning (RL) algorithm to perform integrated process planning and scheduling. We present two RL-designs that deal with different situations of live decision-making. We have evaluated the learning efficiency and scalability of the RL-designs on a virtual aluminum cold rolling mill developed by the SMS Group,11https://www.sms-group.com/plants/cold-rolling-mills-for-aluminum. in the context of the BaSys 4.2 project.22https://www.eclipse.org/research/projects/basys_42/. The results show that the first RL-design is suitable for deriving schedules for individualized production with small lots where process plans must be re-calculated frequently, while the second RL-design is optimal for production with large job quantities where jobs arrive continuously.