Model-based learning on state-based potential games for distributed self-optimization of manufacturing systems

Abstract

In this paper, we propose a novel approach of model-based learning on state-based potential games (MB-SbPGs) that enables distributed self-optimization of manufacturing systems with a sample-efficient approach, where the interactions between the SbPG players and the actual systems can be significantly reduced. Most self-learning methods necessitate such interactions frequently to find the optimal solutions due to the exploratory and iterative behaviours of the players throughout the learning process. Nevertheless, in industrial environments, these interactions are generally expensive and time-consuming, as well as have potential risks. One possible approach to address this challenge is by employing an accurate digital representation of the system. However, developing an accurate digital representation is often a complex task. Therefore, the proposed MB-SbPG approach offers a solution that effectively learns the dynamics of the relevant systems and optimizes them within virtual environments. To achieve the stated objectives, the MB-SbPG approach consists of two primary steps. Firstly, deep learning models are designed to accurately predict system dynamics. Secondly, the SbPG players are trained within virtual environments, which enables the distributed self-optimization features of the system. Within the scope of this research, we examine two separate strategies for MB-SbPGs, namely (a) MB-SbPGs with single-step predictors and (b) MB-SbPGs with multi-step predictors. Furthermore, we demonstrate that MB-SbPGs have the potential to enhance memory-based learning on SbPG. We also validate the suitability of reusing the trained networks on MB-SbPGs for transfer learning in adaptable systems. To assess its effectiveness, we implement and validate MB-SbPG in a real industrial control scenario using a laboratory-scale testbed, which ends up with encouraging results.