Reinforcement learning for distributed hybrid flowshop scheduling problem with variable task splitting towards mass personalized manufacturing

Abstract

Mass personalization manufacturing (MPM), an emerging production pattern, aims to improve enterprise profit in modern industries. However, the processing of heterogeneous orders from the consumers complicates such production scheduling problem. In addition, different scale tasks should adopt different splitting strategies in practical manufacturing, which makes the task splitting method more worthy of investigation. Towards MPM, this paper presents a distributed hybrid flowshop scheduling problem with variable task splitting (DHFSP-VTS) to minimize the makespan and total energy consumption simultaneously. Meanwhile, the VTS allows the tasks to be split into different sublots so they can save setup and transfer time. To solve these problems, we present an order modularization processing method that can categorize multiple types of orders into specific generation tasks, and a highly effective reinforcement learning-multiple objective evolutionary algorithm based on decomposition (RL-MOEA/D) is designed. In RL-MOEA/D, there are three features: (1) three initial rules are used for initialization based on the current splitting scheme that can increase the diversity of solutions; (2) the reinforcement learning agent uses the Q-learning mechanism to dynamically select the scheme of task splitting as action; (3) a neighborhood search strategy improves the exploitation ability and expand the solution space. To verify the effectiveness of RL-MOEA/D, the MOEA/Ds based on four splitting schemes and four RL combined meta-heuristics are compared on 18 instances. The results show that RL-MOEA/D can obtain the best optimization and stability of all the other comparison algorithms. Therefore, it’s a new technique to solve DHFSP with large-scale tasks, especially for the problem of MPM.