A multi-agent double Deep-Q-network based on state machine and event stream for flexible job shop scheduling problem

Abstract

With the industry's rapid shift to large-scale personalized development, the need for a ramose multi-agent system capable of flexible job shop scheduling problem (FJSP) is apparent. As such, this paper proposes a novel framework that transforms the combinatorial optimization problem into a multi-stage sequence decision problem and introduces a multi-agent double Deep-Q-network algorithm (MADDQN) for FJSP. The event-driven workshop environment model based on state machine and event stream mechanism is constructed to transform FJSP into a Markov Decision Process (MDP) with more robust general performance and decouple the workshop environment model from the decision analysis model. The developed multi-agent reinforcement learning consisting of job agents and machine agents applies dispatching rules to maximize the cumulative reward of all agents by Boltzmann exploitation to avoid the local optimum. Results of numerical experiments reflect that the proposed MADDQN performs better than the traditional approaches in large-scale instances, and the real-time performance of the scheduling decision is guaranteed.