Independent double DQN-based multi-agent reinforcement learning approach for online two-stage hybrid flow shop scheduling with batch machines

Abstract

Two-stage hybrid flow shop scheduling with batch machines and jobs arriving over time is complex and challenging in various real-world production scenarios. For the online scheduling problem, traditional heuristic rules can quickly respond to dynamically arrived jobs, while with poor and unstable performance. To close the research gap in the problem, this paper proposes an independent double deep-q-network-based multi-agent reinforcement learning (MA-IDDQN) approach to produce an adaptive rule for batch forming and scheduling. Specifically, the online scheduling problem is transformed into a cooperative Markov decision process by defining state space, action space, and reward function for different agents. Then, two agents are constructed and trained via double DQN to address the batch forming task and scheduling task respectively. Meanwhile, multi-agent cooperates through the behavior analysis mechanism among agents. Moreover, we designed a ε-greedy policy considering waiting in batch forming to make a reasonable decision through historical data. To validate the proposed approach, 27 instances with different scales are settled and contrasted. By comparing with frequently-used heuristic rules and other deep reinforcement learning methods, the experimental results demonstrate that the MA-IDDQN can integrate online batch forming and scheduling to minimize the total tardiness time effectively.