Dynamic scheduling of decentralized high-end equipment R&D projects via deep reinforcement learning

Abstract

Distributed research and development (R&D) plays a pivotal role in high-end equipment manufacturing enterprises, significantly impacting interdisciplinary innovation and expediting development cycles. In the uncertain environment characterized by random arrivals of new projects, achieving dynamic scheduling and real-time coordination for decentralized R&D projects is a challenging problem. This paper studies a dynamic decentralized resource-constrained multi-project scheduling problem with product transfers (DDRCMPSP-PT). The objective is to minimize the development cycle after initial local schedules are generated to minimize the individual project makespan. To tackle the problem, we develop a decentralized multi-agent system using the dynamic coordination mechanism for resource assignment (DMAS/DCMRA). An up-to-date deep reinforcement learning (DRL) algorithm, dueling double deep Q-learning (D3QN) with prioritized replay, is employed to select the optimal strategy to resolve the global resource conflicts adaptively. Two priority rules are presented based on the structural properties and make up the action space of the DRL agent together with ten high-quality priority rules. Extensive simulation experiment results based on real enterprise cases reveal that the DMAS/DCMRA outperforms all traditional priority rule-based methods. The proposed priority rules also show competitive performance compared to other priority rules, which validates the applicability and superiority of the proposed method. This research offers a new agile project management technique for organizations running multiple R&D projects and managing crucial bottleneck resources.