Multi-agent deep Q-network-based metaheuristic algorithm for Nurse Rostering Problem

Abstract

The Nurse Rostering Problem (NRP) aims to create an efficient and fair work schedule that balances both the needs of employees and the requirements of hospital operations. Traditional local search-based metaheuristic algorithms, such as adaptive neighborhood search (ANS) and variable neighborhood descent (VND), mainly focus on optimizing the current solution without considering potential long-term consequences, which may easily get stuck in local optima and limit the overall performance. Thus, we propose a multi-agent deep Q-network-based metaheuristic algorithm (MDQN-MA) for NRP to harness the strengths of various metaheuristics. Each agent encapsulates a metaheuristic algorithm, where its available actions represent different perspectives of the problem environment. By combining their strengths and various perspectives, these agents can work collaboratively to navigate and search for a broader range of potential solutions effectively. Furthermore, to improve the performance of an individual agent, we model its neighborhood search as a Markov Decision Process model and integrate a deep Q-network to consider long-term impacts for its neighborhood sequential decision-making. The experimental results clearly show that an individual agent in MDQN-MA can outperform ANS and VND, and multiple agents in MDQN-MA even perform better, achieving the best results among metaheuristic algorithms on the Second International Nurse Rostering Competition dataset.