Timeslot Scheduling with Reinforcement Learning Using a Double Deep Q-Network

Abstract

Adopting reinforcement learning in the network scheduling area is getting more attention than ever because of its flexibility in adapting to the dynamic changes of network traffic and network status. In this study, a timeslot scheduling algorithm for traffic, with similar requirements but different priorities, is designed using a double deep q-network (DDQN), a reinforcement learning algorithm. To evaluate the behavior of the DDQN agent, a reward function is defined based on the difference between the estimated delay and the deadline of packets transmitted at the timeslot, and on the priority of packets. The simulation showed that the designed scheduling algorithm performs better than existing algorithms, such as the strict priority (SP) or weighted round robin (WRR) scheduler, in the sense that more packets arrived within the deadline. By using the proposed DDQN-based scheduler, it is expected that autonomous network scheduling can be realized in upcoming frameworks, such as time-sensitive or deterministic networking.