DRL-Based Deadline-Driven Advance Reservation Allocation in EONs for Cloud–Edge Computing

Abstract

The ongoing roll-out of cloud–edge computing and Internet of Things (IoT) has been simulating the boom of new advance reservation (AR) services, such as bulk-data migration and virtual machine backup, driving the development of substrate elastic optical networks (EONs). These AR requests are initial-delay-insensitive if they are guaranteed to be completed before a predefined deadline. Therefore, the routing, modulation, and spectrum assignment (RMSA) problem is extended to the time-spectrum domain rather than the single spectrum domain. Traditional heuristic RMSA algorithms follow static procedures under handcrafted rules and assumptions, and thus cannot be optimized automatically. To solve this problem, we propose a deep reinforced deadline-driven allocation (DRDA) algorithm. To the best of our knowledge, this work is the first to leverage deep reinforcement learning (DRL) methods to solve the AR resource allocation problem. Moreover, compared with the single experiment scenario of many existing works, the DRDA algorithm is evaluated in both static and dynamic scenarios. Simulation results show that our DRDA algorithm outperforms the other leading algorithms in both static scenario and dynamic scenario.