Building Autonomic Elastic Optical Networks with Deep Reinforcement Learning

Abstract

Conventional schemes for service provisioning in next-generation elastic optical networks (EONs) rely on rule-based policies that suffer from scalability issues and can lead to poor resource utilization efficiency due to the lack of knowledge about the essential characteristics of EONs (e.g., traffic profiles, physical-layer impairments). This article discusses the application of emerging deep reinforcement learning (DRL) techniques in EONs for enabling an autonomic (self-driving) and cognitive networking framework. This new framework achieves self-learning-based service provisioning capabilities by employing DRL agents to learn policies from dynamic network operations. Such capabilities can remarkably reduce the amount of human effort invested in developing effective service provisioning policies for emerging applications, and thus, can facilitate fast network evolutions. Based on the framework, we first present DeepRMSA, a DRL-based routing, modulation, and spectrum assignment (RMSA) agent for EONs. Then, as today's networks are often composed of multiple autonomous systems, we extend the autonomic networking framework to multi-domain EONs by applying multi-agent DRL (where multiple autonomous DRL agents learn through jointly interacting with their environments). Comparisons of the results from numerical simulations show significant advantages of the proposed framework over the existing rule-based heuristic designs.