Adaptive overlay selection at the SD-WAN edges: A reinforcement learning approach with networked agents

Abstract

Software-Defined Wide Area Networks (SD-WANs) have emerged as a promising solution to address the connectivity demands of modern distributed enterprises. However, the effective application of the Software Defined Networking (SDN) paradigm in such broad and dynamic environments remains a significant challenge. In this paper, we present two novel contributions. First, we design a decentralized control plane for SD-WANs that leverages edge-based network monitoring and overlays’ configuration. Then we present a Reinforcement Learning-based orchestration plane that leverages local information for the enforcement of SD-WAN policies. Since traditional approaches suffer either a lack of scalability due to the problem’s complexity or suboptimal performance due to isolated decision-making, the proposed approach leverages a cooperative Multi-Agent Reinforcement Learning framework. Our novel cooperative approach is based on per-site agents that exchange a small amount of information to enhance performance while preserving scalability. To validate the efficacy of our proposed approach, we conducted an extensive experimental evaluation considering diverse SD-WAN scenarios. Results show that our framework is able to satisfy global network policies for a multi-site SD-WAN with different QoS requirements and cost constraints.