A Deep Reinforcement Learning Approach for Deploying SDN Switches in ISP Networks from the Perspective of Traffic Engineering

Abstract

Nowadays, a hybrid Software-Defined Network (hybrid SDN), which combines the robustness of the distributed network and the flexibility of the centralized network, is a prevailing network architecture. The performance of Traffic Engineering (TE) in a hybrid SDN is largely influenced by the location of SDN switches. To derive the optimal location of SDN switches, the previous SDN switches deployment strategies mainly focused on manually designing heuristics to search for the SDN deployment sequence and only take a static Traffic Matrix (TM) into consideration. However, the manually-designed heuristics cannot capture the complex intrinsic relations among the location of SDN switches, network topology, and dynamic traffic demands. In addition, the SDN deployment strategy optimized under a single TM exhibits poor performance in a dynamic environment. Therefore, in this paper, we propose a Deep Reinforcement Learning (DRL)-based algorithm SEED to intelligently learn the SDN deployment strategy under multiple TMs. Specifically, to capture the dynamic traffic information, we first cluster the historical traffic demand matrices for obtaining the representative Traffic Matrices (TM) that can depict the dynamic traffic. Then, to intelligently learn the intrinsic relations between the topology, TMs, and the location of SDN switches, we design a DRL agent under multiple TMs by interacting with the environment in a trial and error manner. The extensive experiments on three real network topologies and traffic demands demonstrate that our proposed SDN switches deployment strategy can better adapt to the dynamic traffic and better improve the TE performance than the other deployment strategies.