Multi-Agent Reinforcement Learning With Contribution-Based Assignment Online Routing In SDN

Abstract

Emerging applications place critical QoS requirements on the Internet. Networks need to guarantee different quality of service (QoS) requirements for different data flows for various Internet services. Improvements in traffic classification techniques, software-defined networking (SDN), and programmable network devices make it possible to quickly identify user requirements and control the routing of fine-grained traffic. In this paper, we propose CBR, an online routing algorithm using multi-agent deep reinforcement learning. CBR uses GCN to extract topology features, designs different reward functions to learn appropriate routing policies for different types of traffic demands, and organizes agents to generate routes in a hop-by-hop approach. In addition, to address the challenge of not being able to distinguish whether the actions made by each agent are critical or not due to shared reward values, CBR designs a new baseline to indicate their contribution level. Finally, to ensure reliability and speed up training, we use pre-training to learn shortest path rules to obtain initial parameters to speed up training and introduce a routing alternative mechanism to provide security for online routing. We conducted Mininet-based experiments using Abilene and GEANT network topologies. The experimental results show that CBR is able to simultaneously meet the demands of different service types for their requested traffic while performing well in terms of reliability in the case of link failures.