A generic intelligent routing method using deep reinforcement learning with graph neural networks

Abstract

Routing optimization is a well-known and established topic with the fundamental goal of operating networks efficiently. Traditional optimization heuristics may suffer from performance penalty as it mismatches actual traffic, while artificial intelligence (AI) which has undergone a renaissance recently is gradually being applied to the network optimization and has shown excellent advantages. Especially deep reinforcement learning (DRL) is investigated as a key technology for routing optimization with the goal of enabling networks self-driving. Therefore, we contributed in this paper a novel approach for practical intelligent routing method using DRL with GNN, which could be easily implemented as a northbound application on the SDN controller. Our method can not only output continuous control actions for routing optimization but also learn from some networks and generalize to other unseen ones. In order to emphasize the generalization and practicality of the intelligent routing method, we deploy it in a real SDN network for experimentation rather than simulation. The results show that the method can keep on optimizing the routing of traffic in other networks of different topologies after the training is stable. And compared with hop-based OSPF, the optimal load-balancing algorithm and the recent intelligent routing DROM, it reduces network delay by 16.1%, 19.6% and 14.3%, respectively, but at the expense of flow-table space within the acceptable range.