GROM: A generalized routing optimization method with graph neural network and deep reinforcement learning

Abstract

Routing optimization, as a significant part of Traffic Engineering (TE), plays an important role in balancing network traffic and improving quality of service. With the application of Machine Learning (ML) in various fields, many neural network-based routing optimization solutions have been proposed. However, most existing ML-based methods need to retrain the model when confronted with a network unseen during training, which incurs significant time overhead and response delay. To improve the generalization ability of the routing model, in this paper, we innovatively propose a routing optimization method GROM which combines Deep Reinforcement Learning (DRL) and Graph Neural Networks (GNN), to directly generate routing policies under different and unseen network topologies without retraining. Specifically, for handling different network topologies, we transform the traffic-splitting ratio into element-level output of GNN model. To make the DRL agent easier to converge and well generalize to unseen topologies, we discretize the huge continuous traffic-splitting action space. Extensive simulation results on five real-world network topologies demonstrate that GROM can rapidly generate routing policies under different network topologies and has superior generalization ability.