Constructing a DRL Decision Making Scheme for Multi-Path Routing in All-IP Access Network

Abstract

Despite the massive increase in network capacities and devices capabilities, intra-domain TE problems have remained a challenge due to high traffic demand in dynamic network settings. In this paper, we fundamentally adopt and construct the use of Deep Reinforcement Learning (DRL) algorithm to improve end-to-end delay and maximum link utilisation (MLU) in a wireless All-IP access network. In an experimental pseudo setting, we propose a routing method DRL-MPR (Multi-Plane Routing) within a Dueling Deep Q-Network (DDQN) framework as a controller in which by simultaneously allocating multiple robust IP sessions, each session will take the near optimal path/plane. We first revisit Multi-Plane Routing Protocol (MPR) to configure a set of routing paths/Routing Planes (RP) from a physical topology as pre-condition ahead of the traffic injection. We compare our decision making scheme results with some traditional routing methodologies such as OSPF and standard Multi-Plane Routing (MPR). Our simulation results show that DRL obtains lower maximum link utilisation (MLU) and end-to-end delay. For example, in the 19 nodes and 41 links topology, a small traffic load results in 60% maximum link utilisation for DRL-MPR, whereas those of OSPF and MPR are 100% and 94% respectively. In this simulation environment, the traffic demand in the IP access network is gradually increasing to the point of saturated network link utilisation.