Automated Configuration of Router Port Queues via Model-Based Reinforcement Learning

Abstract

The advent of 5G telecommunication systems has increased the need for stringent Quality of Service (QoS) requirements. A key resource in meeting these requirements are routers that perform efficient queue management, congestion control and flow prioritizations for effective network slicing. Configuring routers has traditionally been an expert driven process with static or rule-based configurations for individual flows. However, in dynamically varying traffic conditions, as are proposed in 5G use cases, these traditional approaches can generate sub-optimal configurations. In this paper, we propose a solution to this problem based on model-based Reinforcement Learning (RL) where the environment is modeled as a Partial Order Markov Decision Process (POMDP). The system is trained over different configurations for router ingress queue traffic policing, egress queue traffic shaping and traffic conditions to learn the state transition and observation probabilities of the POMDP model. The optimal policy of the learned model can then be deployed to generate optimal queue configurations automatically, adapting to current traffic conditions. These aspects are demonstrated over a real Ericsson use-case with configured routers for 5G slicing.