A Safe Training Approach for Deep Reinforcement Learning-based Traffic Engineering

Abstract

Traffic engineering (TE) is fundamental and important in modern communication networks. Deep reinforcement learning (DRL)-based TE solutions can solve TE in a data-driven and model-free way thus have attracted much attention recently. However, most of these solutions ignore that TE is a real-world application and there are challenges applying DRL to real-world TE like: (1) Efficiency. Existing learning-from-scratch DRL agent needs long-time interactions to find solutions better than traditional methods. (2) Safety. Existing DRL-based solutions make TE decisions without considering safety constraints, poor decisions may be made and cause significant performance degradation. In this paper, we propose a safe training approach for DRL-based TE, which tries to address the above two problems. It focuses on making full use of data and ensuring safety so that DRL agent for TE can learn more quickly and possibly poor decisions will not be applied to real environment. We implemented the proposed method in ns-3 and simulation results show that our method performs better with faster convergence rate compared to other DRL-based methods while ensuring the safety of the performed TE decisions.