EPC-TE: Explicit Path Control in Traffic Engineering with Deep Reinforcement Learning

Abstract

Segment Routing (SR) provides Traffic Engineering (TE) with Explicit Path Control (EPC) by steering data flows passing through a list of SR routers along a desired path. However, large-scale migration from a pure IP network to a full SR one requires prohibitive hardware replacement and software update. Therefore, network operators prefer to upgrade a subset of IP routers into SR routers during a transitional period. This paper proposes EPC-TE to optimize TE performance in hybrid IP/SR networks where partially deployed SR routers coexist with legacy IP routers. We propose a concept of key nodes to achieve EPC over desired paths and a criterion to select which IP routers to upgrade first under a pre-defined upgrading ratio. EPC-TE leverages Deep Reinforcement Learning (DRL) to inference the optimal traffic splitting ratio across multiple controllable paths between source-destination pairs. EPC-TE can achieve comparable TE performance as a full SR network with an upgrading ratio less than 30%. Extensive experimental results with real-world topologies show that EPC-TE significantly outperforms other baseline TE solutions in minimizing maximum link utilization.