MATE: When multi-agent Deep Reinforcement Learning meets Traffic Engineering in multi-domain networks

Abstract

As the global network evolves into a large-scale interconnection system with multiple distributed domains, scalability and efficiency have become equally important for Traffic Engineering (TE). Traditional centralized TE suffers from significant computational complexity and privacy concerns, while distributed TE results in suboptimal routing decisions due to its lack of a coordination mechanism. This paper proposes MATE, an innovative, scalable, and efficient TE framework for multi-domain networks featuring a hierarchical control plane. MATE designs a three-stage workflow to optimize routing decisions for inter-domain traffic while ensuring their QoS guarantees. First, in the bottom-up topology abstraction stage, MATE aggregates network resources within each domain, creating an abstract view of the global network state while protecting each domain’s internal information. Second, in the top-down decision-making stage, MATE computes a QoS-constrained domain sequence based on the global network state and then converts it into multi-path routing in all related domains. Third, MATE conducts parallel inferences across relevant domains on traffic split ratios for multi-path routing using well-designed multi-agent reinforcement learning. We evaluate MATE on both real-world and synthetic topologies under various traffic patterns. In a large-scale topology encompassing 16 domains, MATE achieves near-optimal link utilization across 97% network scenarios, with an approximation ratio of below 1.3. The experimental results demonstrate MATE’s superiority in fulfilling QoS requirements, minimizing maximum link utilization, and maintaining robustness against traffic pattern variations and random link failures.