Digital twin-assisted service function chaining in multi-domain computing power networks with multi-agent reinforcement learning

Abstract

The emerging computing power network (CPN) is believed to undergo the paradigm reformation of network function virtualization (NFV) and service function chaining (SFC). It is prerequisite to explore the performance upper bound of NFV-assisted CPN before truly deploying the NFV and SFC technologies onto physical networks. Inspired by the application of digital twin (DT) in the industry and due to its advantage in synchronizing physical objects with their virtual replicas, we propose to use the DT to assist the SFC deployment in the multi-domain CPN, with the aid of multi-agent deep deterministic policy gradient (MADDPG) framework. First, we build a dynamic SFC mapping problem in the virtual twin network layer, by modeling the computing power, link bandwidth, delay performance and the VNF ordering as DT objects and constraints, to jointly optimize the energy consumption, end-to-end delay and the VNF re-deploying cost. Then, the prioritized experience replay and re-parameterization trick-empowered centralized training and decentralized execution MADDPG framework is utilized to learn the SFC deployment, by taking each domain controller as one agent. Finally, numerical experiments are carried out to validate the effectiveness of MADDPG in the cross-domain SFC deployment. For performance verification, the deployment success rate, number of crossed domains, energy consumption, end-to-end latency and load balancing degree are all taken as metrics, to show the performance of MADDPG compared to other learning frameworks.