Double-Agent Reinforced vNFC Deployment in EONs for Cloud-Edge Computing

Abstract

With an effective service provisioning strategy that relies on Network Function Virtualization (NFV), cloud-edge computing can enhance the Quality of Service (QoS) in Elastic Optical Networks (EONs). NFV emerges as a promising technology to provide flexible services by orchestrating different virtual Network Function Chains (vNFCs). However, the vNFC deployment process is complex and involves two stages, i.e., allocating diverse Virtual Network Functions (VNFs) onto different physical nodes and routing suitable paths for Virtual Links (VLs). How to coordinate the above two stages to deploy vNFCs efficiently in EONs for cloud-edge computing is extremely important. Previous vNFC deployment algorithms are heuristic policies that disregard heterogeneous characteristics in cloud-edge computing. Recently, single-agent Deep Reinforcement Learning (DRL) can provide adaptive allocation schemes by perceptual learning from the environment. Whereas, its performance is not as good as expected since it makes the process of VFNs deployment and VLs deployment independent. Therefore, we propose a Double-Agent Reinforced vNFC Deployment algorithm (DARD) to integrate the VNFs and VLs deployment stages by two cooperative DRL agents. Additionally, we formulate the vNFC deployment problem as a Mixed Integer Linear Programming (MILP) model to achieve optimal solutions. The performance of DARD is evaluated in both static and dynamic scenarios. Simulation results show that DARD can achieve approaching performance with the MILP model. Moreover, it performs better than the other five state-of-art algorithms.