Deep Reinforcement Learning for Provisioning Virtualized Network Function in Inter-Datacenter Elastic Optical Networks

Abstract

In today’s datacenters (DCs), IT resources virtualization is leveraged to realize Network Function Virtualization (NFV) over general-purpose servers. Meanwhile, most of the service providers (SPs) are planning to use Virtual Network Functions (VNFs) to provide agile and flexible network services. In service provisioning, the VNF selection and mapping greatly affect IT resource utilization in DCs and spectrum resource utilization in optical networks. This paper proposes a Deep Reinforce Learning (DRL)-based algorithm for VNF provisioning. By selecting appropriate VNFs for the service requests, the algorithm intelligently guarantees efficient reusing of deployed VNFs while consuming fewer spectrum resources in inter-DC elastic optical networks (EONs). To facilitate the decision-making of the DRL agent, we first decompose the complex VNF-based service chaining (VNF-SC) into several VNF components (VNFCs), which can be solved one-by-one in turn. Then, a feature matrix-based encoding scheme is designed to represent the set of the VNFCs, the available DCs for the VNFCs, and the VNFC being operated, i.e., the input of neural networks. In addition, considering the complexity and difficulty of the VNF-SC provisioning problem, Double Deep Q Network (DDQN) is introduced in the proposed algorithm. Finally, compared with the benchmark heuristics, the extensive simulation results in different network topologies show that the proposed algorithm can reduce the IT and spectrum resource consumption by at least 9.6% and 1.6%, which proves the effectiveness of the proposed DRL-based VNF provisioning algorithm.