A Two-Stage GCN-Based Deep Reinforcement Learning Framework for SFC Embedding in Multi-Datacenter Networks

Abstract

Network Function Virtualization (NFV), which decouples network functions from hardware and transforms them into Virtual Network Functions (VNFs), is a crucial technology for data center (DC) networks. A service function chain (SFC) is composed of an ordered set of VNFs and virtual links (VLs) connecting them. To optimize the resource allocation in DC networks, we need to efficiently map SFCs onto the physical network. Nevertheless, the dynamics and diversity of SFC requests in multi-datacenter (MDC) networks pose a significant challenge in embedding SFCs. To overcome this challenge, we design a two-stage graph convolutional network (GCN) assisted deep reinforcement learning (DRL) scheme. This framework aims to maximize the overall acceptance ratio of SFC requests while minimizing the total cost in an MDC network. In the first stage, we propose a GCN-based DRL algorithm as a coarse granularity solution to the SFC embedding problem from the macro perspective. This solution outlines a local observation scope (LOS) for each agent in the multi-agent system of the second stage, where all agents simultaneously handle SFC requests from their respective DCs using a multi-agent framework from the micro perspective. Numerical evaluations show that, compared to state-of-the-art methods, the proposed scheme improves the acceptance ratio by approximately 13% compared with the Kolin algorithm and 18% compared with the DQN algorithm and saves the cost by around 28% compared with the Kolin and the DQN.