Stable QoE-Aware Multi-SFCs Cooperative Routing Mechanism Based on Deep Reinforcement Learning

Abstract

The explosive development of the Internet of Things (IOT) has stimulated the sudden and dynamic demand pattern of potential Service Function Chain (SFC) traffic. The virtual network built based on the requirements of Multiple SFCs (Multi-SFCs), sharing the resource, should provide better Quality of Experience (QoE) for Multi-SFCs according to the real-time network status. However, simultaneous Multi-SFCs in the routing process compete for the VNF resources of the same node and preempt the same link. The instability of QoE caused by node invalidation and link faults is highlighted. Therefore, to provide a more efficient and stable service to Multi-SFCs, quantitative QoE utility and stability models and cooperative Multi-SFCs path allocation are needed. Considering the limited resources, this paper designs a stable QoE-aware Multi-SFCs Cooperative Routing Mechanism (CRM) based on Deep Reinforcement Learning (DRL). Firstly, we model the problem of Multi-SFCs cooperative routing aiming at optimizing diverse QoEs of utility, stability and delay, and formulate it as a multi-objective optimization problem. Moreover, the QoE stable queueing model based on node and link availability is formulated by Lyapunov drift, which can potentially mitigate the QoE fluctuations. Then we associate triple Dueling Deep Q-Networks (DDQNs) and propose a cooperative computing multi-task DRL to obtain the optimal path allocation policy. Experiments are conducted to verify the effectiveness and the results show that our method outperforms its peers on QoE, delay, throughput and stability.