DRL approach for online user-centric QoS-Aware SFC embedding with dynamic VNF placement

Abstract

In this paper, we tackle online Service Function Chaining (SFC) embedding issue in Software Defined Network (SDN)/Network Function Virtualization (NFV) enabled networks using Deep Reinforcement Learning (DRL) approach. This approach investigates the problem of Quality of Experience (QoE)/Quality of Service (QoS) driven SFC embedding with dynamic Virtual Network Functions (VNF) placement. We particularly focus on video streaming traffic requiring a given Mean Opinion Score (MOS) quantified in terms of QoE. In this regard, we propose a DRL algorithm based on Deep-Q-Network (DQN) (named DQN_QoE/QoS_SFC algorithm) implementing an optimization problem maximizing QoE while meeting QoS requirements. Our algorithm named DQN_QoE/QoS_SFC improves previous work given in Chen et al. (2018) dealing with the same issue through bringing performance evaluation of the training phase by testing the learning quality. This is achieved through using a QoE Threshold Score (QoETh−Sc) to be attained on average by the DQN_QoE/QoS_SFC agent along the last 100 runs of the training phase while maximizing the expected cumulative reward. We conduct simulation experiments through two performance metrics related to the SFC request namely QoE (QoE) and Rejection Ratio (RR). We investigate also how these metrics evolve for the proposed algorithm when compared with two standard baseline algorithms (Violent search, Random search) during learning phase to achieve satisfactory training quality. We highlight how DQN_QoE/QoS_SFC algorithm behaves along the training process and how it attempts reaching a near-optimal solution as closer as possible to the optimal solution provided by Violent search algorithm.