Deep Reinforcement Learning-based Dynamic SFC Deployment in IoT-MEC Networks

Abstract

Mobile Edge Computing (MEC) is a promising paradigm processing the real-time Internet of Things (IoT) service requests at the edge of the network which is close to IoT terminals. These IoT service flows can be expressed as Service Function Chain Requests consisting of several ordered Virtual Network Functions (IoT-SFCRs). Due to the dynamic nature of IoT network which is caused by the mobility of IoT terminals and the randomness of IoT service requests, it is a challenging problem to perform the dynamic SFC deployment for IoT-SFCRs. In this paper, we decompose the dynamic SFC deployment problem in IoT-MEC networks as two sub-problems, Virtual Network Function (VNF) placement sub-problem and routing path determining sub-problem. We use the Markov Decision Process (MDP) to model the two sub-problems. In order to minimize the weighted sum of resource consumption cost and end-to-end delay of IoT-SFCRs as well as consider the load balance of the network, a Deep Reinforcement Learning-based (DRL) joint heuristic algorithm which place VNF in a batch way to obtain the global optimize solutions. Compared with the state-of-the-art methods, numerical results show that our proposed algorithm can increase the success acceptance rate of IoT-SFCRs by 17% and the average reward of IoT-SFCRs by 23.8% under three typical types of network.