Q-Learning based SFC deployment on Edge Computing Environment

Abstract

Reinforcement learning (RL) has been used in various path finding applications including games, robotics and autonomous systems. Deploying Service Function Chain (SFC) with optimal path and resource utilization in edge computing environment is an important and challenging problem to solve in Software Defined Network (SDN) paradigm. In this paper we used RL based Q-Learning algorithm to find an optimal SFC deployment path in edge computing environment with limited computing and storage resources. To achieve this, our deployment scenario uses a hierarchical network structure with local, neighbor and datacenter servers. Our Q-Learning algorithm uses an intuitive reward function which does not only depend on the optimal path but also considers edge computing resource utilization and SFC length. We defined regret and empirical standard deviation as evaluation parameters. We evaluated our results by making 1200 test cases with varying SFC-length, edge resources and Virtual Network Function's (VNF) resource demand. The computation time of our algorithm varies between 0.03~0.6 seconds depending on the SFC length and resource requirement.